Selective electronic content projection
By identifying and reporting the actions of the presenter device user, changing the rendering of electronic content by the target device in real time, solving the problem of insufficient dynamic and smooth projection in the prior art, and achieving flexible and efficient projection interaction in large-scale workplace environments.
Patent Information
- Application Number
- CN202080076829.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2019-11-08
- Filing Date
- 2020-10-30
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-10-30
AI Technical Summary
The technology used by existing devices to project electronic content in large workplaces or enterprise environments is not dynamic and smooth enough, especially when deployed across multiple members and across multiple locations, and often requires specific networking and pairing processes.
By identifying and reporting the actions of the presenter device user during the projection session, the rendering of electronic content by the target device in real time is changed, multiple projection modes and dynamic target device selection are supported, eliminating peer-to-peer network requirements.
It realizes more dynamic and smooth electronic content projection interaction in large-scale workplace environments, improves projection flexibility and adaptability, and reduces network connection limitations.
Smart Images

Figure CN114730326B_ABST
Abstract
Description
Background Art
[0001] Although there are various consumer-oriented devices that can be used to perform media streaming projection such as online movie services, and there are screen projection functions provided by various teleconferencing applications and services, in general, these solutions are not well suited for workplace or enterprise environments where it is desired to deploy projection target devices to a large number of members and / or across many locations. For example, many consumer-oriented devices operate with a "same network" requirement, such as performing peer-to-peer communications. Similarly, many projection schemes require a one-to-one pairing / authorization process to be performed between a specific pair of presentation devices that control the projection session and projection target devices (which may include, for example, a large format screen suitable for remote viewing by multiple people). Improved projection technology is needed to obtain more dynamic and smooth projection interactions in newer workplace scenarios. Summary of the invention
[0002] According to a first aspect of the present disclosure, a system for projecting electronic content includes one or more processors and a machine-readable medium including instructions, the instructions, when executed by the one or more processors, cause the one or more processors to: identify a first action performed by a user of a presenter computing device at a first time during a projection session that affects the rendering of a first electronic content item being projected to a first target computing device via the projection session. The instructions may also cause the one or more processors to: determine that the presenter computing device is operating in a first projection mode at the first time. In addition, the instructions may cause the one or more processors to: change the rendering of the first electronic content item by the first target computing device by causing the first action to be reported to the first target computing device in real time, based at least on determining that the presenter computing device is operating in the first projection mode at the first time. The instructions may also cause the one or more processors to: identify a second action performed by the user at a second time during the projection session that affects the rendering of the first electronic content item, wherein the second time is different from the first time. In addition, the instructions may cause the one or more processors to: determine that the presenter computing device is operating in a second projection mode different from the first projection mode at the second time. Additionally, the instructions may cause the one or more processors to determine, based at least on a determination that the presenter computing device is operating in the second projection mode at the second time, that the second action is not to be reported in real time to the target computing device.
[0003] According to a second aspect of the present disclosure, a method for projecting electronic content may include: identifying, at a first time during a projection session, a first action performed by a user of a presenter computing device that affects rendering of a first electronic content item being projected to a first target computing device via the projection session. The method may also include: determining that the presenter computing device is operating in a first projection mode at the first time. The method may also include: changing the rendering of the first electronic content item by the first target computing device by causing the first action to be reported in real time to the first target computing device, based at least on the determination that the presenter computing device is operating in the first projection mode at the first time. In addition, the method may include: identifying, at a second time during the projection session, a second action performed by the user that affects rendering of the first electronic content item, wherein the second time is different from the first time. Likewise, the method may include: determining that the presenter computing device is operating in a second projection mode different from the first projection mode at the second time. In addition, the method may include: determining that the second action will not be reported in real time to the target computing device, based at least on the determination that the presenter computing device is operating in the second projection mode at the second time.
[0004] This summary is provided to introduce a selection of concepts in a simplified form that are further described in the detailed description below. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that solve any or all disadvantages noted in any part of this disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0005] The accompanying drawings depict one or more implementations according to the present teachings by way of example only and not limitation. In the drawings, the same reference numerals refer to the same or similar elements. In addition, it should be understood that the accompanying drawings are not necessarily drawn to scale.
[0006] Figure 1A and Figure 1B An example of environment 100 is illustrated in which a first attendee is initiating a projection session of an electronic content item from a first participant computing device to a first projection target computing device.
[0007] Figure 2A and Figure 2B An exemplary system configured to implement the electronic content projection techniques described herein is illustrated. Figure 2A The system is illustrated. Figure 2B The diagram shows the Figure 2A An exemplary implementation of a teleconferencing service in the system shown in . Figure 2CThe diagram shows the Figure 2A An exemplary implementation of a participant computing device in a system such as that shown in Figure 1A and Figure 1B The first participant computing device is shown in . Figure 2D The diagram shows the Figure 2A An exemplary implementation of a projection target computing device in a system such as that shown in Figure 1A and Figure 1B The first projection target computing device shown in .
[0008] Figure 3A , Figure 3B , Figure 3C , Figure 3D and Figure 3E The diagram shows Figure 1A and Figure 1B Further details of initiation of a projection session are shown in . Figure 3A An example of a user interface (UI) presented on a display of a first participant computing device is shown, which is used by a first attendee to initiate a projection session, such as in FIG. Figure 1A As shown in . Figure 3B shows the response to Figure 3A An example of a UI related to identification of candidate projection target computing devices presented upon actuation of the UI controls presented in . Figure 3C The diagram shows Figure 3B , where the UI presents a projection target computing device list showing projection target computing devices identified for a first participant computing device. Figure 3D An example of a UI presented by the first participant computing device for selecting an electronic content item for projection is illustrated. Figure 3E An example is illustrated in which the selected electronic content item is being projected to the first projection target computing device, resulting in the system causing the selected electronic content item to be rendered by native application 243a on a display of the first projection target computing device.
[0009] Figure 4 An alternative approach to selecting an electronic content item for projection is illustrated, wherein a projection session is initiated from a native application used to render the electronic content item.
[0010] Figure 5A and Figure 5B Another approach to selecting an electronic content item for projection is illustrated, in which the electronic content item is selected from an electronic content list prior to selecting a candidate projection target computing device.
[0011] Fig. 6A , Figure 6B and Figure 6CIn the illustrated example, the presenter performs an additional activity on the first participant computing device that changes the content of the display of the first participant computing device without changing the rendering of the electronic content item by other endpoints of the projection session. Fig. 6A In the embodiment, the display is used to render the electronic content item on the first participant computing device, such as in Figure 3E As shown in . Figure 6B shows the response to Fig. 6A An example of performing additional activities by actuating a UI control in . Figure 6C Various examples are shown in which the display is used for additional activities while the projection session continues.
[0012] Fig. 7A , Figure 7B , Figure 7C , Fig.7D , Fig. 7E and Figure 7F An example is illustrated of a presenter device recognizing an action performed by a second attendee and transmitting a corresponding encoded action and a corresponding response from a second projection target computing device, including an effect associated with a projection mode selectively changed by the presenter. Fig. 7A An example of rendering an electronic content item on a display of a presenter device, which renders the electronic content item on a display of a second cast target computing device, is illustrated. Figure 7B An example is shown in which a presenter has added a second content portion, resulting in a second cast target computing device rendering the second content portion via the cast session. Figure 7C shows the areas in which the presenter must Figure 7B Examples of different projection mode operations in which the presenter device will not automatically generate and distribute various encoding actions. Fig.7D In the example shown, Figure 7C While the projection mode is operating, the presenter has further added content portions and also selectively chooses to project the initially unprojected content portions. Fig. 7E Pictured Fig.7D The content of the Fig.7D The projection selection UI control presented in the embodiment of the present invention is projected to the second projection target computing device by actuation of the projection selection UI control presented in the embodiment of the present invention. Figure 7F The diagram shows the response to Figure 7C Example of projected mode operation without generating encoded navigation actions.
[0013] Figure 8 is a flow diagram illustrating an implementation of an exemplary process for projecting an electronic content item to a project-target computing device.
[0014] Fig. 9is a block diagram illustrating an exemplary software architecture, portions of which may be used in conjunction with the various hardware architectures described herein, which may implement any of the features described herein; Fig.10 is a block diagram illustrating components of an exemplary machine configured to read instructions from a machine-readable medium and perform any of the features described herein. DETAILED DESCRIPTION
[0015] In the following detailed description, many specific details are set forth by way of example in order to provide a thorough understanding of the relevant teachings. However, it should be apparent that the present teachings can be practiced without such details. In other cases, well-known methods, processes, components and / or circuits have been described at a relatively high level without details to avoid unnecessarily obscuring various aspects of the present teachings. In the following materials, indications of direction (such as "top" or "left") are used only to provide a reference frame during the following discussion and are not intended to indicate a desired, desired, or intended direction of the described article unless explicitly indicated.
[0016] The term "projection" relates to real-time document presentation, wherein interaction between a presenting user ("presenter") and electronic content (such as, but not limited to, a document created using a content creation application) via a first computing device is graphically rendered via one or more computing devices other than the first computing device. Typically, projection involves identifying and transmitting pixel areas on a display of the first computing device (e.g., pixel areas that have changed from a previous frame) for graphical rendering by one or more other computing devices. In lieu of such a pixel-based approach, the scheme described herein involves the use of a native application that is designed for a particular form of electronic content (e.g., rendering a word processing document using a word processor application, or rendering as a spreadsheet document using a spreadsheet application), and the native application is configured to interpret and render the electronic content using computing resources of the computing device on which the native application is running.
[0017] Figure 1A and Figure 1B An example of a first environment 100 is illustrated in which a first attendee 120 is initiating a projection session of a first electronic content item 144 (which may be referred to as “electronic content” or “content”) from a first participant computing device 130 (which may be referred to as a “participant device,” “user computing device,” or “user device”) to a first projection target computing device 110 (which may be referred to as a “projection target device,” “target device,” or “projection target”). It will be appreciated that a file or document is an example of an electronic content item. Figure 1A and Figure 1BIn the example shown in , environment 100 is a conference room, where a first target device 110 and a table 102 are permanently located. A first attendee 120 and a second attendee 122 and a third attendee 124 are seated at table 102. Attendees 120, 122, and 124 may attend a scheduled or unscheduled meeting together in first environment 100. In this particular example, attendees 120, 122, and 124 are the only attendees of the meeting. Figure 1A At the time shown in , the first display device 112 (which may be referred to as a "display") included in the first target device 110 is not used in conjunction with the conference. For example, the first target device 110 may be in a reduced power sleep state in which the display 112 is powered off. The display 112 provides a large display area that is well suited for viewing from a distance.
[0018] exist Figure 1A , a first attendee 120 wishes to display and discuss a first electronic content item 144 that is accessible to other attendees 122 and 124 via a first participant device 130. In this example, the first participant device 130 is in the form of a handheld portable smart phone computing device, but the first participant device 130 may be embodied in other forms, such as, but not limited to, a tablet computer, a notebook or laptop computer, a desktop computer, and / or a console computing device (e.g., a console provided on the table 102). The first participant device 130 includes a first native application that is configured to locally obtain and render the first electronic content 144 (e.g., in the form of an electronic file stored in a local storage device contained in the first participant device 130 or retrieved from a network accessible storage service) using computing resources contained in the first participant device 130. However, while the first participant device 130 may be effective for personal use to view and / or edit the first electronic content 144, in Figure 1A and Figure 1B In the context of the illustrated meeting, it may be more efficient to render the first electronic content 144 via the first target device 110 for viewing by all attendees 120, 122, and 124. Figure 1A , a first attendee 120 is interacting with a first user interface (UI) 140 presented on a display 132 included in a first participant device 130. Via the first UI 140, the first attendee 120 requests that a first electronic content 144 be projected to a first target device 110.
[0019] In response to the first attendee 120 and Figure 1A The interaction of the first UI 140 in Figure 1BIn the example, projection of first electronic content 144 has begun in a first projection session, where first attendee 120 is the "presenter" of the projection session. The first electronic content 144 (in this example, for a program such as Microsoft PowerPoint TM 132) is rendered by a second native application (which may be the first native application) running on the first participant device 130 in a second UI 142 on the display 132. Concurrently, and during the casting session for the first electronic content 144, the first electronic content 144 is rendered by a third native application running on the first target device 110 in a third UI 150 on the display 112. By using the third native application to render the first electronic content 144 on the first target device 110, the rendering resolution, quality, and capabilities are not those rendered to the first participant device 130, the first native application, or the second native application, as would typically occur when casting is performed by “screenshotting” graphical content of the display 132; for example, the screenshot resolution and / or quality may be significantly reduced due to the first participant device 130 being in a portrait orientation rather than a landscape orientation.
[0020] During the projection session, actions performed by attendee 120 that affect rendering of first electronic content 144 on first participant device 130 are identified, encoded, and transmitted by first target device 110. In response to receiving the encoded action, first target device 110 performs an equivalent action via a third native application and thereby affects rendering of first electronic content 144 by first target device 110 in parallel with the action performed on first participant device 130. The encoded action may include a navigation action that changes a sub-portion of the electronic content being rendered. For slides rendered by a presentation application, navigation actions may include "next slide" (continue to the slide or page immediately after the currently rendered slide or page), "previous slide" (continue to the slide or page immediately before the currently rendered slide or page), and / or "go to page" (continue to any identified slide or page) actions. For some electronic content, navigation actions may include a "scroll" action for scrolling up or down through a view of the electronic content that extends beyond the currently displayed portion. The encoding actions may include editing actions that change portions of the first electronic content 144, such as, but not limited to, adding or removing text, adding, removing, resizing, and moving graphical components, and formatting actions (such as, but not limited to, character formatting and paragraph formatting). The encoding actions may include annotation actions, such as, but not limited to, highlighting of components, inking, and presentation and / or movement of visual pointers. In some examples, annotation actions are applied only during the rendering of the first electronic content 144 during the projection session, and the first electronic content 144 itself is not modified. In some implementations, annotations may be rendered via an overlay UI rather than by a native application. By modifying the rendering of the first electronic content 144 on the display 112 in response to actions received in real time, the first target device 110 provides an engaging canvas for collaborative discussion of the first electronic content 144.
[0021] Figure 2A , Figure 2B , Figure 2C and Figure 2D Portions of an exemplary system 200 configured to implement the electronic content projection techniques described herein are illustrated. Figure 2A and Figure 2B The examples shown in Figure 1A and Figure 1B Any of the features described. Figure 2AA plurality of attendees 220 located in various environments 210 are shown, including a fourth attendee 220a, a fifth attendee 220b, and a sixth attendee 220c in a second environment 220a, a seventh attendee 220d and an eighth attendee 220e in a third environment 210b, and a ninth attendee 220f in a fourth environment 210c. In addition, there are a plurality of target devices 240, including a second target device 240a located in the second environment 210a, a third target device 240b located in the third environment 210b, and three target devices 240c, 240d, and 240e located in various locations and environments.
[0022] Likewise, a portion of the attendees 220 are associated with corresponding participant devices 230, wherein attendees 220a, 220b, 220d, and 220f are associated with corresponding second participant devices 230a, third participant devices 230b, fourth participant devices 230d, and fifth participant devices 230f, respectively. When participant devices 230 are used to participate in a teleconference session, the associated attendees 220 may also be referred to as "users" of the system 200 and / or "participants" of the teleconference session. Each of the environments 210a, 210b, and 210c may be considered to be "remote" from one another. For example, from the perspective of the fourth attendee 220a in the second environment 210a, the third environment 210b and the third target device 240b, the participant device 230d, and the attendees 220d and 220e are each "remote" to the attendee 220a; similarly, the second environment 210a and the second target device 240a, the participant devices 230a and 230b, and the attendees 220a, 220b, and 220c are each "local" to the fourth attendee 220a. It is understood that the number of elements shown for the system 200 (such as the number of attendees 220, the number of participants, the number of environments 210, the number of participant devices 230, and the number of target devices 240) are merely examples, and other numbers of such elements may be included in the system 200.
[0023] As in Figure 2A , each of the participant devices 230 and the target devices 240 are configured to interact with the teleconferencing service 260 via one or more networks 250. It will be appreciated that different devices 230 and 240 may communicate with the teleconferencing service 260 via different network arrangements, including different network technologies (e.g., various devices 230 and 240 may utilize wired, mobile, satellite, and / or WiFi "last mile" network connections), different network operators, and geographically different networks (e.g., environments 210a, 210b, and 210c may be located in different countries and therefore rely on different network segments).
[0024] Figure 2B An exemplary implementation of a teleconferencing service 260 is illustrated. The teleconferencing service 260 utilizes and, in some implementations, manages user data 261 for various user service accounts identified by the teleconferencing service 260, including, in this example, first user data 262a for a first user service account associated with a fourth attendee 220a, second user data 262b for a second user service account associated with another of the attendees 220, and additional user data 261 for other user service accounts. In some implementations, the user data 261 includes organizational information for the user service accounts, such as, but not limited to, identification of teams, groups, organizations, hierarchies, roles, and / or locations that have been associated with various users. In some implementations, the teleconferencing service 260 is configured to interact with a directory service 290, such as a service that implements a Microsoft Active Directory service, to obtain organizational information for the user service accounts.
[0025] Teleconferencing service 260 also includes user authenticator 263, which is configured to authenticate a computing device (such as participant device 230) and / or a software instance running on the computing device (such as a software instance running on a server) for a user service account associated with teleconferencing service 260. Figure 2C credential-based authentication is performed by the teleconferencing application 232 running on the exemplary participant device 230 shown in . Thus, interactions between the participant device 230 and the teleconferencing service 260 are completed in association with a particular user service account, thereby allowing the teleconferencing service 260 to perform authorization procedures for various projection-related operations, such as identifying target devices 240 that allow the user service account to be used for projection. In some implementations, the user authenticator 263 supports the use of credentials obtained from an external authentication service (e.g., a token issued by a single sign-on (SSO) service), and can associate a user account identifier provided by such a service with a corresponding user service account in the user data 261. In some implementations, the user authenticator 263 is configured to issue an access token to a computing device that has been successfully authenticated, and the teleconferencing service 260 is configured to receive and process the access token in subsequent communications for authentication and authorization operations.
[0026] The teleconferencing service 260 maintains target device data 264 for the various target devices 240 that have registered with the teleconferencing service 260. In this example, the target device data 264 includes first target device data 265a associated with the second target device 240a, second target device data 265b associated with another of the target devices 240, and additional target device data 264 for other target devices 240. In some implementations, the target device data 264 includes organizational information for the target devices 240, such as, but not limited to, identification of organizational entities, such as, but not limited to: users, roles, teams, groups, organizations, levels, and / or locations that have been associated with the various target devices 240. In some implementations, the target devices 240 can be associated with organizational entities; for example, the target devices 240 can be associated with a particular floor to facilitate identifying candidate target devices for projection.
[0027] The teleconferencing service 260 can be configured to provide an administrative interface (not shown) for managing target device data 264 in order to establish and maintain desired usage policies for the target device 240. In some implementations, the teleconferencing service 260 is configured to interact with the directory service 290 to obtain and maintain organizational information for the target device 240. In some implementations, the teleconferencing service 260 includes a target device authenticator 266 that is configured to allow the target device 240 and / or a software instance running on the target device 240 (such as on a server running ... Figure 2D The teleconferencing application 242 shown in FIG. 2 registers its availability with the teleconferencing service 260 when it is available for projection. In some implementations, the target device authenticator 266 is configured to perform authentication based on security credentials for the target device 240.
[0028] In some implementations, the teleconferencing service 260 includes a proximity detector 267 that is configured to identify which target devices 240, if any, are physically close to and / or may be close to a particular participant device 230 and / or associated attendee 220 based on at least one or more proximity signals received from the participant devices 230 and / or target devices 240. In some examples, the identification can be based on location information obtained from the user data 261 and / or the target device data 264. For example, a proximity signal (such as a wireless network access point identifier) can be used to identify the floor of a building where the proximity device 230 is located, and the proximity detector 267 can be configured to select the identified target device 240 based on the target device data 264 indicating that it is located on the same floor. In some examples, the proximity detector 267 is configured to obtain and apply proximity signals from other systems, such as, but not limited to, a physical access control service 292, which is configured to control and track physical access to a structure or a portion of a structure. For example, proximity identification can be based on an assumption that an attendee 220 is present on a particular floor of a building based on the attendee's recent use of a physical access control device (e.g., an RFID tag) to access that floor. Device usage activity of computing devices at known locations can be applied to similar effect. Note that in some implementations, proximity detector 267 can be implemented in part or in whole by one or more participant devices 230, one or more target devices 240, and / or teleconferencing service 260.
[0029] In some implementations, the teleconferencing service 260 includes a candidate target identifier 268 that is configured to identify a target device 240 that is currently a candidate target device for projection by a participant device 230. In some implementations, the candidate target identifier 268 is configured to use a proximity detector 267 to identify target devices 240 that are currently in physical proximity, and to select some or all of the candidate target devices from those devices that are in physical proximity. However, in some examples, there are situations where non-proximity target devices 240 are identified as candidate target devices. In some implementations, a participant device 230 identifies a specific target device 240 for processing by the candidate target identifier 268. For example, an identifier obtained for a target device 240 (e.g., based on a printed or displayed code) can be received from a participant device 230 that is considered as a candidate target device.
[0030] In some implementations, the teleconferencing service 260 includes a user / target projection policy 269 that stores policies for identifying allowed and / or disallowed target devices 240 for a user service account. In some implementations, the user / target projection policy 269 includes a whitelist and / or blacklist for a particular organizational entity; for example, the use of a particular target device 240 or group of target devices 240 for projection may be limited to user service accounts associated with a specified team or location (such as a particular building or floor). The target identifier 268 can be configured to obtain and apply a user / target projection policy 269 associated with a user service account and / or target device 240. The application of the user / target projection policy 269 can be based on the user data 261, the target data 264, and / or information obtained from the directory service 290.
[0031] Telephone conference service 260 maintains and uses conference session data 270 for currently active telephone conference sessions, and in some examples, for scheduled telephone conference sessions. Figure 2B In FIG. 1 , conference session data 270 includes ad hoc session data 271 for an unscheduled telephone conference session. For example, an ad hoc session may be instantiated as a session that should be scheduled in Figure 1B In some implementations, the teleconferencing service 260 supports scheduling sessions in advance, including scheduling sessions, identifying locations, sending attendee invitations, and / or identifying resources for sessions, and maintains associated scheduled session data 272. For a currently active session, the conference session data 270 may include information identifying current participants, current participant devices 230, whether the session has a projection component, which target devices 240 are being used to render the projection component, and participants acting as presenters for the projection component.
[0032] The teleconferencing service 260 includes a session manager 273 that is configured to instantiate and manage active teleconferencing sessions. The session manager 273 is configured to receive and respond to requests received from the participant devices 230 and the target devices 240, and to issue commands to the participant devices 230 and the target devices 240. The session manager 273 is configured to instantiate and manage session components, such as, but not limited to, a projection component and an audio-visual component (e.g., the reception and distribution of audio-visual streams from and to multiple endpoints). With respect to the projection component, the session manager 273 is configured to identify (and, in some examples, provide) electronic content that is being projected, so that the endpoint renders the electronic content using an associated native application, receive an action stream encoding actions from the presenter device, and distribute the action stream to non-rendering rendering endpoints for modifying their rendering of the electronic content accordingly. In some examples, a session may be primarily limited to the projection component and not include, for example, an audio-visual component, such as with respect to Figure 1BAs the example shown in might do.
[0033] Figure 2C An exemplary implementation of a participant device 230 is shown, which may be used to Figure 2A . In this example, the participant device 230 is configured to be able to act as a presenter device that transmits encoded actions, and also to act as a projection display endpoint that receives and responds to encoded actions generated by the presenter device. The participant device 230 includes a display 231, as shown in Figure 1A and Figure 1B . The participant device 230 is also configured to run a first teleconferencing application 232 that is configured to interact with the teleconferencing service 260 to perform projection. In some examples, the first teleconferencing application 232 is primarily directed to interacting with various functions available via the teleconferencing service 260, including projection. In some examples, the first teleconferencing application 232 is primarily directed to other functions, such as content creation, but is a "projection-aware" application that is capable of interacting with the teleconferencing service 260 to at least perform projection. For example, the first teleconferencing application 232 can be a content creation application that is capable of performing projection of currently open electronic content.
[0034] Whether used as a presenter endpoint or a projection display endpoint, participant device 230 is configured to run a native application 233a associated with the electronic content being projected to render the electronic content on display 231. In some examples, multiple native applications 233 (each of which is configured to render a corresponding content type) are available on participant device 230, and participant device 230 is configured to run a native application 233 corresponding to the content type of the electronic content being projected. In some examples, native application 233 is not configured to generate and / or process encoding actions for projection, and a native application interface 232a is used, which is configured to interact with native application 233 in order to generate and / or process encoding actions for projection. Although as described in Figure 2C As shown in , the native application interface 232a can be implemented as part of the teleconferencing application 232, but in some implementations, the native application interface 232a can be implemented as a standalone application, a wrapper application and / or a plug-in component for the native application 233, and can be configured to interact directly with the teleconferencing service 260 for projection.
[0035] In some implementations, participant device 230 may include one or more output devices 234 that native application 233 is configured to use to render electronic content. For example, rendering of electronic content may result in output of an audio signal by a speaker included in participant device 230.
[0036] In some implementations, the participant device 230 may include a proximity transceiver 235 (which in various examples may be configured to sense a proximity indication signal, generate a proximity indication signal, or sense and generate a proximity-related signal), and the system 200 (e.g., via the teleconferencing application 232) is configured to utilize the proximity transceiver 235 to generate a proximity indication signal and / or process the sensed proximity indication signal. In some examples, the proximity transceiver 235 (a portion of which may be implemented by the teleconferencing application 232) is configured to provide a proximity signal to the teleconferencing service 260 (e.g., via the teleconferencing application 232) for use by the proximity detector 267, the proximity signal reflecting the proximity between the participant device 230 and the target device 240. An exemplary proximity indication signal is a short-range radio transmission, such as, but not limited to, a Bluetooth transmission or a near field communication (NFC) transmission, which indicates the presence of a projection-enabled device. Another example of a proximity indication signal is a wireless access point identifier (such as, but not limited to, a service set identifier (SSID) or a media access control (MAC) address). For some proximity indication signals, the proximity transceiver 235 can also provide a proximity signal based on the measurement result based on the measurement result, and the measurement result is such as but not limited to the received strength of the signal (such as but not limited to the RSSI value) and / or the noise level (such as but not limited to the signal-to-noise ratio (SNR) value). The proximity transceiver 235 can be configured not to provide a proximity signal for the proximity indication signal when such a measurement result does not meet the threshold condition (e.g., the minimum RSSI threshold). The proximity transceiver 245 can be configured to generate a proximity signal based on the successful decoding, decryption and / or verification (e.g., using a cryptographic hash) of the data content of the received proximity indication signal. For some proximity indication signals, the proximity transceiver 245 can be configured to provide an estimated proximity (e.g., based on the RSSI value).
[0037] Participant device 230 includes one or more input devices 236, such as, but not limited to, a keyboard and / or a pointer device (e.g., a mouse or trackpad). When participant device 230 operates as a presenter device, participant device 230 (e.g., via native application 233 that renders electronic content) can respond to various user inputs received from a user via (one or more) input devices 236, resulting in the generation of associated encoding actions. For example, keyboard input can cause text to be added to the projected electronic content on the presenter device, the corresponding encoding action is provided to target device 240, and target device 240 updates its rendering of the electronic content to include the added text. When participant device 230 operates as a projection display endpoint, in some examples, participants are able to asynchronously interact with the projected electronic content through user input submitted via input device 236. For example, participants can navigate to other parts of the document and / or can make independent changes to the document.
[0038] In some examples, the participant device 230 includes a local storage device 237, such as, but not limited to, a hard drive or a flash memory device. In some examples where the participant device 230 operates as a presenter device, the electronic content is stored in the local storage device 237 as a local file 238 before being selected for projection; for example, as a file 238 created by a content creation application running on the participant device 230. In such examples, the process of projecting the electronic content includes distributing the locally stored file 238 to any other endpoints that render the electronic content during the projection session. In some examples, a participant may edit a first electronic content while it is being projected to the participant, and in response to detecting the edit, the system 200 may be configured to create a second electronic content item that is different from the first electronic content item and reflects the edit that has been applied to the first electronic content item. For example, the second electronic content item may be stored locally in the local storage device 237 as a file revision 239. This allows the participant to retain the edits after the projection session is completed, which may include notes made by the participant during the projection session.
[0039] Figure 2D An exemplary implementation of a target device 240 is shown, which may be used to Figure 2A Any of the target devices 240 shown in FIG. Target device 240 includes a display 241, such as that shown in FIG. Figure 1A and Figure 1B Target device 240 also includes a teleconferencing application 242 and a native application 243a, which can be configured much like the native applications 243a and 243b, respectively, for the target device 110. Figure 2C230 as described above. However, the teleconferencing application 242 is configured to interact with the teleconferencing service 260 as the target device 240 (e.g., by authenticating with the teleconferencing service 260 via the target device authenticator 266). In various implementations, the target device 240 may include a native application interface 242a, multiple native applications 243, (one or more) output devices 244, and / or a proximity transceiver 245, which may be configured much like the native applications 232 and native applications 233a for the participant device 230 in FIG. Figure 2C As described above with respect to the native application interface 232a, native application 233, output device 234, and proximity transceiver 235 of the participant device 230 in FIG.
[0040] Return to reference Figure 2A In some implementations, system 200 includes or is configured to interact with a storage service 280 that is configured to store files and provide the files to participant devices 230 and target devices 240 via network(s) 250. In some examples, storage service 280 maintains files associated with a user account. For example, in Figure 2A 2, storage service 280 maintains first user files 282a associated with a first user account associated with attendee 230a, including file 284a and file revision 286a, and maintains second user files 282b associated with a second user account associated with another of attendees 230, including file 284b and file revision 286b. In some implementations, storage service 280 supports token-based access, allowing access to a file or set of files for a computing device presenting an associated token. In some examples, the token may only provide read-only access to the file or set of files. In some examples, a presenter selects electronic content maintained in storage service 280 for projection, resulting in the generation of an associated token (or multiple tokens) and distribution to projection endpoints for each endpoint to access and retrieve the electronic content from storage service 280. In some implementations, both the teleconferencing service 260 and the storage service 280 support the use of the same credentials obtained from an external authentication service (e.g., a token issued by an SSO service), and a user account identifier provided by such a service can be associated with the corresponding user service account of the teleconferencing service 260 and the user account of the storage service 280, thereby simplifying the system 200's access to files maintained by the storage service 280.
[0041] Figure 3A-6B The diagram shows the Figure 1A and Figure 1B Another example of the first environment 100 is shown in FIG. Figure 3A-6BIn the discussion of FIG. 1 , the first environment 100, the first target device 110, the first participant device 130, and the attendees 120, 122, and 124 correspond to the first environment 100, the first target device 110, the first participant device 130, and the attendees 120, 122, and 124, respectively. Figure 2A 2. Thus, the first participant device 130 and the first target device 110 are described as being associated with the teleconferencing service 260 and the second environment 210a, the second target device 240a, the second participant device 230a, and the attendees 220a, 220b, and 220c. Figure 2A and Figure 2B In addition, the first participant device 130 includes a Figure 2C The features shown and described in the figure, and the first target device 110 includes Figure 2D The features shown and described in.
[0042] Figure 3A , Figure 3B , Figure 3C , Figure 3D and Figure 3E The diagram shows Figure 1A and Figure 1B Further details of initiation of a projection session are shown in . Figure 3A An example of a fourth UI 310 presented on the display 132 of the first participant device 130 is shown, which is used by the first attendee 120 to initiate a projection session, such as for Figure 1A In some examples, the fourth UI 310 can be presented by the teleconferencing application 232 running on the first participant device 130. In this example, the teleconferencing application 232 also displays a fifth UI 320 that displays an activity feed provided by the teleconferencing service 260 and is used to initiate presentation of the fourth UI 310. In other items, the first UI control 312 is provided in the fourth UI 310 for initiating the projection session.
[0043] Figure 3B shows the response to Figure 3A 312 in the actuation of the first UI control 312 in the sixth UI 330 presented. In this example, the sixth UI 330 is presented by the teleconferencing application 232 and includes a first UI indicator 332, which is presented to indicate that the system 200 is identifying Figure 3B , the candidate target devices available to the first participant device 130 at the time shown in . In various implementations, the identification of the candidate target devices is performed in whole or in part by the first participant device 130, the teleconference service 260, the first target device 110, and / or the other target devices 240. Figure 2B The techniques described in candidate target identifier 268 in .
[0044] For the system 200, a prior explicit one-to-one pairing of a first target device 110 with a first participant device 130 or first attendee 120 (such as, for example, a user service account maintained by the teleconferencing service 260) pre-authorizing use of the first target device 110 for projection from the first participant device 130 does not need to be performed in advance. This allows the various target devices 240 to be viewed and used as shared or open devices that are readily available for projection upon request. In some implementations, the system 200 is configured to identify candidate target devices based on (one or more) correspondences between organizational information obtained for the user service account (under which the first participant device 130 has been authenticated with the teleconferencing service 260) and organizational information obtained for target devices 240 (including the first target device 110). In some examples, the correspondence can be determined based on a user / target device projection policy 269, as described in detail in the example of FIG. 269. Figure 2B For example, certain target devices 240 may not be available for projection to the first attendee 120 because those target devices 240 do not satisfy the user / target device projection policy 269 with the first attendee 120 .
[0045] In some implementations, the identification of the candidate target devices can be limited, at least in part, to a subset of the target devices 240 that are determined by the system 200 to be physically proximate to or likely to be proximate to the first participant device 130 and / or the first attendee 120. For example, the system 200 can be used in conjunction with Figure 2B The proximity detector 267 in Figure 2C The proximity transceiver 235 (included in the first participant device 130) and / or Figure 2D The techniques described in the proximity transceiver 245 (included in the first target device 110) in FIG.
[0046] In some examples, the system 200 is configured to identify physically proximate target devices via a plurality of different techniques (which techniques may be separately available based on whether an associated proximity signal applied by a given technique is available for a particular device), determine that a plurality of proximity detection techniques may be used in conjunction with the participant device 230 and / or the attendee 220, and prioritize or otherwise prioritize the application of one or more of the available proximity detection techniques based on their granularity or range. For example, for the participant device 230, if the available proximity detection techniques include a first technique for identifying a first proximate target device in the same building (e.g., based on physical access control records), a second technique for identifying a second proximate target device on the same floor of the building (e.g., based on WiFi access point access records), a third technique for identifying a third proximate target device within a short-range wireless transmission range (e.g., via a Bluetooth signal), and a fourth technique for identifying a fourth proximate target device in the same room (e.g., via an acoustic signal), the system 200 may select the fourth proximate target device or both the third and fourth proximate target devices based on a higher granularity and / or a shorter range of its associated techniques than the other techniques. In some examples, target device 240 may be included and / or ranked in the candidate target devices based on a measurement-based proximity signal for target device 240 and / or an estimated proximity between target device 240 and participant device 230 .
[0047] In some examples, in response to obtaining an identifier for the target device 240 from the participant device 230 (e.g., from a printed code located on or near the target device 240 or a code displayed by the target device 240), the system is configured to determine that the participant device 230 and the target device 240 are in proximity at this time. In some implementations, the system 200 is configured to automatically include the target device 240 as a candidate target device in response to the identifier for the target device 240 having been obtained by the participant device 230 at approximately the same time.
[0048] In some implementations, the system 200 is configured to determine that an attendee 220 associated with a participant device 230 (based on an associated user account) is scheduled to attend an event at or about the current time (e.g., based on a scheduled session record 272 or an electronic calendar record) at a location identified for the event (e.g., in a location field of a record for the event), and / or the attendee 220 is determined to be participating in the event (e.g., for an attendee 220 who joins the session without a prior invitation), and in response to the determination, include one or more target devices 240 at the identified location (e.g., determined based on the target device data 264) as identified candidate target devices. For example, if the attendee 220 is participating in a scheduled session where the corresponding scheduled session record 272 identifies a first conference room as the location for the scheduled session, the target device 240 identified as being located in the first conference room will be included as a candidate target device. The system 200 may perform this operation for attendees 220 who are not actually at the identified location, which would allow the remote attendee to easily project to a target device 240 at the identified location to facilitate other attendees of the session who are not at that location to view the projected content.
[0049] In some implementations, the system 200 is configured to include one or more target devices 240 in the candidate target devices for an attendee 220 of a session (or a participant device 230 being used by an attendee) that are identified or will be identified as candidate target devices for another attendee 220 of the session (or a participant device 230 being used by another attendee). For example, in a session including a first attendee 220a in a second environment 210a (using a second participant device 230a) and a fifth attendee 230d in a third environment 210b (using a fourth participant device 230d), the candidate target devices identified for the attendee 230a may include the second target device 240a (e.g., based on proximity to the second participant device 230a) and the third target device 240b (e.g., based on its proximity to the fourth participant device 230d, which will be a candidate target device for the fourth participant device 230d). This allows an attendee 220 to easily project to a target device at a remote location to facilitate viewing of the projected content by another attendee 220 at the remote location.
[0050] In some examples, system 200 is configured to not include target device 240 in the candidate target devices based on system 200 determining that target device 240 is currently being used for projection or another function.
[0051] In some implementations, the system 200 is configured to automatically identify when a candidate target device is available to a participant device 230 and cause an indication of the candidate target device being available to be presented by the participant device 230 without receiving a request from the participant device 230 to initiate a projection session. In some examples, causing the indication to be presented can be based on determining that a plurality of attendees 220 are present in the same environment 210 as the participant device 230, as this may represent an environment in which projection will facilitate discussion of electronic content among the plurality of attendees 220. In some examples, causing the indication to be presented can be based on determining the type of environment 210 in which the participant device is located. For example, a conference room may be considered a more discreet environment 210 for projection than a more public environment 210, even though the environment 210 also has a target device 240 available for projection.
[0052] Figure 3C The diagram shows Figure 3B An example of the sixth UI 330 shown in FIG. 1 , wherein the sixth UI 330 presents a list of candidate target devices
[0053] 334, which shows the candidate target devices identified for the first participant device 130, as discussed above.
[0054] Table 334 includes one or more target device selection UI controls, including a first target device selection UI control 336. In some implementations, system 200 is configured to arrange the target device selection UI controls according to the estimated proximity and / or the technique used to identify it as a candidate target device. Figure 3C In the example shown in , the candidate target device list 334 is visually divided into a first portion 340 and a second portion 342, the first portion 340 presenting candidate target devices determined to be physically close to the first participant device 130, and the second portion 342 presenting candidate target devices identified by other means (e.g., identified as being close to the remote attendee). In this example, the first portion 340 is sorted according to the estimated proximity of each candidate target device, wherein the first target device selection UI control 336 corresponds to the candidate target device estimated to be closest to the first participant device 130. With such an arrangement of the candidate target device list 334, the first participant 120 is able to more easily select the desired target device 240 for the projection session. In this example, actuating the first target device selection UI control 336 results in selecting the corresponding target device 240 for the projection session and continuing to initiate the projection session. In some implementations, multiple target device selection UI controls can be selected simultaneously using the sixth UI control 330, thereby causing multiple target devices 240 to be used as projection display endpoints for the projection session at the same time.
[0055] Depending on the circumstances in which the first participant 120 is initiating the casting session, the first participant 120 may need to select an electronic content item for casting. Figure 3D An example of a seventh UI 350 for selecting an electronic content item for projection presented by the first participant device 130 (eg, via the telecommunication application 232) is illustrated. In various examples, the seventh UI 350 may be presented before or after the sixth UI 330. Figure 3D In the example shown in , the seventh UI 350 includes a recent content list 352 presenting a plurality of electronic content items identified by the system 200 that were recently accessed by the first participant 120 as respective content selection UI controls 354 corresponding to respective electronic content items, including a first content selection UI control 354a corresponding to the first electronic content item. In some examples, such as in Figure 3D As shown in FIG. 3 , a content search UI control 356 is included in the seventh UI 350, providing another UI (not shown) for searching electronic content items (e.g., by file name and / or content). In this example, actuation of the content selection UI control 354 results in selection of the corresponding electronic content item for the casting session and continued initiation of the casting session. Figure 3D The example of the seventh UI 350 shown in is merely illustrative and is not intended to limit how the first participant 130 identifies the electronic content item for projection. For example, on a notebook computer, a file browser UI component may be more effective. As another example, a voice-driven digital assistant may be used to select the electronic content item for projection.
[0056] In some implementations, if an electronic content item is selected for projection in advance, the content type of the selected electronic content item may be used to identify candidate target devices. For example, in response to the target device 240 itself not supporting rendering of the content type, the target device 240 may be excluded from the identified candidate projection targets. In some examples, if the target device 240 is selected before the electronic content item, and the selected target device 240 itself does not support rendering of the content type of the selected electronic content item, the first participant device 130 may request to select a different target device 240.
[0057] In response to identification of the target device 240 and the electronic content item for the projection session (e.g., via the sixth UI 330 and the seventh UI 350, but in some examples, the target device 240 and / or the electronic content item may be implicitly identified by the system 200), the system 200 begins rendering the selected electronic content item on the selected target device 240. Figure 3EAn example is illustrated in which an electronic content item 370 is being projected to a first target device 110, causing the system 200 to cause the electronic content item 370 to be rendered by the native application 243a on the display 112 of the first target device 110. The first target device 110 is configured to make real-time changes to the rendering of the electronic content item 370 in response to actions performed at and reported by the first participant device 130. For the projection session, the system 200 is also configured to cause the electronic content item 370 to be rendered on the first participant device 130 (which is in the Figure 3E The electronic content item 370 is rendered in a ninth UI 360 on a display 112 of a presenter device (as used for the projection session) at the time shown. In some implementations, the native application 233a renders the electronic content item 370 on the presenter device. However, in some examples, other schemes can be used to render the electronic content item 370 on the presenter device 370, such as, but not limited to, rendering by the teleconferencing service 260, another network accessible service, and / or the target device 110.
[0058] The system 200 configures each of the endpoint devices for the projection session (in Figure 3E , the presenter device 130 and the target device 110) obtain and render the electronic content item 370. In the example where the electronic content item 370 is stored as a local file 238 in the local storage 237 of the presenter device 130 prior to the projection session, the system 200 is configured so that the local file 238 is distributed to and rendered by other endpoints of the projection session. In the example where the electronic content item 370 is stored as a remote file 284a in the storage system 280, the system 200 is configured so that the remote file 284a is obtained by all endpoints of the projection session. For example, a universal resource identifier (URI) for the remote file 284a can be provided by the system 200 to the endpoints along with a token that allows each endpoint to access the remote file 284a.
[0059] The system 200 is configured to instantiate, maintain, and control the projection session via network-based interactions between the presenter device 130 and the teleconferencing service 260, and between the target device 110 (and other projection rendering endpoints) and the teleconferencing service 260. For example, encoded actions generated by the presenter device 130 in response to actions of the first participant 120 via the presenter device 130 are received by the teleconferencing service 260 and distributed to the projection rendering endpoint as an encoded action stream. In some examples, this can be implemented in a similar manner to the reception and distribution of audio-visual streams by the teleconferencing service 260 for sessions including audio-visual components. With this approach, there is no peer-to-peer network or associated "same network" requirement for the presentation device 130 and the target device 110, as coordination and communication for the projection session is provided by the teleconferencing service 260. It also allows for greater flexibility and diversity in network connectivity for endpoint devices, and also allows for greater flexibility and diversity in the availability and selection of candidate target devices for the presentation device, as network architecture restrictions are eliminated.
[0060] In some examples, native applications 233 and 243 are full-featured content creation applications suitable for creating new content items, such as, but not limited to, Microsoft Word, Microsoft Excel, and Microsoft PowerPoint. In some examples, system 200 is configured to operate native application 233 or 243 in "projection mode". In some examples, native application 233 or 243 presents a limited UI in response to operating in projection mode, presenting fewer distracting UI elements when rendering electronic content item 370. In some examples, native application 233 or 243 allows attendee 220 to dynamically switch between a limited UI and a more full-featured UI, such as allowing editing of electronic content item 370. In some examples, native application 233 or 243 is configured to detect actions that affect the rendering of electronic content item 370 and encode the detected actions to be distributed via the projection component of the session in response to operating in projection mode. In some examples, in response to operating in projection mode, native application 233 or 243 is configured to change its rendering in response to encoding actions (e.g., in an encoding action stream) received from conferencing service 260. By using native application 233 or 243 for rendering electronic content item 370, electronic content item 370 can be rendered according to the capabilities and characteristics of each endpoint device.
[0061] Additionally, in some examples, a participant 220 (e.g., a presenting participant or a non-presenting participant) may freely navigate to a selected portion of and / or edit an electronic content item 370 via a native application 233 or 243. The system 200 may be configured to detect edits to the electronic content item 370 as the electronic content item 370 is projected to the participant 220, and in response to the detection, the system 200 may be configured to create a new electronic content item that is different from the electronic content item 370 and reflects the edits that have been applied to the electronic content item 370. For example, the new electronic content item may be stored as a local file revision 239 or a remote file revision 286b. In an example where a non-presenting participant 220 has navigated to a portion of the electronic content item 370 that is different from the presenter 130, the system 200 may be configured to present to the non-presenting participant 220 a UI control that returns the rendering of the electronic content item 370 to the same portion as the presenter 130. Thus, the non-presenting participant 220 is able to freely view and / or edit other portions of the electronic content item 370 and easily resynchronize with the presenter 130 .
[0062] In some implementations, the system 200 may utilize document type-specific, application type-specific, and / or application-specific action sets and / or encoding of actions. For example, the information that needs to be communicated to synchronize the rendering for the (one or more) applications in question may first be defined. Then, a data structure or other means may be created to represent the information, as well as any additional information required by the content rendering endpoint to fully respond to the encoded action. The information to be communicated may depend on the application and type of the content in question. For example, for presenting a document (e.g., a PowerPoint document), the current slide number and the current animation step may be required so that the endpoint renders to follow the presenter's navigation in the document and the interaction with the document. Similarly, in a word processing document, the page / paragraph / line currently presented by the presenter may be included in the encoded navigation action. After determining the information to be communicated in the encoded action, the implementer may determine how to represent the information. The information may be modeled as a data structure, but may also be presented in another form. The presenter device and the projection rendering endpoint need to be able to create and consume encoded actions, respectively. When the presenter device detects the occurrence of a new action, it generates a corresponding encoded action that is sent by the presenter device to the teleconferencing service 260 and then to the projection rendering endpoint, which responds accordingly to the encoded action to change the rendering of the electronic content item 370. In many cases, the communication of these encoded actions consumes less network bandwidth than conventional bitmap-based conventional projection schemes.
[0063] In some implementations, in the event that the presenter device 130 has not yet joined the teleconference session via the teleconference system 260, the teleconference system 260 is configured to instantiate the projection session just as it instantiates an ad hoc teleconference session that includes an audiovisual component, but as a session limited to the projection component, with the presenter device 130 and the target device 110 as endpoints for the projection component. In the event that the target device 110 has not yet joined the teleconference session via the teleconference system 260, the system 200 is configured to join the target device 110 to the teleconference session. In the event that the teleconference system 260 has already instantiated the teleconference session, it is simpler to move from that session limited to the projection component between the two endpoints to join the remote participant 220 to the session and add the audiovisual component to allow communication between the local and remote environments 110 via the facilities provided by the teleconference system 260. In such an implementation, the same ad hoc session continues to be used by the teleconference system 260.
[0064] In some examples, the system 200 is configured to support having multiple participants 220 in a projection session, and is also configured to fluidly change presenters from one participant 220 to another participant 220. In some examples, although the initial presenter device is local to the target display 110, due to the change to a different participant 220 as the presenter, the participant 220 that is remote from the target device 110 can generate encoded actions that the target device 110 is responding to. Since the native application 233 at the participant device 230 has the complete native state of the electronic content item 370 being projected, the switch of the presenter role from one participant 220 to another participant 220 can occur without interrupting the projection session. In addition, since each endpoint renders the electronic content item 370 locally, rather than displaying pixel regions collected from the presenter device, differences in the rendering appearance between one presenter device and the next (e.g., due to differences in screen resolution) do not change the rendering of the electronic content item 370 by the other endpoints.
[0065] Figure 4 An alternative approach to selecting an electronic content item 370 for projection is illustrated, wherein the projection session is initiated from a native application 233 that is used to render the electronic content item 370. Figure 4 In the example shown in , the native application 233 renders the electronic content item 370 on the display 132 in the tenth UI 410. Much like previously in Figure 3B, in some implementations, the system 200 is configured to automatically identify when a candidate target device is available for a participant device 130, and in response to determining that the candidate target device is available, cause the participant device 130 to present a first cast initiation UI control 420 in the tenth UI 410. In some examples, the first cast initiation UI control 420 also presents a visual indication that the system 200 has determined that the candidate target device is available for the participant device 130. In response to actuation of the first cast initiation UI control 420, the electronic content item 370 rendered in the tenth UI 410 is selected for casting, and the process of initiating a casting session continues; for example, as in Figure 3A and / or Figure 3B In some examples, rendering in Figure 4 The native application 233 of the electronic content item 370 in is also used to render the electronic content item 370 during the casting session.
[0066] Figure 5A and Figure 5B Another approach to selecting an electronic content item 370 for projection is illustrated, in which the electronic content item 370 is selected from the electronic content list 512 before selecting a candidate target device. Figure 5A , the eleventh UI 510 is being presented on the display 132 (e.g., via the teleconferencing application 232). In this example, the eleventh UI 510 includes Figure 3D 350 in the seventh UI 350. The recent content list 512 includes a plurality of content selection UI controls 514, each of which corresponds to a respective electronic content item and includes a respective content action UI control 516 for performing an action involving the electronic content item, as shown by the first content selection UI control 514a including the first content action UI control 516a.
[0067] Figure 5B The diagram shows the response to Figure 5A 516a is actuated; and a content item action list 524 includes a second cast initiation UI control 526, which is the same as the first content action UI control 516a actuated in the previous control. Figure 4 The first cast initiation UI control 420 in FIG. 1 operates similarly.
[0068] Fig. 6A , Figure 6B and Figure 6C In the illustrated example, the presenter 120 performs additional activities on the presenter device 130 that change the content of the display 132 of the presenter device 130 without changing the rendering of the electronic content item 370 by other endpoints of the projection session. Fig. 6A In FIG. 1 , the display 132 is used to render the electronic content item 370 on the ninth UI 360, as shown in FIG. Figure 3E In this example, the ninth UI 360 includes a UI control 610. In response to actuation of the UI control 610, the presenter device 130 begins to perform additional activities.
[0069] Figure 6B shows the response to Fig. 6A In this example, the additional activity involves the teleconferencing application 232 presenting a request for the Figure 3D 350 for selecting an electronic content item shown in . In this example, the seventh UI 350 includes a UI control 358 that can be actuated to navigate to a different UI provided by the teleconferencing application 232. The rendering of the electronic content item 370 has changed (in this example, the electronic content item 370 is not rendered on the display 132), but the projection session is not interrupted. The presenter device 130 is configured to: when the projection session is active, present a projection session UI control 620 that allows the presenter 120 to perform one or more operations related to the projection session, and also provide a visual indication that the projection session remains active. Figure 6B In the example shown in , the projection session UI controls 620 include a projection return UI control 622, and the presentation device 130 is configured to return to the projection return UI control 622 in response to actuation of the projection return UI control 622. Fig. 6A 360 shown in . In some examples, the projection session UI controls 620 may include one or more projection navigation UI controls 624, and the presentation device 130 is configured to transmit an associated encoded navigation action for the projection session in response to actuation of the projection navigation UI control 624 (and in some environments, perform the associated navigation action to render the electronic content item 370 by the presenter device 130). Thus, the presenter 120 is able to perform basic navigation through the electronic content item 370 without having to return to the ninth UI 360. In some examples, the projection navigation UI controls 624 are determined based on the content type of the electronic content item 370. In Figure 6B In the example shown in , the projection navigation UI control 624 includes a back navigation UI control 624a and a forward navigation UI control 624b for navigating between successive slides of the electronic content item 370. In some examples, such as in Figure 6BAs shown in , the projection session UI controls 620 include a projection termination UI control 626, and the presentation device 130 is configured to terminate the projection component of the session in response to actuation of the projection termination UI control 626 (which may also result in termination of the associated teleconference session, depending on whether other components (such as an audio-visual component) are active in the session).
[0070] Figure 6C A different example is shown in which the display 132 is used for additional activities while the projection session continues. In this example, the presenter 120 has moved to the home screen of the presenter device 120, showing icons for other applications that can be run on the presenter device 120. Figure 6C In FIG. 6 , the presenter device 130 is configured to present the projection session UI controls 630 while the projection session remains active, as in FIG. Figure 6B The projection session UI control 620 in is very similar. Figure 6C In the example, the projection session UI control 630 is presented as a colored "pill" around the display of the current time. It also serves as a projection return UI control 632, which is used for Figure 6B The projection return UI control 622 in the Fig. 6A In some examples, when another application (e.g., Figure 6C When presenting the UI on the display 132 (eg, an application corresponding to one of the icons in ), the projection session UI controls 630 may also be shown, thereby continuing to provide a discreet and easily available UI control 630 for returning to the ninth UI 360 to control the projection session.
[0071] Fig. 7A , Figure 7B , Figure 7C , Fig.7D , Fig. 7E and Figure 7F An example is illustrated in which the sixth participant device 704 recognizes an action performed by the tenth attendee 702 and transmits a corresponding encoded action and a corresponding response by the fourth target device 712, including an effect associated with a projection mode that can be selectively changed by the tenth attendee 702. Figures 7A-7E In the discussion of FIG. 7 , the first environment 700, the fourth target device 712, the sixth attendee device 704, and the tenth attendee 702 correspond to the first environment 700, the fourth target device 712, the sixth attendee device 704, and the tenth attendee 702, respectively. Figure 2A 220a, the third target device 240b, the fourth participant device 230d, and the seventh attendee 220d are shown in FIG. Thus, the sixth participant device 704 and the fourth target device 712 are described as being associated with the teleconferencing service 260 and in FIG. Figure 2A and Figure 2BIn addition, the sixth participant device 704 includes Figure 2C and the fourth target device 712 includes the features shown and described in Figure 2D Likewise, in Figures 7A-7E Examples may include targeting Figure 1A-6C and Figure 8 Any of the features described. Figures 7A-7E , wherein the sixth participant device 704 operates as a presenter device.
[0072] Fig. 7A A thirteenth UI 710 is illustrated in which a second electronic content item 708 is rendered on a display 706 of a presenter device 704 and a fourteenth UI 730 is illustrated in which the second electronic content item 708 is rendered on a display 714 of a fourth target device 712. In this example, the thirteenth UI 710 is presented by a native application 233a, which is a content creation application that presents a full-featured UI including a UI control 712. The native application 233a is rendering the second electronic content item 708 as a first rendered content 716 in a first content rendering portion 715 of the thirteenth UI 710. Additionally, in this example, the presenter device 704 is configured to present a projection mode UI control 750 for changing the operation of the presenter device 704 between two or more different projection modes, wherein actions performed on the presenter device 704 can be selectively provided to other endpoints of the projection session under the control of the tenth attendee 702. In this example, the projection mode UI control 750 is included in the thirteenth UI 710. Fig. 7A In the example, the first visual treatment 752 (in this example, displaying an emphasis border, a check mark, and the text "Projection Change") is applied to the projection mode UI control 750 to visually indicate that the presenter device 704 is operating in the first projection mode, in which the encoded actions are generated without restriction. In this example, the native application 243a renders the second electronic content item 708 as the second rendered content 734 in the second content rendering portion 732 of the fourteenth UI 730. As shown in Fig. 7A As shown in FIG. , the first rendered content 716 and the second rendered content 734 can have different appearances, such as in FIG. Fig. 7A , which may include differences in line breaks for text elements. The first content portion 718 of the second electronic content item 708 is rendered by both devices 704 and 710.
[0073] continue Fig. 7A , Figure 7BIn the illustrated example, the presenter 702 has added a second content portion 740 to the second electronic content item 708 using the presenter device 704, resulting in the fourth target device 712 rendering the second content portion 740 via the projection session. As indicated by the first visual treatment 752 applied to the projection mode UI control 750, the presenter device 704 is operating in the first projection mode, in response to which it distributes an encoding action corresponding to the addition of the second content portion 740.
[0074] continue Figure 7B , Figure 7C An example is shown in which the presenter 702 has actuated the projection mode UI control 750, and in response, the presenter device 704 has changed from operating in a first projection mode to operating in a second projection mode, in which the presenter device 704 will not automatically generate and distribute coded actions that it would have generated and distributed when operating in the first projection mode. Figure 7C , a second visual treatment 754 (in this example, displaying a non-emphasized border, a large "X", and the text "Do not project changes") is applied to the projection mode UI control 750 to visually indicate that the presenter device 704 is operating in the second projection mode. The presenter 702 can actuate the projection mode UI control 750 to return to the first projection mode. In response to operating in the second projection mode, although the presenter 702 has also added a third content portion 742 (a comment on the second content portion 740) and a fourth content portion 744 (additional text) displayed by the presenter device 704, the corresponding encoded actions are not generated or distributed in response to operating in the second projection mode. Therefore, the rendering of the second electronic content 708 by the fourth target device 710 remains the same as that of the first target device 710. Figure 7B In some implementations, a third visual process 756 can be applied to the portion of the content that is not projected to provide an indication that the portion of the content has not been projected.
[0075] continue Figure 7C For example, Fig.7DAn example is illustrated in which, while continuing to operate in the second projection mode, the presenter 702 has also added the fifth content portion 746 to the second electronic content 708, and is also selectively selected to project the fifth content portion 746 that was not initially projected. In response to operating in the second projection mode, the encoding action corresponding to the addition of the fifth content portion 746 is not generated in real time. In this example, after adding the fifth content portion 746, the presenter 702 has selected the fifth content portion 746 and actuated the projection selection UI control 766 using a pointer 768 via a projection control UI element 762 included in the right-click menu UI 750 so that the fifth content portion 746 that was not initially projected is projected. In some implementations, the content portions that were not projected when operating in the second projection mode will remain unprojected, even when the projection mode UI control 750 is actuated to return to the first projection mode. In some implementations, the projection disable flag is stored in the file containing the electronic content items for the content portions that were not projected to persist the projection disable state of those content portions in subsequent projection sessions. In some implementations, the content creation application is configured to allow the user to specify portions of content that are disabled for projection, both outside of and prior to the projection session. For example, "reserved" slides in a slide deck are marked as disabled for projection to prevent viewing by other participants 220, wherein the presenter 702 can dynamically cause those slides to be projected on demand if desired.
[0076] continue Fig.7D For example, Fig. 7E The diagram shows the response to Fig.7D , the fifth content portion 746 has been projected to the fourth target device 710. In response to the actuation of the cast selection UI control 766, the presenter device 704 has generated and distributed an encoding action corresponding to the addition of the fifth content portion 746 (which may be different from the encoding action generated in real time when the presenter device 704 was operating in the first projection mode), resulting in the fourth target device 712 rendering the fifth content portion 746 via the projection session, as shown in Fig. 7E In some implementations, the renderer 702 can effectively “un-project” the selected content portion via a similar UI interaction, resulting in an encoded action being distributed that removes the selected content portion at the non-rendering endpoint rather than removing the selected content portion at the rendering device 702.
[0077] continue Fig. 7E For example, Figure 7F An example of not generating a coded navigation action in response to operating in the second projection mode is illustrated. While continuing to operate in the second projection mode, the presenter 702 has Fig. 7E708 (at the beginning of the second electronic content item 708) to a different location. However, in response to operating in the second projection mode, no corresponding encoded navigation action is generated. Therefore, the rendering of the second electronic content 708 by the fourth target device 710 is different from the Fig. 7E Remain unchanged. In some examples, the thirteenth UI 710 provides an indication of the position last provided to the endpoint via the projection session (not shown). In some examples, a UI control (not shown) is provided that can be actuated to return the presenter device 704 to the position last provided to the endpoint. In some examples, in response to returning to the first projection mode, the presenter device 704 can automatically return the presenter device 704 to the position last provided to the endpoint.
[0078] In some implementations, a UI (not shown) may be provided to the presenter 702 to allow for selectively enabling or disabling the generation of coded actions for specific types of actions (“content action types”) when operating in the second projection mode. Via the UI, for example, the presenter 702 may individually select whether to project content editing actions (and / or specific types of content editing actions, such as adding / removing comments), navigation actions, and / or annotation actions. Thus, annotation projection may continue to be performed in the first and second projection modes, while projection of content editing and navigation actions is not automatically performed in the second projection mode, such as in Figure 7C , Fig.7D and Figure 7F As shown in .
[0079] Figure 8 is a flow chart illustrating an implementation of an exemplary process 800 for projecting an electronic content item to a target computing device. In some examples, some or all of process 800 may be combined with Figures 1A-7F , Fig. 9 and Fig.10 The invention can be performed with any combination of the features discussed, but it can also be performed with any other features described in this document. Figure 8 In the process 800, a first operation 810 may include identifying, at a first time during the projection session, a first action performed by a user of a presenter computing device that affects rendering of a first electronic content item being projected to a first target computing device via the projection session. In a second operation 820, the process 800 may include determining that the presenter computing device is operating in a first projection mode at the first time. In a third operation 830, the process 800 may include: based at least on determining that the presenter computing device is operating in the first projection mode at the first time, changing the rendering of the first electronic content item by the first target computing device by causing the first action to be reported in real time to the first target computing device.
[0080] In a fourth operation 840, process 800 includes identifying a second action performed by a user that affects rendering of a first electronic content item at a second time during the projection session, wherein the second time is different from the first time. In a fifth operation 850, process 800 includes determining that the presenter computing device is operating in a second projection mode different from the first projection mode at the second time. In a sixth operation 860, process 800 includes determining that the second action will not be reported to the target computing device in real time based at least on determining that the presenter computing device is operating in the second projection mode at the second time.
[0081] In this paper, we present the Figure 1A-8 Detailed examples of the described systems, devices, and techniques are provided to illustrate the present disclosure and its benefits. Such usage examples should not be construed as limitations on the logical process embodiments of the present disclosure, nor should variations of the user interface methods from those described herein be considered outside the scope of the present disclosure. In some embodiments, in Figure 1A-8 The various features described in the present invention are implemented in corresponding modules, which may also be referred to as and / or include logical units, components, units and / or mechanisms. The modules may constitute software modules (eg, codes embodied on a machine-readable medium) or hardware modules.
[0082] In some examples, the hardware module can be implemented mechanically, electronically, or using any suitable combination thereof. For example, the hardware module may include a dedicated circuit or logic unit configured to perform a specific operation. For example, the hardware module may include a dedicated processor, such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC). The hardware module may also include a programmable logic unit or circuit temporarily configured by software to perform a specific operation, and may include a portion of machine-readable medium data and / or instructions for such a configuration. For example, the hardware module may include software included in a programmable processor that is configured to run a set of software instructions. It will be appreciated that the decision to implement the hardware module mechanically, in a dedicated and permanently configured circuit, or in a temporarily configured circuit (e.g., configured by software) may be driven by cost, time, support, and engineering considerations.
[0083] Therefore, the phrase "hardware module" should be understood to cover a tangible entity that can perform a specific operation and can be configured or arranged in a specific physical manner, that is, a physical structure, a permanent configuration (e.g., hard wiring) and / or a temporary configuration (e.g., programming) that operates in a certain way or performs a specific operation described in this article. As used in this article, a "hardware-implemented module" refers to a hardware module. Considering the example that a hardware module is temporarily configured (e.g., programmed), each hardware module in the hardware module does not need to be configured or instantiated at any one time instance. For example, in the case where a hardware module includes a programmable processor that is configured to be a special-purpose processor by software, the programmable processor can be configured as different special-purpose processors (e.g., including different hardware modules) at different times. Software can configure one or more specific processors accordingly, for example, to constitute a specific hardware module at one time instance and to constitute different hardware modules at different time instances. A hardware module implemented using one or more processors can be referred to as "processor-implemented" or "computer-implemented."
[0084] Hardware modules can provide information to other hardware modules and receive information from other hardware modules. Therefore, the described hardware modules can be considered to be communicatively coupled. In the case where multiple hardware modules exist simultaneously, communication can be achieved by signal transmission (e.g., through appropriate circuits and buses) between two or more hardware modules. In embodiments where multiple hardware modules are configured or instantiated at different times, communication between such hardware modules can be achieved, for example, by storing and retrieving information in a memory device accessible to multiple hardware modules. For example, one hardware module can perform an operation and store the output in a memory device, and then another hardware module can access the memory device to retrieve and process the stored output.
[0085] In some examples, at least some of the operations of the method can be performed by one or more processors or processor-implemented modules. In addition, one or more processors can also be operated to support the execution of related operations in a "cloud computing" environment or as "software as a service" (SaaS). For example, at least some of the operations can be performed by multiple computers (as an example of a machine including a processor) and / or between multiple computers, where these operations can be accessed via a network (e.g., the Internet) and / or via one or more software interfaces (e.g., application program interface (API)). The execution of specific operations in the operations can be distributed between processors, not only residing in a single machine, but also deployed across multiple machines. The processor or the module implemented by the processor can be located in a single geographical location (e.g., in a home or office environment, or in a server farm), or can be distributed across multiple geographical locations.
[0086] Fig. 9 is a block diagram 900 illustrating an exemplary software architecture 902, portions of which may be used in conjunction with the various hardware architectures described herein that may implement any of the features described above. Fig. 9 is a non-limiting example of a software architecture, and it should be understood that many other architectures may be implemented to facilitate the functionality described herein. Software architecture 902 may be implemented in a system such as Fig.10 1000, which includes a processor 1010, a memory 1030, and an input / output (I / O) component 1050. A representative hardware layer 904 is illustrated and can represent, for example, Fig.10 The machine 1000 of the embodiment of the present invention. The representative hardware layer 904 includes a processing unit 906 and associated executable instructions 908. The executable instructions 908 represent executable instructions of the software architecture 902, including implementations of the methods, modules, etc. described herein. The hardware layer 904 also includes a memory / storage device 910, which also includes executable instructions 908 and accompanying data. The hardware layer 904 may also include other hardware modules 912. The instructions 908 maintained by the processing unit 908 may be part of the instructions 908 maintained by the memory / storage device 910.
[0087] Exemplary software architecture 902 can be conceptualized as layers, each providing various functions. For example, software architecture 902 may include layers and components, such as operating system (OS) 914, library 916, framework 918, application 920, and presentation layer 944. In operation, application 920 and / or other components within a layer may invoke API calls 924 to other layers and receive corresponding results 926. The illustrated layers are representative in nature, and other software architectures may include additional or different layers. For example, some mobile or dedicated operating systems may not provide framework / middleware 918.
[0088] OS 914 can manage hardware resources and provide public services. OS 914 may include, for example, kernel 928, service 930 and driver 932. Kernel 928 may serve as an abstraction layer between hardware layer 904 and other software layers. For example, kernel 928 may be responsible for memory management, processor management (e.g., scheduling), component management, networking, security settings, etc. Service 930 may provide other public services for other software layers. Driver 932 may be responsible for controlling underlying hardware layer 904 or interface with it. For example, driver 932 may include display driver, camera driver, memory / storage device driver, peripheral device driver (e.g., via universal serial bus (USB)), network and / or wireless communication driver, audio driver, etc., depending on hardware and / or software configuration.
[0089] The library 916 may provide a common infrastructure that may be used by the application 920 and / or other components and / or layers. The library 916 generally provides functions for use by other software modules to perform tasks, rather than interacting directly with the OS 914. The library 916 may include a system library 934 (e.g., a C standard library) that may provide functions such as memory allocation, string manipulation, and file operations. In addition, the library 916 may include an API library 936, such as a media library (e.g., supporting the presentation and manipulation of image, sound, and / or video data formats), a graphics library (e.g., an OpenGL library for rendering 2D and 3D graphics on a display), a database (e.g., SQLite or other relational database functions), and a web library (e.g., WebKit that may provide web browsing functions). The library 916 may also include a variety of other libraries 938 to provide many functions for the application 920 and other software modules.
[0090] Framework 918 (sometimes also referred to as middleware) provides a higher level common infrastructure that can be used by applications 920 and / or other software modules. For example, framework 918 can provide various graphical user interface (GUI) functions, advanced resource management, or advanced location services. Framework 918 can provide a wide range of other APIs for applications 920 and / or other software modules.
[0091] Applications 920 include built-in applications 940 and / or third-party applications 942. Examples of built-in applications 940 may include, but are not limited to, contact applications, browser applications, location applications, media applications, messaging applications, and / or game applications. Third-party applications 942 may include any application developed by an entity other than a supplier of a particular platform. Applications 920 may use the functions available via OS 914, library 916, framework 918, and presentation layer 944 to create a user interface to interact with a user.
[0092] Some software architectures use virtual machines, such as virtual machine 948. Virtual machine 948 provides an execution environment in which applications / modules can run as if they were on a hardware machine (such as Fig.10 The virtual machine 948 may be hosted by a host OS (e.g., OS 914) or a hypervisor and may have a virtual machine monitor 946 that manages the operation of the virtual machine 948 and interoperation with the host operating system. A software architecture that may be different from the software architecture 902 outside the virtual machine runs within the virtual machine 948, such as an OS 950, a library 952, a framework 954, an application 956, and / or a presentation layer 958.
[0093] Fig.101 is a block diagram illustrating components of an exemplary machine 1000 configured to read instructions from a machine-readable medium (e.g., a machine-readable storage medium) and perform any of the features described herein. The exemplary machine 1000 is in the form of a computer system within which instructions 1016 (e.g., in the form of software components) for causing the machine 1000 to perform any of the features described herein can be run. In this way, the instructions 1016 can be used to implement the modules or components described herein. The instructions 1016 cause the unprogrammed and / or unconfigured machine 1000 to operate as a specific machine configured to perform the described features. The machine 1000 can be configured to operate as a standalone device or can be coupled (e.g., networked) to other machines. In a networked deployment, the machine 1000 can operate in the capacity of a server machine or a client machine in a server-client network environment, or as a node in a peer-to-peer or distributed network environment. The machine 1000 may be embodied as, for example, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a gaming and / or entertainment system, a smartphone, a mobile device, a wearable device (e.g., a smart watch), and an Internet of Things (IoT) device. Furthermore, although only a single machine 1000 is illustrated, the term "machine" includes a collection of machines that individually or jointly execute instructions 1016.
[0094] The machine 1000 may include a processor 1010, a memory 1030, and I / O components 1050 that may be communicatively coupled via, for example, a bus 1002. The bus 1002 may include multiple buses that couple various elements of the machine 1000 via various bus technologies and protocols. In an example, the processor 1010 (including, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an ASIC, or a suitable combination thereof) may include one or more processors 1012a to 1012n that may execute instructions 1016 and process data. In some examples, one or more processors 1010 may execute instructions provided or recognized by one or more other processors 1010. The term "processor" includes a multi-core processor that includes cores that can execute instructions simultaneously. Although Fig.10 Multiple processors are shown, but machine 1000 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors each with a single core, multiple processors each with multiple cores, or any combination thereof. In some examples, machine 1000 may include multiple processors distributed among multiple machines.
[0095] The memory / storage 1030 may include a main memory 1032, a static memory 1034 or other memory and a storage unit 2036, which may be accessed by the processor 1010 via the bus 1002. The storage unit 1036 and the memories 1032, 1034 store instructions 1016 that embody any one or more of the functions described herein. The memory / storage 1030 may also store temporary, intermediate and / or long-term data for the processor 1010. The instructions 1016 may also reside, in whole or in part, in the memories 1032, 1034, in the storage unit 1036, in at least one processor 1010 (e.g., in a command buffer or cache), in a memory in at least one I / O component 1050, or any suitable combination thereof during execution thereof. Thus, the memories 1032, 1034, the storage unit 1036, the memory in the processor 1010, and the memory in the I / O component 1050 are examples of machine-readable media.
[0096] As used herein, "machine-readable medium" refers to a device that is capable of temporarily or permanently storing instructions and data that cause the machine 1000 to operate in a particular manner, and may include, but is not limited to: random access memory (RAM), read-only memory (ROM), buffer memory, flash memory, optical storage media, magnetic storage media and devices, caches, network-accessible or cloud storage devices, other types of storage devices, and / or any suitable combination thereof. The term "machine-readable medium" applies to a single medium or a combination of multiple media that are used to store instructions (e.g., instructions 1016) for operation by the machine 1000, such that the instructions, when executed by one or more processors 1010 of the machine 1000, cause the machine 1000 to perform one or more of the features described herein. Thus, a "machine-readable medium" may refer to a single storage device, as well as a "cloud-based" storage system or storage network that includes multiple storage devices or devices. The term "machine-readable medium" does not include the signal itself.
[0097] I / O components 1050 may include a variety of hardware components suitable for receiving input, providing output, generating output, transmitting information, exchanging information, capturing measurements, etc. The specific I / O components 1050 included in a particular machine will depend on the type and / or function of the machine. For example, a mobile device (such as a mobile phone) may include a touch input device, while a headless server or IoT device may not include such a touch input device. Fig.10The specific examples of I / O components illustrated in the figure are by no means limiting, and other types of components may be included in the machine 1000. The grouping of the I / O components 1050 is merely to simplify the discussion, and the grouping is by no means limiting. In various examples, the I / O components 1050 may include a user output component 1052 and a user input component 1054. The user output component 1052 may include, for example, a display component (e.g., a liquid crystal display (LCD) or a projector) for displaying information, an acoustic component (e.g., a speaker), a tactile component (e.g., a vibration motor or a force feedback device), and / or other signal generators. The user input component 1054 may include, for example, an alphanumeric input component (e.g., a keyboard or a touch screen), a pointing component (e.g., a mouse device, a touch pad or other pointing instrument), and / or a tactile input component (e.g., a physical button or touch screen that provides a position and / or a touch force or a touch gesture) configured to receive various user inputs, such as user commands and / or selections.
[0098] In some examples, I / O component 1050 may include biometric component 1056, motion component 1058, environment component 1060, and / or position component 1062, as well as a wide range of other physical sensor components. Biometric component 1056 may include, for example, components that detect body expressions (e.g., facial expressions, vocal expressions, hand or body gestures, or eye tracking), measure biosignals (e.g., heart rate or brain waves), and identify people (e.g., via voice, retina, fingerprint, and / or face-based recognition). Motion component 1058 may include, for example, acceleration sensors (e.g., accelerometers) and rotation sensors (e.g., gyroscopes). Environment component 1060 may include, for example, lighting sensors, temperature sensors, humidity sensors, pressure sensors (e.g., barometers), acoustic sensors (e.g., microphones for detecting ambient noise), physical proximity sensors (e.g., infrared sensing of nearby objects), and / or other components that may provide indications, measurements, or signals corresponding to the surrounding physical environment. Position component 1062 may include, for example, a position sensor (e.g., a global positioning system (GPS) receiver), an altitude sensor (e.g., a barometric pressure sensor from which altitude may be derived), and / or an orientation sensor (e.g., a magnetometer).
[0099] I / O components 1050 may include communication components 1064 that implement a variety of technologies operable to couple machine 1000 to network 1070 and / or device 1080 via respective communication couplings 1072 and 1082. Communication components 1064 may include one or more network interface components or other suitable devices that interface with network 1070. Communication components 1064 may include, for example, components suitable for providing wired communication, wireless communication, cellular communication, near field communication (NFC), Bluetooth communication, Wi-Fi, and / or communication via other modalities. Device 1080 may include other machines or various peripherals (e.g., via USB coupling).
[0100] In some examples, communication component 1064 can detect an identifier or include a component suitable for detecting an identifier. For example, communication component 1064 can include a radio frequency identification (RFID) tag reader, an NFC detector, an optical sensor (e.g., a one-dimensional or multi-dimensional barcode or other optical code), and / or an acoustic detector (e.g., a microphone that recognizes an audio signal of a tag). In some examples, location information can be determined based on information from communication component 1062, such as, but not limited to, a geographic location via an Internet Protocol (IP) address, a location via Wi-Fi, cellular, NFC, Bluetooth, or other wireless station identification and / or signal triangulation.
[0101] The disclosed implementations can incorporate any features, components, methods, systems, software, and other aspects described in the following documents: U.S. Patent Application Publication Nos. 2005 / 0091302 (titled “Systems and methods for projecting content from computing devices” and published on April 28, 2005), 2005 / 0091359 (titled “Systems and methods for projecting content from computing devices” and published on April 28, 2005), 2013 / 0110937 (titled “REAL TIME DOCUMENTPRESENTATION DATA SYNCHRONIZATION THROUGH GENERIC SERVICE” and published on May 2, 2013), 2014 / 0310327 (titled “SYNCHRONIZATION ACROSS THE GENERIC SERVICE” and published on May 2, 2013), and 2015 / 0110939 (titled “SYSTEMS AND METHODS FOR PROJECTING CONTENT FROM COMPUTING DEVICES” and published on April 28, 2005). APPLICATIONS", and published on October 16, 2014), 2016 / 0072857 (titled "ACCESSIBILITY FEATURES IN CONTENT SHARING", and published on March 10, 2016), 2016 / 0182603 (titled "Browser Display Casting Techniques", and published on June 23, 2016), 2016 / 0373512 (titled "DELEGATION OF RENDERING BETWEEN A WEB APPLICATION AND A NATIVE APPLICATION", and published on December 22, 2016), 2018 / 0048590 (titled "CONTROL OF CASTING TO A MEDIA RENDERER", and published on February 15, 2018), 20180109566 (titled "Universal Casting Service” and published on April 19, 2018), 2018 / 0241755 (titled “Meeting Join for Meeting Device” and published on August 23, 2018), 2019 / 0065457 (titled “APPLICATION / DOCUMENT COLABORATION IN AMULTI-DEVICE ENVIRONMENT” and published on August 23, 2018),and published on February 28, 2019) and 2019 / 0295439 (titled “CROSS-APPLICATION FEATURE LINKING AND EDUCATIONAL MESSAGING” and published on September 26, 2019); and U.S. Patent No. 9,276,972 (titled “Real-time media optimization over remote sessions” and published on March 1, 2016).
[0102] Although various embodiments have been described, this description is intended to be exemplary rather than restrictive, and it should be understood that more embodiments and implementations within the scope of the embodiments are possible. Although many possible feature combinations are shown in the drawings and discussed in this detailed description, many other combinations of the disclosed features are also possible. Unless specifically limited, any feature of any embodiment can be used in combination with any other feature or element in any other embodiment or replace any other feature or element. Therefore, it should be understood that any feature of the features shown and / or discussed in this disclosure can be implemented together in any suitable combination. Therefore, except according to the attached claims and their equivalents, the embodiments are not limited. In addition, various modifications and changes can be made within the scope of the attached claims.
[0103] Although the foregoing has described what is considered to be the best mode and / or other examples, it should be understood that various modifications may be made therein, and the subject matter disclosed herein may be implemented in various forms and examples, and these teachings may be applied in many applications, only some of which are described herein. The appended claims are intended to claim any and all applications, modifications, and changes that fall within the true scope of the present teachings.
[0104] Unless otherwise indicated, all measurements, values, ratings, positions, dimensions, sizes and other specifications set forth in this specification (including the claims that follow) are approximate, not exact, and are intended to be within a reasonable range consistent with the functions to which they are related and with customary practice in the art to which they are related.
[0105] The scope of protection is limited only by the following claims. The scope is intended and should be interpreted to be consistent with the ordinary meaning of the language used in the claims when interpreted in light of this specification and subsequent prosecution history and to include all structural and functional equivalents. Nonetheless, no claim is intended to cover subject matter that fails to meet the requirements of Sections 101, 102, or 103 of the Patent Act, nor should it be interpreted in such a manner. Any unexpected admission of such subject matter is hereby disclaimed.
[0106] Except as stated above, nothing stated or described is intended or should be construed as conferring any component, step, feature, object, benefit, advantage, or equivalent to the public, whether or not mentioned in the claims.
[0107] It should be understood that the terms and expressions used in this article have the ordinary meanings given to them by the corresponding investigations and research fields corresponding to them, unless a specific meaning is set forth in this article. Relational terms such as "first" and "second" can be used only to distinguish one entity or action from another, without requiring or implying any actual such relationship or order between such entities or actions. The terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including the list of elements includes not only those elements, but may also include other elements that are not explicitly listed or elements inherent to such processes, methods, articles or devices. In the absence of further restrictions, an element starting with "one" or "an" does not exclude the presence of additional identical elements in the process, method, article or device including the element.
[0108] The Abstract of the present disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the above detailed description, it can be seen that various features are grouped together in various examples in order to simplify the disclosure. This disclosure method should not be interpreted as reflecting an intention that the claims require more features than are expressly recited in each claim. On the contrary, as reflected in the following claims, the subject matter of the present invention does not lie in all the features of a single disclosed example. Therefore, the following claims are hereby incorporated into the detailed description, wherein each claim stands on its own as a separately claimed subject matter.
Claims
1. A system comprising one or more processors and a machine-readable medium comprising instructions that, when executed by the one or more processors, cause the one or more processors to: identifying, at a first time during a casting session, a first change action performed on a presenter computing device that affects rendering of a first electronic content item being cast to a first target computing device via the casting session; causing a display associated with the presenter computing device to display the first electronic content item with the first change action applied; determining that the presenter computing device is operating in a first projection mode at the first time; based at least on a determination that the presenter computing device is operating in the first projection mode at the first time, changing the rendering of the first electronic content item by the first target computing device by causing the first change action to be reported to the first target computing device; identifying a second change action performed on the presenter computing device to the first electronic content item at a second time during the casting session, wherein the second time is different from the first time; causing the display associated with the presenter computing device to display the first electronic content item with the second change action applied; determining that the presenter-computing device is operating in a second projection mode different from the first projection mode at the second time; and Based at least on determining that the presenter computing device is operating in the second projection mode at the second time, the first target computing device maintains the rendering of the first electronic content item unchanged without applying the second change action, while the display displays the first electronic content item with the applied second change action.
2. The system according to claim 1, in: The first change action performs a first edit on the first electronic content item; The second change action performs a second edit on the first electronic content item; as well as In response to identifying the first edit when the presenter computing device is operating in the first projection mode and identifying the second edit when the presenter computing device is operating in the second projection mode, rendering the first edit to the first electronic content item only on the first target computing device.
3. The system according to claim 1, in, The instructions further cause the one or more processors to: adding a first content portion to the first electronic content item based on the first change action; adding a second content portion to the first electronic content item based on the second change action; rendering, at the presenter computing device, the second content portion with a first visual treatment indicating that the second content portion has not been projected; as well as The first content portion is rendered at the presenter computing device without the first visual processing.
4. The system according to claim 1, in, The instructions further cause the one or more processors to: The first target computing device is caused to render the first electronic content item using a first native application running on the first target computing device, wherein the first native application is configured to interpret and render the first electronic content item using computing resources provided by the first target computing device.
5. The system according to claim 1, in, The instructions further cause the one or more processors to: identifying, at a third time during the casting session, a third change action performed on the presenter computing device that causes an annotation to be rendered for the first electronic content item, wherein the third time is different than the second time; determining that the presenter computing device is operating in the second projection mode at the third time; determining that the third change action is of an annotation content action type; and Based at least on determining that the third change action is of the annotation content action type, rendering of the first electronic content item by the first target computing device is changed by causing the third change action to be reported to the first target computing device.
6. The system according to claim 1, in, The instructions further cause the one or more processors to: identifying a participant computing device other than the presenter computing device as a non-presenter endpoint for the projection session; responsive to identifying the participant computing device as a non-presenter endpoint of the projection session, causing the participant computing device to render the first electronic content item using a native application running on the participant computing device, wherein the native application is configured to interpret and render the first electronic content item using computing resources provided by the participant computing device; as well as Rendering of the first electronic content item by the participant computing device is changed by causing the first change action to be reported to the participant computing device.
7. The system according to claim 1, in, The instructions further cause the one or more processors to: adding a second content portion to the first electronic content item based on the second change action; receiving, at the presenter computing device, a selection of the second content portion; receiving, at the presenter computing device, a command to project the selected second content portion at a third time that the second content portion remains unprojected; as well as Rendering of the first electronic content item by the first target computing device is changed to include the second content portion by causing a second change action corresponding to the addition of the second content portion to be reported to the first target computing device.
8. A method for projecting electronic content, the method include: identifying, at a first time during a casting session, a first change action performed on a presenter computing device that affects rendering of a first electronic content item being cast to a first target computing device via the casting session; causing a display associated with the presenter computing device to display the first electronic content item with the first change action applied; determining that the presenter computing device is operating in a first projection mode at the first time; based at least on a determination that the presenter computing device is operating in the first projection mode at the first time, changing the rendering of the first electronic content item by the first target computing device by causing the first change action to be reported to the first target computing device; identifying a second change action performed on the presenter computing device to the first electronic content item at a second time during the casting session, wherein the second time is different from the first time; causing the display associated with the presenter computing device to display the first electronic content item with the second change action applied; determining that the presenter-computing device is operating in a second projection mode different from the first projection mode at the second time; and Based at least on determining that the presenter computing device is operating in the second projection mode at the second time, the first target computing device maintains the rendering of the first electronic content item unchanged without applying the second change action, while the display displays the first electronic content item with the applied second change action.
9. The method according to claim 8, in: The first change action performs a first edit on the first electronic content item; The second change action performs a second edit on the first electronic content item; as well as In response to identifying the first edit when the presenter computing device is operating in the first projection mode and identifying the second edit when the presenter computing device is operating in the second projection mode, rendering the first edit to the first electronic content item only on the first target computing device.
10. The method according to claim 8, further comprising: include: adding a first content portion to the first electronic content item based on the first change action; adding a second content portion to the first electronic content item based on the second change action; rendering, at the presenter computing device, the second content portion with a first visual treatment indicating that the second content portion has not been projected; as well as The first content portion is rendered at the presenter computing device without the first visual processing.
11. The method according to claim 10, further comprising: include: receiving, at the presenter computing device, a selection of a third content portion of the first electronic content item; receiving, at a third time at the presenter computing device, a command not to project the selected third content portion; storing a cast disable flag for the third content portion in a file in response to receiving the command and prior to the casting session; At the presenter computing device, based at least on the projection disabling flag for the third content portion being stored in the file, rendering the third content portion with the first visual treatment.
12. The method according to claim 8, further comprising: include: in response to the second changing action, changing from rendering a first portion of the first electronic content item at the presenter computing device to rendering a different second portion of the first electronic content item at the presenter computing device; as well as Returning to rendering the first portion of the first electronic content item at the presenter computing device in response to transitioning to operating in the first projection mode.
13. The method according to claim 8, further comprising: include: The first target computing device is caused to render the first electronic content item using a first native application running on the first target computing device, wherein the first native application is configured to interpret and render the first electronic content item using computing resources provided by the first target computing device.
14. The method according to claim 8, further comprising: include: during rendering of the first electronic content item by an endpoint computing device of the casting session for the casting session using a third native application executed by the endpoint computing device, detecting that a first edit has been made to the first electronic content item using the third native application; as well as In response to the detecting of the first edit, a second electronic content item is created that is different from the first electronic content item and reflects that the first edit has been applied to the first electronic content item.
15. The method according to claim 8, further comprising: include: adding a second content portion to the first electronic content item based on the second change action; receiving, at the presenter computing device, a selection of the second content portion; receiving, at the presenter computing device, a command to project the selected second content portion at a third time that the second content portion remains unprojected; as well as Rendering of the first electronic content item by the target computing device is changed to include the second content portion by causing a second change action corresponding to the addition of the second content portion to be reported to the target computing device.
