Computer-Implemented Method

Abstract

A computer-implemented method is provided for operating a third-party application (114) using a video end system (100). The video end system (100) is for video conferencing and includes video conferencing hardware (130) and a control interface device (120). The computer-implemented method includes providing an application environment (112) for executing the third-party application (114). The application environment (112) is configured to route control signals from the control interface device (120) to the third-party application (114) and to route media signals between the third-party application (114) and the video conferencing hardware (130) and / or a content stream of a video conference call.

Description

[Technical Field]

[0001] The present invention relates to a computer-implemented method for running third-party applications using a video end system, a computer-implemented method for installing third-party applications, and a video end system. [Background technology]

[0002] A video conferencing system typically comprises two or more endpoints, each configured to communicate with the other endpoints over a data or telephone network or other connection for conducting two-way or multi-way video and / or audio conference calls.

[0003] Endpoints, which may also be referred to as video end systems, are configured to enable users to communicate with other users over a network by sending data streams, including audio, video, and other content streams (e.g., data files, text, or screen sharing), from one endpoint to another in a two-way conference call. In the case of a multiway conference call, the data streams can be sent to a central or distributed conference network where they are switched or transcoded to enable multiple participants to communicate and share information.

[0004] A video end system may include a single-purpose endpoint device whose primary function is to support video and / or audio conference calls, and typically includes additional video conferencing hardware, including one or more video monitors, video cameras, audio microphones, audio speakers, and controllers, to enable users to conduct video conference calls.

[0005] In some examples, a video end system may be an all-in-one unit in which a single-function endpoint device and video conferencing hardware are included in the same device, e.g., a desktop unit or handheld device. In other examples, a video end system may include an endpoint device connected to a large screen, a control interface device, and a camera. This configuration may be suitable for a small conference room, for example. In other examples, a video end system intended for a larger room or auditorium may include an endpoint device connected to multiple screens, one or more control interface devices, and multiple cameras and microphones. It should be noted that in all of these examples, the endpoint device and the control interface device may be the same device.

[0006] The software running on the video endpoint devices of a video end system is traditionally proprietary software designed for a single use of video conferencing, i.e., for sharing video and content streams with other endpoints. In some cases, a video end system may be used simply to display content from a computer, such as a laptop computer, to attendees in a local conference room without establishing a conference call.

[0007] Due to the single-purpose nature of such endpoint devices, the endpoint device cannot be used to perform additional functions for which the endpoint device has not already been programmed. If additional functionality is required that is outside the capabilities of the single-purpose endpoint device, an external device, such as a computer or laptop computer, may be used to perform the additional functionality. As a result, the external device can share content generated by the additional functionality as a content stream in a video conference call.

[0008] Thus, the present inventors have recognized a need to leverage traditional video end system hardware for functions other than those designed by the video end system manufacturer.

[0009] Video end systems are implemented by providing video conferencing applications that run on standard real-time embedded operating systems (as opposed to proprietary operating systems such as VxWorks, RTLinux, or Windows CE). Because their design is based on standards, mass-market operating systems (such as Android) have the advantage of supporting a wide variety of hardware and eliminating licensing costs. Standard operating systems (such as Android) also support the installation of a wide variety of third-party applications. However, such third-party applications are typically configured to run on tablets and smartphones that utilize a single display, including a touch screen. In contrast, video end systems may have multiple displays and separate control interface devices, which means that the video end system may not be compatible with most third-party applications.

[0010] The present invention has been conceived in light of the above considerations. Summary of the Invention

[0011] According to a first aspect of the present invention, there is provided a computer-implemented method for operating a third party application using a video end system, the video end system being for conducting video conference calls, the video end system comprising video conferencing hardware and a control interface device, The computer-implemented method comprises: providing an application environment for executing a third-party application, said application environment comprising: The method may further include providing a control interface device configured to route control signals from the control interface device to the third party application and route media signals between the third party application and the video conferencing hardware and / or the content stream of the video conferencing call.

[0012] By providing an application environment for running third-party applications, it is advantageous that third-party applications can be used with the video end system without experiencing problems due to incompatibility of the third-party applications with the video end system. This allows users of the video end system (which may be, for example, a conference room device) to use the video end system for additional functionality beyond video conferencing. For example, the video end system can be used to host local and / or in-conference collaborative applications to enhance the user's conference experience. The method of the first aspect can also provide an opportunity for third-party application developers to develop new applications specific to video conferencing using the video end system.

[0013] Importantly, the present method allows third party applications to be run as native applications hosted locally on the video end system, in contrast to existing methods that may involve the video end system having to access a browser to use a remotely hosted application.

[0014] The computer-implemented method may additionally include installing a third-party application in the application environment, and optionally running the third-party application in the application environment, so that the functionality of the video end system can be extended by running new applications.

[0015] In one or more embodiments, the application environment may include a control signal interface, i.e., programming configured to route control signals from the control interface device to a third-party application.

[0016] In one or more embodiments, the application interface may include a media signal interface, i.e., programming configured to route media signals between a third-party application and the video conferencing hardware and / or content stream of a video conferencing call.

[0017] In the present invention, a video end system is intended to refer to a dedicated video conferencing system that includes specialized (ie, dedicated) hardware and / or software for conducting video conference calls.

[0018] The video end system may include system software configured to operate the video end system and enable conducting conference calls using the video end system. As such, a computer-implemented method may include operating the video end system to conduct video and / or audio conference calls using the system software. The system software may be an application or computer program, including proprietary software, provided with and configured to operate with the video end system.

[0019] The video end system may be managed by a standard operating system. For example, the standard operating system may be a mass-market operating system such as Android. The operating system may be configured to support a collection of mass-market applications produced by third parties designed to run on mass-market hardware such as tablets or smartphones. Other examples of standard operating systems include Huawei's HarmonyOS or Apple's iOS.

[0020] The video end system can be configured to run a single operating system. Therefore, the use of an application environment can enable a single operating system to run system software for video conferencing and third-party applications. Thus, both functions can be run on the same device (e.g., a video endpoint, as described below) without requiring an additional computing unit to be introduced to run the third-party applications.

[0021] A third-party application may be an application, computer program, or software package that provides additional functionality. In particular, a third-party application may have functionality other than supporting video conference calls. For example, a third-party application may include a collaboration application. For example, the application may be one or more of a document sharing application, a whiteboard application, a multimedia presentation application, a training application, a review application, a polling application, or a question submission application. The application may be configured to run locally or remotely over the Internet to include other parties in the video call. A third-party application may be provided by or downloaded from an external source (i.e., the third-party application is not part of the system software originally provided with the video end system to enable conference calls).

[0022] Third-party applications can be configured to run on mass-market hardware such as tablets or smartphones (i.e., third-party applications can be configured to be compatible with hardware other than the video end system).

[0023] An application environment is intended to refer to a framework or platform configured to accommodate the deployment of a single application or two or more applications. An application environment includes a predefined collection of computing resources that host applications. An application environment can be configured to provide a virtualized or standardized device interface to third-party applications to enable the third-party applications to run even when the physical device (originally intended for use with the application) is disconnected or the physical device is different from what the application expects. For example, the virtualized device interface may be configured to reroute data to a different physical device while the application is running.

[0024] In some examples, one or more (eg, multiple) application environments may be provided for running multiple (respective) third-party applications.

[0025] A control interface device, sometimes referred to as a controller, may be any device with user input controls or a user interface to allow a user to control the video end system and / or third-party applications hosted in the application environment. For example, the control interface device may be a tablet with a touch screen. A user can issue control signals to the video end system by pressing buttons on the touch screen. For example, a user can use the touch screen to start and end a conference call, add or remove parties from a conference call, adjust the volume level of speakers in the video end system, configure display screen settings, etc. Other examples of control interface devices include a keyboard, a mouse, a control panel with mechanical buttons, a voice control device, etc.

[0026] Multiple control interface devices may be provided that are connected to the rest of the video end system (e.g., via a wired or wireless connection). For example, the control interface devices may be personal devices such as smartphones or computers. Thus, the computer-implemented method may further include connecting the personal devices to the video end system and routing control signals between the personal devices and the third-party applications. The connection between the personal devices and the video end system may be a wireless connection or a wired connection, such as a connection via Wi-Fi or Bluetooth. The computer-implemented method may further include terminating the connection between the personal devices and the video end system. In this way, the personal devices can be temporarily paired with the video end system in order to control the third-party applications via the devices. Thus, each user of the video end system can control natively installed third-party applications from their own devices.

[0027] The videoconferencing hardware may include loudspeakers and microphones, and the media signals may include audio streams. Specifically, the videoconferencing hardware may include one or more loudspeakers (e.g., as broadcasting speakers or as personal headphones) for generating audio content of the conference call and / or audio content from third-party applications received via the application environment. The videoconferencing hardware may include one or more microphones for providing audio streams of the conference call and / or content streams to third-party applications via the application environment.

[0028] The video conferencing hardware may include a camera and / or a display screen for sharing images in a video conference call, and the media signal includes a video stream. Specifically, the video stream may be communicated from the camera to the content stream of the conference call and / or to a third-party application via the application environment. One or more additional video streams may be received from the content stream of the conference call and / or from a third-party application via the application environment to the display screen. Of course, the video end system may include multiple cameras and display screens.

[0029] Each component of the videoconferencing hardware (i.e., camera, display screen, microphone, loudspeaker, etc.) can be provided as one or more integrated units or as separate devices. The videoconferencing hardware and control interface devices can also be provided as one or more integrated units or as separate devices. For example, a controller with a touch screen can serve as both a user input device and a display screen.

[0030] The application environment can be provided on the video end system. However, in other examples, the video end system may communicate with a cloud service (or a separate server on a local network), and the application environment may be provided on the cloud service (or a separate server). Hosting the application environment on a cloud service can advantageously improve the performance of the video end system, as potentially resource-intensive third-party environments do not need to be hosted and run on the video end system's local hardware. Also, when provided by a cloud service, the application may be completely isolated from the corporate network on which the video end system is located. This allows untrusted applications to be run without risk to the corporate network.

[0031] The video end system may include processing elements for executing system software and operating the video end system. The application environment may be hosted on the processing elements of the video end system or may be connected to the processing elements (e.g., via an internet connection to a cloud service).

[0032] For example, a video end system may comprise an endpoint device configured to communicate with video conferencing hardware and a control interface device, the endpoint device being configured to perform the computer-implemented method of the first aspect.

[0033] The endpoint devices may be dedicated computing devices with a processor, such as a computer or local server, that are connected (e.g., by wired or wireless connections) to the videoconferencing hardware and control interface devices. The endpoint devices may include a network interface for sending and receiving content streams for the videoconference call.

[0034] In this example, the endpoint device may include system software for operating the video end system and enabling the performance of conference calls, and the application environment may be provided natively on the endpoint device within the application environment, where the computer-implemented method of the first aspect may be performed by the endpoint device.

[0035] In some examples, the application environment may be provided on a control interface device (or one of multiple control interface devices), where the control interface device may be configured to perform the computer-implemented method of the first aspect. In this example, the control interface device may also be considered as an endpoint device for operating the video end system to enable the conference call. In other words, the control interface device and the endpoint device may be provided as a single unit. In this example, the single unit may include system software for operating the video end system and enabling the conference call to be conducted.

[0036] This example is useful in a scenario where one participant in a video call wants to download and run a third-party application without allocating other resources of the video end system to run the third-party application. This example can be advantageous when the video end system is running an OS that is not compatible with the third-party application, but the control interface is running a different OS that may be compatible with the third-party application.

[0037] As mentioned above, the control interface device and the video conferencing hardware may also be provided as a single unit. In this example, the video end system may be a single desktop or handheld unit, which may also be referred to as a video end device.

[0038] In some examples, when an application environment is provided on a control interface device, the video end system may include one or more additional control interface devices. The application environment may be configured to route control signals from the one or more additional control interface devices to third-party applications. Thus, even if a third-party application is installed on a control interface device, the third-party application can still be controlled from other devices. The application environment may be configured to route media signals to the one or more additional control interface devices.

[0039] In still further examples, as described above, the control interface device and the videoconferencing hardware may comprise separable units that communicate with each other (e.g., via a wired or wireless connection). Thus, control signals can be routed between the endpoint device and the control interface device via a wired or wireless connection. In this manner, a user can access and control natively hosted third-party applications from a separate device, such as the user's own personal device.

[0040] If the application environment is provided on a cloud service, control signals can be routed between the control interface device and a third-party application hosted on the cloud. In these examples, the video end system can include an endpoint device that communicates with the application environment provided on the cloud service. The control interface device can be connected (wired or wirelessly) to the endpoint device. Thus, control signals can be routed between the control interface device and the third-party application hosted on the cloud via the endpoint device. In this manner, a permanent or temporary connection can be formed between the control interface device and an endpoint device that has already been connected to an application environment provided on a cloud service. In this manner, a user can use any suitable control interface device to launch and control an application, even if the application is hosted on the cloud.

[0041] The control interface device and the endpoint device can communicate via a cloud-based relay service. Thus, in this example, the application environment can be located on the endpoint device, and the application environment can receive control signals from the control interface device via the cloud-based relay service. Here, the endpoint device can itself be a first control interface device, and a second control interface device can communicate with the first control interface device via the cloud-based relay service.

[0042] An advantage of the endpoint devices and control interface devices communicating through the cloud may be that the wireless network to which the controller is connected may be different from the network to which the video end system is connected and may be firewalled from this network, so there may not be direct communication between the two devices. Thus, by communicating through the cloud, the endpoint devices and control interface devices may be able to communicate even if they are connected to separate networks.

[0043] The application environment may include a data mapping interface configured to relay control signals from the control interface device to the third-party application and to relay media signals between the third-party application and the video conferencing hardware and / or the content stream of the video conferencing call. The data mapping may be a virtual interface that defines a boundary to the application environment and may act as a virtualization layer by relaying data between the third-party application within the application environment and software and hardware external to the application environment.

[0044] The media signals may include video data and / or audio data forming the content stream of the video conference call (e.g., audiovisual signals operating in a camera-off mode). In some examples, the media signals may include other types of data passed to and from third-party applications. For example, the media signals may include computer files or text (e.g., other types of audiovisual-related data, e.g., still images, computer files, text that can be shared over a video link). The control signals may include signals for selectively enabling and disabling the routing of media signals between the third-party application and the videoconferencing hardware and / or the content stream of the videoconference call.

[0045] For example, the media signals may include video from a third-party application to enable remote viewing of the third-party application via videoconferencing hardware and / or the content stream of the videoconference call. To that end, the computer-implemented method may include receiving a remote view control signal from a control interface device, enabling or disabling the remote view of the third-party application on one or more videoconferencing hardware units, and / or enabling or disabling the remote view of the third-party application for one or more channels of the videoconference call based on the remote view control signal. In this manner, a user of the video end system can determine on which display screen the third-party application is displayed and which remote participants in the videoconference call can view the third-party application.

[0046] The application environment can be configured to convert control signals from the control interface device having a first format into application control signals for controlling a third-party application having a second format (i.e., using a data mapping interface). In other words, the computer-implemented method can include using the data mapping interface to convert (i.e., reformat) control signals from the control interface device having a first format into application control signals for controlling a third-party application having a second format. For example, the application environment can be configured to emulate gestures or key presses in an application upon receiving user control signals from the control interface. This can include emulating function buttons, a virtual keyboard, a dial pad, a directory lookup, or other navigation elements for a third-party application in the application environment. In some examples, this functionality can be used to send control signals from a control interface device that is separable from the endpoint device to a third-party application hosted on the endpoint device.

[0047] In this manner, a replacement user interface (UI) that maps user interface (UI) signals from a first system (e.g., on a control interface device) capable of running a first OS to a second, different system (i.e., in the application environment of the endpoint device) capable of running a second, different OS can be provided for running third-party applications in the application environment. In some examples, a custom replacement UI may be provided for a particular third-party application to be used in the new environment. The custom replacement UI may be provided by a custom configuration service, as discussed below.

[0048] For example, the control interface device (or one or more control interface devices) may include a keyboard for sending control signals that are keyboard events. The application environment can be configured to convert the keyboard events into application control signals for controlling the third-party application. In other words, the computer-implemented method can include converting control signals that include keyboard events from the control interface device having a first format into application control signals for controlling the third-party application having a second format. The application control signals may be configured to mimic control signals expected by the third-party application. For example, keyboard events may be converted into function signals that correspond to function buttons normally provided by the third-party application.

[0049] The control interface device may include a touchscreen having a first size for sending control signals, and the third-party application may be configured to receive application control signals from a touchscreen having a second size. For example, the application may be intended to run on a smartphone, and the control interface device may have a much larger screen. In this case, the application environment may be configured to scale or map control signals from the control interface device to application control signals for controlling the third-party application. In other words, the computer-implemented method may include scaling control signals from the control interface device to form application control signals for controlling the third-party application. For example, the application environment may be configured to receive position and movement information from the touchscreen of the control interface device and proportionally scale the position and movement information to correspond to the application touchscreen.

[0050] The control interface device can include a user interface environment for executing an application user interface associated with the third-party application. For example, the control interface device can include a screen or a touch screen, and the application user interface can be configured to display a representation of the third-party application on the screen. The representation of the third-party application can include function buttons, a virtual keyboard, a dial pad, a directory lookup, or other navigation elements for navigating and / or controlling the third-party application.

[0051] Similarly, the application environment can be configured to convert a video signal from a third-party application having a first format into a media signal for the videoconference call content stream and / or videoconference hardware having a second format. In other words, the computer-implemented method can include converting a media signal from a third-party application having a first format into a media signal for the videoconference call content stream and / or videoconference hardware having a second format. For example, the media signal may include a video signal, and the application environment may be configured to reformat the video data to map screen sizes, video formats (e.g., compressed vs. uncompressed), pixel counts, etc., between formats compatible with the application and the videoconference hardware and / or videoconference call content stream. In other examples, the media signal may include an audio signal, an image, file data, etc., that is converted from a first format to a second format compatible with the third-party application.

[0052] The application environment may include a data mapping interface formed from an application programming interface (API) provided by a third-party application.

[0053] In some examples, the application environment can include customized instructions corresponding to a third-party application. For example, the application environment can be configured to implement a custom replacement UI for a particular third-party application, configured to map control and / or media signals between the third-party application and the control interface device. This can enable a control interface device having a different operating system and / or UI format than the endpoint device or the third-party application to control and / or view the third-party application.

[0054] The customized instructions may be stored in a central configuration service. The central configuration service may be located on a remote server accessible by the video end systems. In this case, the computer-implemented method may include retrieving the customized instructions from the central configuration service and implementing or generating a data mapping interface based on the customized instructions. For example, the central configuration service may include customized UI instructions and a curated app store of respective third-party applications for installation on the video end systems.

[0055] The video end systems can be in communication with a central monitoring service, and the computer-implemented method can include monitoring third-party applications from the central monitoring service. The central monitoring service can be located on a remote server. The central monitoring service can be the same service as the central configuration service. For example, the central monitoring service can be configured to monitor third-party applications, where the third-party applications are approved applications that are allowed to be installed on the video end systems. In a further example, the central monitoring service can be configured to select which apps from a curated store are installed on a particular video end system, which control interface devices are used with each app and / or each video end system, and which control permissions for the third-party apps can be enabled.

[0056] Additionally, the third-party application itself may be monitored to ascertain and / or log how the third-party application behaves and / or how well the third-party application operates. For example, error reports may be issued to a central configuration service associated with the particular third-party application. The third-party configuration service can be configured to issue updated, customized instructions for interfacing to the third-party application.

[0057] The video end system may include an administrative interface for allowing an administrator to enable or disable approval for downloading third-party applications. For example, if a user wants to install a new third-party application, the computer-implemented method may include first requesting approval from the user to download the new third-party application. The approval may then be received as a password entered via the control interface device.

[0058] In some examples, enabling or disabling approval of third party application downloads on the video end system can be done by a central configuration service.

[0059] The video end system can be configured to communicate with a management server or cloud service (e.g., the central configuration service discussed above) to obtain a list of approved applications. The video end system can then download a third-party application only if the third-party application is included on the list. In some examples, the list of approved applications may be presented to a user, from which an application may be selected for installation for operation by the video end system.

[0060] An application environment including a third-party application may be sandboxed, and by sandboxing the application environment, the third-party application's access to computer resources is restricted. For example, a third-party application may be restricted from full access to CPU, memory, file system, network, audio resources, or display resources. In this way, potentially malicious activity of untrusted third-party applications can be reduced.

[0061] Sandboxing can be achieved by hosting third-party applications on a guest virtual LAN or by hosting the application environment on a cloud server to further isolate third-party applications from the rest of the corporate network or video end system.

[0062] In other examples, the third-party application may be a trusted application and may be provided with direct access to computing resources such as the CPU, memory, file system, network, audio resources, or display resources, which is beneficial for increasing the efficiency and performance of the trusted third-party application by removing delays that may be introduced by a virtualization layer. In a second aspect of the present invention, there is provided a computer-implemented method of installing a third party application for use by a video end system, the video end system being for video conferencing, the method comprising: generating an application environment, The application environment is relaying control signals from the control interface device to the third-party application; relaying media signals between a third-party application and the video conferencing hardware and / or content stream of the video conferencing call; a generating step configured to: installing a third-party application within the application environment; A computer-implemented method may be provided, including:

[0063] In some examples, installing the third party application for use by the video end system may include installing the third party application on the video end system, hi other examples, installing the third party application for use by the video end system may include installing the third party application on a cloud server in communication with the video end system.

[0064] The computer-implemented method of the second aspect may further include downloading customized instructions corresponding to the third-party application from a central configuration service to form the application environment.

[0065] In some examples, the computer-implemented method may further include communicating with a management server or a cloud service to obtain a list of approved applications, and downloading the third-party application only if the third-party application is included in the list. Thus, instances of potentially malicious third-party applications may be reduced. Also, the owner of the video end system may control whether potentially resource-intensive third-party applications may be downloaded to the video end system.

[0066] The computer-implemented method can include receiving user authentication information to enable installation of third-party applications, thereby improving security of the video end system and protecting it from downloading potentially malicious third-party applications by unauthorized users.

[0067] For the avoidance of doubt, it should be noted that the computer-implemented method of the second aspect and optional features thereof may form additional steps of the computer-implemented method of the first aspect.

[0068] In a further aspect of the present invention, there may be provided a computer program product comprising instructions which, when executed by a computer, cause the computer to perform the method of any one of the preceding aspects. The computer program product may take the form of a non-tangible computer program product comprising said instructions.

[0069] A further aspect of the present invention may provide a computer readable storage medium having stored thereon a computer program according to the preceding aspect of the present invention. In another aspect of the present invention, there is provided an application environment configured to host third party applications for operation by a video end system, the application environment comprising: Routing control signals from a control interface device of the video end system to a third-party application; Routing media signals between third-party applications and the video conferencing hardware and / or content streams of the video conference call; An application environment can be provided that is configured to:

[0070] In some examples, the application environment may be configured to run on a video end system, while in other examples, the application environment may be configured to run on a cloud service that communicates with the video end system.

[0071] In another aspect of the present invention, there can be provided a video endpoint system for video conferencing, comprising video conferencing hardware and a control interface device, the video conferencing hardware optionally comprising a screen, a camera and a loudspeaker, and the video endpoint system configured to perform the method of any one of the first or second aspects.

[0072] The present invention includes combinations of the described aspects and preferred features except where such combinations are expressly not permitted or explicitly avoided.

[0073] Embodiments and experiments illustrating the principles of the present invention will now be discussed with reference to the accompanying figures. [Brief explanation of the drawings]

[0074] [Figure 1] 1 shows a video conferencing system comprising two video end systems according to the prior art; [Figure 2] 1 illustrates a video end system according to an embodiment of the present invention; [Figure 3] FIG. 3 is a diagram illustrating the video end system of FIG. 2. [Figure 4] FIG. 2 illustrates a video end system according to another embodiment of the present invention. [Figure 5] FIG. 2 illustrates a video end system according to another embodiment of the present invention. [Figure 6] FIG. 2 illustrates a video end system according to another embodiment of the present invention. [Figure 7] FIG. 2 illustrates a video end system according to another embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0075] Aspects and embodiments of the present invention will now be discussed with reference to the accompanying drawings. Further aspects and embodiments will be apparent to those skilled in the art. All documents mentioned in this text are incorporated herein by reference.

[0076] FIG. 1 shows a diagram illustrating a typical video conferencing system comprising a video end system 1 connected to a second video end system 1′ according to the prior art.

[0077] Each video end system 1, 1' comprises an endpoint device 3, 3' connected to a control interface device 2, 2' and video conferencing hardware comprising a display screen 5, 5' and a camera 4, 4'.

[0078] In this example, the first video end system 1 and the second video end system 1' are connected to each other via the Internet 6 to enable two-way communication. In this way, video footage from a camera 4 or other media content (e.g., audio, computer files, text) from the first end system 1 can be received by the second video end system 1' and displayed on a screen 5'. Similarly, content from the second video end system 1' can be received by the first video end system 1.

[0079] The endpoint device 3 may be a local processing resource with proprietary software running on a mass-market operating system such as Android. The endpoint device 3 is configured to control the video end system 1 and enable the video end system to conduct video conference calls. In some examples, the video end system 1 may include multiple local processing resources distributed across multiple devices. In some examples, the video end system 1 may be connected to remote processing resources via the Internet 6.

[0080] The control interface device 2 comprises a user interface and may optionally be a separate device or built into the same unit as the endpoint device 3. For example, the entire video end system may be a personal device (e.g., equipped with a touch screen), or in the case of a conference room system, the control interface device 2 may be one of several control interface devices (e.g., touch screens) positioned around a conference room table. If the control interface device 2 is a separate unit, it may run a mass-market operating system such as Android. Other examples of the control interface device 2 may include a keyboard, a mouse, function buttons, or other user controls.

[0081] The control interface device 2 is configured to receive user input and relay the user input to the endpoint devices as control signals for controlling hardware and conducting the conference call.

[0082] In this example, the control interface device 2, camera 4, and display screen 5 are connected to the endpoint device 3 via wired or wireless data connections. For example, the data connections may be wired connections, including USB or powered Ethernet such as 802.3at, or wireless connections, including Wi-Fi or Bluetooth. The data connections between each device in the video end system 1 may be direct point-to-point connections, or indirect connections, including data relaying through one or more servers on the Internet 6.

[0083] A user who wants to run a third-party application on a prior art video end system may be unable to do so due to incompatibility between the video end system and the third-party application. Therefore, the present inventors have devised a method for running a third-party application using a video end system configured for video conferencing.

[0084] In the following examples, similar features are given corresponding reference numerals and corresponding descriptions apply, except where such descriptions are expressly not permitted or explicitly avoided.

[0085] FIG. 2 shows a diagram illustrating a video end system 100 according to one embodiment of the present invention.

[0086] In this example, video end system 100 comprises a video endpoint device 110 connected to video conferencing hardware 130 and a control interface device 120. Endpoint device 110 is connected to the Internet 160 for exchanging content streams of video conference calls with remote parties.

[0087] Endpoint device 110 is configured to send and receive media signals from video conferencing hardware that may form part of the content stream of a video conference call. For example, in Figure 2, the video conferencing hardware includes camera 1332 and display screen 1334 for recording and displaying video footage.

[0088] The endpoint device 110 is also configured to receive control signals from the control interface device 120 for controlling the video end system 100 .

[0089] In this example, the endpoint device 110 is configured to execute a third-party application 114 hosted within an application environment 112 provided on the endpoint device 110 .

[0090] In this example, an additional user interface environment 122 is provided on the control interface device 120 to host the application user interface of the third party application. For example, if the control interface device 120 includes a touch screen, a video feed from the third party application 114 can be provided to the application user interface running in the user interface environment 122, such that a remote view of the third party application 114 can be displayed on the touch screen.

[0091] The application environment 112 on the endpoint device 110 is also connected to a management agent service 170 via the Internet 160. The management agent service 170 contains information about the identity and permitted behavior of each video end system 100 under its control. Examples of permitted behavior may include details about how each application 114 may be executed, such as a list of authorized third-party applications that may be run on the video end system 100, whether user approval from the user is required, whether the application 114 may be shared with any or specific remote parties, and / or whether the application 114 must be restricted in terms of device or network access. The management agent service 170 is configured to send configuration management controls to the application environment 112.

[0092] Figure 3 shows another diagram of the video end system 100 of Figure 2. In Figure 3, the video conferencing hardware 130 additionally includes a speaker 1336 and a microphone 1338 for playing and recording audio data for the content stream of the video call. The video end system 100 also includes a network interface 150 for connecting to the Internet 160.

[0093] As shown in FIG. 3, the application environment 112 includes an environment interface 116 for relaying signals between the third-party application 112 and other components of the video end system 100 .

[0094] The environment interface 116 (which may also be referred to as the data mapping interface 116) is configured to route control signals 124 from the control interface device 120 to the third-party application 114. The environment interface 116 is also configured to route media signals 140 between the third-party application 112 and the videoconferencing hardware 130 and / or the content stream of the videoconference call via the network interface 150. In some examples, the application environment 112 may be configured to relay other types of data, such as data files, text, etc., between the third-party application 114 and the rest of the video end system.

[0095] In the illustrated example, media signal 140 includes video stream 142 received from a camera, which is relayed via data mapping interface 116 to third party application 112 and to network interface 150 to form part of the content stream of the video call. Also, as part of the content stream of the video call, video stream 144 received from third party application 114 (or from network interface 150) via data mapping interface 116 is relayed to display screen 1134. Similarly, audio stream 148 received from microphone 1338 is relayed via data mapping interface 116 to third party application 112 and to network interface 150 to form part of the content stream of the video call, and audio stream 146 received from third party application 112 (or from network interface 150) is provided to speaker 1336.

[0096] During use, a user may request installation of a third-party application 112 (e.g., an Android application) on the video end system 100 via the user interface of the control interface device 120. The control interface device 120 then contacts the intended application hosting device (i.e., the endpoint device 110 in this example) and instructs the hosting device to generate an application environment 112 for running the third-party application 114, and then downloads the third-party application 114 into the application environment 112.

[0097] The third-party application 114 may have prerequisites for running on a typical device. Generating the application environment 112 includes allocating CPU and memory resources, as well as interfaces such as a display, keyboard, and touch screen for the virtual device and / or application environment 112. The prerequisites of the third-party application 114 are provided so that when the third-party application 114 is launched into the generated application environment 112, the application environment 112 emulates, from the perspective of the third-party application 114, a typical device such as a conventional mass-market tablet or phone.

[0098] The user interface of the video end system 100 located on the control interface device 120 can then be used to connect to the third party application 112 in a variety of ways as described herein.

[0099] In one example, a virtual video stream can be relayed over a connection between endpoint device 110 and control interface device 120 so that the application video output of third-party application 114 can be displayed on control interface device 120. If the aspect ratio and resolution of the screen provided on control interface device 120 are similar to the requirements of third-party application 120, the video view can be displayed full screen. In another example, user interface environment 122 on control interface device 120 may be configured to display the application video output in a window on the screen of control interface device 120. In some examples, a user of control interface device 120 may zoom the video using a zoom gesture, such as a two-finger pinch. This example is useful for setting up third-party application 114, for example, by entering user credentials, where the information should not be shared with other attendees in the conference room.

[0100] In another example, video output from third-party application 114 may be spliced ​​or redirected to display screen 1334 or to the content stream of a video conference call. If display screen 1334 includes a touch screen, touch screen gestures may be relayed to third-party application 114.

[0101] In some examples, if the endpoint device 110 includes a graphics processing unit (GPU), and / or a display, and / or a touchscreen on the same unit, the third-party application 112 hosted on the endpoint device 110 can be given direct access (i.e., without a virtualization layer) to the GPU, display, and / or touchscreen. This can be beneficial to improve the efficiency and performance of the third-party application 114.

[0102] The data mapping interface 116 can be configured to convert the control signal 124 from the control interface device 120 having a first format into an application control signal 124′ for controlling the third-party application 112 having a second format.

[0103] For example, control interface device 120 may include a physical touchpad or a virtual touchpad emulated by the touchscreen of control interface device 120. Touch gestures from the virtual touchpad or the touchpad of control interface device 120 may be relayed as touch gestures to third-party application 112 via the connection between control interface device 120 and endpoint device 110 via data mapping interface 116. In this example, application environment 112 may be configured to accurately map gesture locations between the touchpad or the virtual touchpad of control interface device 120 and a target virtual display of third-party application 112 having a different size, taking into account any active cropping, zooming, screen size differences, and aspect ratios between the touchscreen of control interface device 120 and the target virtual display of third-party application 112.

[0104] In another example, the endpoint device 110 or the control interface device 120 may be programmed with customized instructions associated with a particular third-party application 112 to form an application environment and / or user interface environment 122. The customized instructions can be configured to provide an enhanced user interface on the control interface device 120 to replace the traditional user interface of that third-party application 112. In this way, a user interface more appropriate for the hardware of the video end system 100 can be provided instead of a user interface of a third-party application that may be better suited to different hardware (e.g., a smartphone).

[0105] For example, the enhanced user interface provided on the control interface device 120 may include application-specific function buttons, a virtual keyboard, a dial pad, a directory lookup, or navigation elements configured to control the third-party application 114. The application environment 112 may be configured to translate control signals 124 from the enhanced user interface into control signals 124′ that emulate the gestures and key presses expected by the third-party application 114.

[0106] Customized instructions for a particular third-party application 112 can be provided by a central server (e.g., on the Internet 160), and the customized instructions can be downloaded by the endpoint device 110 and / or the control interface device 120. The customized instructions may include a combination of structured data, code, and / or portable code. In some examples, the customized instructions may comprise an application programming interface (API) provided by the third-party application 114.

[0107] In some examples, a list of approved third-party applications that are permitted to be run by the video end system 100 may be centrally managed by a management server or a cloud service. The video end system 100 may connect to the management server or cloud service to obtain the list of approved applications. If the desired third-party application is included in the list of approved applications, the endpoint device 110 can download the third-party application 114 from a publicly accessible repository (e.g., from the Google Play Store), from a private repository such as a private server, from the management server itself, or from a cloud service. This is beneficial for reducing the possibility of a malicious third-party application 112 being installed on the video end system 100.

[0108] Additionally or alternatively, an administrative interface may be provided (e.g., on the control interface device 120) to allow an administrator to enable or disable approval for particular third-party applications 114. In some examples, a user login or video end system device identifier may be required to enable or disable downloading of a third-party application 114.

[0109] In another embodiment, the management server or management interface can be configured to allow guest applications not included in the list of approved applications to be run by the video end system 100. This can be allowed based on user login or based on a video end system device identifier.

[0110] In a further example, the management server or cloud service may be configured to monitor the usage and behavior of third-party applications 114 .

[0111] In some examples, the endpoint device 110 may be configured to sandbox the third-party application 112, which limits the third-party application's 112 access to the computer resources of the video end system 100. For example, the third-party application 112 may be restricted from full access to the CPU, memory, file system, network, audio resources, or display resources. The third-party application's 112 access to network resources can be limited by hosting the third-party application 112 on a guest virtual LAN, so that network traffic associated with the sandboxed third-party application can be isolated from other network traffic.

[0112] In another example, the sandboxing described above may be performed on a cloud server to further isolate the third-party application 112 from the enterprise network. In this example, the third-party application 112 may only have access to selected media and control signals of the video end system 100.

[0113] 4 shows a diagram illustrating a video end system 200 according to another embodiment of the present invention. In this example, an application environment 212 including a third-party application 214 is provided on a control interface device 220, which also hosts an environment user interface 222. A remote view (i.e., a video stream) of the third-party application 214 is routed by the application environment 212 through the endpoint device 210 to a display screen 2334.

[0114] 5 shows a diagram illustrating a video end system 300 according to another embodiment of the present invention. In this example, an application environment 312 including a third-party application 314 is provided on a cloud service 380.

[0115] An application user interface 322 running on the control interface device 320 is connected via the endpoint device 310 to an application environment 312 on a cloud service 380 via the Internet 360 to control a third-party application 314. A video stream from the third-party application 314 of the application environment 312 is routed from the cloud service 380 via the Internet 360 to the video endpoint device 310 and onto a display screen 3334. If the application user interface 322 requires a video feed from the third-party application 314, the video stream can also be routed to the control interface device 322 via the same connection to allow the application 312 to be viewed (e.g., as part of the user interface) on the control interface device 322.

[0116] 6 is a diagram illustrating a video end system 400 according to another embodiment of the present invention, in which a control interface device 420 is not directly connected to an endpoint device 410 by a wired or wireless link, but instead is indirectly connected to the endpoint device 410 via the Internet 460 using a control relay service 490. Control signals from an application user interface 422 running on the control interface device 420 can be routed to the application environment 412 via the control relay service 490. If the application user interface 422 requires a video feed from the application environment 412, the video feed can be routed over the same connection to allow the third-party application 414 to be viewed (e.g., as part of the user interface) on the control interface device 420.

[0117] 7 is a diagram illustrating a video end system 500 according to one embodiment of the present invention, in which the video end system 500 is participating in a conference call with a remote video end system 500'. In this example, the video output from the third party application 514 is provided by the application environment 512 as a content stream for the video call, allowing the video output of the third party application 514 to be sent to both the local display 5334 and the remote video end system 500' for display on the remote display screen 5334'.

[0118] The features disclosed in the foregoing description, or in the following claims, or in the accompanying drawings, and which, where appropriate, are expressed in their specific form or in terms of means for performing a disclosed function, or a method or process for obtaining a disclosed result, can be utilized to realize the invention in various of its forms, either separately or in any combination of such features.

[0119] While the present invention has been described in conjunction with the exemplary embodiments set forth above, many equivalent modifications and variations will be apparent to those skilled in the art given this disclosure. Accordingly, the exemplary embodiments of the invention set forth above are considered to be illustrative and not limiting. Various modifications to the described embodiments can be made without departing from the spirit and scope of the invention.

[0120] For the avoidance of any doubt, any theoretical explanations provided herein are provided for the purpose of enhancing the understanding of the reader, and the inventors do not wish to be bound by any of these theoretical explanations.

[0121] Any section headings used herein are for organizational purposes only and should not be construed as limiting the subject matter described.

[0122] Throughout this specification, including the claims that follow, unless the context indicates otherwise, the words "comprise" and "include," and variations such as "comprises," "comprising," and "including," will be understood to imply the inclusion of stated integers or steps or groups of integers or steps, but not the exclusion of any other integers or steps or groups of integers or steps.

[0123] It should be noted that, as used in this specification and the appended claims, the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. Ranges may be expressed herein as from "about" one particular value and / or to "about" another particular value. When such a range is expressed, another embodiment includes from the one particular value and / or to the other particular value. Similarly, when values ​​are expressed as approximations, it will be understood that the particular value forms another embodiment by use of the antecedent "about." The term "about" in connection with numerical values ​​is optional and means, for example, + / - 10%.

Claims

1. 1. A computer-implemented method for operating a third-party application using a video end system, the video end system being for conducting video conference calls, the video end system comprising video conferencing hardware and a control interface device, the computer-implemented method comprising: the computer-implemented method comprising: providing an application environment for executing the third-party application, the application environment comprising: routing control signals from the control interface device to the third-party application; routing media signals between the third-party application and the video conferencing hardware and / or content stream of the video conference call; a providing step configured to:

20. A computer-implemented method comprising:

2. The computer-implemented method of claim 1 , wherein the videoconferencing hardware comprises a camera and a display screen, and the media signal comprises a video stream.

3. The computer-implemented method of claim 1 or 2, wherein the videoconferencing hardware includes a loudspeaker and a microphone, and the media signal includes an audio stream.

4. The computer-implemented method of any one of claims 1 to 3, wherein the application environment is provided on the video end system.

5. The computer-implemented method of claim 4 , wherein the application environment is provided on the control interface device.

6. The video end system, the control interface device; and a video endpoint device in communication with the videoconferencing hardware; 5. The computer-implemented method of claim 4, wherein the application environment is provided on the video endpoint device.

7. 7. The computer-implemented method of claim 6, wherein the control interface device and the video endpoint device communicate via a cloud-based relay service, and the application environment is configured to receive the control signals from the cloud-based relay service.

8. The computer-implemented method of any one of claims 1 to 3, wherein the video end system is in communication with a cloud service, and the application environment is provided on the cloud service.

9. the control signals include signals for selectively enabling and disabling the routing of the media signals between the third-party application and the video conferencing hardware and / or the content stream of the video conference call. A computer-implemented method according to any one of claims 1 to 8.

10. the application environment is configured to convert the control signals from the control interface device having a first format into application control signals for controlling the third-party application having a second format; A computer-implemented method according to any one of claims 1 to 9.

11. the control interface device comprises a keyboard for sending control signals that are keyboard events; the application environment is configured to convert the keyboard events into application control signals for controlling the third-party application. The computer-implemented method of claim 10.

12. the control interface device comprises a touch screen having a first size for sending control signals; the third-party application is configured to receive application control signals from a touch screen having a second size; the application environment is configured to scale the control signals from the control interface device into application control signals for controlling the third-party application.

12. A computer-implemented method according to claim 10 or 11.

13. the application environment is configured to convert the media signals from the third-party application having a first format into media signals for the content stream of the video conference call and / or the video conference hardware having a second format. A computer-implemented method according to any one of claims 1 to 12.

14. the application environment is formed from an application programming interface (API) provided by the third-party application; A computer-implemented method according to any one of claims 1 to 13.

15. The computer-implemented method of any one of claims 1 to 14, wherein the application environment comprises customized instructions corresponding to the third-party application.

16. the customized instructions are stored in a central configuration service; The computer-implemented method of claim 15 , wherein the computer-implemented method includes obtaining the customized instructions from the central configuration service.

17. the video end system is in communication with a central monitoring service; The computer-implemented method of any one of claims 1 to 16, wherein the computer-implemented method comprises monitoring the third-party application from the central monitoring service.

18. 18. The computer-implemented method of claim 1, wherein the video end system comprises an administration interface for allowing an administrator to enable or disable approval for downloading third-party applications.

19. 19. The computer-implemented method of claim 1, wherein the application environment including the third-party application is sandboxed, and sandboxing the application environment limits access to computer resources by the third-party application.

20. 1. A computer-implemented method for installing a third-party application for use by a video end system, the video end system being for video conferencing, the computer-implemented method comprising: the computer-implemented method comprising: generating an application environment, The application environment: routing control signals from a control interface device to the third-party application; routing media signals between the third-party application and the video conferencing hardware and / or content stream of the video conference call; a generating step configured to: installing the third-party application within the application environment; 20. A computer-implemented method comprising:

21. downloading customized instructions corresponding to the third-party application from a central configuration service to create the application environment; 21. The computer-implemented method of claim 20, further comprising:

22. communicating with a management server or cloud service to obtain a list of approved applications; downloading the third-party application only if the third-party application is included in the list; 22. The computer-implemented method of claim 20 or 21, further comprising:

23. receiving user authentication information to enable installation of the third-party application; The computer-implemented method of any one of claims 20 to 22, further comprising:

24. 1. A computer program comprising instructions, The instructions, when the computer program is executed by a computer, cause the computer to perform the method of any one of claims 1 to 23. Computer program.

25. A video end system for video conferencing, comprising: Video conferencing hardware, a control interface device; It is equipped with The video end system is configured to perform the method according to any one of claims 1 to 23. Video end system.

Images