Video conferencing end-system

Abstract

A video end-system (250a) for serving as an endpoint (203a) in a video conference, the video end-system (250a) comprising one or more processors configured to conduct a video conference by performing the following steps: establishing a logical association with a media source (220) communicatively connectable with the video end-system (250a); receiving media from the media source (220), wherein the media is directed to the video end-system (250a) in dependence on the logical association; and inserting the media into a video conference established with another video conferencing endpoint (203b).

Description

[0001] VIDEO CONFERENCING END-SYSTEM

[0002] FIELD OF THE INVENTION

[0003] The present invention relates to video conferencing systems, in particular to a video endsystem that can receive media from a media source for insertion into a video conference.

[0004] BACKGROUND

[0005] Video conferencing systems typically comprise two or more endpoints, each endpoint being configured to communicate with other endpoints via a data or telephony network, or other connections, for implementing a two-way or multiway video and / or audio conference call.

[0006] An endpoint, which may also be referred to as a video end-system, is configured to enable users to communicate with other users over a network by sending data streams, such as audio, video and other content streams (for example, data files, text, or screen sharing) from one endpoint to another in a two-party conference call. For a multiway conference call, the data streams may be transmitted to a centralised or distributed conferencing network where they are switched or transcoded, allowing multiple participants to communicate and share information. A video end-system may comprise a single purpose endpoint device whose main function is to support video and / or audio conference calls. The video end-system may comprise separate video conferencing hardware, which may include one or more video monitors, video cameras, audio microphones, audio speakers and a controller for enabling users to control the video end-system. These devices will typically be connected permanently to the video end-system and remain part of the video end-system’s configuration.

[0007] On some occasions, it may be advantageous to provide content streams into a video conference from an external device. Video conferencing end-systems will typically allow this by means of a cable, such as an HDMI cable, or a network stream using a protocol such as AirPlay, Google Cast or Miracast. Figure 1 shows a diagram of a typical known video conferencing system 100 featuring a laptop 110 being used to inject media into the content stream of a video conference. Two video end-systems 103 are connected to respective controllers 102, displays 101 , 105 and cameras 104. They form a connection 107 across the internet 106 to stream video and audio from the cameras and microphones to establish communication over the screens 101 , 105. The laptop 110 is connected via a traditional High-Definition Multimedia Interface (HDMI) cable into the video end-system 103 and provides a video and / or audio feed to the content stream that is shown on displays 101 , 105.

[0008] Establishment of a cabled connection is generally straightforward. Establishment of a networked connection may require additional configuration, typically including device discovery, device trust, selection of the exact video source (such as screen or application window) of the sending device, then video selection on the end-system.

[0009] A meeting room may however already contain another fixed device, such as another video conferencing end-system or an interactive display system that is connected to the corporate network and is managed by centralised network management system. It is therefore undesirable that such a device requires a complex association procedure before it can be used as a content stream for the first video end-system.

[0010] It is desirable to develop an improved mechanism for granting a video end-system access to a media source at an auxiliary device for inclusion in a video conference.

[0011] SUMMARY OF THE INVENTION

[0012] According to a first aspect, there is provided a video end-system for serving as an endpoint in a video conference, the video end-system comprising one or more processors configured to conduct a video conference by performing the following steps: establishing a logical association with a media source communicatively connectable with the video end-system; receiving media from the media source, wherein the media is directed to the video end-system in dependence on the logical association; and inserting the media into a video conference established with another video conferencing endpoint. The one or more processors may be configured to establish a logical association with the media source communicatively connectable with the video end-system by communicating with a video conferencing server. The video conferencing server may be at a cloud server. The video conferencing server may be implemented by one or more servers at a location remote from the video end-system and the auxiliary device and that is / are in communication with the video end-system and the auxiliary device over the internet. Alternatively, the logical association with the media source may be established in other ways than by communicating with a video conferencing server, such as such as via Bluetooth, acoustic pairing, proximity pairing or user interaction.

[0013] The media may be inserted into the video conference established with another video conferencing endpoint by communicating with the video conferencing server. The video conferencing server may comprise a management server, at which the logical association may be established, and optionally a conference server, via which a media stream (optionally including the media from the media source) can be sent to the other video conferencing endpoint as part of the video conference. The management server and / or the conference server may be remote from the video end-system. They may be separate entities to the video end-system. The management server may be a separate entity to the conference server. In some embodiments, the media stream (optionally including the media from the media source) may be sent to the other video conferencing endpoint via an alternative connection (for example a point-to-point / wired connection).

[0014] The media source may be remote from the video end-system. The media source may be wirelessly connectable with the video end-system. The media source may be an auxiliary computing device communicatively connectable with the video end-system. The media source may be implemented by one or more processors of (or associated with) an auxiliary computing device communicatively connectable with the video end-system.

[0015] The one or more processors of the video end-system may be configured to implement a first software entity for supporting the video conference, the first software entity being capable of receiving media input from any of multiple input channels. The video endsystem may be capable of implementing a second software entity for receiving media from the media source and supplying it to the first software entity by emulating one of the input channels. The one or more processors may be configured to establish a control channel and / or a media channel between the video end-system and the media source. The one or more processors may be configured to establish a control channel and / or a media channel between the video end-system and the auxiliary computing device. The control channel and / or the media channel may be established via the video conferencing server. For example, the control channel and / or the media channel may be established via the management server. The video end-system may receive the media from the media source via the media channel. Alternatively, the control channel may be established with the media source and / or computing device directly.

[0016] The video end-system may comprise a user interface for controlling aspects of the video end-system and / or the media source and / or the auxiliary computing device.

[0017] The user interface may comprise a user input device. The one or more processors may be configured to receive a signal in response to a user input at the user input device and in response to receiving the signal, receive the media from the media source for insertion into the video conference. The user input device may become visible on the user interface when a connection between the computing device and the video end-system is established.

[0018] The video end-system may comprise video conferencing hardware, such as one or more display screens, cameras or microphones. The one or more processors may be further configured to cause the media received from the media source to be output at the video conferencing hardware.

[0019] The one or more processors may be configured to receive the media over a data link that supports the transfer of media data from the media source to the video end-system only. The data link may be a media channel that is separate to one or more video conference channels on which the data for the video conference is sent to and received from the other endpoint. The data link may be via the videoconferencing server (for example, via the management server).

[0020] The video end-system may be configured to establish logical associations with multiple media sources (for example, by communicating with the video conferencing server or in other ways as described above). The video end-system may be configured to receive media from one of the multiple media sources to insert into the video conference in response to a user selection of the one of the multiple media sources and in dependence on a respective one of the logical associations corresponding to the one of the multiple media sources.

[0021] Access to one or more media sources associated to a video end-system at the video conferencing server may be restricted if it is determined that the auxiliary computing device acting as or implementing that media source is out of range of the video endsystem. The proximity of an auxiliary computing device to the video end-system may be determined using techniques such as using Bluetooth or audio pairing. If an auxiliary device is determined to be out of range of the video end-system, this may be indicated at the server so that the media from the media source cannot be inserted into the video conference.

[0022] According to another aspect, there is provided a computing device configured to communicate with a video end-system serving as an endpoint in a video conference, the computing device comprising one or more processors configured to perform the following steps: establishing a logical association between a media source and the video endsystem; and causing transmission of media from the media source to the video endsystem to be inserted into a video conference established between the video end-system and another video conferencing endpoint, wherein the media is directed to the video endsystem in dependence on the logical association.

[0023] The one or more processors may be configured to establish a logical association with the video end-system by communicating with a video conferencing server. The video conferencing server may be at a cloud server. The video conferencing server may be implemented by one or more servers at a location remote from the video end-system and the auxiliary device and that is / are in communication with the video end-system and the auxiliary device over the internet. Alternatively, the logical association with the video endsystem may be established in other ways, such as such as via Bluetooth, acoustic pairing, proximity pairing or user interaction. The computing device may be an auxiliary computing device. The computing device may be remote from the video end-system. The one or more processors may be configured to cause transmission of media from the media source to the video end-system to be inserted into a video conference established between the video end-system and another video conferencing endpoint by communicating with the video conferencing server.

[0024] The media source may be the computing device. The media source may be implemented by one or more processors of (or associated with) the computing device communicatively connectable with the video end-system.

[0025] The one or more processors of the computing device may be configured to implement a first software entity (which may or may not be the same as the first software entity referred to above) capable of outputting media from the media source on any of multiple output channels. The computing device may be capable of implementing a second software entity (which may or may not be the same as the second software entity referred to above) for receiving media from the media source and supplying it to the video end-system by emulating one of the output channels.

[0026] The one or more processors may be configured to process media data output by the media source to form a compressed representation of the media data to supply to the video end-system for insertion into the video conference.

[0027] The one or more processors may be configured to establish a control channel between the computing device and the video end-system. The one or more processors may be configured to establish a control channel between the media source and the video endsystem.

[0028] The computing device may comprise a user interface for controlling aspects of the computing device and / or the media source and / or the video end-system. The one or more processors may be configured to update the user interface when a connection between the computing device (and / or the media source) and the video end-system is established. The user interface may comprise a user input device, wherein in the one or more processors are configured to receive a signal in response to a user input at the user input device and in response to receiving the signal, cause the transmission of the media to the video end-system for insertion into the video conference. The user input device may become visible on the user interface when the connection between the computing device and the video end-system is established. The user interface may be configured to advertise the user input device as a means of sending media from the media source to the video end-system.

[0029] In response to receiving the signal, the one or more processors may be configured to establish a media channel between the media source and the video end-system.

[0030] The computing device and / or the media source may be an interactive whiteboard, a mobile phone, a tablet or a laptop computer. The media source may be configured to output one or more of the following media types: audio, video, image files, computer files and text.

[0031] The logical association may be a persistent logical association for use in multiple video conferencing sessions.

[0032] The video conferencing server may be a cloud server or a local area network (such as a corporate network).

[0033] According to a further aspect, there is provided a computer-implemented method for conducting a video conference at a video end-system configured to act as an endpoint in a video conference, the method comprising performing the following steps: establishing a logical association with a media source communicatively connectable with the video end-system; receiving media from the media source, wherein the media is directed to the video end-system in dependence on the logical association; and inserting the media into a video conference established with another video conferencing endpoint.

[0034] According to a further aspect, there is provided a computer-implemented method for implementation at a computing device configured to communicate with a video endsystem serving as an endpoint in a video conference, the method comprising the following steps: establishing a logical association between a media source and the video endsystem; and causing transmission of media from the media source to the video end- system to be inserted into a video conference established between the video end-system and another video conferencing endpoint, wherein the media is directed to the video endsystem in dependence on the logical association.

[0035] In some further examples, the auxiliary computing device may be configured to run an application. The application may be a third-party application. For example, the auxiliary computing device may be configured to run presentation software or video editing software. This may provide the media source discussed above. An output of the third- party application may be included in a content stream of the video conferencing call, as described above. The third-party application may be provided in an application environment which is configured to provide an interface between the third-party application and the remainder of the auxiliary computing device.

[0036] The auxiliary computing device may comprise a control interface device. The control interface device may comprise or be configured to display the user interface of the auxiliary computing device. The computer implemented method may further comprise: providing an application environment for a third-party application to run on, the application environment configured to: route control signals from the control interface device to the third-party application; and route media signals between the third-party application and the auxiliary computing device hardware and / or a data link to the video end-system for inclusion in a content stream of a video conferencing call. Advantageously, by providing an application environment for running the third-party application, the third-party application may be used with an auxiliary computing device without experiencing issues owing to incompatibilities of the third-party application with the auxiliary computing device.

[0037] The computer implemented method may additionally comprise running the third-party application in the application environment. Accordingly, the functionality of the auxiliary computing device may be expanded by running new applications.

[0038] In one or more examples, the application environment may include a control signals interface, that is to say programming configured to route control signals from the control interface device of the auxiliary computing device to the third-party application. The auxiliary computing device may comprise hardware such as a display screen and / or speakers. In one or more examples, the application interface may include a media signals interface, that is programming configured to route media signals between the third-party application and the auxiliary computing device hardware and / or a data link to the video end-system for inclusion in a content stream of a video conferencing call.

[0039] The third-party application may include a collaborative application. For example, the application may be one or more of a document sharing, whiteboarding, multimedia presentation, training, reviewing, polling or question submission application. The third- party application may be provided by or downloaded from an external source (i.e. the third-party application is not the part of system software which was originally provided with the auxiliary computing device). The third-party application may be configured to run on mass-market hardware such as tablets or smart phones.

[0040] An application environment is intended to refer to a framework or platform which has been configured to accept the deployment of a single application. The application environment comprises a predefined collection of computing resources that host an application. The application environment may be configured to provide the application with virtualised or normalised device interfaces for enabling the application to run even if physical devices (originally intended for use with the application) are disconnected or are different from what the application expects. For example, the virtualised device interfaces may be configured to re-routed data to different physical devices whilst the application is running. In some examples, multiple application environments may be provided to operate multiple respective applications.

[0041] The application environment may be provided on a processor of the auxiliary computing device or on a processing element which is connected to the processor (e.g. via an internet connection to a cloud service). In some examples, the application environment may be provided on the control interface device of the auxiliary computing device.

[0042] The application environment may comprise a data mapping interface which is configured to relay control signals from the control interface device to the third-party application; and relay media signals between the third-party application and the auxiliary computing device hardware and / or a data link to the video end-system for inclusion in a content stream of a video conferencing call. The data mapping may be a virtual interface which defines a boundary to the application environment and act as a virtualisation layer by relaying data between the third-party application inside the application environment and software and hardware which is external to the application environment.

[0043] The media signals may include video data, and / or audio data (e.g an audiovisual signal operating in a mode with a camera turned “off”) which can form the content stream of a video conferencing call. In some examples the media signals may include other types of data which may pass to and from the third-party application. For example, the media signals may include computer files, or text (e.g. other types of audiovisual related data, for example stills, computer files, text that can be shared over a video link). The control signals may comprise signals for selectively enabling and disabling the routing of the media signals between the third-party application and the auxiliary computing device hardware and / or a data link to the video end-system for inclusion in the content stream of the video conferencing call.

[0044] For example, the media signals may comprise video from the third-party application for enabling remote view of the third-party application via the auxiliary computing device hardware and / or a data link to a video end-system for inclusion in the content stream of the video conferencing call. Therefore, the computer implemented method may comprise receiving remote view control signals from the control interface device, and enabling or disabling remote view of the third-party application on one or more video conferencing hardware units and / or enabling or disabling remote view of the third-party application for one or more channels of a video conferencing call based on the remote view control signals. In this way, a user of the video end-system may decide which display screens to display the third-party application on and which remote participants of the video conferencing call can view the third-party application.

[0045] The application environment may be configured to transform the control signals from the control interface device of the auxiliary computing device having a first format to application control signals for controlling the third-party application having a second format (i.e. using a data mapping interface). In other words, the computer implemented method may comprise: using the data mapping interface to transform (i.e., reformat) the control signals from the control interface device having a first format to application control signals for controlling the third-party application having a second format. For example, the application environment may be configured to emulate gestures or key-presses in the application upon receipt of user control signals from the control interface. This may include emulating function buttons, a virtual keyboard, a dial pad, a directory lookup, or other navigation elements for the third-party application in the virtual environment.

[0046] For example, the control interface device (or one or the control interface devices) for the auxiliary computing device may comprise a keyboard for sending control signals which are keyboard events. The application environment may be configured to transform the keyboard events into application control signals for controlling the third-party application. In other words, the computer implemented method may comprise the step of transforming the control signals comprising keyboard events from the control interface device having a first format to application control signals for controlling the third-party application having a second format. The application control signals may be configured to mimic control signals expected by the third-party application. For example, the key board events may be transformed into function signals corresponding to function buttons normally provided by the third-party application.

[0047] The control interface device may comprise a touch screen having a first size for sending control signals, and the third-party application may be configured to receive application control signals from a touch screen having a second size. For example, the application may be intended to be run on a smart phone and the control interface device may be a much larger screen. In this case, the application environment may be configured to scale or map the control signals from the control interface device to the application control signals for controlling the third-party application. In other words, the computer implemented method may comprise the step of scaling the control signals from the control interface device to form application control signals for controlling the third-party application. For example, the application environment may be configured to receive position and movement information from the control interface device touchscreen and scale the position and movement information proportionally to correspond to the application touchscreen.

[0048] The control interface device may comprise a user interface environment for running an application user interface associated with the third-party application. For example, the control interface device may comprise a screen or a touch screen and the application user interface may be configured to display a representation of the third-party application on the screen. The representation of the third-party application may include function buttons, a virtual keyboard, a dial pad, a directory lookup, or other navigation elements for navigating and / or controlling the third-party application. Similarly, the application environment may be configured to transform video signals from the third-party application having a first format to media signals for the content stream of the video conferencing call and / or the video conferencing hardware having a second format. In other words, the computer implemented method may comprise the step of transforming media signals from the third-party application having a first format to media signals for the content stream of the video conferencing call and / or the auxiliary computing device hardware having a second format. For example, the media signals may include video signals and the application environment may be configured reformat video data to map screen sizes, video formats (e.g. compressed vs uncompressed), numbers of pixels etc between formats compatible with the application and the auxiliary computing device hardware and / or the content stream of the video conferencing call. In other examples, the media signals may include audio signals, images, or file data etc which are transformed from a first format compatible with the third-party application to a second format.

[0049] The application environment may comprise a data mapping interface which is formed from an application programming interface (API) provided by the third-party application.

[0050] In some examples, the application environment may comprise customised instructions corresponding to the third-party application. The customised instructions may be stored in a centralised configuration service. The centralised configuration service may be located on a remote server which is accessible by the auxiliary computing device. In this case, the computer implemented method may comprise retrieving the customised instructions from the centralised configuration service and implementing or creating the data mapping interface based on the customised instructions.

[0051] In another aspect, there may be provided an application environment configured to host a third-party application for operation by an auxiliary computing device, wherein the application environment is configured to: route control signals from a control interface device of the auxiliary computing device to the third-party application; and route media signals between the third-party application and auxiliary computing device hardware and / or a data link to the video end-system for use in a content stream of a video conferencing call.

[0052] The video end-system and / or the computing device may have any of the optional features described above or be configured to perform the steps of the computer-implemented methods above.

[0053] According to a further aspect, there is provided a computer program comprising instructions that when executed by a computer cause the computer to perform the methods above.

[0054] According to a further aspect, there is provided a computer-readable storage medium having stored thereon computer readable instructions that when executed at a computer (for example, comprising one or more processors) cause the computer to perform the methods above. The computer-readable storage medium may be a non-transitory computer-readable storage medium. The computer may be implemented as a system of interconnected devices.

[0055] BRIEF DESCRIPTION OF THE FIGURES

[0056] The present invention will now be described by way of example with reference to the accompanying drawings. In the drawings:

[0057] Figure 1 shows a diagram of a typical video conference where a laptop connected to the video end-system via a cable provides a content stream that is shared over the video conference;

[0058] Figure 2a shows a diagram of an auxiliary device being used as a media source for a conference;

[0059] Figure 2b shows an example of connections and signals between various entities in the system to insert media from a media source into a video conference;

[0060] Figure 3 schematically illustrates an example of an auxiliary computing device;

[0061] Figure 4 shows a diagram of a video conferencing system according to another example; Figure 5 shows a diagram of a video conferencing system according to a further example; Figure 6 shows a diagram of a video conferencing system according to another example. Figure 7 schematically illustrates the steps of an exemplary method for conducting a video conference at a video end-system;

[0062] Figure 8 schematically illustrates the steps of an exemplary method for implementation at an auxiliary computing device configured to communicate with a video end-system;

[0063] Figure 9 illustrates an exemplary device configured to operate as a video end-system and / or an auxiliary computing device and some of its associated components.

[0064] DETAILED DESCRIPTION

[0065] The present invention relates to sharing media from a media source with a video endsystem that can be inserted into a video conferencing media stream to form a logically composite system for video conferencing.

[0066] Figure 2a shows an example of a video conferencing system 200. In this example, two video end-systems 250a, 250b are communicatively connectable to each other, allowing a video conference to be established between the two video end-systems.

[0067] In the following examples, the term video end-system is used to refer to a dedicated video conferencing system. The video end-system comprises a video conferencing endpoint and may also comprise function specific hardware and software for performing video conferencing calls. The endpoint may be a uniquely addressable point on a network that can engage in a call with another video conferencing endpoint. The video end-system may be a shared video end-system that is configurable for use by multiple individual users. For example, the shared video end-system may be located in a corporate meeting room which is available to be reserved and used by multiple users for making conference calls.

[0068] The video end-system comprises one or more processors. The video end-system may comprise a main processor. The main processor may be a processing element of the video end-system which is configured to run system software for operating the video endsystem and perform video conference calls. The main processor may be configured to communicate with video conferencing hardware and receive commands from a controller of the video end-system. The main processor may be a separate endpoint device such as a single-purpose computing device or a local server, which is connected, for example by wired or wireless connection, to the video conferencing hardware and a controller of the video end-system. As such, the device comprising the main processor may also be referred to as an endpoint device. In other examples, the main processor may form part of a single unit which also includes the controller and / or the video conferencing hardware. The main processor may comprise a network interface for sending and receiving a content stream of a video conferencing call.

[0069] The video end-systems shown in Figure 2a can therefore be used for performing video conferencing calls. In this example, each video end-system 250a, 250b comprises a video conferencing endpoint 203a, 203b. In this example, the endpoints 203a, 203b are communicatively connectable with each other over a communication network 206. The network 206 may, for example, be the internet, or a corporate network. The video endsystem 250a may establish a video conference with the endpoint of the other video endsystem 250b via the communication network 206 (i.e. by sending media data over the communication network).

[0070] Infrastructure is provided to support video conferencing between the video end-systems in association with an auxiliary media source, as described herein. This infrastructure is shown generally as a video conferencing server 215 in Figure 2a. The infrastructure may be entirely located remotely from the video end-systems, and may communicate with the endpoints of the video end-systems over a network 206 such as the internet. The communication may be wireless. Alternatively, part or all of the infrastructure may be located on common premises with one or more of the video end-systems, for example in a corporate managed implementation. The infrastructure may be provided by one or more computing devices or servers. Each server may have one or more processors for implementing its functions. The servers may be located close to each other or in dispersed locations. In one example, one physical or cloud server 222 (for example a live conference server) may support the establishment and maintenance of a video conference between two video end-systems, and another physical or cloud server 221 (a central management server) may support the establishment of a logical association between a video end-system and a media source, as described below. In another embodiment both of those functions may be provided by a single video conferencing server, that server providing a live conference service and a central management service. In some embodiments, the server 222 may not be present as part of the video conferencing server 215 and the connection between the video end-systems 250a and 250b for sending the media as part of the video conference may be via a point-to-point or cabled connection. In the present specification the term video conferencing server refers to a server that is configured to provide support for one or more of the video conferencing functions as described herein between the video end-systems.

[0071] In the example shown in Figure 2a, the video end-system is communicatively connectable with a conference server 222. This connection is via the communication network 206, such as the internet. The video end-system may be one of a plurality of video endsystems in communication with the conference server 222. The video end-system may establish a video conference with one or more other video conferencing endpoints which may be part of respective video end-systems via the server 222. The video end-system is communicatively connectable with one or more other video conferencing endpoints via the network 206. A media stream can be sent between the video end-system 250a and one or more other endpoints over the communication network 206 via channel 207 in Figure 2a.

[0072] One or more processors of the video end-system (for example the main processor) may be configured to implement one or more software entities which are configured to operate the video end-system and enable the performance of a conference call using the video end-system. The video end-system may be operated to conduct a video and / or audio conferencing call by running system software. The system software may be an application or a computer program which includes proprietary software which is provided with and configured to operate with the video end-system. The video end-system may be managed by a standard operating system. For example, the standard operating system may be a mass-market operating system such as Windows, Apple iOS, Linux or Android.

[0073] One or more processors of each video end-system (for example, the main processor) may be configured to implement a first software entity for supporting the video conference. The first software entity may be an application, computer program or software package running on the video end-system. The first software entity may be capable of receiving media input from any of multiple input channels. The multiple input channels may be, for example, input channels for media output by the video conferencing hardware (such as the camera or microphone), or one or more input channels a port such as a High-Definition Multimedia Interface (HDMI) port to which an additional media source may be connected and media output by the media source may be provided for inclusion in the video conference. The media input to the input channels may be transmitted to the other video end-system during the video conference. The one or more processors may also be configured to implement additional software entities (such as applications), as will be described in more detail below.

[0074] Each video end-system may comprise video conferencing hardware. The video conferencing hardware may comprise one or more devices which are in communication with the one or more processors of the video end-system that are used for video conferencing. For example, the video conferencing hardware may comprise a camera and / or a display screen for sharing images and / or a video stream in a video conferencing call. Specifically, a video stream may be communicated from the camera via one of the media input channels to be included in the video stream of a video conferencing call. One or more additional video streams may be received from the video stream of the video conferencing call and displayed on the display screen. In some examples, the video endsystem may comprise multiple cameras and display screens.

[0075] The video conferencing hardware may also comprise a loudspeaker and / or a microphone for sending and receiving an audio stream of a conference call. Specifically, the video conferencing hardware may comprise one or more audio loudspeakers (e.g. as area broadcasting speakers or as loudspeakers included in personal headphones) for outputting audio content of a conferencing call. The video conferencing hardware may comprise one or more audio microphones from providing an audio stream for the content stream of a conferencing call. In further examples, the control interface and video conferencing hardware may comprise separable units which are in communication with each other (for example via wired or wireless connections).

[0076] Each component of the video conferencing hardware (i.e., cameras, display screens, microphones, loudspeakers, etc) may be provided as one or more integral units, or as separate devices. The video conferencing hardware and the control interface may also be provided as one or more integral units or as separate devices. For example, a controller having a touch screen may serve as a display screen as well as the control interface. Each video end-system may also comprise a controller, which may also be referred to as a control interface or a control interface device. The controller may be any device comprising user input controls, such as a user interface, for enabling users to control the video end-system. For example, the control interface device may be a tablet comprising a touch screen. The control interface device may comprise a user input device via which a user may use to issue control signals to the video end-system by engaging with a user input device (for example, by pressing buttons) on the touch screen. Other examples of a control interface device may include a keyboard, a mouse, a control panel with mechanical buttons and a voice control device. For example, a user may use the control interface device to begin and end a conference call, add or remove parties from a conference call, adjust volume levels of speakers in the video end-system and configure display screen settings. Multiple control interface devices may be provided which are connected to (for example via a wired or wireless connection) to the remainder of the video end-system.

[0077] The controller and the video conferencing hardware may also be provided as a single unit. For example, the video end-system may be a single desktop or handheld unit which may also be referred to as a video end-device. In yet further examples, the control interface and video conferencing hardware may comprise separable units which are in communication with each other, for instance via the endpoint (e.g. via wired or wireless connections).

[0078] In the example shown in Figure 2a, each respective endpoint 203a, 203b is connectable to a respective controller 202a, 202b, displays 201 a, 201 b, 205a, 205b and a respective camera 204a, 204b. In this example, the endpoints 203a, 203b form a connection 207 across the network 206 (for example, the internet) to stream video and audio from the cameras 204a, 204b and / or microphones to establish communication. The audio and video may be output at the displays 201 a, 201 b, 205a, 205b and / or speakers.

[0079] The video end-system is associated with a centralised management system (also referred to as a centralised service or server), shown at 221 in Figure 2a. The endpoint 203a is communicatively connectable with a media source. In one embodiment the media source may be a physical device that is dedicated to serving media. It could, for example, be a camera, microphone or digital whiteboard. In another embodiment the media source may be a computing device that can serve media and also perform other functions. The computing device may, for example, be a tablet, smartphone or other personal computing device. The computing device may store in non-transient form code executable by one or more processors of the computing device for serving media. For example, the code could implement a whiteboard function that can receive drawing input from a user and output a media image representing that drawing input. Alternatively, the code could implement a slide presentation function that can output images of presentation slides stored on the device. In this embodiment the media source may be an application defined by the stored code and executed by the processor(s) of the device.

[0080] In the example shown in Figure 2a, the endpoint 203a is communicatively connectable with an auxiliary computing device 220 which serves as a media source. The auxiliary computing device may be remote from the video end-system 250a. It may be a separate entity. The auxiliary computing device 220 may be, for example, an interactive whiteboard, a mobile phone, a tablet or a laptop. The auxiliary device 220 is also communicatively connectable with the network 206. The device 220 is communicatively connectable with the central management system 221. The endpoint 203a may be connectable with the device 220 via a direct channel 225 and can also communicate with the device 220 via the management system 221 , via channels 223 and 224 shown in Figure 2a. The centralised management system 221 will be described in more detail below.

[0081] The one or more processors of the computing device 220 are configured to implement a first software entity capable of outputting media from the media source on any of multiple output channels. The one or more processors of the computing device may also be capable of implementing one or more other software entities, as will be described in more detail below. The first software entity may be an application, computer program or software package running on the auxiliary computing device 220. The application may be implemented by one or more processors of the computing device 220. For example, the application may be one or more of a document sharing, whiteboarding, multimedia presentation, training, reviewing, polling or question submission application. The media source may be configured so as to output media via one or more of the multiple output channels. The media may include video data, and / or audio data. In some examples the media may include other types of data which may be received from the media source. For example, the media may include computer files, still images or text.

[0082] It may be desirable to send media from the media source to the video end-system for inclusion in a video conference established between the endpoint 203a and another endpoint 203b. For example, a user may wish to share notes or a drawing from a virtual whiteboard, or a tablet. Thus, the media source may provide an auxiliary media stream for inclusion in the video conference.

[0083] The auxiliary device 220 and video end-system may establish a control connection directly (for example via 225 in Figure 2a) or via the central management service 221 .

[0084] The auxiliary device 220 may be connectable via channel 223 to the centralised service 221. The centralised service 221 may manage connections and / or logical associations between the video end-system and one or more media sources, such as the auxiliary computing device 220. A logical association may be instantiated by the storing of data defining a link between the end-system and a respective media source. Once a logical association has been instantiated, it may be used to assist and / or permit the establishment of an active media link between the end-system and the respective media source. The logical association may comprise an address of the end-system and / or the respective media source. In one mode of operation, when the end-system is to establish an active media link with a media source it may interrogate the server 221 to retrieve the address of a media source with which the end-system has a stored logical association. The end-system may then use that address to access that media source over a network. In another mode of operation the end-system may request the server 221 to establish a media link with a media source, and in response the server 221 may determine the address of a media source for which the end-system has a stored logical association and may then set up a media link using that address. Channel 223 may be a control channel and / or a media channel. The endpoint 203a may also be connectable to the centralised service 221 via channel 224. Channel 224 may be a control channel and / or a media channel. The one or more processors of the video end-system may therefore be configured to establish a control channel between the video end-system and the auxiliary device and a media channel between the video end-system and the auxiliary computing device.

[0085] Channels 223 and 224 are channels via which media and / or control signals can be sent from the device 220 to the video end-system, via the server 221. Channel 223 is a separate channel to the channel 207 on which the media stream for the video conference is sent to the other endpoint 203b via the network 206. Thus, the channel on which media and / or control signals are sent between the video end-system and the media source at the auxiliary device 220 is a separate channel to that used to send media to the other endpoint 203b during the video conference.

[0086] There may be a secure connection between the video end-system and the centralised service 221 . There may be a secure connection between the computing device 220 and the centralised service 221. A secure connection between the video end-system and the centralised management service 221 may be established via channel 223. A secure connection between auxiliary device 220 and centralised management service 221 may be established via channel 223. The communication channel 223 between the auxiliary device and the centralised management system and the communication channel 224 between the video end-system and the centralised management system can enable the two devices to securely exchange parameters such as identity, network address and information regarding capabilities. They may also be used to exchange session keys for secure direct communications.

[0087] Within the centralised service 221 , there is configured an association between the auxiliary device 220 and the endpoint 203a. By associating the auxiliary device with a centralised service and creating a logical association between the auxiliary device and the video end-system in the centralised service, simplified access to auxiliary media streams for use within a video conference may be achieved. This association between the devices can allow the centralised management system to record details of the auxiliary device, such as its location and authorization. This can allow media from the media source to be easily directed to the video end-system in dependence on the logical association. The logical association may allow the video end-system to be authorised to access to a media source at an auxiliary device without requiring procedures including device discovery, device trust, selection of the exact media source (such as screen or application window) of the sending device.

[0088] Furthermore, the centralised management system can allow persistent logical associations to be established between auxiliary devices, for example auxiliary devices that are collocated with video end-systems, such that the logical associations can be used to supply media output by the auxiliary device to the video conference across multiple different video conference calls. That is, the logical association formed at the video conferencing server between the computing device and the video end-system may persist across multiple video conferencing calls between the video end-system and one or more other video conferencing endpoints and may not be formed for each individual video conferencing session.

[0089] The logical association between the video end-system and the computing device 220 may also allow capabilities of the computing device 220 and the video end-system to be exchanged by recording details of the capabilities of each device at the centralised service.

[0090] During a video conference, the video end-system may also discover one or more new auxiliary devices that are within a communicable range of the video end-system, for example via Bluetooth or audio pairing, and establish a logical association at the video conferencing server between media sources at the one or more new devices and the video end-system. Existing logical associations may also be updated if one or more properties, such as location or authorization of the auxiliary device, change.

[0091] As mentioned above, the central service may store logical associations for multiple auxiliary devices, such as auxiliary devices that are collocated with video end-systems (for example, within the same room or building), so that devices which are commonly used do not need to be configured for each video conference call)

[0092] The list of auxiliary computing devices associated to a video end-system at the video conferencing server may be restricted during a particular time, for example so that devices which are out of range of the video end-system are restricted from actually being available for use as media sources. The proximity of an auxiliary computing device to the video end-system may be determined using techniques such as using Bluetooth or audio pairing. If an auxiliary device is determined to be out of range of the video end-system, this may be indicated at the centralised management service so that the device is not available for use to share its media.

[0093] As mentioned above, the one or more processors of the video end-system may be configured to implement a first software entity for supporting the video conference. The first software entity may be capable of receiving media input from any of multiple input channels. For example, the first software entity may receive media from the camera 204a. The one or more processors of the video end-system (for example, the main processor) may be capable of implementing a second software entity for receiving media from the media source (for example via server 206, via channels 223 and 224) and supplying it to the first software entity by emulating one of the input channels. For example, the video end-system may emulate a HDMI port to which, traditionally, a computing device such as a laptop may be connected to supply media for inclusion in the video conference. Put generally, one way media from the media source may be provided to a software entity supporting a video conference is by software emulation of an input to the software entity. This approach allows the media to be provided using the techniques described herein even if the software entity does not have an input port that is preconfigured to receive data from the media source. The input that is to be emulated may be supported by a logical port provided by an operating system of the end-system. In conventional operation a driver for that port may link the logical port with a pre-existing input such as a physical port, for example an HDMI or USB port. An alternative driver may be implemented. That driver may provide to the same logical port media from the media source. The software entity can then process data received at the logical port in the normal way, even though that data comes from the media source. In this mode of operation, the input from the media source may be termed a virtual port.

[0094] The one or more processors of the computing device 220 may be configured to implement a first software entity capable of outputting media from the media source on any of multiple output channels. The first software entity may be an application, such as a meeting collaboration application, such as a virtual whiteboard or presentation system. This application is configured to provide media that is output to the local physical devices of the computing device 220, such as screens or speakers, via any of the multiple output channels. The computing device may also be capable of implementing a second software entity for receiving media from the media source and supplying it to the video end-system by emulating one of the output channels. This may allow the media to be redirected to the video end-system.

[0095] The centralised service 221 may act so as to emulate media output devices on the auxiliary device to redirect output media to the video end-system and emulate media input devices on the video end-system to accept redirected media from the auxiliary device.

[0096] Put another way, the processor(s) and other hardware implementing a physical endpoint for a video end-system will typically be configured to run software that implements logical video conferencing endpoints for one or more video conferencing platforms. In order for that software to operate in normal use it takes inputs from physical receiving or recording entities that are connected to physical ports of the hardware. For example, it may receive audio from a microphone that is attached to an audio-supporting port of the hardware, or video from a video-supporting port of the hardware. Those may be physical ports such as HDMI or USB ports which are exposed at the exterior of the hardware to allow the receiving or recording devices to be connected to them. The video conferencing software may be provided by an operator of the videoconferencing platform, who may be independent from the manufacturer of the hardware. The video conferencing software may be configured to support a protocol specified by an operator of the video conferencing platform. Thus, the video conferencing software may be configured to receive audio and video data from physical ports of the video conferencing endpoint hardware. As described herein, the operation of the endpoint can be enhanced by further software running on the processor(s) of the hardware. That software may be an application, a driver or part of an operating system running on the hardware. That further software can emulate a physical port of the hardware, preferably a physical port that is or may be exposed at the exterior of the hardware - for example a USB or HDMI port. By emulating the port, the further software presents data to a software entity running on the hardware that is consuming data received by the port in such a way that the consuming software can consume that data as if it had been received at the physical port. The further entity can implement a logical interface whereby it can receive data from a source external to the hardware. For example, the further entity may receive that data over a wireless data link to a wireless transceiver comprised in the hardware. That may be a WiFi (IEEE 802.11 ) link or a Bluetooth link, for instance. A video source (which includes a still image source) and / or and audio source (which may be the same entity) can connect to that logical interface to provide the data to the further entity. That data may be video and / or audio data. In this way, a source of media (for example, video and / or audio) can provide data to the video conferencing software as if it were connected to the port whose data the video conferencing software can consume. The video conferencing software can behave as if the media source were connected to that port, when in fact the media source might not be connected to that port and instead be providing media data to the further software which is emulating the port from which the video conferencing software is configured to receive media data.

[0097] In the example shown in Figure 2a, the logical association between the video end-system 250a and the auxiliary device 220 is established at the centralised service 221 , which in this example is at the cloud, that is to say it is implemented by one or more servers at a location remote from the video end-system and the auxiliary device and that is / are in communication with the video end-system and the auxiliary device over the internet. However, the system may also operate without communicating with a video conference server and / or without a cloud. For example, the logical association between the video end-system and the auxiliary device may also be established via other techniques, such as via Bluetooth, acoustic pairing, proximity pairing or user interaction.

[0098] Figure 2b shows an example of connections and / or signals sent between various entities in the system 200 to insert media from the media source at auxiliary device 220 into a video conference between the video end-system 250a and another video end-system. For simplicity, only the endpoint 203a of the video end-system 250a is shown in Figure 2b. The processor at the endpoint 203a can implement multiple software entities. In this example, the processor implements a first software entity 241 for supporting the video conference (i.e. for sending media between endpoint 203a and the end point of another video conferencing system, such as endpoint 203b in Figure 2a) and a second software entity 242 that can emulate one of the input channels on which the first software entity

[0099] 241 is capable of receiving media input, as described above. The second software entity

[0100] 242 receives media from the media source at auxiliary device 220 and supplies it to the first software 241 entity by emulating one of the input channels. The second software entity may operate independently of the first software entity. The first and / or second software entities may be independent from the manufacturer of the hardware.

[0101] In this example, the logical association between the media source and the video endsystem is established by communicating with management service 221 at the cloud over the internet. The two-way link shown at 251 between the second software entity 242 and the media source at the auxiliary device 220 establishes an authenticated connection between the endpoint 203a and the media source. Using the signal shown at 252, the second software entity 242 requests a sharing session ID from the management service 221 . Using the signal shown at 253, the management service 221 transmits the sharing session ID to the second software entity 242. Using the signal shown at 254, the second software entity 242 transmits the sharing session ID to the auxiliary device 220. The two- way link shown at 255 establishes a connection between the media source at auxiliary device 220 and the management service 221 using the sharing session ID, Using the signal shown at 256, the media source at auxiliary device 220 sends media to the second software entity 242 via the management service 221 using the sharing session ID. Using the signal shown at 257, the second software entity 242 sends the media to the first software entity 241 for supporting the video conference. The first software entity 241 can then include the media in a video conference established with another video end-system.

[0102] In the example shown in Figure 2b, the media transits the cloud and is routed to the endpoint 203a for insertion into the video conference. However, in other implementations, the media may not transit the cloud and may be sent from the media source to the endpoint of the video end-system via a direct channel. For example, the endpoint 203a and the auxiliary device 220 may be directly connected via a wireless or a wired connection and the media for insertion to the video conference may be sent from the media source at auxiliary device 220 to the endpoint 203a of the video end-system via this direct connection.

[0103] Media output by the media source can be routed to be output at video conferencing hardware of the video end-system and inserted into a video conference established with another video conferencing endpoint in order to conveniently share the media output by the media source with remote parties. The one or more processors of the video end-system may be configured to receive a control signal from the control unit 202a or the auxiliary computing device 220 for enabling remote viewing of the media via the video conferencing hardware and / or the content stream of the video conference. For example, the one or more processors of the video end-system may receive control signals from the control interface device 202a, and enable or disable remote viewing of the media on one or more video conferencing hardware units and / or enable or disable remote viewing of the media for one or more channels of a video conference call based on the control signals. In this way, a user of the video end-system may control which components of the video conferencing hardware of the video end-system the media is output on and / or which remote participants of the video conference call can view the media (i.e. which endpoints the media is sent to).

[0104] Optionally, the video end-system may establish a user interface extension for controlling aspects of the auxiliary device. Optionally, the computing device may establish a user interface extension on the auxiliary device for controlling aspects of the video end-system.

[0105] The control devices 202a, 202b of each video end-system may each comprise a user interface. The auxiliary device 220 may also comprise a user interface. In some implementations, the user interfaces may each comprise a touch screen on a display screen. The user may interact with one or more user input devices on the user interface (for example, buttons displayed on the touch screen). The user interfaces for the video end-system and the computing device 220 may be configured to provide simplified access to the auxiliary media streams for use within a videoconference, as will be described in more detail below.

[0106] The user interface of the video end-system may comprise a control for requesting the media source at the auxiliary device to send its current media output(s) to said video endsystem for inclusion in the current video conference, or to cease sending the current media output(s) if the current media output(s) are currently being inserted into the video conference.

[0107] The user interface of the video end-system, which may be part of the controller 202a of the video end-system, may comprise a user input device. For example, where the user interface is in the form of a touchscreen, the user input device may comprise a button on the touchscreen. The user input device may alternatively comprise a button, switch or lever on a mechanical control panel. The one or more processors of the video endsystem may be configured to receive a signal in response to a user input at the user input device. In response to receiving the signal, the video end-system can receive the media from the media source for insertion into the video conference. The one or more processors of the video end-system may establish a media channel between the video end-system and the media source (for example, via the network 206) in response to receiving the signal.

[0108] In some implementations, there is included in the user interface of the auxiliary device a control for sending the current media output(s), such as audio and video, to the video end-system to be sent to the remote parties in the current videoconference, by means of a content stream, or to cease sending the current media output(s) to the video endsystem. The user interface of the computing device 220, may comprise a user input device. For example, where the user interface is in the form of a touchscreen, the user input device may comprise a button on the touchscreen. The user input device may alternatively comprise a button, switch or lever on a mechanical control panel. The one or more processors of the auxiliary computing device may be configured to receive a signal in response to a user input at the user input device. In response to receiving the signal, the auxiliary computing device can send the media from the media source to the video end-system for insertion into the video conference.

[0109] The user interfaces of the auxiliary device and / or the end-system may be updated as a result of the establishment of a connection between the video end-system and the device 220. This user interface update may be triggered either by negotiation between the auxiliary device and the end-system, or may come as an instruction from the centralised service 221 .

[0110] The updated user interface may allow the user to select the auxiliary device as a media source for the current video conferencing call, for example by pressing a button on the user interface. When the user inputs a request to the user input device, a media channel may be created between the auxiliary device 220 and the end-system 203a which allows media from the media source to be transported from the auxiliary device 220 into the content stream of the end-system 203a and sent to the other endpoint 203b via the video conference communication channel 207.

[0111] In another example, the input device (for example, a button) may be created as a response to detection of a virtual port on the video end-system. According to the usual media selection criteria of the user interface, the virtual port may be selected and the act of selecting the virtual port may result in the creation of the media stream from the auxiliary device to the video end-system. The user interface of the auxiliary computing device may advertise the virtual port as a means of sending the media from the computing device to the video end-system.

[0112] Figure 3 shows an example of the auxiliary device 220 in more detail. In this example, an application 532 is launched in an environment 530 running on the device 220. The application 532 may correspond to the first software entity implemented by the computing device discussed above.

[0113] The environment may include a control signals interface, that is to say programming configured to route control signals from the user interface of the auxiliary computing device and / or the video end-system to the application. The environment may include a media signals interface, that is programming configured to route media signals between the application and the video conferencing hardware of the video end / system and / or a content stream of a video conference.

[0114] The environment may comprise a data mapping interface which is configured to relay control signals from the user interface of the auxiliary computing device and / or the video end-system to the application, and relay media signals between the application and the video conferencing hardware and / or a content stream of a video conferencing call. The data mapping interface may be a virtual interface which defines a boundary to the application environment and act as a virtualisation layer by relaying data between the application inside the application environment and software and hardware which is external to the application environment.

[0115] The environment can present a virtualised or normalised operating system interface to the application 532 whilst interfacing to the physical devices (for example media output devices such as screens and speakers) provided by the device 220. This virtualisation layer comprises a virtualised media output layer 531 and a virtualised media input layer 533.

[0116] The user interface of the auxiliary computing device 220 is shown at 534. The user interface 534 may provide the interface as programmed in the application 532, such as touch-screen controls as required by the application. In addition, the user interface 534 may be augmented to control video routing, such as sending the video and audio output to the video end-system for inclusion in a video conference. The environment 530 is in communication with the cloud service 221 and can use this connection to determine user interface adaptations, either by negotiation with the cloud service, or by negotiation with the video end-system. In this example, the media output layer 531 provides video and audio routing and compression. When an indication is received by the video end-system (for example via the server 206) from the user interface 534 that the user desires to send media from the application into a video conference, a signal can be sent either via the cloud service 221 (via channels 223 and 224 in Figure 2a) or via a direct control channel to the video-conferencing system (such as 225 in Figure 2a, which may be a wireless connection) to establish a media channel and instruct the video end-system to incorporate that media channel as a content media source for the video-conference (in connection 207). In this way, the device can always be available to the video end-system to be used as a media source. The video conferencing server may be hard pre-configured with multiple potential devices before a video conferencing call. One or more of the devices can be activated to send media data to be incorporated into the video conference.

[0117] The application environment may be configured to transform media signals from the media source having a first format to media signals for the content stream of the video conference and / or the video conferencing hardware having a second format. In other words, the media source may be configured to output media signals having a first format, which may be transformed to media signals having a second format for the content stream of the video conferencing call and / or the video conferencing hardware. For example, the output media signals may include video signals and the application environment may be configured reformat video data to map screen sizes, video formats (e g. compressed vs uncompressed), numbers of pixels etc between formats compatible with the output device(s) of the auxiliary computing device and the video conferencing hardware and / or the content stream of the video conferencing call. In other examples, the media signals may include audio signals, images, or file data etc which are transformed from a first format compatible with the output device(s) of the auxiliary computing device to a second format.

[0118] Therefore, an audio and / or video output from the application can be provided by the application environment to be included in a content stream for a video call established between endpoints 203a and 203b in Figure 2a, enabling the video output of the application to be sent both to the local display 205a and to the remote endpoint 203b to be displayed on a remote display screen 205b and / or the audio output to be output at local and remote speakers.

[0119] In one implementation, the media output ports, such as video and audio, of the auxiliary device may be duplicated onto corresponding virtual devices. By this means, the application running on the auxiliary device that is providing the media streams is unaware of the presence of the media integration with the video end-system.

[0120] In another implementation, the application running on the auxiliary device may be wrapped in an environment that emulates media input and output services to the application whilst simultaneously reformatting the media, such as by scaling or cropping, or completely replacing the user interface, to make the media suitable for routing to local physical devices and to create media streams for use by the video end-system. The environment may be configured to route control signals from the user interface of the computing device and / or the video end-system to the application, and route media signals between the application and the video end-system.

[0121] As mentioned above, in some implementations, the auxiliary device may process the media output by the media source to provide a compressed representation of the media to send to the video end-system. The virtual devices or wrapping environment (i.e. the application environment) mentioned above may utilize media compression algorithms, such as H.265 and Advanced Audio Coding (AAC), to compress the media output from the application to give more suitable streams for transmission to the video end-system. In some implementations, there may be provided one or more virtual media source devices on the video end-system. These virtual media source devices emulate the media source at the auxiliary device. The media from the media source at the auxiliary computing device can be presented via the virtual media source devices to the video endsystem firmware. The virtual media sources may also optionally decompress the media. By this means, the video end-system firmware, which typically will already understand raw media streams being introduced from physical media ports such as HDMI, requires minimal adaption to take the media streams from a media source at a separate device and inject them into a video conference in a similar fashion to that of a physically provided media stream (for example, a media source connected to the video end-system via a HDMI port on the system).

[0122] This may be implemented as an additional media sink component incorporated into an existing media presentation system of the auxiliary device, and / or an additional media source device incorporated into the video end-system. The source and sink may be associated via the logical association established at the centralised management service. The media can be sent from the sink to the source in dependence on the logical association.

[0123] Put another way, in a conventional mode of operation a video end-system may receive media from devices that are connected to it, typically though dedicated physical ports. Those devices may, for example, be cameras, microphones or digital whiteboards. The endpoint may be capable of supporting a video conference with a remote endpoint supported by a conference server 222 or an alternative connection between the endpoints. The conference server 222 may perform tasks such as authenticating the endpoints or their users, forwarding video conference traffic between the endpoints and buffering video conference traffic in case of retransmission. As an alternative way to provide media to one of the endpoints, the media may be provided over a data connection (e.g. a packet-based connection) with a media provider. The central management server 221 can assist in linking the media provider to the endpoint. That endpoint can then add media from the media provider to the video conference traffic and send that traffic to the server for forwarding to another end-point in the video conference. The central management server could be implemented as a single physical processing unit or could be a logical function that is distributed between multiple physical processing units. Figures 4 to 6 show video conferencing systems according to further implementations in which an auxiliary computing device is configured to operate a third-party application. Each of the features described herein in relation to Figures 4 to 6 may be combined with any or all of the features described above in relation to Figures 2 and 3.

[0124] Figure 4 shows a diagram of some components of a video conferencing system. A video end-system shown at 300 is communicatively connectable with an auxiliary computing device 350 which is configured to operate a third-party application. The third-party application provides a media source at the auxiliary computing device. In this example, the video end-system 300 comprises a video endpoint device 310, connected video conferencing hardware 330 and a control interface device 320. The endpoint device 310 is connected to the internet 360 for exchanging a content stream of a video conferencing call with remote parties. The endpoint device 310 is configured to send and receive media signals to and from the video conferencing hardware, which may form part of the content stream of the video conferencing call. In Figure 4 the video conferencing hardware 330 includes a camera 3332 and a display screen 3334 for recording and displaying video footage. The endpoint device 310 is also configured to receive control signals from the control interface device 320 for controlling the video end-system 300.

[0125] In this example, the auxiliary computing device 350 is configured to run a third-party application 354 which is hosted inside an application environment 352 provided on the auxiliary computing device 350. Additionally, the application environment 352 on the auxiliary computing device 350 is connected to a management agent service 370 via the internet 360. The management agent service 370 contains information regarding the identities and permitted behaviour of each auxiliary computing device that is communicatively connectable with the video end-system 300. As described above, logical associations between the video end-system and one or more auxiliary computing devices can be established at the service 370. Examples of permitted behaviour may include a list of authorised third-party applications that may be run on the auxiliary computing device 350, details regarding the manner in which each application 354 may be run such as whether user authorisation from a user is required, whether the application 354 may be shared with any or specific remote parties, and / or whether the application 354 must be restricted in terms of device or network access. The management agent service 370 is configured to send configuration management controls to the application environment 352.

[0126] Figure 5 shows another example of a video conferencing-system. In Figure 5 the video conferencing hardware 330 of the video end-system 300 additionally includes a speaker 3336 and a microphone 3338 for playing and recording audio data for the content stream of a video call. Additionally, the video end-system 300 comprises a network interface 358 for connecting to the internet 360. As shown in Figure 5, the application environment 352 comprises an environment interface 356 for relaying signals between the third-party application 354 and other components of the auxiliary computing device 350. The environment interface 356 (which may also be referred to as a data mapping interface 356) is configured to route control signals from a control interface device 325 of the auxiliary computing device to the third-party application 354. The environment interface 356 is also configured to route media signals 385, 386, where 385 is an audio signal and 386 is a video signal, between the third-party application 354 and the video endpoint 310 via the server 370, so that the media can be output to one or more components of the video conferencing hardware 330 and / or the content stream of a video conferencing call via the network interface 380. The audio stream 385 and the video stream 386 received from the third-party application 352 via the data mapping interface 316for inclusion in the content stream of a video call are relayed to the speaker 3336 and the display screen 3334 respectively. Likewise an audio stream 348 received from the microphone 3338 and a video stream 347 received from the camera 3332 are relayed to the endpoint 310 to form part of the content stream of a video call that is sent to another endpoint via network interface 358 and the internet 360.

[0127] In use, a user may request installation of the third-party application 354 (e.g. an Android application) on the auxiliary computing device 350 via the user interface of the control interface device 325. The control interface device 325 then instructs the auxiliary computing device to create an application environment 352 for the third-party application 354 to run on and then downloads the third-party application 354 into the application environment 352 on the auxiliary computing device 350.

[0128] The third-party application 354 may have pre-requisite requirements for running on a typical device. Creating the application environment 352 may include allocating CPU and memory resources as well as virtual devices and / or interfaces such as a display, keyboard and touch-screen for the application 354. When the third-party application 354 is launched into the created application environment 352, the prerequisite requirements of the third-party application 354 are provided such that the application environment 352 emulates a typical device such as a conventional mass-market tablet or phone from the point of view of the third-party application 354.

[0129] The user interface of the auxiliary computing device may then be used to connect to the third-party application 352 in a variety of ways. In one example, a virtual video stream may be relayed across the connection between the application environment 352 and the control interface device 325 such that an application video output of the third-party application 354 may be displayed on the control interface device 325. The video view may be displayed full-screen if the aspect ratio and resolution of a screen provided on the control interface device 325 is similar to the requirements of the third-party application. In other examples, a user interface environment on the control interface device 325 may be configured to display the application video output in a window on a screen of the control interface device 325. In some examples, a user of the control interface device 325 may make use of zoom gestures such as two-finger pinch to zoom the video. This example is useful for setting up a third-party application 354, e.g. by entering user credentials, where the information should not be shared with the other attendees in a meeting room. In another example, a video output from the third-party application 354 may be connected to or redirected to the display screen 3334 or to the content stream of a video conferencing call. If the display screen 3334 includes a touch-screen, the touch-screen gestures may be relayed back to the third-party application 354.

[0130] In some examples, when the auxiliary computing device 350 includes a graphics processing unit (GPU), and / or a screen, and / or a touch screen on a same unit, the third- party application 354 hosted on the device 350 may be given direct access to the GPU, the display and / or the touch-screen (i.e. , without a virtualisation layer). This can be useful for improving the efficiency and performance of the third-party application 354.

[0131] The data mapping interface 356 may be configured to transform control signals from the control interface device 325 having a first format to application control signals for controlling the third-party application 354 having a second format. For example, the control interface device 325 may include a physical touch-pad or a virtual touch-pad e ulated by a touch-screen of the control interface device 325. Touch gestures from the touch-pad of the virtual touch-pad may be relayed over the connection between the control interface device 325 and the third-party application 354 via the data mapping interface 356 as touch gestures. In this example, the application environment 352 may be configured to correctly map the gesture positions between the touchpad or virtual touchpad of the control interface device 325 and a target virtual display of the third-party application 354 which has a different size, taking into account any active crop, zoom, screen size differences and aspect ratios between the touch screen of the control interface device 325 and the target virtual display of the third-party application 354.

[0132] In another example, the auxiliary computing device 350 may be programmed with customised instructions associated with a specific third-party application 354 for forming the application environment and / or the user interface environment. The customised instructions may be configured to provide an enhanced user interface on the control interface device 325 to replace a conventional user interface of that third-party application 354. In this way a user interface that is more appropriate for the hardware of the auxiliary computing device 350 may be provided, in place of the user interface of the third-party application which may be better suited to different hardware (e.g. a smart phone).

[0133] For example, the enhanced user interface provided on the auxiliary computing device may include application specific function buttons, a virtual keyboard, a dial pad, a directory lookup, or navigation elements which are configured for controlling the third- party application 354. The application environment 352 may be configured to convert control signals from the enhanced user interface into control signals which emulate gestures and key-presses expected by the third-party application 354.

[0134] The customised instructions for a particular third-party application may be provided by a central server, (e.g., on the internet 360) which may be downloaded by the auxiliary computing device. The customised instructions may include a combination of structured data, code, and / or portable code. In some examples, the customised instructions may comprise an application programming interface (API) provided by the third-party application. In some examples, a list of authorised third-party applications that are permitted to be operated by the auxiliary computing device may be centrally managed by a management server or cloud service, such as service 221 or 370 herein. The auxiliary computing device may connect to the management server or cloud service to retrieve the list of authorised applications. If a desired third-party application is included in the list of authorised applications, then the auxiliary computing device may download the third-party application from a publicly accessible repository (for example from the Google Play Store) or a from private repository such as a private server or from the management server itself or from the cloud service. This is useful for reducing the likelihood of malicious third-party applications being installed on the auxiliary computing device. Additionally or alternatively, a management interface may be provided to enable administrators to enable or disable authorisation for specific third-party applications. In some examples, a user login or video end-system device identifier may be required to enable or disable the download of a third-party application.

[0135] In another example, the management server or the management interface may be configured to permit guest applications that are not included in the list of authorised applications to be operated by the auxiliary computing device. This may be permitted based on a user login or based on a video end-system device identifier. In further examples, the management server or cloud service may be configured monitor the usage and behaviour of the third-party applications. In another example, the application environment containing the third-party application can be provided on a control interface device of the auxiliary computing device, which may also host a user interface environment for a user interface application.

[0136] Figure 6 shows a diagram of a video end-system 500 that is communicatively connected to an auxiliary computing device 550 according to another example. The video endsystem 500 comprises an endpoint 510, a control interface device 520, a camera 5332 and a display screen 5334. In this example, the application environment 512 containing the third-party application 514 is provided on a cloud service 580 with which the auxiliary computing device 550 is communicatively connected. A video stream from the third-party application 514 in the application environment 512 is routed from the cloud service 580 via the internet 560 to the auxiliary computing device 550. The video stream from the application can then be routed to the video endpoint device 510 via the service 577 which stores the logical association between the auxiliary computing device 550 and the video end-system 500 and on to the display screen 5334 and can be included in the media stream to another endpoint, as described above.

[0137] Figure 7 shows the steps of an exemplary computer-implemented method for conducting a video conference at a video end-system configured to act as an endpoint in a video conference. At step 701 , the method comprises establishing a logical association with a media source (which may be at an auxiliary computing device) communicatively connectable with the video end-system. At step 702, the method comprises receiving media from the media source, wherein the media is directed to the video end-system in dependence on the logical association. At step 703, the method comprises inserting the media into a video conference established with another video conferencing endpoint (which may be done by communicating with the video conferencing server). This method can advantageously allow media from a media source at an auxiliary computing device to be inserted into a video conference in dependence on a logical association established between the media source and the video end-system.

[0138] Figure 8 shows the steps of an exemplary computer-implemented method for implementation at a computing device configured to communicate with a video endsystem serving as an endpoint in a video conference. At step 801 , the method comprises establishing a logical association between a media source (which may be at or associated with the computing device) and the video end-system. At step 802, the method comprises causing transmission of media from the media source to the video end-system to be inserted into a video conference established between the video end-system and another video conferencing endpoint (which may be done by communicating with the video conferencing server), wherein the media is directed to the video end-system in dependence on the logical association.

[0139] The device 901 shown in Figure 9 may be configured to operate as a processing element of the video end-system or the auxiliary computing device described herein. The device 901 comprises a processor 902, a memory 903 and a transceiver 904. The processor may be implemented as dedicated hardware in the device, such as a processing chip. The memory is arranged to communicate with the respective processor. Memory may be a non-volatile memory. Each device may comprise more than one processor and more than one memory. The memory may store data that is executable by the processor. By executing program code contained in such data, the one or more processors may perform functions as described herein. The memory may store such program code in a non- transitory manner. The processor may be configured to operate in accordance with a computer program stored in non-transitory form on a machine readable storage medium. The computer program may store instructions for causing the processor to perform its methods in the manner described herein. The device 901 also comprises a transceiver 904 for receiving and / or sending data from and / or to one or more of the other entities, such as the centralised service 221 , communication network 206, or between video conferencing endpoints via channel 207 in Figure 2a. Each device also comprises a power source, such as a battery, or is connectable to mains power.

[0140] The approach described herein can enable media integration between a set of video endsystems and other networked auxiliary media source devices using a centralised management system at a video conferencing server, or in other ways. The centralised management system can allow persistent logical associations to be established between auxiliary devices that are collocated with video end-systems, such that the logical associations can be used to supply media to the video conference across multiple different video conference calls. This method is compatible with standard video conferencing software and hardware and existing meeting room presentation systems, such as an interactive whiteboard, and thus allows to easily integrate new capability within pre-existing video conferencing hardware and software.

[0141] The applicant hereby discloses in isolation each individual feature described herein and any combination of two or more such features, to the extent that such features or combinations are capable of being carried out based on the present specification as a whole in the light of the common general knowledge of a person skilled in the art, irrespective of whether such features or combinations of features solve any problems disclosed herein. The applicant indicates that aspects of the present invention may consist of any such individual feature or combination of features. In view of the foregoing description, it will be evident to a person skilled in the art that various modifications may be made within the scope of the invention.

Claims

CLAIMS1 . A video end-system for serving as an endpoint in a video conference, the video endsystem comprising one or more processors configured to conduct a video conference by performing the following steps: establishing a logical association with a media source communicatively connectable with the video end-system; receiving media from the media source, wherein the media is directed to the video end-system in dependence on the logical association; and inserting the media into a video conference established with another video conferencing endpoint; wherein the one or more processors of the video end-system are configured to implement a first software entity for supporting the video conference, the first software entity being capable of receiving media input from any of multiple input channels, and wherein the video end-system is capable of implementing a second software entity for receiving media from the media source and supplying it to the first software entity by emulating one of the input channels.

2. The video end-system as claimed in any preceding claim, wherein the one or more processors are configured to establish a control channel between the video end-system and the media source.

3. The video end-system as claimed in any preceding claim, wherein the video endsystem comprises a user interface for controlling aspects of the video end-system and / or the media source.

4. The video end-system as claimed in claim 3, wherein the user interface comprises a user input device, wherein in the one or more processors are configured to receive a signal in response to a user input at the user input device and in response to receiving the signal, receive the media from the media source for insertion into the video conference.

5. The video end-system as claimed in any preceding claim, wherein the video endsystem comprises video conferencing hardware, wherein the one or more processors arefurther configured to cause the media received from the media source to be output at the video conferencing hardware.

6. The video end-system as claimed in any preceding claim, wherein the one or more processors are configured to establish the logical association with the media source by communicating with a video conferencing server.

7. The video end-system as claimed in claim 6, wherein the video conferencing server is a cloud server or a local area network.

8. The video end-system as claimed in any preceding claim, wherein the one or more processors are configured to receive the media over a data link that supports the transfer of media data from the media source to the video end-system only.

9. The video end-system as claimed in any preceding claim, wherein the video endsystem is configured to establish logical associations with multiple media sources, and wherein the video end-system is configured to receive media from one of the multiple media sources to insert into the video conference in response to a user selection of the one of the multiple media sources and in dependence on a respective one of the logical associations corresponding to the one of the multiple media sources.

10. An auxiliary computing device configured to communicate with a video end-system serving as an endpoint in a video conference, the computing device comprising one or more processors configured to perform the following steps: establishing a logical association between a media source and the video endsystem; and causing transmission of media from the media source to the video end-system to be inserted into a video conference established between the video end-system and another video conferencing endpoint, wherein the media is directed to the video endsystem in dependence on the logical association; wherein the one or more processors of the computing device are configured to implement a first software entity capable of outputting media from the media source on any of multiple output channels, and wherein the computing device is capable of implementinga second software entity for receiving media from the media source and supplying it to the video end-system by emulating one of the output channels.

11. The computing device as claimed in claim 10, wherein the one or more processors are configured to process media data output by the media source to form a compressed representation of the media data to supply to the video end-system for insertion into the video conference.

12. The computing device as claimed in claim 10 or claim 11 , wherein the one or more processors are configured to establish a control channel between the media source and the video end-system.

13. The computing device as claimed in any of claims 10 to 12, wherein the one or more processors are configured to establish the logical association with the video end-system by communicating with a video conferencing server.

14. The computing device as claimed in any of claims 10 to 13, wherein the computing device comprises a user interface for controlling aspects of the media source and / or the video end-system.

15. The computing device as claimed in claim 14, wherein the one or more processors are configured to update the user interface when a connection between the media source and the video end-system is established.

16. The computing device as claimed in claim 14 or claim 15, wherein the user interface comprises a user input device, wherein in the one or more processors are configured to receive a signal in response to a user input at the user input device and in response to receiving the signal, cause the transmission of the media to the video end-system for insertion into the video conference.

17. The computing device as claimed in claim 16, wherein the user interface is configured to advertise the user input device as a means of sending media from the media source to the video end-system.

18. The computing device as claimed in claim 16 or claim 17, wherein in response to receiving the signal, the one or more processors are configured to establish a media channel between the media source and the video end-system.

19. The video end-system as claimed in any of claims 1 to 9 or the computing device as claimed in any of claims 10 to 18, wherein the media source is an interactive whiteboard, a mobile phone, a tablet or a laptop computer.

20. The video end-system as claimed in any of claims 1 to 9 or the computing device as claimed in any of claims 10 to 18, wherein the media source is configured to output one or more of the following media types: audio, video, image files, computer files and text.21 . The video end-system as claimed in any of claims 1 to 9 or the computing device as claimed in any of claims 10 to 18, wherein the logical association is a persistent logical association for use in multiple video conferencing sessions.

22. A computer-implemented method for conducting a video conference at a video endsystem configured to act as an endpoint in a video conference, the method comprising the following steps: establishing a logical association with a media source communicatively connectable with the video end-system; implementing a first software entity for supporting a video conference, the first software entity being capable of receiving media input from any of multiple input channels; implementing a second software entity for receiving media from the media source and supplying it to the first software entity by emulating one of the input channels; receiving media from the media source, wherein the media is directed to the video end-system in dependence on the logical association; and inserting the media into a video conference established with another video conferencing endpoint.

23. A computer-implemented method for implementation at an auxiliary computing device configured to communicate with a video end-system serving as an endpoint in a video conference, the method comprising the following steps:establishing a logical association between a media source and the video endsystem; implementing a first software entity capable of outputting media from the media source on any of multiple output channels; implementing a second software entity for receiving media from the media source and supplying it to the video end-system by emulating one of the output channels; and causing transmission of media from the media source to the video end-system to be inserted into a video conference established between the video end-system and another video conferencing endpoint, wherein the media is directed to the video endsystem in dependence on the logical association.

24. A computer program comprising instructions that when executed by a computer cause the computer to perform the method of claim 22 or claim 23.

25. A computer-readable storage medium having stored thereon computer readable instructions that when executed by a computer cause the computer to perform the method of claim 22 or claim 23.

Images