Method and system for making functional devices available to meeting participant

Abstract

To provide a method, a device, a system, a program, and a storage device that allow a meeting participant to use a functional device.SOLUTION: In a computer system, a parent unit 100 includes a transmitter and a user laptop 160 includes a memory, a display, and an operating system. A peripheral device 90 is coupled to a user laptop via a generic communication protocol and includes a receiver 130 and at least one fixed or configurable endpoint of a functional device (video cam 91, speakerphone 92) exposed on the peripheral device. The parent unit 100 and the peripheral device each transmit and receive data through a communication network from the functional device to the user laptop via at least one fixed or configurable endpoint using the generic communication protocol for communication between the user laptop and the peripheral device.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present invention relates to a method, device and system for making functional devices available to meeting participants, and software for carrying out such a method. [Background technology]

[0002] background Enabling a group of people to communicate with each other for a specific purpose is one of the fundamental aspects of collaboration, problem solving, negotiation, teaching, education, etc. To support communication, there has been a proliferation of electronic communication tools, such as synchronous and asynchronous conferencing, online chat, instant messaging, audio conferencing, video conferencing, data conferencing, application sharing, remote desktop sharing, electronic meeting systems, collaboration (coordination) tools, project management systems, knowledge management systems, and social software systems.

[0003] One typical format is a lecture or seminar, which often involves a presentation using presentation software. Often, a traditional one-person presentation or lecture has been delivered to an audience that is in a rather passive mode as far as determining, assembling, enhancing or modifying the information to be presented is concerned.

[0004] Like other business processes, meetings are moving towards digitalization. People are increasingly using computer technology alone and in combination with broadband networks to support meeting objectives before or during the actual meeting. For example, distributing files using email allows people to read them before the meeting.

[0005] However, certain features such as microphones and speakers tend to be of poor quality when provided by portable devices such as laptops and mobile phones. The larger the meeting, the greater the need to ensure that high quality audio and visual signals are available to meeting participants. Summary of the Invention [Means for solving the problem]

[0006] Summary of the Invention SUMMARY OF THE PRESENT EMBODIMENTS Embodiments of the present invention relate to methods, devices and systems for making functional devices available to meeting participants, and software for carrying out such methods.

[0007] An embodiment of the present invention provides a system for connecting a processing device to a functional device connected to or within a parent unit of a communication network, said processing device having a memory, a display and an operating system, said system comprising: a first peripheral device configured to be coupled to said processing device via a generic communication protocol, said parent having a transmitter, said first peripheral device having a receiver and at least one fixed or configurable endpoint of said functional device exposed on said first peripheral device; The parent device and the first peripheral device each include the processing device and the first and configured to transmit and receive data through the communication network from the functional device to the processing device via the at least one fixed or configurable endpoint using the generic communication protocol for communication between peripheral devices.

[0008] Said communication network is preferably a wireless network, which allows for "sharing or combining" at least a selection of functions / endpoints of the second peripheral device, i.e. a one-to-many scenario.

[0009] The data being transmitted or received, ie streaming data carrying images, audio, etc., may be in a raw or unaltered format.

[0010] There may be more than one first peripheral device, for example at least two first peripheral devices may be used to transmit data from the functional device to at least two processing devices.

[0011] A functional device may operate solely with a vendor-specific driver. The functional device may be a data generating device such as one or more of a microphone, a speakerphone, a speaker, a display, a touch screen, a projector, a camera, a video camera, a webcam, etc.

[0012] The at least one fixed or configurable endpoint of the functional device exposed on the first peripheral device may be a human interface device.

[0013] The processing device may be configured to communicate with the at least one fixed or configurable endpoint with a pre-installed generic driver that is a human interface device driver.

[0014] The at least one fixed or configurable endpoint of the functional device exposed on the first peripheral device may be a mass storage device.

[0015] The processing device may be configured to communicate with the at least one fixed or configurable endpoint with a pre-installed generic driver that is a mass storage device driver.

[0016] The at least one fixed or configurable endpoint of the functional device exposed on the first peripheral device may be a composite device.

[0017] The processing device may be configured to communicate with the at least one fixed or configurable endpoint with a pre-installed generic driver that is a composite device driver.

[0018] Embodiments of the invention include combining endpoints, and the invention includes a system in which two endpoints are combined and exposed to the processing device as one endpoint.

[0019] The "exposure" or "enumeration" feature is (e.g. a button) decides which endpoint it sees. For "exposure", this can be achieved with a "descriptor field". When a device is able to expose a peripheral or other device, it means that the particular device configures one or more endpoints with particular descriptor or "descriptive fields." A point is defined by a number of descriptive fields.

[0020] The basic form of the descriptive field is "USB over IP" and the actual transfer of unaltered data can be accomplished by tunneling.

[0021] Exposure may be realized in a "descriptor field". An endpoint is a capability / function of a "physical device" (second peripheral, or functional device), for example if a phone is a second peripheral, it may have several endpoints, e.g. a microphone or a loudspeaker or an IP connection, etc.

[0022] The "distinction" between an endpoint and a second peripheral / functional device defines the endpoint.

[0023] The "embodiments of the present invention" "provide" a system that allows the use of original vendor drivers for each functional device (and associated endpoints), for example in the context of mirroring, such that the functional device can be operated only with vendor-specific drivers.

[0024] The system may include means for encoding, and optionally encrypting, the data. Thus, the system may include an encoder for encoding, and optionally encrypting, the data.

[0025] The processing device may be configured to host unified communications between two or more processing devices. The unified communications may include Skype® calls or Skype® The first peripheral device may be configured to provide a functional device for unified communications between two or more processing devices. For example, the functional device may be a speakerphone, a microphone, a speaker, a video camera, a webcam, a camera, or other audio source.

[0026] The system may be configured to share data from other processing devices in the meeting on the call, which may be enabled through a local view window by the sharing person or host using a manual action.

[0027] The system may be configured to select the camera, for example, via a manual action in the UC client.The system may be configured to synchronize a sharing state of the first peripheral device with a sharing state of the UC client.

[0028] Sharing of a function / endpoint may be sharing of a room display. Combining resources may be using two room loudspeakers (e.g. to improve quality), however in this case the user can only see "one room loudspeaker". In the latter case, the combining feature is also included if there are more than two participants / users in the room. Sharing can also be considered as selection of a function / endpoint of a particular functional device.

[0029] Embodiments of the present invention may use multiple endpoints, at least one of which may be a fixed or configurable endpoint, and sharing may be achieved across multiple processing devices. This is for the VR feature, i.e. the use of "multiple first peripherals" (e.g. buttons) is also supported.

[0030] The system may be configured to provide a multi-user wireless speakerphone, where a first peripheral device may be connected to each of the processing devices such that the speakerphone is accessible to all processing devices in a meeting without any coordination or action being taken to access the device.

[0031] The system may be configured to provide separate controls, for example to provide controls for the peripheral devices to control the volume of the parent unit's audio output and to mute the room microphone, such that muting one microphone also mutes it for all other peripheral devices connected to the processing device in the meeting.

[0032] The audio signal must be picked up by a microphone or may be configured such that the parent unit must inject an audio signal in any audio feed, such as a microphone signal, into the first peripheral device connected to the processing device.

[0033] The system may be configured to expose to the user the same types of functional devices that are connected to the parent device and to allow the user to use drivers provided by the vendors of the functional devices installed on the processing device.

[0034] The system may be configured such that when a plurality of processing devices communicate with a parent unit, a functional device connected to the parent unit is exposed to all of the plurality of peripheral devices. The functional device may be exposed originally.

[0035] The exposure may be achieved in a descriptor field. The system may be configured to enable any processing device connected to the communications network through a first peripheral device to view any audiovisual content displayed, provided or projected on that processing device in the meeting room, referred to as the "local view."

[0036] The processing device that is the receiver of the local view may have, but is not limited to, the following capabilities: * Ability to open a window and view meeting room content in it. * Ability to zoom in on content. o Local view allows users to participate in a blackboard or annotation session from their own processing device.

[0037] The system may be configured to provide alternative methods and systems if there are too few first peripheral devices in a particular meeting room.

[0038] The system may be configured to provide a warning if the first peripheral device is attempting to connect to an incorrect parent unit.

[0039] In another aspect, the present invention provides a method for connecting a processing device to a functional device connected to or in a parent unit of a communication network, the processing device having a memory, a display and an operating system, the parent unit having a transmitter and the first peripheral device having a receiver, the method comprising: coupling a first peripheral device to the processing device via a generic communication protocol; providing at least one fixed or configurable endpoint of said functional device exposed on said first peripheral device; and transmitting data from the parent unit through the communications network from the functional device to the processing device via the at least one fixed or configurable endpoint using the generic communications protocol for communication between the processing device and the first peripheral device, and receiving the data at the first peripheral device.

[0040] The communication network is preferably a wireless network, which allows for sharing or combining at least a selection of the functions / endpoints of the second peripheral device, i.e. a one-to-many scenario.

[0041] The data transmitted or received, ie streaming data carrying images, audio, etc., may be in raw or unaltered format.

[0042] There may be more than one first peripheral device. At least two first peripheral devices can be used to transmit data from the functional device to at least two processing devices.

[0043] A functional device may operate solely with a vendor-specific driver. The functional device may provide any one or more of the following: a microphone, a speakerphone, a speaker, a display, a touch screen, a projector, a camera, a video camera, a webcam.

[0044] The method may include providing the at least one fixed or configurable endpoint of the functional device exposed on the first peripheral device as one of a human interface device, a mass storage device, a composite device, a microphone, a speakerphone, a speaker, a display, a touch screen, a camera, a video camera, or a webcam.

[0045] The data may be encoded and / or optionally encrypted. Unified communications between two or more other processing devices may be hosted on the processing device.

[0046] The first peripheral device may provide a functional unit for said unified communication between two or more processing devices.

[0047] The method may include exposing to the processing device the same type of functional device as that connected to the parent unit, and using at least one driver for the functional device that is installed on the processing device.

[0048] The exposure may be achieved with a "descriptor field." In another aspect, a peripheral device configured to be coupled to a processing device via a generic communication protocol is provided, the peripheral device having a receiver and at least one fixed or configurable endpoint of a functional device exposed on a first peripheral device; The receiver of the first peripheral device is configured to receive data from the functional device over a communications network via the at least one fixed or configurable endpoint and transmit data to the processing device using the generic communications protocol for communication between the processing device and the peripheral device.

[0049] Said communication network is preferably a wireless network, which allows for "sharing or combining" at least a selection of functions / endpoints of the second peripheral device, i.e. a one-to-many scenario.

[0050] The at least one fixed or configurable endpoint of the functional device exposed on the first peripheral device may be one of a human interface device, a mass storage device, a composite device, a microphone, a speakerphone, a speaker, a display, a touch screen, a projector, a camera, a video camera, or a webcam. The exposure may be achieved with a "descriptor field."

[0051] A functional device may operate solely with a vendor-specific driver. In another aspect, there is provided a computer program product for performing any of the steps of the method of claim 10 when executed on a processor. The computer program may be stored by using a non-transitory signal storage means. The non-transitory signal storage means may be an optical disk such as a CD-ROM or DVD-ROM, a magnetic disk such as a hard disk, a solid state memory such as a flash memory, a magnetic tape or the like. [Brief description of the drawings]

[0052] [Figure 1A] FIG. 1 illustrates an embodiment of the present invention. [Figure 1B] FIG. 1 illustrates an embodiment of the present invention. [Figure 1C] FIG. 1 illustrates an embodiment of the present invention. [Diagram 2] FIG. 2 illustrates a message flow diagram of how a peripheral device is connected to a processing device in an embodiment of the present invention. [Diagram 3] FIG. 2 illustrates a message flow diagram of how a peripheral device is connected to a processing device in an embodiment of the present invention. [Figure 4] FIG. 11 shows an arrangement of components that can be used in embodiments of the present invention. This drawing is FIG. 11 of WO2013 / 037980, entitled "Electronic tools and methods with audio for meetings," which application is incorporated by reference herein with respect to FIG. 11 and in its entirety. [Diagram 5] FIG. 2 illustrates a configuration of processing devices for use in an ongoing unified communications meeting according to one embodiment of the present invention. [Figure 6] FIG. 1 illustrates an embodiment of the present invention in which a peripheral device is coupled to a processing device. [Figure 7] FIG. 1 illustrates an embodiment of the present invention in which a peripheral device is coupled to a processing device. [Figure 8] FIG. 1 illustrates an embodiment of the present invention in which a peripheral device is coupled to a processing device. [Figure 9] FIG. 1 illustrates an embodiment of the present invention in which a peripheral device is coupled to a processing device. [Figure 10] FIG. 1 illustrates an embodiment of the present invention in which a peripheral device is coupled to a processing device. [Figure 11] FIG. 1 illustrates an embodiment of the present invention in which a peripheral device is coupled to a processing device. [Figure 12] FIG. 1 illustrates an embodiment of the present invention in which a peripheral device is not coupled to a processing device. [Figure 13] FIG. 1 illustrates an embodiment of the present invention in which a peripheral device is not coupled to a processing device. [Figure 14]FIG. 1 illustrates an embodiment of the present invention in which a peripheral device is not coupled to a processing device. [Figure 15] FIG. 2 is a diagram showing a message flow used in any of embodiments 1 to 11 and 17 according to an embodiment of the present invention. [Figure 16] FIG. 11 is a diagram showing a message flow used in any of the embodiments 12 to 14 according to an embodiment of the present invention. [Figure 17] FIG. 13 illustrates yet another embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0053] definition "Plug and Play" is a computer bus or device specification A term used to describe the feature of plug-and-play, which makes hardware components in a system easy to discover and does not require physical device configuration or user intervention in resolving resource conflicts. A plug-and-play device can be added to the bus of a computing system (either at run-time or at shutdown) and the newly added device, and possibly the rest of the computing system, will automatically configure itself so that the newly added device functions from both a hardware and software perspective.

[0054] Examples of plug-and-play interfaces include (non-exhaustive list): Firewire (IEEE-1394), PCI, mini PCI, PCI Express, mini PCI Express, PCMCIA, PC Card, Universal Serial Val (USB), SDIO Card, HDMI (registered trademark), DisplayPort, Bluetooth (registered trademark), etc.

[0055] "Auto-configuration" is automatic configuration without manual intervention, without setting any switches or jumpers, and without the need for software configuration. An example of an auto-configuring device is a USB device. Examples of auto-configuration protocols are DHCP, Zeroconf, and Bonjour.

[0056] A plug-and-play device has, by default, auto-configuration software to make it plug-and-play. As an example, a USB device becomes plug-and-play by including the correct auto-configuration software (e.g., host driver, host stack, application software). Auto-configuration can also be software alone and is not limited to physical devices.

[0057] "Hot swapping and hot plugging" are terms used to describe the ability to replace components in a computer system without shutting down the system. More specifically, hot swapping describes replacing components without significant interruption to the system, and hot plugging describes adding components that expand the system without significant interruption to the system's operation. A well-known example of this functionality is the Universal Serial Bus (USB), which allows users to add or remove peripheral components such as mice, keyboards, or printers. Other examples are, for example, eSATA, PCIe, and FireWire.

[0058] A "portable application" (portable app), often referred to as standalone, is a computer software program designed to run without being installed on the target machine. This type of application is often stored on a removable medium such as a CD, USB flash drive, flash card, or floppy disk. A program is stored on a portable storage device, which stores its program files, configuration information and data only on the storage medium. It is a program that can be stored on an electronic device, such as a USB flash drive, an iPod, a memory card, a portable hard drive or other portable electronic device, and runs on a computer or other processing device coupled to the electronic device without making any permanent configuration changes to the host computer. All such programs have a zero footprint. This means that all temporary files, registry entries, and any other changes to the machine only exist while the program is running.

[0059] To be considered a portable application for purposes of this invention, a software program must: It does not require any kind of formal installation to be performed on a permanent storage device of a computer, but rather must be capable of being stored on a removable storage device, such as a USB flash drive, an iPod, a memory card, a portable hard drive, or other portable electronic storage device, and therefore be capable of being used on multiple computers.

[0060] The settings are stored with the software and preferably are portable with the software (i.e., they are written to an electronic device such as a USB drive). The settings are not stored in a registry or any other central system database on the computer.

[0061] This is done by leaving a zero (or near zero) "footprint" on any PC it is used after, i.e. all temporary files / registry settings must be avoided or at least deleted once the program exits, and user created files can be stored directly on the same removable media when the application is stored.

[0062] A portable application does not leave its files or settings on the host computer on which it is executed. For example, the application does not write to the Windows registry or store its configuration files (such as INI files) in the user's profile. It stores its configuration files in the program's directory instead. Another requirement is that the application must store these in a relevant format, since file paths are often different on different computers that vary due to differences in Windows drive letter assignments. Preferably, such programs do not require a launcher program to copy necessary settings and files to the host computer when the application starts and back to the application's directory when the application closes, because this could leave remnants on the hard drive behind in the event of a power outage.

[0063] "Electronic meeting systems" (EMS) must be distinguished from typical groupware, on the one hand, and Web conferencing systems, on the other. In fact, there is some overlap between non-essential features of the products in the above categories. The main difference with groupware is the intensity of collaboration. EMS must be distinguished from systems that allow multiple users to simultaneously present the contents of their individual computer screens on a remote display.

[0064] "Groupware" supports collaboration within a group where individual contributions remain identifiable. In contrast, EMS supports collaboration within a group where the group as a whole It enables them to collaboratively produce accountable results. In business processes, groupware and electronic meeting systems are complementary, with groupware supporting teams in research and documentation prior to an EMS session or during the implementation of the results of such a session.

[0065] The "Web Conferencing System" and the "Electronic Meeting System" are complementary in the context of online meetings or workshops, where an EMS extends a Web Conferencing System by providing interactive tools for generating and documenting group results, while a "Web Conferencing System" complements an EMS with screen sharing and audio conferencing capabilities that are necessary for synchronous online meetings but are not available in an EMS.

[0066] "Data conferencing" means a real-time communication session between two or more participants who share computer data. Interaction and presentation devices such as screens, keyboards, mice, cameras, etc. may be shared. This is a term used to distinguish it from video conferencing and audio conferencing. Data may include screens, documents, graphics, drawings, and applications that are viewable by participants in the meeting.

[0067] "Application sharing" is an element of remote access under the umbrella of collaboration software that allows two or more users to access a shared application or document in real time simultaneously from their respective computers. Typically, the shared application or document runs on a host computer, and remote access to the shared content is provided from the host computer to other users. Application sharing must be distinguished from systems that do not allow collaboration between different users on an application, but do allow multiple users to simultaneously project the contents of their individual computer screens onto a remote display.

[0068] The term "any media content" means that a user can generate, create, or select any media content suitable for display. This is distinct from a client voting or selecting the media content displayed by others in a meeting or presentation. This term refers to client-oriented, client-granted rights and privileges for the display of content, rather than a central presenter providing content that is provided to members of a meeting.

[0069] "Unified communication system or tool" means voice or audio-visual content communication such as that provided by "Skype" or "Skype for Business". Such software can replace the voice and / or visual data provided by a host processing device. A unified communication tool can be described as a collection of tools for implementing VOIP, (web) conferencing, shared whiteboard, message exchange (e.g. chat), file transfer, or presence. A unified communication system or tool can be any communication system that includes Voice-Over-Internet-Protocol (VoIP), text or instant messaging (e.g. AIM, Blauk, eBuddy, Gadu-Gadu, IBM Lotus Sametime, ICQ, iMessage, IMVU, Lync, MXit, Paltalk, Skype, Tencent QQ, Windows Live Messenger® or MSN Messenger®, Wireclub, Xfire, and Yahoo! Messenger®, email, Twitter (e.g., Tweets), Digital Service Protocol (DSP), etc. A unified communication system or tool can be used for video conferencing by utilizing a video conferencing cloud service that includes a video conferencing node. The video conferencing system enables communication between one or more users located at a first video conferencing endpoint and one or more users located at a second video conferencing endpoint.

[0070] "Screen scraping" in the sense of the present invention means a process for This means processing rather than just reading the framebuffer and rendering it on a display. Screen scraping for presentation is described in US 2002 / 0196378 to Slobodin et al., which is incorporated herein by reference.

[0071] "Auto composition" or "auto layout" refers to the automatic nature of multiple graphics / video sources being rendered on the central display in a way that a user intuitively expects to happen, without user intervention.

[0072] A "wireless" or "wireless communication network" may be any network that does not use cable links between nodes, for example using RF, optical, or infrared such as IrDA, diffuse infrared, WLAN, WiMax, WiFi, WiFi Direct, LiFi, ultrasonic, ZigBee, or Bluetooth for communication, or any other wireless communication network known to those skilled in the art. Any communication network disclosed below may preferably be a wireless network. This allows for "sharing or combining" at least a selection of the functions / endpoints of the second peripheral device, i.e., a one-to-many scenario.

[0073] "Computer" generally refers to a processing device, i.e., having a processing engine capable of various digital processes such as rendering graphic images for display. A computer may be in the form of a workstation, personal computer, laptop, palmtop, PDA, smartphone, tablet, etc. Generally, a computer has memory such as volatile RAM. Non-volatile memory such as a hard disk, optical disk, or solid state memory may be included in the computer or may be a peripheral device. Although most computers today are electronic computers, the term "computer" also includes optical-based computing devices.

[0074] The term "pre-installed generic driver" is intended to mean a driver that is installed on a processing device such as a computer as a standard driver, e.g., installed with an operating system installed. Such a driver is standard for the operating system and can drive a standard class of peripheral devices coupled or connected to the processing device. There is no need to install a specific driver for such peripheral devices. Such a generic driver may be a human interface driver (HID) or mass storage device driver having predefined software components configured to drive mass storage, CD-ROM, keyboard, etc., or a combination thereof. Such a device may be a readable and writable computer peripheral memory device such as a USB memory stick, flash memory, external hard drive, etc.

[0075] A "host device" is a processing device that desires to share with a first peripheral device or dongle.

[0076] A "functional device" is a second peripheral device connected to the parent device in some way. The system has the function of transparently exposing the second peripheral device connected to the parent device to the first peripheral device as if it were attached to the processing device to which the first peripheral device is connected. This exposure is achieved by a "descriptor field". A functional device can be operated only by a vendor-specific driver.

[0077] An "endpoint" can be described as a data source or sink and is defined for a USB device, which may be a physical or virtual device. In the present invention, an endpoint must be broadly interpreted as a data source or sink. Thus, data can be stored or emitted at an endpoint. An endpoint can act as a kind of buffer and can be defined for a physical or virtual device. Data stored at an endpoint may be received from a host, such as a USB host, or it may be waiting to be sent to the host. An endpoint is defined by a number of descriptive fields. For example, an endpoint may be configured to support different transfer types, such as four transfer types. These transfers are defined in the USB specification as, for example, control transfers, interrupt transfers, isochronous transfers, and bulk transfers. A person skilled in the art can use these transfer types in the present invention as needed. Within the limitations of the hardware, the endpoints can be configured using middleware, for example USB middleware. The endpoints may be limited to a particular transfer type.

[0078] A USB host client can send data to, say, endpoint 1. Data from the USB host is sent to OUT endpoint 1. The running software then reads the data when it's ready. To return data, it must be written to IN endpoint 1, because the software does not have free access to the USB bus, since it is controlled by the USB host. Data in IN endpoint 1 stays there until the host sends an IN packet to endpoint 1 requesting data.

[0079] Restrictions may be imposed on the number of endpoints. Each endpoint may have one transfer direction. Certain endpoints may, for example, be defined for control transfers only and not assigned to any other function.

[0080] A control transfer may be a bidirectional transfer reserved for the host to send and request configuration information to and from a device using specific IN and OUT endpoints.

[0081] Each control transfer may include multiple transactions. A control transfer may have several phases. 1. The SETUP phase sends a setup packet to define the request and specify the amount of data to be transferred in the data phase. 2. The DATA phase is optional. If there is a data phase, it always begins with a transaction containing a DATA1 packet. The transaction type then alternates between DATA0 and DATA1 until all the required data has been transferred. 3. A STATUS phase is a transaction that contains a zero-length DATA1 packet. If the data phase is IN, the status phase is OUT, and vice versa.

[0082] Interrupt transfers have a finite latency to or from a device. In USB, interrupt transfers or interrupt pipes have a defined polling rate. Typically, interrupt transfer data consists of event notifications, characters, or coordinates from a pointing device.

[0083] Isochronous transfers are used to send real-time information, such as audio and video data, and can be sent at a constant rate. A dedicated portion of the USB bandwidth is allocated to a USB isochronous data stream to ensure that data can be delivered at the desired rate. An isochronous pipe sends a new data packet every frame regardless of whether the last packet was successful or unsuccessful. No error detection is required for isochronous transfers. Any errors in the electrical transmission are not corrected.

[0084] Bulk transfers are used for data that is not of type control, interrupt, or isochronous. Reliable data exchange is guaranteed with error detection at the hardware level.

[0085] Bulk transfers use all available bandwidth after other transfers have finished. If the bus is very busy, bulk transfers may be delayed.

[0086] "A particular device exposes a peripheral or other device" means that the particular device configures one or more endpoints with particular descriptor fields.

[0087] Description of the Preferred Embodiments Embodiments of the present invention enable a wireless connection between a user processing device, such as a laptop, computer, PDA, smartphone, etc., and a wireless network master. This wireless connection may be made by an integrated wireless transceiver in the user processing device, such as a laptop, computer, PDA, smartphone, etc. Alternatively, in some embodiments of the present invention, the wireless connection is made through a first peripheral device that is plugged into, coupled to, connected to, or attached in some manner to the user processing device, or is provided by other hardware or software.

[0088] An embodiment of the present invention allows a user, e.g., a participant in a meeting room, to utilize at least one, several, or all functional devices attached to a parent unit, the at least one functional device having an electronic output, such as a digital electronic output, and capable of working in some way with other devices plugged into, coupled to, connected to, or attached to the parent unit, or provided by other hardware or software. The at least one functional device may be provided by plugging into a second peripheral device, e.g., a dongle, or by a wirelessly connected second peripheral device, or by connecting to the at least one functional device with some software that may be downloaded or pre-installed. The at least one functional device may have electronic output provided by a touch screen, microphone, speakerphone, camera, and may function with devices having electronic inputs, such as a display or speaker. An embodiment of the present invention provides a functional device for shared use by participants in a classroom or meeting room.

[0089] The at least one functional device coupled or connected or attached to the parent unit, either via a serial connection, e.g., a serial plug and play connection, e.g., USB, or via a wireless connection, e.g., a Near Field Communication (NFC) connection, e.g., a Bluetooth connection, an IR connection, etc., is made accessible and usable by one or more users in the meeting room or classroom. Preferably, this is done in a transparent and standard manner. In various cases, the system may function as a wireless (e.g., USB) hub, and in other cases, a first peripheral device or other hardware or software may act as the combined functionality of the functional device, e.g., a second peripheral device coupled or connected or attached to the parent unit. A virtual device having one function is exposed.

[0090] The embodiment of the present invention utilizes a base unit, which includes at least a transmitting function, and usually includes a transmitting and receiving function such as a transceiver function. The base unit is a base unit of a network such as a wireless network, and thus may have a transmitter, a receiver, or a transceiver. The base unit includes a digital processing engine such as an FPGA, a CPU, a GPU, etc., memory, and input / output ports, and can be connected, for example by plugging in, to any functional device, such as one or more second peripheral devices with or without visual or audio capabilities that can be used in a meeting room or classroom. The one or more functional devices, such as a second peripheral device, for use with the parent unit may include one, some, or all of the following: a television or other video provider providing images, such as video, for example, through an HDMI, DVI, USB, or other connection; an image capture device, such as a camera providing images, such as captured video, for example, through a USB or HDMI or other connection; an audio device, such as a speakerphone or one or more microphones using, for example, a 3.5 mm audio jack through a USB connection, SPDIF or other connection, providing audio, such as sound or music, for example, through a USB connection or other connection; a display device, such as a touch screen, for example, using a USB or other connection, integrated through a USB connection; and any other functional device or first peripheral device that can be connected, e.g., plugged into, a first device, such as the parent unit, and that has an electronic output, for example, a digital electronic output.

[0091] An embodiment of the present invention includes a device such as a first peripheral device that includes at least a receiving function and typically includes a transmitting and receiving function such as a transceiver function. The first peripheral device (at least the receiver device) may be hardware, software, or a combination of these, e.g., having wireless or wired connectivity capabilities, that can be connected to a parent unit through a wireless connection such as Wi-Fi, Bluetooth, Wi-Gig, Li-Fi, NFC or similar connection, or through a wired link, e.g., USB, Ethernet, etc. Thus, the parent unit can expose and make available any functional device, such as a second peripheral device connected to the parent unit, to one or more first peripheral devices (e.g., receiver devices), e.g., simultaneously to multiple first peripheral devices (e.g., receiver devices) in one of the following ways: 1. Exposing the or each functional device (such as the or each second peripheral device) connected to the parent unit to one or more first peripheral devices or user processing devices, for example in a one-to-one or one-to-many relationship to the first peripheral devices or user processing devices, preferably without interpreting or processing electronic signals from each functional device, for example from the second peripheral device, for example by tunneling packets from the parent unit to one or more endpoints of the first peripheral devices or user processing devices. 2. Interpreting, processing and transforming electronic signals from said at least one functional device, e.g., second peripheral device, e.g., mimicking said at least one functional device, e.g., second peripheral device, in software, hardware or both at the end of the first peripheral device (e.g., at the end of the receiver device). Mimicking means that a device identical to the second peripheral device connected to the parent device is exposed at the endpoint, but the communication is not simply tunneled as in the first and second methods. 3. Combining one or more functional devices, e.g. a second peripheral device or a portion of a second peripheral device, and exposing it to one or more first peripheral devices, e.g. a receiver device, through custom hardware, software, or both. 4. Incorporating one or more functional devices, such as a second peripheral device or a portion of a second peripheral device, and exposing its functionality to the end user through proprietary software. 5. Combining any of the above methods to provide functionality of a functional device, for example a second peripheral device, to one or more first peripheral devices, such as a receiver device.

[0092] Where a functional device, e.g., a second peripheral device, is exposed natively or as a hardware imitation of such a device, an endpoint, such as a USB endpoint on the first peripheral device or user processing device, can be utilized that can expose a corresponding second peripheral device, such as a USB device connected or plugged into the parent unit (as in the case of a hub), without the need to install proprietary software or drivers to support this.

[0093] When exposing a custom device, i.e. a device developed specifically to function in the application of the present invention, these same endpoints of the first peripheral device, such as the receiver hardware, can be utilized, or this can be done in software using drivers.

[0094] The first peripheral device can be used to set up a connection, such as a wireless or wired connection, between the first peripheral device and the parent unit, for example by acting as a hub, such as a wireless hub, of which a wireless USB hub is an example. Whatever second peripheral device is connected or plugged into the parent unit, the second peripheral device, e.g., selected from the non-exhaustive list of TV, video device, audio device, speakerphone, camera, display, ..., will be exposed to the first peripheral device in any of the five ways.

[0095] Advantages of embodiments of the invention may be one, some, or all of the following.

[0096] In one implementation, all of the second peripheral devices, which may be USB devices, are mirrored one-to-one without requiring any additional drivers or software to be installed on the user's processing device other than the second peripheral vendor's drivers. The second peripheral devices can be exposed to the user's processing device either directly by the hardware of the first peripheral device or by using a software virtual hub on the first peripheral device or on the user's processing device.

[0097] In another implementation, different secondary peripheral devices (e.g., having electronic outputs and / or inputs, e.g., speakers, microphones) can be combined and connected to the same parent unit. Additional functionality such as echo cancellation, conversion, encoding / decoding, etc. can be given to the parent unit. Only one secondary peripheral device needs to be exposed to the user's processing device through the first peripheral device.

[0098] A combination of the above two methods can be provided for one or a combination of devices or functions. Depending on whether a device or function needs to be available to multiple users at once, different approaches can be used in one final solution, and there are technical feasibility possibilities such as simple mirroring of devices.

[0099] Some software can be added to the client software running on user processing devices to facilitate the configuration of these devices (e.g. Set as default device, for example). You can also add some custom controls (mute, pan / tilt / zoom…). o Some control can be added to the first peripheral device. * Such control can be eliminated by implementing some processing on the parent unit (e.g. automatic gain control for the audio signal, framing of participants to make PTZ unnecessary).

[0100] Unlike a normal hub, multiple users can use the functional devices, and only one can plug into one cable. Each user processing device that has a first peripheral device plugged into it can use those functional devices, and the parent device makes sure that all streams are correctly captured, played, processed, combined, and / or enhanced.

[0101] · It is possible to adjust to the unpredictability of wireless communication channels by giving different priorities (speed, throughput, …) to different streams. For example, prioritizing an audio stream over a video stream to improve the call experience.

[0102] If there is a VOIP call such as a Skype call in the meeting, the first peripheral device can be connected, e.g. plugged into the user processing device and use all the functional devices connected to the parent unit, e.g. a more advanced camera giving a wide room view, a more advanced microphone and a more advanced speaker. This is especially useful when there are multiple people in a room who want to answer a conference call such as a VOIP or Skype call.

[0103] 1, group 90 is a microprocessor or FPGA based device that includes one or more peripheral devices 91-93 that can be connected, e.g., plugged into, parent device 100. Peripheral devices 91-93 can be connected to parent device 100 by standard connections such as Firewire (IEEE-1394), PCI, mini PCI, PCI Express, mini PCI Express, PCMCIA, PC Card, Universal Serial Bus (USB), SDIO card, HDMI, DisplayPort, Bluetooth, or other serial connections.

[0104] By using group 90, for example, peripheral devices Webcam 91 Speakerphone 92 Multiple HID endpoints 93 can be exposed.

[0105] The master unit 100 comprises an electronic digital processor, such as a microprocessor or an FPGA-based device, and includes, for example: A web interface 110 which is an interface for the parent unit 100 by which the system can be configured. One or more drivers 111, for example two specific drivers for webcam 91 and speakerphone 92, executed on the parent unit 100 to receive / send data from / to the group 90 and USB peripherals 91-93. · Additional processing for both audio and webcam encoding, mixing, (de)multiplexing and other processing is performed in blocks 112, 114, 115. · Data may be encrypted or decrypted in the decryptor / encryptor 113, 116 before being transmitted over the wireless link 127 to the first peripheral device 130. The control module 117 of the parent device 100 and the first peripheral device 130 are A control channel 128 between the 00 and a first peripheral device 130 is used to perform various functions such as keep alive, identification, crypto key exchange, etc. · The Unified Protocol block 118 is a functional block that abstracts the main functionality of the system: screen sharing and other non-media streaming (eg mouse pointer). · Encryption and decryption modules 119, 120 are provided for the main communication with the first peripheral device 130 and other applications. An encoder such as H264 encoder 121 provides a local view block 123 (which is one or more functional devices that emit a signal from the parent device 100 towards the user processing device 160) that captures, encodes and transmits the rendered image on the main display 126 back to the first peripheral device 130 and other clients. An audio processing block 124 is provided for audio content. A video decoder 125 is provided for decoding and rendering screen content to a display 126.

[0106] The first peripheral device 130 is a microprocessor or FPGA-based device that includes, for example: · Webcam video from the webcam 91 of the parent unit 100 is decoded in block 30 and exposed to the processing device 160 through a driver such as USB Video Endpoint and H264 driver 31 via USB Video. · The audio signal from the speakerphone 92 connected to the base unit 100 is decoded in block 32 and processed and exposed as a microphone signal from the USB speakerphone endpoint 92. Audio signals coming from the processing device 160 through the speakerphone USB endpoint are encrypted in block 35 and sent to the base unit 100 via the wireless connection 127. The control HID (Human Interface Device) 51 is a mirror of the additional HID USB endpoints that the group 90 and secondary peripheral devices 91 - 93 expose to the parent device 100 . A vendor-specific HID endpoint, shown as data pump 42, is used for channel 52, which may be a high-bandwidth data channel between the parent unit 100 and vendor-specific software, such as software 70 running on processing device 160, via the first peripheral device 130. Screen video and audio signals for the display 126 may be sent over this channel 52, which is preferred over through a control HID endpoint in the first peripheral device 130. Screen input for the display 126 that is captured (block 123) and encoded (block 121) in the parent unit 100 is also sent to the vendor-specific software running on the processing device 160, so that other users, such as multiple users or meeting participants, each having a plugged-in processing device 160 and optionally a first peripheral device 130, can see what is on the display 126 on their own processing device 160 display (not shown). A number of basic components such as an operating system (OS) and drivers for controlling light indicators such as an LED ring on the first peripheral device 130, a bandwidth measurement module and other e.g. wireless wi-fi modules, an update / upgrade module, a secure boot module, secure key storage, a debug and test module, etc. are provided in blocks 45-50. The audio capture module 43 exposes the speaker endpoints to the processing device 160. The module 43 captures this audio and passes it back to the vendor software 70 running on the processing device 160 where it is time-stamped (block 69) and, after alignment with the video data, sent to the parent unit 100, for example through the data pump 42. This allows for improved time stamping and lip sync. Cut. In a less preferred embodiment, the audio can be captured and processed on the first peripheral device 130 and sent immediately to the parent unit 100 .

[0107] A microprocessor or FPGA processing device 160, such as a computer, laptop, smart phone, personal assistant (PDA), etc., runs proprietary software 60, 70, including, for example, third party software (eg, Skype) 60. The third party software 60 runs on the processing device 160 (e.g., communication software such as Skype (registered trademark)), and the processing device uses a video endpoint and / or a speakerphone endpoint (both a speakerphone microphone and a speakerphone speaker) from at least one functional device 91-93 (e.g., a webcam 91 attached to the parent unit 100). Block 70 contains the dedicated software developed for the present invention, which - Software for decoding (61) and decoding (62) the pipeline, which then shows the screen data of the display 126 from the parent device 100 in a software window. Data from the final stage (local view 63), following the arrow 71, is captured by communication software such as Skype® client software 60. The unified protocol block 64 is the module that handles the control layer and controls (indirectly) the media path, the connection to the first peripheral device 130 and the connection to the parent unit 100 . The pipelines 65 to 67 are for screen capture (67), encoding (66), and encryption (65) for screen sharing with the parent device 100. - The pipelines 68, 69 receive the speaker audio signal back from the first peripheral device 130 (block 68), perform processing (e.g. time stamping, block 69) and send it out through the HID endpoint ("Data Pump 52"). The endpoints that the software does not use in this case are the control HIDs from the second peripheral device. If some software is executed from the vendor of the second peripheral device, this software can communicate with the associated second peripheral device through these HID endpoints (e.g. pan / tilt / zoom of a camera). · The speaker endpoint (content audio arrow 53) is used by the processing device 160 Operating System (OS) to transmit all (non-speakerphone) sound / audio signals.

[0108] Connecting a first peripheral device to a user processing device The first peripheral device 130 can be connected to the user processing device 160 by the method 300 shown in FIG. 2. This connection may be a serial connection, such as a USB connection, such as USB 1, 2, 3 or -C, and the peripheral device 130 is connected, for example plugged into, a plug-and-play serial port provided for the USB connection. The first peripheral device 130 may include a user input device, for example an actuator. The first peripheral device 130 may be configured as a connection unit and may be a physical device in the form of a connector for a plug-and-play interface of the user processing device. It may have a base and flexible data and power connections attached to the base. The base may have an actuator, for example a button configured to be an input device that can be activated by the user.

[0109] The flexible data and power connections must be configured so that the base can lie flat on the meeting table regardless of the orientation of the connector that needs to be inserted into the plug-and-play interface. Preferably, the base contains the electronics and optionally The first peripheral unit has a persistent storage for storing applications such as portable applications in the first peripheral device 160. Network configuration parameters for WiFi connection to the parent unit 100 can also be stored in the memory. The first peripheral unit may have a processing engine (e.g. CPU, GPU, FPGA), a wireless transmitter / receiver for WiFi or LiFi, etc., a plug-and-play interface such as a USB interface, a light ring such as an LED ring or strip as a visual indicator. The portable application may be stored in the first peripheral device 160, i.e. in the base, or may be downloaded from another pluggable device such as a flash drive, downloaded from the parent unit 100, etc. The visual indicator is to allow user feedback from the connected unit of the status of some activity.

[0110] Some examples of actuator activation that can be used in any of the embodiments of the present invention are voice activation (handclaps, voice recognition, computer sounds, music, ...), remote control via wirelessly connected devices (IR, Bluetooth, WiFi, LiFi...), light activation, pressure activation, e.g. pressing with a finger or hand, touch activation, proximity (a "near touch" to the actuator or bringing the actuator close to some object), biometric readers such as fingerprint readers, iris scanners, DNA analyzers, e.g. keypads for entering key codes such as passwords.

[0111] The method 300 includes, in step 301, connecting the first peripheral device 130 to the processing device 160, for example by plugging in, which results in a kernel boot step 302 and an exchange of initialization scripts in step 303. The first peripheral device 130 is discovered to be a USB device in step 306, and the USB protocol stack is loaded in step 304, followed by the wireless connection protocol stack in step 305. Parameters for the wireless connection are entered, for example from a memory on the first peripheral device 130 or by a user, and the association of the wireless connection is completed in step 307. The client software that needs to be loaded on the processing device 160 is then loaded and activated in step 308, for example by a user action. In step 310, the first peripheral device 130 is ready to share data with the display screen 126 attached to the parent unit 100.

[0112] 3, if a launcher program is available, e.g., stored on the first peripheral device 130 or stored on the processing device 160 or can be retrieved from a remote location such as a URL on the Internet or from a LAN server, it can automatically launch the client software in step 328. Steps 321-327, 329, and 330 are identical to steps 301-307, 309, and 310 in FIG.

[0113] For example, the client software may be the software 70 of FIG. 1 running on the processing device 160. Other software such as communication software 60, e.g., Skype, may be pre-installed on the processing device 160. The client software 70 may be for processing video data from the processing device 160, and the client software runs on the processing device 160 using a processing engine such as a CPU, GPU or FPGA. The client software 60 may be screen scraping software for screen scraping the screen of the processing device 160. The client software 60 may be stored as a software program in a memory such as mass storage on the first peripheral device 130. The software program may be loaded into the processing device 160 when coupled to the processing device 160, either automatically or by user action as described above. The software 60 is preferably executed on the processing device 160. When the software 60 is executed on the processing device 160, it preferably leaves a zero footprint upon termination. When the software 60 is executed on the processing device 160, it captures video data available on the processing device 160, for example from a presentation or video running on the processing device 160. For example, the software 70, when executed on the processing device 160, is configured to screen scrape and read one or more video frame buffers, for example from a graphics card in the processing device 160. Typically, the processing device 160 has its own display and therefore has a graphics card or equivalent of a buffer for storing video data. This read video information is time-stamped with a clock time (block 69) so that the captured video signal can later be synchronized with an associated audio signal. The video signal is then encoded in a video encoder (block 66) for transfer through a plug-and-play interface using a generic driver, for example through a USB interface using a pre-installed generic driver, for example a pre-installed generic human interface device (HID) (arrow 53). The data is transmitted to a wireless communication network 127, such as a WiFi or LiFi network, routed through the communication network 127 to the parent unit 100, and inserted into a suitable composition, such as an OpenGL-based composition in a compositor, for display on a display device 126.

[0114] Send data from the processing device to the parent device With reference to FIG. 4, audio data, audio on a processing device 160, such as a computer, laptop, smartphone, PDA, tablet, etc., is sent through a port using a generic driver, for example, through a USB port 8 using a standard built-in generic audio driver such as the UAC driver 7. In the first peripheral device 130, audio packets are read from a generic port, for example, the USB port 11, by a dedicated audio device 14. These packets are then processed by any of a mixer, a rate converter, an echo canceller, a noise canceller, etc. Any of the mixing, rate conversion, echo cancellation, noise cancellation can be performed using an ALSA driver 18. The ALSA driver gives the packets to a dummy audio device 16, such as an audio scraper. In this audio device 16, the audio packets may be time-stamped from a clock 15, which may be synchronized with the clock 9 of the processing device 31. This information is then encoded in an encoder 17 and packetized in a packetizer 19 before being transferred to the communication network between the first peripheral device 130 and the parent unit 100. The network may be a wireless network, such as a WiFi or LiFi network. For this purpose, a suitable transmitter 21 is provided in the first peripheral device 130. In the parent unit 100, the audio information stream is recovered in a suitable communication interface, such as a WiFi access point 22. The audio is then unpacked in an unpacker 23 and decoded in a decoder 24, before being fed to an audio mixer 28. In order to synchronize the audio and video streams, they are sent together to a synchronizer 27, where the two streams are synchronized and thus lip-sync is maintained.

[0115] Send data from the processing device to the parent device An embodiment of the present invention can capture data, such as visual or audio signals, and provide the captured data to one or more processing devices 160 so that the captured data can be shared with other participants. These visual or audio signals can be captured, provided, or obtained from functional devices such as a webcam 91, a speakerphone 92, or even captured from a display 126. For example, a functional device can be a microphone that is appropriately positioned in the meeting to receive good audio data. In addition, the audio may be used with a central speaker system that may be of higher quality than the audio processing device 160 of the meeting participants. Such a speaker system may be, for example, a four-channel surround speaker system. This may also avoid the difficulties associated with multiple microphones and speakers that can result in annoying positive feedback.

[0116] 1-4 to provide additional functionality that can be shared by all participants in a meeting having a processing device 160, optionally with a plugged-in first peripheral device 130. In an embodiment of the invention, fixed USB endpoints on the first peripheral device 130 are provided for basic functionality. These are fixed and are a combination of vendor-specific endpoints and multiple standard endpoints, which can be interpreted or understood as custom drivers, default OS drivers, and / or host applications as described with reference to FIG. 4 for screen sharing and audio.

[0117] The configurable USB endpoints are configured when pairing the first peripheral device 130 with the parent device 100 or through the wireless connection 127 between the processing device 160 and the parent device 100. Once configured, the USB endpoints expose the second peripheral devices 91-93 that can be interpreted as custom drivers, vendor specific drivers or default OS drivers. These can be used by either host or third party applications. These endpoints may be exact copies of the peripheral devices among the second peripheral devices 91-93 connected to the parent device 100 or may be dedicated designated endpoints.

[0118] Communication between these USB endpoints and the processing device 160 can occur through standard enumeration of USB endpoints when the first peripheral device 130 is connected, e.g., plugged in, to the processing device 160. All information on how to communicate with the parent unit 100 and which endpoints to expose can be stored in the first peripheral device 130 and set up by firmware of the first peripheral device 130.

[0119] The decision as to which endpoints of the first peripheral device 130 to expose can be made by the parent unit 100, which may be configured to analyze which second peripheral devices are connected to the parent unit 100 (which connections can be achieved through USB connections or otherwise) and create settings for the associated USB endpoints on the first peripheral device 130.

[0120] As mentioned above, in embodiments of the present invention, data may flow in both directions, ie, to or from processing device 160, during operation. 1. Data from a second peripheral device connected to the parent unit 100 can stream into the parent unit 100 or can be captured by the parent unit 100, such as data for the display 126. Any such data can be interpreted using firmware in the parent unit 100 and possibly processed, enhanced, encoded, and / or encrypted to generate processed data. In some embodiments, this processed data is then transmitted over the wireless link 127 to the first peripheral device 130, which can decode, decode, process, and / or interpret the data, if necessary, and make it available through one or more of its USB endpoints (e.g., fixed or configured endpoints). This data is then captured by the operating system of the processing device 160 and made available to a host application or third party running on the processing device 160 through custom or standard drivers. The meeting application or other processing devices 160 of the other participants in the meeting may be made available. 2. Data, which may be from a host application and / or one or more third party applications running on the processing device 160, can be sent through a handle made available by the system. This data is then sent through the system (standard generic) driver and USB port to the appropriate USB endpoint of the first peripheral device 130. The first peripheral device 130 then receives this data via firmware on the first peripheral device 130, processes, enhances, encodes and / or encrypts this data, and sends it over the wireless link 127 to the parent unit 100. The parent unit 100 receives this data via firmware on the parent unit 100, decodes, processes, decodes and / or enhances it, and forwards it to the appropriate first peripheral device, which may be connected or attached to the parent unit 100 through a serial connection such as USB, HDMI or other interface.

[0121] Such a multi-participant meeting is shown diagrammatically in Figure 6 with four processing devices 160 (160-1, 160-2, 160-3, 160-4), which may be laptops. Three of the processing devices (160-1, 160-2, 160-3) are each connected to a first peripheral device 130, for example by plugging in a USB connection. One processing device 160-4 is hosting a Unified Communication (UC) call, such as a Skype or Skype for Business call. Two of the processing devices, a first processing device (160-1, 160-2), have a WiFi connection to the parent device 100 and are sharing media to the primary screen 126. Processing devices 160-3 and 160-4 are sharing a unified communications (UC) call, such as a Skype call or a Skype for Business call.

[0122] The parent device 100 is connected to one or more of the following: Webcam 91 4K Touch Screen 94 95 cameras connected via USB or HDMI USB Speakerphone 92 USB Microphone 96 and Speaker Set 97 (connects via audio jack, s / pdif, USB, HDMI, etc.) The host processing device 160-4 has pre-installed UC client software. This software interacts with a USB endpoint on the first peripheral device 130 connected to the host, e.g., plugged in. This type of client software uses a speakerphone USB endpoint as the default audio device (e.g., microphone plus speaker). An embodiment of the present invention is under control of all audio or visual signals sent from and injected into the processing device 160.

[0123] The speakerphone 92 can be used by any of the users with at least the first peripheral device 130 connected and functional. No action needs to be taken to access this device. Separate controls may be available on the first peripheral device, for example to control the volume of the parent unit 100's audio output and to mute the room microphone 96 if desired. When muted, the microphone 96 is also muted for all other buttons or apps using this device.

[0124] UC client software 60 running on the host processing device 160-4 does not inject audio content. The microphone 96 can pick up this signal, or the parent unit 100 can inject it into the microphone signal to the first peripheral device 130 connected to the host 160-4. Because of the low latency, there is no need to lip-sync with the content video (even if it is shared).

[0125] A second peripheral device of the same type as that plugged into the parent unit 100 is exposed to the user so that the user can use the vendor-provided drivers. If one of the processing devices 130 is sharing in the room, the content audio is also shared in the room.

[0126] The processing device with the first functioning peripheral device can wirelessly access the room camera 96. All other devices connected to the parent unit 100 (e.g., the HID control device 93) are essentially exposed.

[0127] Any user connected to the system, for example through a first peripheral device, can see the content projected on the display of the user's processing device in the meeting room, i.e., a "local view." A local view receiver may have one, some, or all of the following capabilities, but is not limited to these:

[0128] The ability to open a window and view the meeting room content in this window, whether this is the stream itself, annotations, or a blackboard session.

[0129] Ability to zoom in on content. Additional measures can be added to the feature to ensure image quality when zooming in.

[0130] The local view allows users to initiate and / or participate in blackboard or annotation sessions from their own device.

[0131] Ability to take screenshots and / or pause the local view. Figures 6-15 and 17 show how various functions can be made available to participants in a room, for example a meeting. There are four basic ways, each of which is an embodiment of the invention. a) Using generic drivers in the OS 164 of the host processing device 160 and endpoints such as mass storage or HID endpoints, client software running on the host processing device 160 can be used to transfer data without the host processing device 160 being aware that the end device is a functional device such as a camera, touch screen, microphone, loudspeaker, speakerphone, etc. within or connected to the parent unit 100. b) Connect an endpoint on the first peripheral device of the functional device to or within the parent device 100. This means that the host processing device 160 detects that it is connected to the functional device and interacts with the functional device as if it were such a device. c) Connect an endpoint on the first peripheral device of the functional device to or within the parent device 100. This means that the host processing device 160 detects that it is connected to a functional device, but the client software needs to enable a vendor-specific driver to interact with the functional device. d) Connect an endpoint on the first peripheral device of the functional device to or within the parent device 100. This means that the host processing device 160 detects that it is connected to the functional device, but does not know how the processing device 160 interacts with the functional device. Pre-processing is performed on the parent device 100 so that the parent device can process the first peripheral device. One method is to create a common endpoint on the first peripheral device so that the processing device can cooperate with any of the functional devices that are the subject of pre-processing on the parent device 100.

[0132] 6 shows a processing device 160 that communicates with a first peripheral device 130 connected thereto, and a parent unit 100 that performs wireless communication 127 with the first peripheral device 130. Various functional devices are connected to the parent unit 100, such as a display 126, a speakerphone 92 (or a microphone and speaker set 96, 97) and a video cam 91 (or other camera 95).

[0133] The generic driver allows the client software 70 to send and receive data to and from the mass storage device 132 on the first peripheral device 130 via the USB interface 131. The client software 70 also has a USB endpoint 134, which may be a USB HID endpoint or a vendor-specific endpoint if necessary. This endpoint 134 is the endpoint through which the client software 70 sends data for the display 126 to the parent unit 100, which may be image processed (blocks 124, 125 in FIG. 1) before sending it to the group 90, for example via an HDMI connection. In this way, the HID endpoint 134 makes the HDMI display 126 and its audiovisual signals available to any processing device 160 to which the first peripheral device is attached when in the meeting room. However, the display 126 is not exposed to the processing device 160, i.e., it is provided as a HID endpoint and not as a display. The video cam 91 may be exposed to the first peripheral device 130 as an endpoint (in the manner described for FIG. 1) and the video data is encoded in block 122 in FIG. 1. 6, the video data is encoded in the parent unit 100 and sent via the first peripheral device 130 to a driver 162 of the operating system 164 of the processing device 60. The processing device 60 exposes the encoded data from the video cam 91 to the processing device 160, making the video data available for a UC call, e.g., Skype®.

[0134] Data from the speakerphone 92 may be processed at the base unit 100 and / or at the first peripheral device 130 and mimicked to the processing device 160. The microphone of the speakerphone 92 generates a signal that is sent to the processing device 160 via the USB speakerphone endpoint 136 on the first peripheral device, so that noise reduction or echo cancellation can be performed at the base unit 100 as described with respect to FIG. 1 (block 115). When the processing device 160 recognizes the speakerphone endpoint 136 as a speakerphone, as if the speakerphone were plugged into the processing device 160, the microphone data is made available to the UC call. The microphone data is also made available to all other users who have a processing device 160 to which the first peripheral device is connected. In this embodiment, the speaker of the speakerphone has a USB endpoint 136 on the first peripheral device 130.

[0135] Figure 7 shows a similar configuration to that of Figure 6, except that there are two USB endpoints 136 and 138 for a speakerphone and speaker, respectively. As mentioned above, the HID endpoint 134 allows any processing device 160 to which the first peripheral device 130 is attached to utilize the HDMI display 126 when in the meeting room to provide audiovisual signals to the display 126. However, the display 126 is not exposed to the processing device 160, i.e., it is provided as an HID endpoint rather than as a display.

[0136] The two endpoints 136, 138 receive audio signals from two different sources. Audio from a US call can be handled via the speakerphone endpoint 136 and processed at the base unit as described with respect to FIG. 6. However, if there is additional audio data that is not part of the UC call, this data is sent via the speaker endpoint 138. Since there is no second peripheral device in the group 90 that is a standalone speaker, the base unit 100 mixes the audio signals for the speakerphone and the speaker before sending all the audio data to the speaker of the speakerphone 92. If there is a separate speaker 97 in the group 90, the base unit 100 can send this audio data to this speaker. As a result, a virtual speaker is exposed to the processing device 160.

[0137] Figures 8-11 and 17 illustrate various embodiments of the present invention with examples of how to make a touch screen 99 available in a meeting room. When a touch screen is touched it generates a signal related to the coordinates of the touch location. In the meeting this needs to be sent to the processing device 160 so that the actual signal can be interpreted if it is provided to different screens on the processing device 160. For meetings where participants use a variety of processing devices a way needs to be found to provide the touch coordinates to all these different processing devices. Figures 8-11 disclose the above four basic methods.

[0138] In the embodiment of FIG. 8, there is a group 90, a parent unit 100, a first peripheral device 130, and a processing device 160. There are two functional devices connected to or within the parent unit 100: a display 126 and a touch screen 99. The touch screen 99 has a touch screen endpoint 139 at the first peripheral device 130. The processing device 160 recognizes this endpoint and can interact with it if the OS 164 has an associated driver 162. This driver can be downloaded if necessary. In this embodiment, each processing device 160 must download the associated driver 162 at the meeting or already have it installed.

[0139] In the embodiment of FIG. 9, there is a group 90, a parent unit 100, a first peripheral device 130, and a processing device 160. There are two functional devices connected to or within the parent unit 100: a display 126 and a touch screen 99. The touch screen 99 has a touch screen endpoint 139 in the first peripheral device 130. The processing device 160 recognizes this endpoint and can interact with it if the parent unit 100 is configured to convert the coordinates of the touch into coordinates that the processing device 160 can use, i.e., if the OS 164 has an associated driver 162. This driver can be downloaded if necessary. In this embodiment, each processing device 160 needs to download or already have installed the associated driver 162 at the meeting, but the conversion function of the parent unit 100 can reduce the number of different drivers that would be required.

[0140] In the embodiment of FIG. 10, there is a group 90, a parent unit 100, a first peripheral device 130, and a processing device 160. There are two functional devices connected to or within the parent unit 100: a display 126 and a touchscreen 99. The touchscreen 99 has a touchscreen endpoint 139 at the first peripheral device 130. The processing device 160 is configured to recognize this endpoint and to translate the coordinates of a touch by the parent unit 100 into coordinates that the processing device 160 can use. 9. If the endpoint is connected to a processing device 160, i.e., if the OS 164 has an associated driver 162, it can communicate with this endpoint. This driver is a standard driver that can be distributed, e.g., the driver can be downloaded as needed. In this embodiment, each processing device 160 at the meeting needs to download or already have installed the associated driver 162, but the translation functionality of the parent unit 100 can further reduce the number of different drivers that would be required compared to the embodiment of FIG. 9.

[0141] In the embodiment of Fig. 11, there is a group 90, a parent unit 100, a first peripheral device 130 and a processing device 160. There are two functional devices connected to or within the parent unit 100: a display 126 and a touch screen 99. The touch screen 99 is handled by the client software 70 as previously mentioned, i.e. the client software running on the first peripheral device 130 is configured to interact with the touch screen 99 through the USB interface 131 and to interact with the OS 164 of the processing device 160. This means that the touch screen 99 is not exposed to the processing device 160. The client software 70 is configured to simulate touch events and provide the correct coordinates to the OS 164 of the processing device 160. This is therefore a proprietary solution that requires the client software 70 to be configured to interact with each type of touch screen 99 through the USB interface 131.

[0142] Figures 12-14 relate to embodiments of the invention where there are multiple users. Firstly, each participant in a meeting can use any of the embodiments described in Figures 8-11. Thus, the invention provides a multi-user solution.

[0143] In Fig. 12, the processing device 60' does not have the first peripheral device 130, e.g. none is available. In such a case, the client software 70' may be configured to directly interact with the parent unit 100 through the wireless connection 127', the client software 70' being configured to provide the display device 126 to the processing unit 160' and to enable interaction therebetween, providing any simulation or transformation as necessary. The client software 70' may be configured to simulate any of the embodiments described with respect to Figs. 1-11 without the use of the first peripheral device 130. The client software 70' may simulate, for example, a UB interface of the type used in any of the embodiments of Figs. 1-11. The processing device 160 may use the methods described with respect to any of Figs. 1-11.

[0144] In FIG. 13, the processing device 60' does not have the first peripheral device 130, e.g. none is available. In such a case, the client software 70' may be configured to directly communicate with the parent unit 100 through a wireless connection 127', the client software 70' being configured to provide the display device 126 to the processing unit 160' and to enable communication therebetween, providing any simulation or transformation as necessary. This is as described with respect to FIG. 12. However, the group 90 has a microphone and speaker combination, such as a speakerphone 92 or a microphone and speaker combination 96, 97. The processing device 160 may utilize any of the embodiments of FIGS. 1-11. The client software 70' of the processing device 160' may be configured to directly provide video data to the parent unit 100, where the video data is processed, e.g. decoded and composited, and provided to the display 126 through a suitable connection, such as an HDMI connection.

[0145] In the processing device 160', the client software 70' and the driver 127' can provide a simulated speakerphone 92 to the OS 164' to generate virtual microphones and virtual speakers. Third party software 60' for unified communications such as Skype® calls is provided with these virtual devices, which are then used for the call.

[0146] In Fig. 14, the processing device 60' does not have the first peripheral device 130, e.g. none is available. In such a case, the client software 70' may be configured to communicate directly with the parent unit 100 through a wireless connection 127', the client software 70' being configured to present the display device 126 to the processing unit 160' and to enable communication therebetween, providing any simulation or transformation as necessary. This is as described with respect to Fig. 13. However, the group 90 has a video cam 91 connected to the parent unit 100, e.g. via a USB connection. The processing device 160 may utilize any of the embodiments of Figs. 1-11. The client software 70' of the processing device 160' may be configured to allow the parent unit 100 to receive the video data directly.

[0147] In the processing device 160', the client software 70' and the driver 162' can provide a simulated video cam 91 to the OS 164' to create a virtual video cam device. Third party software 60' for unified communications such as Skype® calling is provided with all the virtual devices, which are then used for the call.

[0148] FIG. 15 illustrates a messaging flow diagram applicable to any of the embodiments described with respect to FIGS.

[0149] FIG. 16 illustrates a messaging flow applicable to any of the embodiments described with respect to FIGS. 12-14 where a first peripheral device is not used.

[0150] FIG. 17 illustrates yet another embodiment of the present invention including a processing device 160 having a processor and memory for executing and optionally storing a client software 70 including a host application as well as a third party application 60 that may be configured to perform a unified communication call such as a Skype® call or a Skype® for Business call. The processing device 160 has a serial port such as a USB port and a generic or custom driver for communicating with a first peripheral device 130 via the port. The peripheral device 130 has a processor and memory for executing and optionally storing firmware for providing a wireless link 127 to the parent unit 100 and for providing at least one configurable or fixed endpoint of a functional device 90 connected to or within the parent unit 100. The parent unit has a processor and memory for executing and optionally storing firmware for providing a connection to the wireless link 127 and for providing a port to the functional device 90. The functional device 90 has a processor and memory and executes and optionally stores firmware for providing a link and interface to the parent unit 100 and for providing data to be transmitted to the at least one configurable or fixed endpoint on the peripheral device 130.

[0151] The method of the present invention may be implemented by a computer system. The present invention may perform its functions using a processing engine, which is preferably implemented by one or more microprocessors, FPGAs, or central processing units (CPUs) and / or It has digital processing capabilities such as those provided by a Graphics Processing Unit (GPU) and is configured to perform its functions by being programmed with software, i.e., one or more computer programs. Reference to software may encompass any type of program in any language that is directly or indirectly executable by a processor, either through compiled or interpretable languages. Implementation of any of the methods of the present invention may be performed by logic circuits, electronic hardware, processors, or circuits, which may encompass any type of logic or analog circuitry, and may be integrated to any degree, including but not limited to general purpose processors, digital signal processors, ACIS, FPGAs, discrete components, or transistor logic gates.

[0152] A processing device may have memory (such as non-transitory computer readable media, RAM and / or ROM), an operating system, optionally a display such as a fixed format display, a port for a data input device such as a keyboard, a pointer device such as a "mouse", serial or parallel ports for communicating with other devices, a network card, and connections for connecting to any of a network.

[0153] The software may be embodied in a computer program product configured to perform the functions of any of the methods of the invention, for example as listed below, when the software is loaded into a computer and executed on one or more processing engines, such as a microprocessor, ASIC, FPGA, etc. Thus, any of the embodiments of the invention may incorporate a computer system capable of executing one or more computer applications in the form of computer software.

[0154] The methods described above with respect to the embodiments of the present invention may be performed by one or more computer application programs executed on a computer system by being loaded into memory and running on or in association with an operating system such as Windows, Linux, Android, etc., provided by Microsoft Corporation, USA. The computer system may include a main memory, preferably a random access memory (RAM), and may also include a non-transitory hard disk drive and / or a removable non-transitory memory, and / or a non-transitory solid state memory. The non-transitory removable memory may be an optical disk, such as a compact disk (CD-ROM or DVD-ROM), a magnetic tape, which is read and written by a suitable reader. The removable non-transitory memory may be a computer readable medium on which computer software and / or data is stored. The non-volatile storage memory may be used to store persistent information that should not be lost even if the computer system is powered down. Application programs may use information and store it in the non-transitory memory.

[0155] The software embodied in the computer program product is configured to perform the following functions when the software is loaded into one or more corresponding devices and executed on one or more processing engines, such as microprocessors, ASICs, FPGAs, etc., and these functions include: connecting a processing device to a functional device connected to or within a parent unit of a communication network, the processing device having a memory, a display and an operating system, the parent unit having a transmitter and the first peripheral device having a receiver; coupling a first peripheral device to a processing device via a generic communication protocol and providing at least one fixed or configurable endpoint of a functional device exposed on the first peripheral device; Sending data from a parent device through a communication network from the functional device to the processing device via at least one fixed or configurable endpoint using a generic communication protocol for communication between the processing device and the first peripheral device, and receiving the data at the first peripheral device.

[0156] The software embodied in the computer program product is configured to perform the following functions when the software is loaded into one or more corresponding devices and executed on one or more processing engines, such as microprocessors, ASICs, FPGAs, etc., and these functions include: Sending or receiving data in raw or unaltered format, i.e. streaming data carrying images, audio, etc.; Connecting a functional device to or within a base unit of a wireless communication network, Running functional devices with only vendor-specific drivers.

[0157] The software embodied in the computer program product is configured to perform the following functions when the software is loaded into one or more corresponding devices and executed on one or more processing engines, such as microprocessors, ASICs, FPGAs, etc., and these functions include: Exposure achieved through the descriptor field, transmitting data from the functional device to the at least two processing devices using at least two first peripheral devices; The combination of two endpoints and exposing them to a processing device as a single endpoint.

[0158] The software embodied in the computer program product is configured to perform the following functions when the software is loaded into one or more corresponding devices and executed on one or more processing engines, such as microprocessors, ASICs, FPGAs, etc., and these functions include: the functional device enabling one or more of a microphone, a speakerphone, a speaker, a display, a touch screen, a projector, a camera, a video camera, a webcam; Indicating at least one fixed or configurable endpoint of a functional device exposed on the first peripheral device as one of a human interface device, a mass storage device, a composite device, a microphone, a speakerphone, a speaker, a display, a touch screen, a projector, a camera, a video camera, or a webcam.

[0159] The software embodied in the computer program product is configured to perform the following functions when the software is loaded into one or more corresponding devices and executed on one or more processing engines, such as microprocessors, ASICs, FPGAs, etc., and these functions include: Encoding and / or optionally encrypting the data; hosting on a processing device unified communications between two or more other processing devices; the first peripheral device providing a functional unit for unified communication between two or more processing devices; or Exposing to the processing device the same type of functional device as is connected to the parent device, and using at least one driver for the functional device installed in the processing device.

[0160] Any of the above software may be installed on any of the servers or nodes of the network. The present invention can be realized as a computer program product compiled for a processing engine in any one of the following: The computer program product can be stored on a non-transitory signal storage medium, such as an optical disk (CD-ROM or DVD-ROM), a digital magnetic tape, a magnetic disk, a solid-state memory such as a USB flash memory, a ROM, etc.

Claims

1. 1. A system for connecting a processing device to a functional device connected to or contained within a parent unit of a communications network, the processing device having a memory, a display and an operating system, the system comprising: a first peripheral device configured to be coupled to the processing device via a generic communication protocol, the parent having a transmitter, the first peripheral device having a receiver and at least one fixed or configurable endpoint of the functional device exposed on the first peripheral device; the parent unit and the first peripheral device are each configured to transmit and receive data over the communications network from the functional device to the processing device via the at least one fixed or configurable endpoint using the generic communications protocol for communication between the processing device and the first peripheral device.

2. The system of claim 1 , wherein data transmitted or received, optionally streaming data carrying images, audio, etc., can be in a raw or unaltered format.

3. The system according to claim 1 or 2, wherein the functional device is connected to or located within a master unit of a wireless communication network.

4. 2. A system according to any one of the preceding claims, wherein the exposure is achieved by a descriptor field.

5. 2. A system according to any one of the preceding claims, wherein the functional devices are capable of operating only with vendor specific drivers.

6. 2. A system according to any one of the preceding claims, configured to transmit the data from the functional device to at least two processing devices using at least two first peripheral devices.

7. 10. A system according to any one of the preceding claims, wherein two endpoints are combined and exposed to the processing device as one endpoint.

8. 10. The system of claim 1, wherein the functional device is one or more of a microphone, a speakerphone, a speaker, a display, a touch screen, a projector, a camera, a video camera, a webcam.

9. 10. The system of claim 1, wherein the at least one fixed or configurable endpoint of the functional device exposed on the first peripheral device is one of a human interface device, a mass storage device, a composite device, a microphone, a speakerphone, a speaker, a display, a touch screen, a projector, a camera, a video camera, or a webcam.

10. 10. A system according to any one of the preceding claims, further comprising means for encoding, and optionally encrypting, the data.

11. 2. A system according to any one of the preceding claims, wherein the processing device is configured to host unified communications between two or more other processing devices.

12. The system of claim 11 , wherein the first peripheral device is configured to provide a functional unit for the unified communication between two or more processing devices.

13. 11. The system of claim 1 , further comprising at least one driver for a functional device installed on the processing device, the driver being configured to expose to the processing device a functional device of the same type as that connected to the parent unit.

14. 2. A system according to any one of the preceding claims, wherein the functional device is a second peripheral device.

15. 2. A system according to any one of the preceding claims, wherein the functional device is a data capture device.

16. 1. A method for connecting a processing device to a functional device connected to or within a parent unit of a communication network, the processing device having a memory, a display and an operating system, the parent unit having a transmitter and a first peripheral device having a receiver, the method comprising: coupling a first peripheral device to the processing device via a generic communication protocol and providing at least one fixed or configurable endpoint of the functional device exposed on the first peripheral device; and transmitting data from the parent device through the communications network from the functional device to the processing device via the at least one fixed or configurable endpoint using the generic communications protocol for communication between the processing device and the first peripheral device, and receiving the data at the first peripheral device.

17. The method according to claim 16, wherein the data transmitted or received, i.e. streaming data carrying images, audio etc., can be in raw or unaltered format.

18. The method according to claim 16 or 17, wherein the functional device is connected to or located within a master unit of a wireless communication network.

19. The method according to any one of claims 16 to 18, wherein the functional device is operated exclusively with a vendor-specific driver.

20. The method according to any one of claims 16 to 19, wherein the exposure is realizable with a descriptor field.

21. The method of any one of claims 16 to 20, comprising transmitting said data from said functional device to at least two processing devices using at least two first peripheral devices.

22. A method according to any one of claims 16 to 21, wherein two endpoints are combined and exposed to the processing device as one endpoint.

23. The functional device is any one of a microphone, a speakerphone, a speaker, a display, a touch screen, a projector, a camera, a video camera, and a webcam. The method of any one of claims 16 to 22, further comprising providing one or more.

24. 24. The method of claim 16, further comprising providing the at least one fixed or configurable endpoint of the functional device exposed on the first peripheral device as one of a human interface device, a mass storage device, a composite device, a microphone, a speakerphone, a speaker, a display, a touch screen, a projector, a camera, a video camera, or a webcam.

25. The method of any one of claims 16 to 24, further comprising the step of encoding and / or optionally encrypting said data.

26. The method of any one of claims 16 to 25, further comprising hosting on the processing device unified communications between two or more other processing devices.

27. 27. The method of claim 26, further comprising the first peripheral device providing a functional device for the unified communication between two or more processing devices.

28. A method according to any one of claims 16 to 27, comprising exposing to the processing device functional devices of the same type as those connected to the parent unit, and using at least one driver for the functional devices installed on the processing device.

29. A peripheral device configured to be coupled to a processing device via a generic communication protocol, the peripheral device having a receiver and at least one fixed or configurable endpoint of a functional device exposed on the first peripheral device; A peripheral device, wherein the receiver of the first peripheral device is configured to receive data from the functional device over a communications network via the at least one fixed or configurable endpoint using the generic communications protocol for communication between the processing device and the peripheral device, and to transmit the data to the processing device.

30. The peripheral device of claim 29 , wherein the communication network is a wireless communication network.

31. A peripheral device according to claim 29 or 30, wherein the data transmitted or received, i.e. streaming data carrying images, sound etc., can be in raw or unaltered format.

32. The peripheral device according to any one of claims 29 to 31, wherein the functional device can be operated only with a vendor-specific driver.

33. 33. The peripheral device of claim 29, wherein the at least one fixed or configurable endpoint of the functional device exposed on the first peripheral device is one of a human interface device, a mass storage device, a composite device, a microphone, a speakerphone, a speaker, a display, a touch screen, a projector, a camera, a video camera, or a webcam.

34. The steps of the method according to any one of claims 16 to 28 when executed on a processor. A computer program product for carrying out any of the above.

35. 35. A non-transitory signal storage means storing a computer program product according to claim 34.

Images