System for making one or more functional devices available in a meeting room
The system wirelessly connects processing devices to meeting room equipment using a generic protocol, allowing high-quality audio and video devices to be shared and used by multiple participants without physical connections or additional software, addressing the integration challenges in existing systems.
Patent Information
- Application Number
- DE202018006950
- Authority / Receiving Office
- DE · DE
- Patent Type
- Utility models
- Current Assignee / Owner
- Priority Date
- 2017-12-29
- Filing Date
- 2018-12-21
- Publication Date
- 2024-12-05
- Estimated Expiration
- 2028-12-31
AI Technical Summary
Existing systems fail to efficiently and flexibly integrate high-quality audio and video devices into meeting environments, requiring participants to be physically connected to the devices, which limits their seamless use across different processing devices.
A system that connects processing devices wirelessly to functional devices via a base unit, using a generic communication protocol to access endpoints without the need for special drivers, enabling sharing and combining device functionalities across multiple participants.
Enables high-quality audio and video devices to be seamlessly integrated and used by multiple participants, enhancing collaboration and communication in meetings without the need for physical connections or additional software installation.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
Technical area
[0001] The present invention relates to a system for making one or more functional devices in a meeting room available to a processing device of a meeting participant. The invention also relates, among other things, to a communication system for meeting rooms, a processing device for use in a system according to the invention, a first peripheral device for use in a system according to the invention, a base unit for use in a system according to the invention, meeting room equipment with a system according to the invention, and a collaboration platform for meetings with a system according to the invention. background
[0002] The ability of ad hoc groups of people to communicate with each other is one of the fundamental aspects of collaboration, problem-solving, negotiation, teaching and education, etc. To support communication, there has been an explosive growth of electronic communication tools such as electronic meeting and conferencing tools (following modern usage, the English term “tool” is used here to refer to software such as utilities that facilitate certain tasks), e.g. synchronous and asynchronous conferencing, online chat, instant messaging, audio conferencing, video conferencing, data conferencing, application sharing, remote desktop sharing, electronic meeting systems, collaborative management (coordination) tools, project management systems, knowledge management systems and social software systems.
[0003] A classic approach is the lecture or seminar, which often involves a presentation using presentation software. The traditional presentation or lecture by a single person has largely been retained, with the listeners in a more passive mode regarding determining, building on, expanding, or modifying the information to be presented.
[0004] Like other business processes, meetings are becoming increasingly digitalized. More and more people are using computer technology alone or in conjunction with broadband networks to achieve their goals before and during the actual meeting. For example, email is being used to distribute files to people for reading in advance.
[0005] However, certain features such as microphones and speakers are typically of low quality when provided by portable devices such as laptops and mobile phones. As meetings become increasingly larger, there is a need to be able to provide meeting participants with high-quality audio and visual signals.
[0006] EP 3 099 009 A1 discloses an electronic meeting tool and a method for exchanging arbitrary media content by and among users during a meeting. For this purpose, a node configuration means is used that is designed to operate a display node of a communications network, wherein the display node is connected to a first display. The node configuration means is designed to receive arbitrary media content selected by the user and to control the display of the arbitrary media content selected by the user on the first display.At least one peripheral device configured to communicate the user-selected media content via the communications network is provided. The peripheral device is a connection unit comprising a connector configured to connect a port of a processing device having a second display, a memory, and an operating system, and a transmitter for communicating with the communications network. A program configured to be loaded onto the processing device and run on the operating system of the processing device is provided. The program is configured to receive the user-selected media content, leaving no trace upon termination (commonly referred to as "leaving no footprint," meaning that no file fragments, registry entries, etc. should remain on the processing device).An input device is provided to enable the user to perform a user action that triggers the transmission of any media content selected by the user to the transmitter via the port.
[0007] US 2016 / 0125838 A1 discloses systems and methods that enable a host device to communicate video information, network information, and USB information over USB using a USB host controller. The video information and network information are encapsulated within USB and communicated by the USB host controller. In some embodiments, the USB information communicated by the USB host controller is further communicated over a non-USB extension medium through an upstream port device and one or more downstream port devices.
[0008] WO 2008 / 080107 A1 discloses a method for improving data throughput for a wireless USB system. The USB system includes wired adapters that wirelessly transmit data between a host system and a wired USB device. The document also discloses a wireless USB hub that acts as a proxy for the wired USB devices, presenting them to a host either as standalone "WUSB devices" (Wireless USB) with their own addresses or as separate functions on an existing WUSB device.
[0009] WO 2012 / 128972 A1 discloses a system that allows PC users to virtualize a local USB device using software configured to emulate the USB device being virtualized. A USB device virtualization client component processes received USB device commands and sends them over a network via TCP / IP to a USB device virtualization server. Brief description of the invention
[0010] In view of the known prior art, the invention is based on the object of making functional devices in a meeting room efficiently and flexibly available to multiple participants without requiring each participant to be physically connected to the devices. In particular, it addresses the problem of how high-quality audio and video devices, displays, and other peripherals can be seamlessly integrated into a meeting environment and used by different participants with different processing devices.
[0011] The object is achieved by a system for making one or more functional devices in a meeting room available to a processing device of a meeting participant, having the features of claim 1. The independent claim 14 relates to a communication system for meeting rooms. The independent claim 15 relates to an apparatus comprising a processor and a memory, the memory containing instructions which, when executed by the processor, cause the apparatus to implement the system according to one of claims 1 to 13. The independent claim 16 relates to a processing device for use in a system according to the invention. The independent claim 18 relates to a first peripheral device for use in a system according to the invention. The independent claim 20 relates to a base unit for use in a system according to the invention.The subordinate claim 22 relates to a meeting room configuration with a system according to the invention. The subordinate claim 23 relates to a collaboration platform for meetings with a system according to the invention. Advantageous embodiments and further developments of the invention are the subject of the subordinate claims.
[0012] The system according to the invention makes it possible to make the functionalities of devices connected to a central base unit accessible to the participants' processing devices via a wireless network. This overcomes the challenge of presenting the devices in such a way that they can be recognized and used by standard operating systems and applications without the need for special drivers or software. At the same time, it creates the flexibility to combine, process, or expand the device functions as needed to meet the requirements of modern collaboration and communication scenarios.
[0013] Embodiments of the present invention provide a system for connecting a processing device to a functional device connected to or in a base unit of a communications network, the processing device having a memory, a display, and an operating system, the system comprising: a first peripheral device that can be coupled to the processing device via a generic communication protocol, wherein the base unit has a transmitter and the first peripheral device has a receiver and at least one fixed or configurable endpoint of the functional device that is made accessible on the first peripheral device; the base unit and the first peripheral device are configured to send and receive data, respectively, via the communication network from the functional device to the processing device via the at least one fixed or configurable endpoint using the generic communication protocol for communication between the processing device and the first peripheral device.
[0014] The communications network is preferably a wireless network. This enables the sharing or combination of at least a selection of functionalities / endpoints from secondary peripheral devices, i.e., the one-to-many scenario.
[0015] The data transmitted or received, i.e. streaming data that carries images, audio, etc., may be in a raw or unmodified format.
[0016] There may be more than one first peripheral device. For example, the data may be transferred from the functional device to at least two processing devices using at least two first peripheral devices.
[0017] A functional device can be operated with a manufacturer-specific driver alone.
[0018] The functional device may be a data generating device such as any one or more of a microphone, a hands-free device, a speaker, a display, a touchscreen, a projector, a camera, a video camera, and a webcam.
[0019] The at least one fixed or configurable endpoint of the functional device made accessible on the first peripheral device may be a user interface device.
[0020] The processing device may be configured to communicate with the at least one fixed or configurable endpoint, wherein the pre-installed generic driver is a driver for a user interface device.
[0021] The at least one fixed or configurable endpoint of the functional device made accessible on the first peripheral device may be a mass storage device.
[0022] The processing device may be configured to communicate with the at least one fixed or configurable endpoint with a pre-installed generic driver that is a mass storage device driver.
[0023] The at least one fixed or configurable endpoint of the functional device on the first peripheral device may be a composite device.
[0024] The processing device may be configured to communicate with the at least one fixed or configurable endpoint with a pre-installed generic driver that is a composite device driver.
[0025] Embodiments of the present invention include combining endpoints: The present invention includes a system in which two endpoints are combined and made accessible to the processing device as one endpoint.
[0026] An accessibility or enumeration property determines which endpoints the first peripheral device (e.g., a button) sees. In terms of accessibility, this can be done with descriptor fields. A specific device can make a peripheral device or another device accessible. This means that the specific device configures one or more endpoints with specific descriptor fields or a descriptive field. An endpoint is defined by a set of descriptive fields.
[0027] A basic form of descriptive fields can be “USB over IP”, and the actual transmission of the unchanged data can be done via so-called “tunnels”.
[0028] Accessibility can be implemented using descriptor fields. Endpoints are the capabilities / functionalities of the physical devices (second peripherals or functional devices). For example, a telephone device is a second peripheral that can have multiple endpoints, such as a microphone, a speaker, an IP port, etc. The distinction between endpoints and second peripherals / functional devices defines the endpoints.
[0029] Embodiments of the present invention provide a system that enables the use of the manufacturer's original driver for each functional device (and its associated endpoint), e.g., in the context of mirroring. Therefore, the functional devices can operate solely with the manufacturer-specific driver.
[0030] The system may include means for encoding and, optionally, encrypting the data. Therefore, the system may include an encoder for encoding and, optionally, encrypting the data.
[0031] The processing device may be configured to host unified communication between two or more processing devices. The unified communication may be a call via an internet-based instant messaging service, such as the Skype and Skype for Business (also called Skype4Business) services provided by Microsoft Corp. The term "unified communication" is explained in more detail below. The first peripheral device may be configured to provide the functional device for unified communication between two or more processing devices. The functional device may be, for example, a speaker, a microphone, a loudspeaker, a video camera, a webcam, a camera, or another audio source.
[0032] The system can be designed to share data from other processing devices in a meeting during the call. A manual action performed by the sharer or the host via a local viewport can be used to activate sharing.
[0033] The system can be configured to select a camera, e.g., via a manual action in UC clients. The system can be configured to synchronize the release state of the first peripheral device with the release state of the UC client.
[0034] Sharing functionalities / endpoints can involve sharing a room display. Combining resources can involve using two room speakers (e.g., to increase quality), but the user only sees "one room speaker." The latter case would also include the combination function if there is more than one participant / user in the room. Sharing can also be conceivable as selecting functionalities / endpoints of a specific functional device.
[0035] Embodiments of the present invention may use multiple endpoints, at least one fixed or configurable endpoint, and the release may be made to multiple processing devices, ie, the use of multiple first peripherals (eg, buttons) is also supported.
[0036] The system may be configured to provide a wireless speakerphone for multiple participants. The speakerphone may be made accessible to each processing device in a meeting, with a first peripheral device connected to each of the processing devices. No moderation or action is required to access this device.
[0037] The system can be designed to provide separate controls, e.g., on the peripheral device to adjust the volume of an audio output from the base unit and to mute a room microphone. Muting a microphone can also mute all other peripheral devices connected to the processing equipment in a meeting.
[0038] Audio signals should be picked up by a microphone, or the base unit may be designed to feed audio signals into any audio feed, e.g. a microphone signal to a first peripheral device connected to a processing device.
[0039] The system may be configured to provide the user with access to the same type of functional device that is connected to the base unit, so that a user can use the drivers provided by a manufacturer of the functional device that are installed on the processing device.
[0040] The system can be designed such that, for a plurality of processing devices that communicate with the base unit, a functional device connected to the base unit is made accessible to all of the plurality of processing devices. The functional device can be made natively accessible.
[0041] Accessibility can be achieved using descriptor fields.
[0042] The system may be designed to enable any processing device connected to the communications network via the first peripheral device to view any audiovisual content displayed, provided or projected in the meeting room on that processing device in what is known as a "local view".
[0043] A processing device that is the recipient of the local view may have, but is not limited to, the following functions: - the ability to open a window and display meeting room content in it; - the ability to enlarge content; - the local view may allow the user to start and / or join a whiteboard or annotation session from their own processing device.
[0044] The system can be designed to provide alternative options and systems when there are too few primary peripherals in a particular meeting room.
[0045] The system may be designed to issue a warning when a first peripheral device attempts to connect to the wrong base unit.
[0046] In a further aspect, the system and device aspects of the present invention allow a method for connecting a processing device to a functional device connected to or in a base unit of a communication network, wherein the processing device has a memory, a display and an operating system, the base unit has a transmitter and the first peripheral device has a receiver, the method comprising: coupling a first peripheral device to the processing device via a generic communication protocol, wherein at least one fixed or one configurable endpoint of the functional device is made accessible on the first peripheral device; Transmitting data from the base unit and receiving the data at the first peripheral device via the communication network from the functional device to the processing device via the at least one fixed or configurable endpoint using the generic communication protocol for communication between the processing device and the first peripheral device.
[0047] The communications network is preferably a wireless network. This enables the sharing or combination of at least a selection of functionalities / endpoints from secondary peripherals, i.e., the one-to-many scenario.
[0048] The data transmitted or received, i.e. streaming data that carries images, audio, etc., may be in a raw or unmodified format.
[0049] There may be more than one first peripheral device. The data can be transmitted from the functional device to at least two processing devices via at least two first peripheral devices.
[0050] A functional device can be operated solely with a manufacturer-specific driver.
[0051] The functional device may provide one or more of a microphone, a speakerphone, a speaker, a display, a touchscreen, a projector, a camera, a video camera, a webcam.
[0052] The method performable by the system and device aspects of the invention may include representing the at least one fixed or configurable endpoint of the functional device accessible on the first peripheral device as one of a user interface device, a mass storage device, a composite device, a microphone, a hands-free device, a speaker, a display, a touchscreen, a projector, a camera, a video camera, or a web camera.
[0053] The data can be encoded and / or optionally encrypted.
[0054] The processing device can host unified communication between two or more other processing devices.
[0055] The first peripheral device may be a functional device for unified communication between two or more processing devices.
[0056] The method feasible thanks to the system and device aspects of the invention may include making the same type of functional device connected to the base unit accessible to the processing device and using at least one driver for the functional device installed on the processing device.
[0057] Accessibility can be achieved using descriptor fields.
[0058] In a further aspect, a peripheral device is provided that can be coupled to a processing device via a generic communication protocol, the peripheral device comprising a receiver and at least one fixed or configurable endpoint of a functional device that is made accessible on the first peripheral device, and wherein the receiver of the first peripheral device is configured to receive data from the functional device via the communication network and to send the data to the processing device via the at least one fixed or configurable endpoint using the generic communication protocol for communication between the processing device and the peripheral device.
[0059] The communications network is preferably a wireless network. This enables the sharing or combination of at least a selection of functionalities / endpoints from secondary peripherals, i.e., the one-to-many scenario.
[0060] The at least one fixed or configurable endpoint of the functional device exposed on the first peripheral device may be one of: a user interface device, a mass storage device, a composite device, a microphone, a hands-free device, a speaker, a display, a touchscreen, a projector, a camera, a video camera, or a webcam. Exposure may be implemented using descriptor fields.
[0061] A functional device can be operated solely with a manufacturer-specific driver.
[0062] A computer program product can be provided for performing the steps of a method performable thanks to the system and device aspects of the invention when executed on a processor. A non-volatile signal storage medium can be used to store the computer program. The non-volatile signal storage medium can be an optical disk such as a CD-ROM or DVD-ROM, a magnetic disk such as a hard disk, a solid-state memory such as a flash memory, a magnetic tape, or the like.
[0063] Further details, features and advantages of the invention will become apparent from the following purely exemplary and non-limiting description of preferred embodiments in conjunction with the drawing comprising 17 drawing figures. Short description of the drawing Fig. 1 shows in the Fig. 1A to C an embodiment of the present invention. The Fig.2 and Fig. 3 illustrates the message flow of how a peripheral device is connected to a processing device in embodiments of the present invention. Fig. Figure 4 shows an arrangement of components that can be used in embodiments of the present invention. This figure is the Fig. 11 of WO 2013 / 037980 A1 with the title “Electronic tools and methods with audio for meetings”. Fig. 5 shows an arrangement of processing devices used in a meeting with ongoing unified communication according to an embodiment of the present invention. The Fig. 6 to 11 show embodiments of the present invention in which a peripheral device is coupled to a processing device. The Fig.12 to 14 show embodiments of the present invention in which a peripheral device is not connected to a processing device. Fig. Figure 15 shows a message flow for use with one of the embodiments as described in connection with the Fig. 1 to 11 and 17. Fig. Figure 16 shows a message flow for use with embodiments as described in connection with the Fig. 12 to 14. Fig. 17 shows another embodiment of the present invention. Definitions
[0064] "Plug and Play" is a term used to describe a feature of a computer bus or device specification that facilitates the locating of a hardware component in a system without requiring physical device configuration or user intervention to resolve resource conflicts. Plug and Play devices can be added to a computer system's bus (while the device is running or powered off), and the newly added device, and potentially the rest of the computer system, are automatically configured to allow the newly added device to function, both from a hardware and software perspective.
[0065] Plug and Play interfaces include, for example (not a complete list): Firewire (IEEE-1394), PCI, Mini PCI, PCI Express, Mini PCI Express, PCMCIA, PC Card, Universal Serial Bus (USB), SOlO cards, HDMI, DisplayPort, Bluetooth, etc.
[0066] Autoconfiguration is the automatic configuration of devices without manual intervention, without setting switches or jumpers, and without the need for software configuration. An example of automatic device configuration: USB devices. Examples of autoconfiguration protocols: DHCP, Zeroconf, Bonjour.
[0067] A "Plug and Play" device has auto-configuration software by default to make it "Plug and Play." For example, USB devices become Plug and Play devices by incorporating the appropriate auto-configuration software (e.g., host driver, host stack, application software). Auto-configuration can also refer to software alone and is not limited to a physical device.
[0068] "Hot swapping" and "hot plugging" are terms that describe the functions of exchanging computer system components without shutting down the system. More specifically, "hot swapping" describes the replacement of components without significant disruption to the system, while "hot plugging" describes the addition of components that would enhance the system without significant disruption to system operation. A well-known example of this functionality is the Universal Serial Bus (USB), which allows the user to add or remove peripheral components such as a mouse, keyboard, or printer. Other examples include eSATA, PCIe, FireWire, etc.
[0069] A portable application, sometimes called a standalone application, is a computer software program that can be run on the target computer without installation. This type of application is stored on a removable device such as a CD, USB flash drive, flash card, or floppy disk, and stores its program files, configuration information, and data only on the storage medium. It is a program that can be stored on an electronic device such as a USB flash drive, iPod, memory card, portable hard drive, or other portable electronic device and can be run on a computer or other processing device connected to the electronic device without requiring any permanent configuration changes to the host computer. All of these programs have no footprint, meaning they do not require any user interface changes.All temporary files, registry entries and all other changes to the computer only exist while the program is running.
[0070] To be considered a portable application within the meaning of the present invention, a software program must: does not require formal installation on a permanent storage device of the computer to run, and the software can be stored on a removable storage device such as a USB flash drive, iPod, memory card, portable hard drive or other portable electronic storage device so that it can be used on multiple computers; the settings are stored with the software and can preferably be carried with it (i.e. they are written to an electronic device such as a USB drive); the settings are not stored in the registry or any other central system database of the computer; Leave a zero (or near-zero) footprint on any PC it is run on after use, meaning all temporary files / registry settings should either be avoided or at least removed once the program is closed, and user-created files can be saved directly to the same removable media on which the application is stored.
[0071] A portable application does not leave its files or settings behind on the host computer on which it is running. For example, the application does not write to the Windows Registry, nor does it save its configuration files (e.g., an INI file) in the user's profile; instead, the configuration files are stored in the program directory. Since file paths often differ on different computers due to different Windows drive letter mappings, another requirement is that applications save them in a relative format. Preferably, such a program does not require a startup program that copies the required settings and files to the host computer when the application starts and moves them back to the application directory when it exits, as this could leave residue on the hard disk in the event of a power failure.
[0072] Electronic meeting systems (EMS) must be distinguished from traditional groupware on the one hand, and web conferencing systems on the other. In reality, there is some overlap between the products in these categories. The main difference from groupware is the intensity of collaboration. EMS must be distinguished from systems that allow the content of a single computer screen to be displayed on a remote screen with multiple users simultaneously.
[0073] Groupware supports collaboration within groups, while individual contributions remain recognizable. In contrast, EMS enables the group to collaboratively achieve a result for which the group as a whole is responsible. In a business process, groupware and electronic meeting systems complement each other: Groupware supports teams in researching and creating documents prior to an EMS meeting or in implementing the results of such a meeting.
[0074] "Web conferencing systems" and "electronic meeting systems" complement each other in online meetings or workshops: EMS extends the web conferencing system with interactive tools for creating and documenting group results. On the other hand, "web conferencing systems" complement EMS with screen sharing and voice conferencing functions, which are required in synchronous online meetings and are not available in EMS.
[0075] "Data conferencing" refers to a communication session between two or more participants who exchange computer data in real time. Interaction and presentation devices such as a screen, keyboard, mouse, camera, etc., can be shared. This term is used to distinguish between video and audio conferencing. The data can include screens, documents, graphics, drawings, and applications that can be viewed by the conference participants.
[0076] Application sharing is an element of remote access encompassed by the term "collaborative software" that allows two or more users to access a shared application or document simultaneously and in real time from their respective computers. Typically, the shared application or document runs on a host computer, and remote access to the shared content is granted to the other users by the host user. Application sharing is to be distinguished from systems that do not allow collaboration between different users but allow the content of individual computer screens to be projected onto a remote screen shared by multiple users simultaneously.
[0077] The term "arbitrary media content" refers to the fact that a user can generate, create, or select any media content suitable for display. This is distinct from one client voting or selecting media content to be displayed by another in a meeting or presentation. This term refers to client-centric, distributed rights and privileges for displaying content, rather than a central moderator who provides content to be presented to meeting participants.
[0078] “Unified communications system(s) or tools” refers to audio or audiovisual communications, such as those provided by the services known as Skype and Skype for Business. Such software may accept audio and / or visual data provided by a host processing device. A unified communications tool can be described as a collection of tools for VOIP, (web) conferencing, collaborative whiteboarding, messaging (e.g., chat), file transfer, or presence. Unified communications systems or tools may utilize a protocol- or standards-defined or specific communication session or interaction, such as Voice-Over-Internet Protocol (“VoIP”), text or instant messaging (e.g.,AlM, Blauk, eBuddy, Gadu-Gadu, IBM Lotus Sametime, ICQ, iMessage, IMVU, Lync, MXit, Paltalk, Skype, Tencent QQ, Windows Live Messenger or MSN Messenger, Wireclub, Xfire and Yahoo! Messenger), email, X (formerly Twitter), Digital Service Protocol (DSP), and the like. Unified communications systems or tools may utilize a video conferencing cloud service that includes a video conferencing node to enable one or more users located at a first video conferencing endpoint to communicate with one or more users located at a second video conferencing endpoint in a video conference.
[0079] "Screen scraping" in the context of the present invention refers to the reading of video frame buffers and their processing, not just their display on a screen. Screen scraping for presentations is described in US 2002 / 0196378 A1, to which reference is made in this regard.
[0080] “Auto-composition” or “auto-layout” refers to the automatic rendering of multiple graphics / video sources on a central display without user intervention, in the way a user would intuitively expect.
[0081] "Wireless" and "wireless communication network" can refer to any network that does not use wired connections between nodes, e.g., using RF, optical, or infrared communication, such as IrDA, diffuse infrared, WLAN, WiMax, WiFi, WiFi Direct, LiFi, ultrasonic, ZigBee, or Bluetooth, or any other wireless communication network known to those skilled in the art. Any communication network described below can be, and preferably is, a wireless network. This enables the sharing or combination of at least a selection of functionalities / endpoints from secondary peripheral devices, i.e., the one-to-many scenario.
[0082] "Computer" generally refers to a processing device, that is, a processing machine capable of performing various types of digital processing, such as rendering graphic images for display. A computer can be in the form of a workstation, personal computer, laptop, palmtop, PDA, smartphone, tablet, etc. A computer usually has volatile memory (RAM). Non-volatile memory such as a hard disk, optical disk, or solid-state storage may be integrated into the computer or be a peripheral device. Currently, most computers are electronic, but the term "computer" also includes optical-based computing devices.
[0083] The term "pre-installed generic driver" refers to a driver that is installed on a processing device such as a computer as a default driver, e.g., installed when the operating system is installed. Such a driver is a standard driver for the operating system and can control a standard class of peripheral devices that are coupled or connected to the processing device. The installation of a specific driver for such a peripheral device is not required. Such a generic driver can be a driver for a human interface device (HID) or a driver for mass storage devices that has pre-defined software components configured to control a mass storage device, CD-ROM, keyboard, etc., or combinations of these devices.Such devices can be readable and writable computer peripheral storage devices such as USB memory sticks, flash memory, external hard drives, or others.
[0084] “Host Device” means a processing device that seeks to share information using a first peripheral or dongle.
[0085] A "functional device" is a second peripheral connected in some way to a base unit. The system is capable of making second peripherals connected to the base unit transparently accessible to the first peripheral, as if it were connected to the processing device to which the first peripheral is connected. This accessibility can be implemented using descriptor fields. A functional device can be operated solely with a manufacturer-specific driver.
[0086] "Endpoints" can be described as data sources or sinks and are defined for USB devices, which can be physical devices or virtual devices. For the purposes of the present invention, endpoints should be understood in the broadest sense as data sources or sinks. Thus, data can be stored or output at an endpoint. An endpoint can be defined as a type of buffer for physical devices or virtual devices. The data stored at an endpoint can either be received from a host, such as a USB host, or waiting to be sent to it. An endpoint is defined by a series of descriptive fields. For example, an endpoint can be configured to support different transfer types, e.g., four transfer types. These transfers can be as defined in the USB specification, e.g.,Control transfers, interrupt transfers, isochronous transfers, and bulk transfers. Those skilled in the art can use these transfer types as needed in the present invention. Within the limits of the hardware, the endpoints can be configured using middleware, e.g., USB middleware. An endpoint can be restricted to a specific transfer type. For example, the client of a USB host can send data to endpoint 1. Coming from the USB host, the data is sent to OUT endpoint 1. The running software then reads the data when it is ready. Returning data must be written to IN endpoint 1 because the software cannot freely access the USB bus, as the USB bus is controlled by the USB host. The data in IN endpoint 1 remains there until the host sends an IN packet to endpoint 1 requesting the data.
[0087] The number of endpoints can be limited. Each endpoint can have a transfer direction. For example, a specific endpoint can be defined only for control transfers and not assigned to any other function.
[0088] Control transfers can be bidirectional transfers reserved for the host to send or request configuration information to or from a device through a specific IN and OUT endpoint.
[0089] Each control transfer can involve a series of transactions. Control transfers can have multiple stages: 1. The SETUP stage transports a setup packet that defines the request and specifies how much data should be transferred in the DATA stage. 2. The DATA stage is optional. If present, it always begins with a transaction containing a DATA1 packet. Then, the transaction type alternates between DATA0 and DATA1 until all required data has been transferred. 3. The STATUS stage is a transaction containing a zero-length DATA1 packet. If the DATA phase was IN, then the STATUS phase is OUT, and vice versa.
[0090] Interrupt transfers have a limited latency to or from a device. With USB, an interrupt transfer or interrupt pipe has a defined polling rate. Interrupt transfer data typically consists of event notifications, characters, or coordinates from a pointing device.
[0091] Isochronous transmissions are used for transmitting real-time information such as audio and video data that can be sent at a constant rate. Isochronous USB data streams are allocated a specific portion of the USB bandwidth to ensure data can be transmitted at the desired speed. An isochronous line sends a new data packet in each frame, regardless of the success or failure of the last packet. Isochronous transmissions do not require error detection. An error in the electrical transmission is not corrected.
[0092] Bulk transfers are used for data that is not of the control, interrupt, or isochronous type. Reliable data exchange is ensured at the hardware level through error detection.
[0093] Bulk transfers consume all the bandwidth available after other transfers complete. If the bus is heavily loaded, a bulk transfer may be delayed.
[0094] “A specific device exposes a peripheral or other device” means that the specific device configures one or more endpoints with specific descriptor fields. Description of preferred embodiments
[0095] Embodiments of the present invention enable a wireless connection between a user processing device, such as a laptop, computer, PDA, smartphone, etc., and a base unit of a wireless network. The wireless connection may be established through an integrated wireless transceiver in the user processing device, such as the laptop, computer, PDA, or smartphone, etc. In some embodiments of the present invention, the wireless connection may also be established via a first peripheral device that is connected, coupled, connected, or attached to the user processing device in some way, or provided by other hardware or software.
[0096] Embodiments of the present invention enable users, e.g., participants in a meeting room, to utilize at least one functional device, some functional devices, or all of the functional equipment connected to the base unit, wherein the at least one functional device has an electronic output, such as a digital electronic output, and can interoperate with other devices plugged into, paired with, connected to, or in any way connected to the base unit, or is provided by other hardware or software. The at least one functional device can be provided either by plugging in a second peripheral device, e.g., a dongle, or by a wirelessly connected second peripheral device, or by connecting to the at least one functional device with software that can be downloaded or pre-installed.The at least one functional device may have an electronic output, such as provided by a touchscreen, a microphone, a speakerphone, or a camera, and may function with devices having an electronic input, such as a display or a speaker. Embodiments of the present invention provide the functional device for shared use by participants in a classroom or meeting room.
[0097] The at least one functional device coupled, connected, or attached to the base unit, e.g. via a serial connection such as a plug-and-play serial connection, e.g. USB, or via a wireless connection such as a near-field communication (NFC) connection, e.g. a Bluetooth connection, an IR connection, etc., is made accessible and usable by one or more users in the meeting room or classroom. This is preferably done in a transparent and standardized manner. In some cases, the system may behave like a wireless (e.g. USB) hub; in other cases, the first peripheral device or other hardware or software makes a virtual device accessible that has the same functionality as or more than the combined functionalities of the functional devices, e.g. the second peripheral devices coupled, connected, or attached to the base unit.
[0098] Embodiments of the present invention utilize a base unit, wherein the base unit comprises at least a transmitting function and typically a transmitting and receiving function, such as a transceiver function. The base unit is a base unit of a network, e.g., a wireless network, and may therefore comprise a transmitter, a receiver, or a transceiver. The base unit may be a hardware box comprising a digital processing engine such as an FPGA, a CPU, a GPU, etc., as well as memory and input / output ports, and is configured to be connected, e.g., by plugging into any functional device, such as one or more second peripherals that may be used in a meeting room or classroom with or without visual or audio capabilities.Functional devices, such as one or more second peripherals that can be used with the base unit, can be one or more of a television or other video device that provides images such as video via an HDMI, DVI, USB or other connection, an image capture device such as a camera, e.g. to provide captured images such as video via a USB or HDMI or other connection, audio devices such as hands-free devices that provide audio such as voice or music via a USB connection or other connection, or speakers via a USB connection that use e.g. a 3.5 mm audio jack, SPDIF or other connection, or one or more microphones that use e.g. a 3.5 mm jack, USB or other connection, display devices such as touchscreens that use e.g.use a USB or other connection, all-in-one devices via a USB connection and any other functional device or first peripheral that can be connected to the first device such as the base unit, e.g. by plugging in, and that has, e.g. an electronic output such as a digital electronic output.
[0099] Embodiments of the present invention comprise a device such as a first peripheral device that includes at least a receiving function and typically a transmitting and receiving function such as a transceiver function. The first peripheral device (which is at least a receiving device) may be hardware, e.g., having a wireless or wired connectivity, software, or a combination of both, that can be connected to the base unit via a wireless connection, such as a Wi-Fi, Bluetooth, Wi-Gig, Li-Fi, NFC, or similar connection, or a wired connection, e.g., a USB, Ethernet, etc. connection. This enables the base unit to make any functional device, such as a second peripheral device connected to the base unit, accessible and available to one or more first peripheral devices (e.g., receiving devices) for simultaneous communication with a plurality of first peripheral devices (e.g.,receiving devices) in any of the following ways:. 1. By making the or each functional device (such as the or each second peripheral device) connected to the base unit accessible in a one-to-one manner or in a one-to-many manner to a plurality of the first peripheral devices or user processing devices, e.g. one or more first peripheral devices or user processing devices, preferably without interpreting or processing the electronic signals from each functional device, e.g. the second peripheral device, e.g. by tunneling the packets from the base unit to the endpoints of the one or more of the first peripheral devices or user processing devices. 2. By interpreting, processing, and translating the electronic signal coming from the at least one functional device, e.g., the second peripheral, and, e.g., imitating the at least one functional device, e.g., the second peripheral, on the first peripheral side (e.g., on the receiving device side), either in software or hardware, or both. Imitating means that the same device as the second peripheral connected to the base unit is presented at the endpoints, although communication is not simply tunneled, as in the methods described above, which can be carried out thanks to the system and device aspects of the invention. 3. By combining one or more functional devices, e.g. second peripheral devices or a part of a second peripheral device, and making them accessible to the one or more first peripheral devices, e.g. receiver devices, by a customized device in hardware, software or both. 4. By combining one or more functional devices, e.g. second peripheral device(s) or part of a second peripheral device, and making its functionalities available to the end user through proprietary software. 5. By combining any of the above-mentioned methods feasible thanks to the system and device aspects of the invention to provide the function of the functional device, e.g. the second peripheral device, to one or more first peripheral devices, such as receiving devices.
[0100] When a functional device, e.g. a second peripheral, is exposed natively or as an emulation of such a device in hardware, endpoints such as USB endpoints on the first peripheral or the user processing device can be used, which can expose the corresponding second peripherals, such as USB devices connected to or plugged into the base unit (as in a hub), so that no proprietary software or drivers need to be installed to support this.
[0101] When making custom devices accessible, ie devices specifically designed to work with the present invention, the same endpoints of the first peripheral devices, such as the receiver's hardware, are used, or this can be done in software with a driver.
[0102] A first peripheral device can be used to establish a connection, such as a wireless or wired connection, between a first peripheral device and the base unit, e.g., by acting as a hub such as a wireless hub, of which a wireless USB hub is an example. Any second peripheral device connected or plugged into or on the base unit, e.g., a second peripheral device selected from the above non-exhaustive list of a television, video device, audio device, hands-free device, camera, screen, etc., is accessible to the first peripheral device in one of the 5 ways described above.
[0103] Advantages of embodiments of the present invention may include one, some, or all of the following: - In one implementation, all second peripherals, which may be USB devices, are mirrored one-to-one, so that no additional drivers or software other than the drivers from the manufacturer of the second peripheral need to be installed on the user's processing device. The second peripherals can be made accessible to the user's processing device either directly through the hardware of the first peripheral or using a virtual software hub on the first peripheral or on the user's processing device. - In another implementation, various second peripherals (e.g., speakers, microphones, e.g., with electronic outputs and / or inputs) can be combined and connected to the same base unit. Additional functions such as echo cancellation, translation, encoding / decoding, etc.) can be provided in the base unit. Only a second peripheral needs to be made accessible to the user's processing device via the first peripheral. - A combination of the two aforementioned approaches can be deployed for a single device or a combination of devices or features. Within a final solution, different approaches can be used depending on whether the device or feature needs to be available to multiple users simultaneously, whether simple device mirroring is technically feasible, etc. - By adding some software to the client software running on the user-side processing device, the configuration for these devices can be simplified (e.g. they can be set as default devices) and some custom controls (mute, pan / tilt / zoom, ...) can be added. -- Some controls can be added to the first peripheral. -- Some processing can be performed in the base unit to eliminate the need for such controls (e.g., automatic gain control for an audio signal, framing participants to eliminate the need for pan / tilt / zoom). - The functional devices can be used by multiple users, unlike conventional hubs, where only one person can plug in a cable. Any user-side device connected to a first peripheral can use these functional devices, and the base unit ensures that all streams are correctly captured, played, processed, combined, and / or enhanced. - Different streams can be assigned different priorities (speed, throughput, ...) to adapt to the unpredictable characteristics of the wireless communication channel, e.g. to prioritize audio streams over video streams to improve the call experience. - In a VOIP call, such as a Skype call in a meeting, the first peripheral device is connected to, e.g. plugged into, a user processing device, and any functional devices connected to the base unit can be used, e.g. a better camera allowing a wider view of the room, a better microphone and better speakers).
[0104] This is particularly advantageous when there are several people in the room who want to participate in the conference call, e.g. a VOIP or Skype call.
[0105] With reference to Fig.1, group 90 is a microprocessor- or FPGA-based device comprising one or more peripherals 91-93 that can be connected to, e.g., plugged into, a base unit 100. The peripherals 91-93 can be connected to the base unit 100 via standard connections such as serial connections, Firewire (IEEE-1394), PCI, Mini PCI, PCI Express, Mini PCI Express, PCMCIA, PC Card, Universal Serial Bus (USB), SDIO cards, HDMI, DisplayPort, Bluetooth, etc.
[0106] Group 90 can be used to access e.g. the peripherals: - a webcam 91, - a hands-free system 92, - a set of HID endpoints 93.
[0107] The base unit 100 has an electronic digital processor, such as a microprocessor or an FPGA-based device, and includes, for example: - a web interface 110, which is an interface for the base unit 100 in which the system can be configured; - one or more drivers 111, e.g. two specific drivers for the webcam 91 and the hands-free device 92, running on the base unit 100 to receive or send data from / to the group 90 and the USB peripherals 91-93; - additional processing is performed in blocks 112, 114, 115 for audio and webcam coding, mixing, (de-)multiplexing and other processing; - the data can be encrypted or decrypted in the encryption / decryption devices 113, 116 before being sent to the first peripheral device 130 via a wireless connection 127; - the control modules 117 in the base unit 100 and 37 in the first peripheral device 130 use a control channel 128 between the base unit 100 and the first peripheral device 130 to perform various functions such as maintenance, identification, exchange of crypto keys, etc.; - the unified protocol block 118 is a functional block that abstracts the main functions of the system: screen sharing and other non-media streams (e.g., mouse pointer); - the encryption and decryption modules 119, 120 are intended for the main communication with the first peripheral device 130 and other applications; - an encoder, such as the H264 encoder 121, and the local display block 123 (which is a functional device or devices since it sends a signal from the base unit 100 to the user processing device 160) are provided to capture the rendered image on the main display 126, encode it and send it back to the first peripheral device 130 and other clients; - audio processing blocks 124 are provided for audio content; - the video decoder 125 is used to decode and display the screen contents on the display 126.
[0108] The first peripheral device 130 is a microprocessor- or FPGA-based device, which includes, for example: - Webcam video coming from the webcam 91 of the base unit 100 is decrypted in block 30 and fed to the processing device 160 via a USB video endpoint using USB video and drivers such as H264 drivers 31. - Audio signals from the hands-free device 92 connected to the base unit 100 are decoded in block 32, processed and made available as a microphone signal from the USB hands-free device endpoint 92. - Audio signals from the processing device 160, coming via a USB hands-free endpoint, are encrypted in block 35 and sent to the base unit 100 via the wireless connection 127. - A control HID 51 (Human Interface Device) is a mirror of additional HID USB endpoints that the group 90 and the second peripheral devices 91-93 have made accessible to the base unit 100. - A manufacturer-specific HID endpoint, referred to as a data pump 42, is used for a channel 52, which may be a high-bandwidth data channel between manufacturer-specific software, such as software 70 running on processing device 160, and base unit 100 via first peripheral 130. Screen video and audio signals for display 126 may be sent via this channel 52. This is preferred over transmission via an HID control endpoint in first peripheral 130. Also, the screen input of display 126, which is captured (block 123) and encoded (block 121) in base unit 100, is sent to the manufacturer-specific software running on processing device 160 so that other users, e.g.a plurality of users or participants in a meeting, each having a processing device 160 and optionally a first peripheral device 130 connected, can display the contents of the display 126 on a display (not shown) on their own processing device 160. - A number of basic components such as the operating system (OS) and the drivers for controlling optical displays such as an LED ring on the first peripheral device 130, a bandwidth measurement module and other Wi-Fi modules, update / upgrade modules e.g. over the air, a secure boot module, a secure key store, a debug and test module, etc. are included in blocks 45 to 50. - The audio capture module 43 makes the speaker endpoint accessible to the processing device 160. The module 43 records this audio data and sends it back to the manufacturer's software 70 running on the processing device 160, where it can be timestamped (block 69) and compared with the video data, and then sent to the base unit 100, for example, via the data pump 42. This enables better time stamping and lip synchronization. - In a less preferred embodiment, audio data may be captured and processed on the first peripheral device 130 and sent directly to the base unit 100.
[0109] A microprocessor or FPGA processing device 160, such as a computer, laptop, smartphone, personal assistant (PDA), etc., runs proprietary software 60, 70, including, for example, third-party software (e.g., Skype) 60. - The third-party software 60 runs on the processing device 160 (e.g., communications software such as Skype) that utilizes the video endpoint of the at least one functional device 91-93 (e.g., the webcam 91 connected to the base unit 100) and / or the hands-free device endpoints (both hands-free microphone and hands-free speaker). - Block 70 comprises software developed for the present invention and used to: -- a pipeline to decode (61) and decode (62), which then displays the screen data of the display 126 of the base unit 100 in a window of the software. The data following the arrow 71 leaving the last stage (local view 63) is captured by the communications software, such as the Skype client software 60. -- The unified protocol block 64 is a module that handles the control layer and controls the media paths, the connection to the first peripheral device 130 and (indirectly) the connection to the base unit 100. -- Pipeline 65-67 is used for screen capture (67), encoding (66) and encryption (65) for sharing the screen with base unit 100. -- The pipeline 68, 69 receives the speaker audio signals (block 68) from the first peripheral device 130, processes them (e.g., timestamp, block 69) and sends them out via the HID endpoint (the “data pump” 52). - Endpoints not used by the software in this case are the control HIDs of the second peripherals. If software from the manufacturer of a second peripheral is running, this software can communicate with the second peripheral in question via these HID endpoints (e.g., pan / tilt / zoom the camera). - The speaker endpoint (the audio content arrow 53) is used by the operating system (OS) of the processing device 160 to send all (non-speakerphone) tones / audio signals. Connecting a first peripheral device to a user processing device
[0110] As in Fig.2, a first peripheral device 130 can be connected to a user processing device 160 by means of a method designated as a whole by 300, which is feasible thanks to the system and device aspects. The connection can be a serial connection, such as a USB connection, such as USB 1, 2, 3 or -C, etc., and the peripheral device 130 is connected, e.g., plugged into a serial plug-and-play port as provided for a USB connection. The first peripheral device 130 can include a user input device, e.g., an actuator. The first peripheral device 130 can be configured as a connection unit and can be a physical device in the form of a connector for a plug-and-play interface of a user processing device. It can have a base and a flexible data and power connection connected to the base. The base can include the actuator, e.g.,a button as an input device that can be operated by a user.
[0111] The flexible data and power connector should be designed so that the base lies flat on the conference table, regardless of the orientation of the plug, which is required for inserting the plug into the plug-and-play interface. The base preferably includes electronics and optionally has a permanent memory for storing an application, e.g., a wearable application. The network configuration parameters for the WiFi connection to the base unit 100 can also be stored in a memory. The first peripheral unit can have a processing unit (e.g., CPU, GPU, FPGA), a wireless transmitter / receiver, such as for WiFi or LiFi, a plug-and-play interface, such as a USB interface, a light ring, such as an LED ring or strip as a visual indicator. The wearable application can be installed on the first peripheral device 160, e.g.,in the base, or it can be downloaded, received from another pluggable device such as a flash drive, downloaded from the base unit 100, etc. The visual indicator serves to provide the user with feedback from the connection unit about the status of an activity.
[0112] Some examples of actuator activation that can be used with any of the embodiments of the present invention: sound activation (clapping hands, voice recognition, computer sound, music, ...), remote control via a wirelessly connected device (IR, Bluetooth, WiFi, LiFi ...), light activation, pressure activation, e.g. by pressing down with the finger or hand, touch activation, proximity ("near-touch" on the actuator or proximity of the actuator to an object), biometric reader such as fingerprint reader, iris scanner, DNA analyzer, keypad, e.g. for entering a keycode or a password.
[0113] The method 300 comprises, in step 301, connecting the first peripheral device 130 to a processing device 160, e.g., by plugging it in. This leads to a kernel boot step 302 and an exchange of initialization scripts in step 303. The first peripheral device 130 is recognized as a USB device in step 306, and the USB protocol stack is loaded in step 304, followed by loading the protocol stack for the wireless connection in step 305. The parameters for the wireless connection are entered, e.g., from a memory of the first peripheral device 130 or by the user, and the assignment of the wireless connection is completed in step 307. Any client software that needs to be loaded onto the processing device 160 is now loaded and started, e.g., by a user action, in step 308. The first peripheral device 130 is now ready in step 310 to exchange data with a display 126 connected to the base unit 100.
[0114] As in Fig. 3, a startup program, which is stored, for example, on the first peripheral device 130 or on the processing device 160 or can be retrieved from a remote location such as a URL on the Internet or from a LAN server, can automatically start the client software in step 328. Steps 321 to 327 and 329 and 330 are the same as steps 301 to 307 and 309 and 310 in Fig. 2.
[0115] The client software can be, for example, the software 70 from Fig.1 running on the processing device 160. Other software such as communications software 60, e.g., Skype, may be pre-installed on the processing device 160. The client software 70 may be for processing video data from the processing device 160, and this client software is executed on the processing device 160, e.g., using a processing engine such as a CPU, GPU, or FPGA. The client software 60 may be screen scraping software that scans the screen of the processing device 160. This client software 60 may be stored as a software program in a memory, e.g., a mass storage device on the first peripheral device 130. This software program is executed, e.g., loaded onto the processing device 160, when connected to the latter, either automatically or through a user action as described above.When software 60 is executing on processing device 160, it is preferred that it leave no footprint upon termination. Software 60, when executing on processing device 160, captures the video data available on processing device 160, e.g., from a presentation or video running on processing device 160. For example, software 70, when executing on processing device 160, is capable of scanning the screen, e.g., to read one or more video frame buffers from a graphics card in processing device 160. Generally, processing device 160 has its own screen and thus a graphics card or equivalent with a buffer for storing video data. This read video data can be timestamped (in block 69) to later synchronize the captured video signal with a corresponding audio signal.The video signal is then encoded in a video encoder (block 66) for transport (arrow 53) via the plug-and-play interface using a generic driver, e.g., via a USB interface with generic pre-installed drivers, e.g., a generic pre-installed driver for a human interface device (HID). The data is transmitted to the wireless communication network 127, e.g., a WiFi or LiFi network, and forwarded via the communication network 127 to the base unit 100 and inserted into a suitable composition, e.g., an OpenGL-based composition in a compositor, for display on the device 126. Sending data from the processing device to the base unit
[0116] With reference to Fig.4, the audio data is sent to the processing device 160, e.g., a computer, laptop, smartphone, PDA, tablet, etc., via a connection with generic drivers, e.g., via a USB port 8, using the standard built-in generic audio driver, e.g., the UAC driver 7. On the first peripheral device 130, the audio packets from the generic port, e.g., the USB port 11, are read by a dedicated audio device 14. These packets are then processed by a mixer, rate converter, echo canceller, noise suppressor, or the like. Mixing, rate conversion, echo cancellation, and noise suppression can be performed using an ALSA driver 18. The ALSA driver forwards the packets to a dummy audio device 16, e.g., an audio scraper.In this audio device 16, the audio packets can be time-stamped by the clock 15, which may be synchronized with the clock 9 of the processing device 31. This information is then encoded in an encoder 17 and packetized in a packetizer 19 before being transmitted to the communications network between the first peripheral device 130 and the base unit 100. The network may be a wireless network such as a WiFi or LiFi network. For this purpose, a suitable transmitter 21 is provided in the first peripheral device 130. On the base unit 100, the audio information stream is retrieved at a suitable communications interface such as the WiFi access point 22. The audio is then unpacked in an unpacker 23, decoded in a decoder 24, and subsequently presented to an audio mixer 28.To synchronize the audio and video streams, both are sent to a synchronizer 27, where the two streams are synchronized, thus maintaining lip synchronization. Sending data from the processing device to the base unit
[0117] Embodiments of the present invention can capture data such as visual or audio signals and bring them to one or more processing devices 160 so that this captured data can be shared with other participants. These visual or audio signals are captured, provided, or received by functional devices such as a webcam 91, a speakerphone 92, or even the display 126. A functional device could, for example, be a microphone connected to a suitable location in a meeting to ensure good reception of audio data. This can be used with a central speaker system, which may be of higher quality than those provided in the processing devices 160 of the meeting participants. Such a speaker system could, for example, be a quadraphonic surround speaker system.This also helps avoid the difficulties associated with multiple microphones and speakers, which can lead to disruptive positive feedback.
[0118] Embodiments of the present invention may include the Fig.1 to 4 to provide additional functionality that can be shared, for example, by all participants in a meeting who have a processing device 160 optionally equipped with a pluggable first peripheral device 130. In embodiments of the present invention, fixed USB endpoints are provided on the first peripheral device 130 for basic functionality. These are fixed and represent a combination of manufacturer-specific endpoints and a number of standard endpoints and can be interpreted or understood as a custom driver, a standard operating system driver and / or a host application and share screen and audio data, as described with reference to Fig. 4 was described.
[0119] The configurable USB endpoints are configured either when pairing a first peripheral device 130 with a base unit 100 or via the wireless connection 127 between the processing device 130 and the base unit 100. Once configured, the USB endpoints expose second peripheral devices 91-93, which can be interpreted as custom drivers, vendor-specific drivers, or standard operating system drivers. These can be used either by a host application or by third-party applications. These endpoints can be an exact copy of those of the second peripheral device 91-93 connected to the base unit 100, or they can be dedicated, specified endpoints.
[0120] Communication of these USB endpoints with the processing device 160 can be achieved by enumerating the USB endpoints by default when the first peripheral device 130 is connected to the processing device 160, e.g., by plugging it in. All information about how to communicate with the base unit 100 and which endpoints to make accessible can be stored on the first peripheral device 130 and configured by the firmware of the first peripheral device 130.
[0121] The decision as to which endpoints the first peripheral device 130 should make accessible may be determined by the base unit 100, which may be configured to analyze which second peripheral devices are connected to the base unit 100 (the connection may be via a USB connection or otherwise) and create a configuration for the corresponding USB endpoint on the first peripheral device 130.
[0122] As mentioned above, in embodiments of the present invention, data may flow in either direction during operation, ie, to or from the processing device 160: 1. Data from a second peripheral device connected to the base unit 100 may flow into the base unit 100 or be captured in the base unit 100, e.g., as data for the display 126. All of this data may be interpreted and possibly processed, enhanced, encoded, and / or encrypted using the firmware of the base unit 100 to generate processed data. In some embodiments, this processed data is then sent to the first peripheral device 130 via a wireless connection 127. This first peripheral device 130 will, if necessary, decode, process, and / or interpret the data and make it available via one or more USB endpoints of this first peripheral device 130 (e.g., either the fixed or the configured ones).This data is then captured by the operating system of the processing device 160 and made available via a custom or standard driver to either the host application or a third-party application running on the processing device 160 or other processing devices 160 of other participants in the meeting. 2. Data, which may originate from the host application and / or one or more third-party applications running on the processing device 160, can be sent via a system-provided handle. The data is then sent via the (generic standard) system drivers and the USB port to the corresponding USB endpoint of the first peripheral device 130. The first peripheral device 130 then receives this data via the firmware of the first peripheral device 130, where it processes, enhances, encodes, and / or encrypts the data and sends it to the base unit 100 via the wireless connection 127.The base unit 100 receives the data via the firmware of the base unit 100, decrypts, processes, decodes and / or enhances the data and forwards it to the corresponding first peripheral device, which can be connected or connected to the base unit 100 via a serial connection such as USB, HDMI or another interface.
[0123] Such a meeting with several participants is in Fig.6 is schematically illustrated with four processing devices 160 (160-1, 160-2, 160-3, 160-4), which may be laptops. Three of the processing devices (160-1, 160-2, 160-3) are each connected to a first peripheral device 130, e.g., by plugging into a USB connection. A processing device 160-4 may be the host of a unified communications (UC) call, such as a Skype call or a Skype for Business call. The first peripheral devices of two of the processing devices (160-1, 160-2) have established a WiFi connection with a base unit 100 and are passing media to the main display 126. The processing devices 160-3 and 160-4 share the unified communications (UC) call, such as the Skype call or a Skype for Business call.
[0124] One or more of the following is / are connected to the base unit 100: - a webcam 91, - a 4K touchscreen 94, - a camera 95, connected via USB or HDMI, - a USB hands-free device 92, - a set of USB microphone 96 and speaker set 97 (connected via an audio jack, s / pdif, USB, HDMI, etc.)
[0125] UC client software is preinstalled on the host processing device 160-4. This software interacts with USB endpoints on the first peripheral device 130, which is connected to the host, for example. This type of client software uses USB endpoints for speakerphones as the standard audio device (e.g., microphone plus speaker). Embodiments of the present invention control all audio or visual signals output from and input to the processing device 160.
[0126] The hands-free device 92 can be used by any user with at least a first connected and functioning peripheral device 130. No action is required to access this device. Separate controls can be provided on a first peripheral device, for example, to adjust the volume of the audio output of the base unit 100 and to mute the room microphone 96 if necessary. When muted, the microphone 96 is also muted for all other buttons or apps that use this device.
[0127] The UC client software 60 running on the host processing device 160-4 does not input any audio content. Microphone 96 can capture this signal, or the base unit 100 can input it into the microphone signal to the first peripheral device 130 connected to the host 160-4. Because the latency is low, lip synchronization with the video content is not required (even when shared).
[0128] The same type of second peripheral as the one plugged into the base unit 100 is accessible to a user so that the user can use the drivers provided by the manufacturer.
[0129] If one of the processing devices 130 is shared in the room, the audio content is also shared in that room.
[0130] A processing device with a functioning first peripheral device can wirelessly access the room camera 95. All other devices connected to the base unit 100 (e.g., HID control devices 93) are made natively accessible.
[0131] Any user connected to the system, for example, via a first peripheral device, has the ability to view the content projected in the meeting room on a display on the user's processing device—i.e., a "local view." The recipient of the local view may have some or all of the following features, but is not limited to: The ability to open a window and view meeting room media content in that window, whether it's the stream itself, annotation, or a whiteboard session. The ability to enlarge the content. The function can be supplemented with additional measures to ensure image quality when zooming The local view could allow the user to start and / or join a whiteboard or annotation session from their own device The ability to take screenshots and / or pause the local view.
[0132] The Fig. Figures 6 to 15 and 17 show ways in which various functions can be made available to participants in a room, e.g., in a meeting. There are four basic methods that are feasible thanks to the system and device aspects of the invention: a) Using a generic driver of the operating system 164 of the host processing device 160 and endpoints such as mass storage or HID endpoints to enable the transfer of data using client software running on the host processing device 160 without the host processing device 160 knowing that the end device is a functional device such as a camera, touchscreen, microphone, speakerphone, speakerphone, etc., connected in or to the base unit 100. b) Having an endpoint on the first peripheral of a functional device connected to or in the base unit 100, which means that the host processing device 160 detects that it is connected to the functional device and interacts with it as if it were such a device. c) Having an endpoint on the first peripheral of a functional device connected to or in the base unit 100, which means that the host processing device 160 detects that it is connected to the functional device, but that the client software must have the manufacturer-specific drivers to interact with the functional device. d) Having an endpoint on the first peripheral of a functional device connected to or within the base unit 100, which means that the host processing device 160 detects that it is connected to the functional device, but that preprocessing is performed on the base unit 100 to allow the processing device 160 to interact with the functional device. One method is to establish a common endpoint on the first peripheral so that the processing device can work with any functional device for which preprocessing exists in the base unit 100.
[0133] Fig.6 shows the processing device 160 in communication with a first peripheral device 130 connected thereto and the base unit 100 in wireless communication 127 with the first peripheral device 130. Various functional devices such as a display 126, a hands-free device 92 (or a microphone and speaker set 96, 97), and a video camera 91 (or another camera 95) are connected to the base unit 100.
[0134] A generic driver enables the client software 70 to transmit and receive data to and from a mass storage device 132 of the first peripheral device 130 via a USB interface 131. The client software 70 also has a USB endpoint 134, which may be a USB HID endpoint and, if appropriate, a manufacturer-specific endpoint. This endpoint 134 is the endpoint for the client software 70 to transmit the data for the display 126 to the base unit 100, where the data can be image processed (blocks 124, 125 of Fig.1) before being transmitted to the group 90, e.g., via an HDMI connection or the like. Thus, the HID endpoint 134 makes the HDMI display 126 and its audiovisual signals available to any processing device 160 with a first peripheral device connected thereto when it is located in a meeting room. However, the display 126 is made accessible on the processing device 160, ie, it is not represented as a display, but as an HID endpoint. The video camera 91 can be made accessible on the first peripheral device 130 as an endpoint (in a manner as in Fig. 1), where the video data e.g. in block 122 of Fig. 1. Alternatively, as in Fig.6, the video data is encoded in the base unit 100 and sent via the first peripheral device 130 to a driver 162 of the operating system 164 of the processing device 60, which makes the encoded data from the video camera 91 accessible on the processing device 160 and makes the video data available for the UC call, e.g., Skype.
[0135] Data from the hands-free device 92 can be processed on the base unit 100 and / or on the first peripheral device 130 and presented to the processing device 160. The microphone of the hands-free device 92 generates a signal that is transmitted to the processing device 160 via a USB hands-free device endpoint 136 on the first peripheral device, wherein any noise cancellation or echo cancellation can be performed in the base unit 100, as described with reference to Fig.1 (block 115). The microphone data is provided to the UC call because the hands-free endpoint 136 is recognized by the processing device 160 as a hands-free device, as if the hands-free device were connected to the processing device 160. Furthermore, the microphone data is provided to all other users who have a processing device 160 with a connected first peripheral device. In this embodiment, the speakerphone has its USB endpoint 136 on the first peripheral device 130.
[0136] Fig. 7 shows a similar configuration as in Fig.6, except that there are two USB endpoints 136 and 138 for a speakerphone and a speakerphone. As before, the HID endpoint 134 makes the HDMI display 126 available to any processing device 160 with a first peripheral device 130 connected to it, if located in a meeting room, to deliver audiovisual signals to the display 126. However, the display 126 is not made accessible on the processing device 160; that is, it is not represented as a display, but as an HID endpoint.
[0137] The two endpoints 136, 138 receive audio signals from two different sources. The audio of the UC call can be Fig.6 via the hands-free endpoint 136 and the processing in the base unit. However, if there is additional audio data that is not part of the UC call, this data is sent via the speakerphone endpoint 138. Since there is no second peripheral device in the group 90 that is a standalone speaker, the base unit 100 mixes the audio signals for the hands-free device and the speakerphone before sending all the audio data to the speakerphone of the hands-free device 92. If there were a separate speaker 97 in the group 90, the base unit 100 could send this audio data to the speakerphone. The result is that a virtual speaker is exposed on the processing device 160.
[0138] The Fig.Figures 8 to 11 and 17 describe various embodiments of the present invention using the example of providing a touchscreen 99 in a meeting room. When a touchscreen is touched, it generates a signal related to the coordinates of the touch position. This must be forwarded to the processing devices 160 in a meeting, with the actual signals having to be interpreted if they are to be applied to another screen of the processing device 160. If there is a meeting with a plurality of processing devices being used by the participants, a method must be found to transmit the touch coordinates to all the different processing devices. Fig. Figures 8 to 11 show the four basic procedures mentioned above.
[0139] In the embodiment of Fig.8, there is a group 90, a base unit 100, a first peripheral device 130, and a processing device 160. Two functional devices are connected to or in the base unit 100: a display 126 and a touchscreen 99. The touchscreen 99 has a touchscreen endpoint 139 in the first peripheral device 130. The processing device 160 recognizes this endpoint and can interact with it if the operating system 164 has the appropriate driver 162. This driver can be downloaded if necessary. This embodiment assumes that each processing device 160 in a meeting has downloaded or installed the appropriate driver 162.
[0140] In the embodiment of Fig.9, there is a group 90, a base unit 100, a first peripheral device 130, and a processing device 160. Two functional devices are connected to or in the base unit 100: a display 126 and a touchscreen 99. The touchscreen 99 has a touchscreen endpoint 139 in the first peripheral device 130. The processing device 160 recognizes this endpoint and can interact with it if the base unit 100 is configured to translate the coordinates of the touch into ones that can be used by the processing device 160, i.e., if the operating system 164 has the appropriate driver 162. This driver can be downloaded if necessary. This embodiment assumes that each processing device 160 in a meeting has downloaded or installed the appropriate driver 162, but the translation function of the base unit 100 can reduce the number of different drivers that may be required.
[0141] In the embodiment of Fig.10 there is a group 90, a base unit 100, a first peripheral device 130 and a processing device 160. Two functional devices are connected to or in the base unit 100, namely a display 126 and a touchscreen 99. The touchscreen 99 has a touchscreen endpoint 139 in the first peripheral device 130. The processing device 160 recognizes this endpoint and can interact with it if the base unit 100 is designed to translate the coordinates of the touch into those that can be used by the processing device 160, ie the operating system 164 has the corresponding driver 162. This driver is a standard driver that can be distributed, e.g. this driver can be downloaded if necessary.In this embodiment, each processing device 160 in a meeting is required to have the appropriate driver 162 downloaded or installed, but the translation function of the base unit 100 can reduce the number of different drivers that may be required even further than in the previous embodiment of FIG. Fig. 9.
[0142] In the embodiment of Fig.11, there is a group 90, a base unit 100, a first peripheral device 130, and a processing device 160. There are two functional devices connected to or in the base unit 100, namely a display 126 and a touchscreen 99. The touchscreen 99 is now handled by the client software 70, as previously described, namely, the client software running on the first peripheral device 130 is configured to interact with the touchscreen 99 via the USB interface 131 and to interact with the operating system 164 of the processing device 160. This means that the touchscreen 99 is not exposed to the processing device 160. The client software 70 is configured to simulate a touch event and communicate the correct coordinates to the operating system 164 of the processing device 160.It is therefore a proprietary solution that requires the client software 70 to be designed to interact with any type of touchscreen 99 via a USB interface 131.
[0143] The Fig. Figures 12 to 14 relate to embodiments of the present invention in which there are multiple users. First, each participant in a meeting can use one of the Fig. 8 to 11. Thus, the present invention provides solutions for multiple users.
[0144] In Fig.12, the processing device 60' does not have a first peripheral device 130, e.g., none is available. In such a case, the client software 70' may be configured to interact directly with the base unit 100 via a wireless connection 127', and the client software 70' is configured to provide any simulations or translations required to present the display device 126 to the processing unit 160' and enable interaction therewith. The client software 70' may be configured to implement any of the Fig. 1 to 11, but without using a first peripheral device 130. The client software 70' may, for example, simulate a USB interface of the type described in one of the embodiments of the Fig. 1 to 11. For the processing device 160, the Fig. The methods described in 1 to 11 may be used.
[0145] In Fig. 13, the processing device 160' does not have a first peripheral device 130, e.g., none is available. In such a case, the client software 70' may be configured to interact directly with the base unit 100 via a wireless connection 127', and the client software 70' is configured to provide any simulations or translations required to present the display device 126 to the processing device 160' and enable interaction therewith, just as with respect to Fig. 12. However, the group 90 has a microphone and speaker combination such as a hands-free device 92 or a microphone and speaker combination 96, 97. The processing device 160 can use any of the embodiments of the Fig.1 to 11. The client software 70' of the processing device 160' may be configured to provide video data directly to the base unit 100, where it is processed, e.g., decoded and assembled, and provided to the display 126 via a suitable connection, such as an HDMI connection.
[0146] For the processing device 160', the client software 70' and a driver 162' can present a simulated hands-free device 92 to the operating system 164', thus creating a virtual microphone and a virtual speaker. The third-party software 60' for unified communications, e.g., Skype, receives these virtual devices, which it then uses for the call.
[0147] In Fig.14, the processing device 160' does not have a first peripheral device 130, e.g., none is available. In such a case, the client software 70' may be configured to interact directly with the base unit 100 via a wireless connection 127', and the client software 70' is configured to provide any simulations or translations required to present the display device 126 to the processing unit 160' and enable interaction therewith, just as with respect to Fig. 13. However, the group 90 has a video camera 91, which is connected to the base unit 100 via a USB connection, for example. The processing device 160 can be any of the embodiments of the Fig. 1 to 11. The client software 70' of the processing device 160' may be configured to receive video data directly from the base unit 100.
[0148] For the processing device 160', the client software 70' and a driver 162' can present a simulated video camera 91 to the operating system 164', thus creating a virtual video camera device. The third-party software 60' for unified communications, e.g., Skype, receives all virtual devices, which it then uses for the call.
[0149] Fig. Figure 15 shows a message flow diagram for each of the Fig. 1 to 11 described embodiments.
[0150] Fig. Figure 16 shows a message flow that is used for each of the Fig. 12 to 14 is applicable when no first peripheral device is used.
[0151] Fig.17 shows another embodiment of the present invention, including a processing device 160 having a processor and memory, and executing and optionally storing client software 70, which also includes a host application and a third-party application 60, which may be configured to execute a unified communications call, such as a Skype call or a Skype for Business call. The processing device 160 has a serial port, e.g., a USB port, and generic or custom drivers for communicating via the port with a first peripheral device 130.The peripheral device 130 has a processor and memory, and executes and optionally stores firmware that provides a wireless connection 127 to a base unit 100 and provides at least one configurable or fixed endpoint of a functional device 90 connected to or connected within the base unit 100. The base unit has a processor and memory for executing and optionally storing firmware for providing a connection to the wireless connection 127 and also for providing a port for the functional device 90. The functional device 90 has a processor and memory, and executes and optionally stores firmware for providing the connection and interface to the base unit 100 and for providing data to be sent to the at least one configurable or fixed endpoint of the peripheral device 130.
[0152] Methods performable thanks to the system and device aspects of the invention can be carried out by a computer system. According to the present invention, a processing engine can be used to perform certain functions. The processing engine preferably has digital processing capabilities, such as those provided by one or more microprocessors, FPGAs, or a central processing unit (CPU) and / or a graphics processing unit (GPU), and is capable of performing the respective functions by being programmed with software, i.e., one or more computer programs. The term "software" can encompass any type of program in any language that can be executed directly or indirectly by a processor, either via a compiled or an interpreted language.The execution of the methods that are feasible thanks to the system and device aspects of the invention can be performed by logic circuits, electronic hardware, processors or circuits, which can include any type of logic or analog circuits integrated to any degree and are not limited to general-purpose processors, digital signal processors, ASICs, FPGAs, discrete components or transistor logic gates and the like.
[0153] A processing device may include memory (e.g., a non-transitory computer-readable medium, RAM and / or ROM), an operating system, optionally a display (e.g., a fixed-format display), ports for data input devices (e.g., a keyboard), a pointing device (e.g., a mouse), serial or parallel ports for communication with other devices, network cards, and connections for connection to any network.
[0154] The software may be embodied in a computer program product configured to perform the functions of any of the methods performable by the system and device aspects of the invention, e.g., as listed below, when the software is loaded onto a computer and executed on one or more processing engines such as microprocessors, ASICs, FPGAs, etc. Thus, any of the embodiments of the present invention may comprise a computer system capable of executing one or more computer applications in the form of computer software.
[0155] The methods performable by the system and device aspects of the invention described above can be carried out by one or more computer application programs running on the computer system by being loaded into memory and running on or in conjunction with an operating system, such as the operating system known as Windows and provided by Microsoft Corp., or other operating systems such as the operating systems known as Linux and Android. The computer system may comprise a main memory, preferably random access memory (RAM), and also a non-volatile hard disk drive and / or a non-volatile removable storage device and / or a non-volatile solid state storage device. A non-volatile removable storage device may be an optical disk such as a compact disc (CD-ROM or DVD-ROM) or a magnetic tape read and written by a suitable reader.Non-volatile memory can be a computer-readable medium on which computer software and / or data are stored. Non-volatile memory can be used to store persistent information that should not be lost when the computer system is shut down. Application programs can access and store information in non-volatile memory.
[0156] The software embodied in the computer program product is designed to perform the following functions when the software is loaded onto the corresponding device or devices and executed on one or more processing machines such as microprocessors, ASICs, FPGAs, etc.: Connecting a processing device to a functional device connected to or in a base unit of a communications network, the processing device having a memory, a display and an operating system, the base unit having a transmitter and the first peripheral device having a receiver; coupling a first peripheral device to the processing device via a generic communication protocol, wherein at least one fixed or one configurable endpoint of the functional device is provided on the first peripheral device; Transmitting data from the base unit and receiving the data at the first peripheral device via the communication network from the functional device to the processing device via the at least one fixed or configurable endpoint using the generic communication protocol for communication between the processing device and the first peripheral device.
[0157] The software embodied in the computer program product is designed to perform the following functions when the software is loaded onto the corresponding device or devices and executed on one or more processing machines such as microprocessors, ASICs, FPGAs, etc.: Transmitting or receiving data, i.e. streaming data carrying images, audio, etc., in a raw or unaltered format; Connecting the functional device to or in a base unit of a wireless communication network; Operating the functional device using only the manufacturer-specific driver.
[0158] The software embodied in the computer program product is designed to perform the following functions when the software is loaded onto the corresponding device or devices and executed on one or more processing machines such as microprocessors, ASICs, FPGAs, etc.: Accessibility implemented with descriptor fields; Transferring the data from the functional device to at least two processing devices using at least two first peripheral devices; Combining two endpoints and making them accessible as one endpoint to the processing device.
[0159] The software embodied in the computer program product is designed to perform the following functions when the software is loaded onto the corresponding device or devices and executed on one or more processing machines such as microprocessors, ASICs, FPGAs, etc.: the functional device unlocks one or more of a microphone, a hands-free device, a loudspeaker, a display, a touchscreen, a projector, a camera, a video camera, a webcam; Representing the at least one fixed or configurable endpoint of the functional device made accessible on the first peripheral device as one of a user interface device, a mass storage device, a composite device, a microphone, a hands-free device, a speaker, a display, a touchscreen, a projector, a camera, a video camera, or a web camera.
[0160] The software embodied in the computer program product is designed to perform the following functions when the software is loaded onto the corresponding device or devices and executed on one or more processing machines such as microprocessors, ASICs, FPGAs, etc.: Encoding and / or optionally encrypting the data; hosting unified communication between two or more other processing devices on the processing device; the first peripheral device represents a functional device for unified communication between two or more processing devices; or Making the same type of functional device connected to the base unit accessible to the processing device, and using at least one driver for the functional device installed on the processing device.
[0161] Any of the above-mentioned software may be implemented as a computer program product compiled for a processing machine in one of the servers or nodes of the network. The computer program product may be stored on a non-transferable signal storage medium such as an optical disk (CD-ROM or DVD-ROM), a digital magnetic tape, a magnetic disk, a solid-state storage device such as a USB flash drive, a ROM, etc.
[0162] Embodiments of the present invention can also be implemented as a collaboration platform for meetings. This collaboration platform comprises the system according to any one of claims 1 to 13 and a software component for managing access to the functional devices 90. The collaboration platform is designed to coordinate the simultaneous access of multiple processing devices 160 to the functional devices 90 via the wireless communication network 127. This enables efficient and flexible collaboration in meeting scenarios where multiple participants need to access shared resources simultaneously. The software component can implement functions such as resource allocation, access control, and synchronization of device functions to ensure smooth collaboration.
[0163] If one analyzes the invention, some of its special features and characteristics can also be listed from a different perspective as follows: a) A system for connecting a processing device to a functional device connected to or in a base unit of a communications network, the processing device having a memory, a display and an operating system, the system comprising: a first peripheral device configured to be coupled to the processing device via a generic communication protocol, wherein the base unit comprises a transmitter and the first peripheral device comprises a receiver and at least one fixed or configurable endpoint of the functional device made accessible on the first peripheral device; the base unit and the first peripheral device are configured to send and receive data, respectively, via the communication network from the functional device to the processing device via the at least one fixed or configurable endpoint using the generic communication protocol for communication between the processing device and the first peripheral device. (b) A system having the features specified in point (a), wherein the data sent or received, optionally streaming data carrying images, audio or similar, may be in a raw or unmodified format. c) A system having the features referred to in points a) or b), wherein the functional device is connected to or in a base unit of a wireless communication network. d) System with features according to any of the above points a) to c), wherein the accessibility is implemented with descriptor fields. e) A system having features as defined in any of points a) to d) above, wherein the functional device can be operated solely with the manufacturer-specific driver. f) A system having features according to any one of points a) to e) above, which is designed to transmit the data from the functional device to at least two processing devices using at least two first peripheral devices. (g) A system having features according to any of points (a) to (f) above, wherein two endpoints are combined and made accessible to the processing device as one endpoint. h) A system having features according to any one of points a) to g) above, wherein the functional device is one or more of a microphone, a hands-free device, a loudspeaker, a display, a touchscreen, a projector, a camera, a video camera, a web camera. i) A system having features according to any one of points a) to h) above, wherein the at least one fixed or configurable endpoint of the functional device made accessible on the first peripheral device is one of a user interface device, a mass storage device, a composite device, a microphone, a hands-free device, a speaker, a display, a touchscreen, a projector, a camera, a video camera, or a web camera. j) System having features according to any one of points a) to i above, further comprising means for coding and optionally for encrypting the data. k) A system having features according to any one of points a) to j) above, wherein the processing device is configured to host unified communication between two or more further processing devices. l) A system having features according to any one of points a) to k) above, wherein the first peripheral device is designed to represent a functional device for unified communication between two or more processing devices. m) System having features according to any one of points a) to i) above, designed to make the same type of functional device connected to the base unit accessible to the processing device, and further comprising at least one driver for the functional device installed on the processing device. n) A system having features according to any one of points a) to m) above, wherein the functional device is a second peripheral device. (o) A system having features according to any of points (a) to (n) above, wherein the functional device is a data acquisition device. p) A peripheral device configured to be coupled to a processing device via a generic communication protocol, the peripheral device comprising a receiver and at least one fixed or configurable functional device endpoint made accessible on the first peripheral device; wherein the receiver of the first peripheral device is configured to receive data from the functional device via a communication network and to send the data to the processing device via the at least one fixed or configurable endpoint using the generic communication protocol for communication between the processing device and the peripheral device. q) Peripheral device having the features specified in point p), where the communication network is a wireless communication network. r) Peripheral device having features according to any of the above points p) of q), wherein the data being sent or received, i.e. streaming data carrying images, audio, etc., may be in a raw or unmodified format. s) Peripheral device with features according to one of the above points p) to r), whereby the functional device can be operated solely with the manufacturer-specific driver. t) A peripheral device having features according to any one of points p) to s) above, wherein the at least one fixed or configurable endpoint of the functional device made accessible on the first peripheral device is one of a user interface device, a mass storage device, a composite device, a microphone, a hands-free device, a loudspeaker, a display, a touchscreen, a projector, a camera, a video camera, or a web camera. Translation of the text in the drawings Fig. 1A Low latency audio Lower audio latency Audio processing: echo cancellation, double stream detection and resampling Audio processing: echo cancellation, dual stream detection and resampling Audio processing: resampling Audio processing: resampling Audio, video over HDMI Audio, video via HDMI Base unit Base unit Button control module Button control module Control Channel Control channel Display Display H264 encoder H264 encoder H264, MJPEG video H264, MJPEG video HID HID Local View Screen Grabber Screen grabber of the local view Microphone microphone Multiplex Salsa and encrypt Multiplexing Salsa and encrypting Peripheral(s) Peripheral device(s) Redesigned web interface Rebuilt web interface Roomdock switch Roomdock switch Salsa and decrypt demux Salsa and decryption demultiplexing Salsa decrypt and demux Salsa decryption and demultiplexing Speakerphone Hands-free system UDP or TCP Datapump UDP or TCP data pump Unified protocol Unified protocol USB audio driver USB audio driver USB Peripheral USB peripheral device UVC UVC V4l2h264 driver V4l2h264 driver Video decoder Video decoder Fig.1B Audio capture Audio recording Audio Latency and resample Manager Audio Latency and Resample Manager Low latency audio Lower audio latency Audio return Audio return Authentication module Authentication module Bandwidth measurement and feedback Bandwidth measurement and feedback Button button Button control module Button control module Content Audio Audio content Control Channel Control channel Control HID Control HID Datapump USB TCP / UDP Data pump USB TCP / UDP Debug and test console Debug and test console Decrypt audio Decrypt audio Encrypt audio Encrypt audio Fig. 1 continued Fig. 1 Continuation H264, MJPEG video H264, MJPEG video HID Datapump HID data pump Rotary ring or button screen Rotating ring or button screen Secure Boot Safe boot Secure Key Storage Secure key storage Skype Audio out Skype audio output Skype audio in Skype audio input UCV MJPEG, H264 UCV MJPEG, H264 UDP or TCP Datapump UDP or TCP data pump Unified protocol Unified protocol Unified Protocol Channel Channel for unified protocol Unified protocol platform Platform for unified protocol Upgrade via USB / WiFi Upgrade via USB / WiFi USB Video, H264 driver USB video, H264 driver Videode crypt Video decryption Voice Audio Voice audio Wifi driver and IP stack WiFi driver and IP stack Fig. 1C Audio receiver Audio receiver Audio Return Audio return Audio / Video Timestamp Audio / video timestamp Client Client Content Audio Audio content Content insert Insert content Control Hid Control HID Fig. 1 continued Fig. 1 Continuation H246 encoder H246 encoder H264 decoder H264 decoder HID Datapump HID data pump Localview insert content ? Insert local view content? Multiplex audio video and encrypt Salsa Multiplex audio and video and encrypt Salsa Salsa Decrypt Deciphering Salsa Screen capture Screen recording Skype Skype Unified protocol Unified protocol UVC MJPEG, h264 UVC MJPEG, H264 Voice Audio Voice audio Fig. 2 Discovered as USB drive Recognized as a USB drive Initialization scripts Initialization scripts Kernel boot finished Kernel boot completed Launch Client Start client Plug-in button Plug-in button Ready to share on screen Ready to share on screen Start loading USB stack Start loading the USB stack Start loading Wi-Fi stack Start loading the Wi-Fi stack System actions System actions USB Reset request received from client USB reset request received from client User actions User actions WiFi association completed WiFi association completed Fig. 3 Current average timing for A + B + D ≈ 28 seconds - in stand-alone mode with launcher Current average time for A + B + D ≈ 28 seconds - in standalone mode with launcher Discovered as USB drive Recognized as a USB drive Initialization scripts Initialization scripts Kernel boot finished Kernel boot completed Launch Client Start client Plug-in button Plug-in button Ready to share on screen Ready to share on screen Start loading USB stack Start loading the USB stack Start loading Wi-Fi stack Start loading the Wi-Fi stack System actions System actions USB Reset request received from client USB reset request received from client User actions User actions WiFi association completed WiFi association completed Fig. 4 ALSA ALSA Audio Decode Decode audio Audio driver Audio driver Audio pack Audio package Audio unpack Unpack audio Base unit Base unit Client PC Client PC Client software Client software Clock Clock Composer composer Encoder encoder External USB dongle External USB dongle HD driver HD driver HD protocol pack HD protocol package HD protocol unpack Unpack HD protocol LCD & button control LCD & button control Mass storage Mass storage mixer mixer Screen scraper Screen scraper Sync Sync USB audio device USB audio device USB HID device USB HID device USB port USB port Video Decoder Decode video Video encoder Video encoder Video memory Video memory Video pack Video package Video scraper Video scraper Video unpack Unpack video WIFI AP WIFI AP WIFI transceiver WIFI transceiver Fig. 5 Hello Hello Fig. 6 3rd Party client Third-party client Base Unit Base unit Client Client Display Display Driver driver Echo canceling noise reduction Echo cancellation Noise reduction Encoding Coding Image processing Image editing Mass Storage Mass storage Mic microphone OS Operating system Peripherals Peripherals Receiver Recipient Speaker Speaker Speakerphone Hands-free system USB USB USB interface USB interface User Laptop User laptop Vendor HID Manufacturer HID Video Cam video camera Wi-Fi Wi-Fi Fig. 7 3rd Party client (eg Skype4Business) Third-party client (e.g. Skype4Business) Base Unit Base unit Client Client Display Display Echo canceling noise reduction sound mixing Echo cancellation Noise reduction Sound mixing Encoding Coding Function of the (HDMI) screen is offered to the end user through the use of the Client The function of the (HDMI) screen is offered to the end user through the use of the client HDMI HDMI Image processing Image editing Mass Storage Mass storage Mic microphone Native device is mimicked to user's laptop: The laptop sees the same endpoint as is plugged into the Base Unit, but processing and interpretation of the stream is done in the Base Unit (and / or receiver) The native device is replicated on the user's laptop: the laptop sees the same endpoint as it is connected to the base unit, but the processing and interpretation of the stream occurs in the base unit (and / or the receiver). OS Operating system Peripherals Peripherals Receiver Recipient Speaker Speaker Speakerphone Hands-free system USB USB USB interface USB interface User Laptop User laptop Vendor HID Manufacturer HID Video Cam video camera Virtual device is exposed to user's laptop The virtual device is made accessible to the user's laptop Wi-Fi Wi-Fi Fig. 8 3rd Party client Third-party client Base Unit Base unit Client Client Display Display Driver driver OS Operating system Peripherals Peripherals Receiver Recipient Touchscreen interface Vendor X Touchscreen interface manufacturer X Translate XYZ Translate XYZ USB USB USB interface USB interface User Laptop User laptop Wi-Fi Wi-Fi Fig. 9 3rd Party client Third-party client Base Unit Base unit Client Client Display Display Driver driver OS Operating system Peripherals Peripherals Receiver Recipient Touchscreen interface Vendor X Touchscreen interface provider X Translate XYZ Translate XYZ USB USB USB interface USB interface User Laptop User laptop Wi-Fi Wi-Fi Fig. 10 3rd Party client Third-party client Base Unit Base unit Client Client Display Display Driver driver OS Operating system Peripherals Peripherals Receiver Recipient Touch screen interface BARCO Touchscreen interface BARCO Touchscreen interface Vendor X Touchscreen interface manufacturer X Translate XYZ Translate XYZ USB USB USB interface USB interface User Laptop User laptop Wi-Fi Wi-Fi Fig. 11 3rd Party client Third-party client Base Unit Base unit Client Client Display Display Driver driver HDMI HDMI OS Operating system Peripherals Peripherals Receiver Recipient Touchscreen interface Vendor X Touchscreen interface provider X Translate XYZ Translate XYZ USB USB USB interface USB interface User Laptop User laptop Wi-Fi Wi-Fi Fig. 12 3rd Party client Third-party client Base Unit Base unit Client Client Display Display Driver driver HDMI HDMI Image processing Image editing Mass Storage Mass storage Mic microphone OS Operating system Peripherals Peripherals Receiver Recipient Speaker speaker USB USB USB interface USB interface User 2 Laptop User 2's laptop User Laptop User laptop Vendor HID Manufacturer HID Video Cam video camera Wi-Fi Wi-Fi Fig. 13 3rd Party client Third-party client Barco Speaker Barco speakers Barco Speaker Phone Barco hands-free system Base Unit Base unit Client Client Display Display Driver driver Echo cancellation mixing noise reduction Echo cancellation Mixing Noise reduction HDMI HDMI Image processing Image editing Mass Storage Mass storage Mic microphone OS Operating system Peripherals Peripherals Receiver Recipient Speaker Speaker USB USB USB interface USB interface User 2 Laptop User 2's laptop User Laptop User laptop Vendor HID Manufacturer HID Video Cam video camera Wi-Fi Wi-Fi Fig. 14 3rd Party client Third-party client Barco Speaker Barco speakers Barco Speaker Phone Barco hands-free system Base Unit Base unit Client Client Display Display Driver driver Echo cancellation mixing noise reduction Echo cancellation Mixing Noise reduction HDMI HDMI Image processing Image editing Mass Storage Mass storage Mic microphone OS Operating system Peripherals Peripherals Receiver Recipient Speaker Speaker USB USB USB interface USB interface User 2 Laptop User 2's laptop User Laptop User laptop Vendor HID Manufacturer HID Video Cam video camera Wi-Fi Wi-Fi Fig. 15 3rd party application Third-party application 3 rd party application uses virtual USB devices Third-party application uses virtual USB devices Barco application shares screen Barco application splits screen Base Unit Base unit Client software Application Client software application Driver configuration Driver configuration Connect to USB endpoint (eg speakerphone or camera) Connection to USB endpoint (e.g. hands-free device or camera) Data decode / processing / encryption Decode / process / encrypt data Data exchange Data exchange Data forward / processing / encryption Data Exchange Data forwarding / processing / encryption Data exchange Data processing / decryption / decode Data Exchange Data processing / decryption / decoding Data exchange Data processing / encoding / encryption Data processing / coding / encryption Driver installed on PC Driver installed on the PC Enumerate USB endpoints Enumerate USB endpoints Execute software Run software Initialization initialization Initialize the driver Initialize the driver Load drivers for endpoints Load drivers for endpoints PC operating system PC operating system Peripheral Peripheral device Set up wireless connection with Base Unit Setting up the wireless connection with the base unit Setting up virtual USB endpoints Setting up virtual USB endpoints Start screen sharing Start screen sharing Start up software on Laptop and connect to Base Unit Start software on the laptop and connect to base unit Fig. 16 3rd party application Third-party application 3 rd party application uses peripheral exposed by USB device Third-party application uses peripherals made accessible by a USB device Barco application shares screen Barco application splits screen Base Unit Base unit Client Application Client application Connect to HID USB endpoint to set up control channel Connect to HID USB endpoint to set up control channel Connect to USB endpoint (eg speakerphone or camera) Connect to USB endpoint (e.g. hands-free device or camera) Data decode / processing / encryption Decode / process / encrypt data Data exchange Data exchange Data forward / processing / encryption Data Exchange Data forwarding / processing / encryption Data exchange Data processing / decryption / decode Data Exchange Data processing / decryption / decoding Data exchange Data processing / encoding / encryption Data processing / coding / encryption Driver installed on PC Driver installed on PC Enumerate USB endpoints Enumerate USB endpoints Execute software Run software Fetch software from USB Device Mass Storage if not installed on PC Get software from USB mass storage device if it is not installed on the PC Initialization initialization Insert device. Boot up OS and Configure USB Interface Plug in the device. Boot the operating system and configure the USB interface. Insert USB device into Laptop Plug USB device into laptop Load drivers for endpoints Load drivers for endpoints PC operating system PC operating system Peripheral Peripheral device Set up wireless connection with Base Unit Set up wireless connection with the base unit Start screen sharing Start screen sharing USB device detected. Enumerate USB endpoints USB device detected. Enumerate USB endpoints Fig. 17 3rd party application Third-party application Base Unit Base unit Configurable USB endpoints Configurable USB endpoints Firmware Firmware Fixed USB endpoints Fixed USB endpoints Host application Host application PC PC System drivers System driver USB Device USB device USB Peripheral USB peripheral device USB port USB port Wireless link Wireless connection QUOTES CONTAINED IN THE DESCRIPTION
[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature
[0000] EP 3 099 009 A1
[0006] US 2016 / 0125838 A1
[0007] WO 2008 / 080107 A1
[0008] WO 2012 / 128972 A1
[0009] WO 2013 / 037980 A1
[0063] US 2002 / 0196378 A1
[0079]
Claims
[1] A system for making one or more functional devices (90) in a meeting room available to a processing device (160) of a meeting participant, wherein: - the functional devices (90) are connected to a base unit (100) of a wireless communication network (127), - the functional devices (90) have one or more fixed or configurable Universal Serial Bus (USB) endpoints, wherein an endpoint comprises a data source or sink, - the processing device (160) has a memory, a display and an operating system (164), - the base unit (100) comprises at least one transmitter and one digital processing unit, - a first peripheral device (130) comprises a receiver, wherein: - the first peripheral device (130) is adapted to be coupled to the processing device (160) via a serial connection, - the first peripheral device (130) and the processing device (160) are configured to communicate via a generic communication protocol over the serial connection, - the base unit (100) and the first peripheral device (130) are configured to make the functional devices (90) accessible and available to the first peripheral device (130) by making at least one of the fixed or configurable endpoints of the functional devices (90) accessible by configuring one or more endpoints with descriptor fields on the first peripheral device (130), - the base unit (100), the first peripheral device (130) and the processing device (160) are configured to transmit data from the functional devices (90) to the base unit (100), from the base unit (100) to the first peripheral device (130) via the wireless communication network (127), and from the first peripheral device (130) to the processing device (160) via the accessible fixed or configurable endpoints using the generic communication protocol. [2] The system of claim 1, wherein the transmission device is configured to transmit or receive data carrying images and / or audio in a raw or unaltered format. [3] The system of claim 1 or 2, wherein the transmission means is configured to transmit the data from the functional device to at least two processing devices, including the processing device and another processing device, using at least two first peripheral devices. [4] The system of any one of claims 1 to 3, wherein the base unit (100) and the first peripheral device (130) are configured to combine two endpoints of one or more functional devices and make them accessible on the first peripheral device as one endpoint for the processing device. [5] The system of any one of claims 1 to 4, wherein the base unit (100) and the first peripheral device (130) are configured to make the same type of functional device accessible to the processing device as is connected to the base unit and to use at least one driver for the functional device installed on the processing device. [6] The system of any one of claims 1 to 5, wherein the base unit (100) and the first peripheral device (130) are configured such that the at least one fixed or configurable endpoint of the one or more functional devices exposed on the first peripheral device is presented as one of a human interface device, a mass storage device, a composite device, a microphone, a hands-free phone, a speaker, a display, a touchscreen, a projector, a camera, a video camera, or a webcam. [7] System according to one of claims 1 to 6, further comprising an encoding and / or encryption device for encoding and / or optionally encrypting the data. [8] The system of any one of claims 1 to 7, wherein the processing device (160) is configured to host a unified communication between two or more further processing devices, the unified communication comprising an audio or audiovisual communication using a protocol- or standard-defined or specific communication session or interaction of software that accepts audio and / or visual data from the processing device. [9] The system of claim 8, wherein the first peripheral device (130) is configured to present a functional device for unified communication between the two or more processing devices. [10] The system of claim 8 or 9, wherein the unified communication comprises a Skype call or a Skype for Business call. [11] The system of any one of claims 8 to 10, wherein the processing device (160) is configured to share data from other processing devices in the unified communication. [12] The system of any one of claims 1 to 11, wherein the base unit (100) comprises processing means configured to provide additional functions such as echo cancellation, translation, encoding / decoding to the functional devices (90). [13] The system of any one of claims 1 to 12, wherein the first peripheral device (130) comprises a controller configured to provide separate controls for the volume of an audio output of the base unit (100) and for muting a room microphone. [14] Communication system for meeting rooms, comprising: - the system according to one of claims 1 to 13, - at least two processing devices (160), - at least two first peripheral devices (130), and - a base unit (100), wherein the communication system is configured to enable shared use of the functional devices (90) by the at least two processing devices (160) via the wireless communication network (127). [15] An apparatus comprising a processor and a memory, the memory containing instructions which, when executed by the processor, cause the apparatus to implement the system of any one of claims 1 to 13. [16] A processing device (160) for use in a system according to any one of claims 1 to 13, the processing device comprising: - a memory, - an advertisement, - an operating system (164), - a serial interface for connecting the first peripheral device (130), - wherein the processing device (160) is configured to communicate with the first peripheral device (130) via a generic communication protocol and to receive data from the functional devices (90) via the accessible fixed or configurable endpoints. [17] The processing device of claim 16, further comprising client software (70) configured to capture, encode, and send screen contents to the base unit (100) via the first peripheral device (130). [18] A first peripheral device (130) for use in a system according to any one of claims 1 to 13, comprising: - a receiver for wireless communication with the base unit (100), - a serial interface for connection to the processing device (160), - wherein the first peripheral device (130) is configured to make at least one of the fixed or configurable endpoints of the functional devices (90) accessible by configuring one or more endpoints with descriptor fields. [19] The first peripheral device of claim 18, further comprising separate control means for controlling the volume of an audio output of the base unit (100) and for muting a room microphone. [20] Base unit (100) for use in a system according to any one of claims 1 to 13, comprising: - at least one transmitter for wireless communication with the first peripheral device (130), - a digital processing unit, - Interfaces for connecting the functional devices (90), - wherein the base unit (100) is configured to receive data from the functional devices (90) and transmit this data to the first peripheral device (130) via the wireless communication network (127). [21] The base unit of claim 20, further comprising processing means configured to provide additional functions such as echo cancellation, translation, encoding / decoding to the functional devices (90). [22] Meeting room equipment, including: - the system according to one of claims 1 to 13, - a plurality of functional devices (90) for shared use, wherein the meeting room equipment is configured so that multiple participants can access the functional devices (90) simultaneously with their respective processing devices (160) and first peripheral devices (130). [23] Collaboration platform for meetings, including: - the system according to one of claims 1 to 13, - a software component for managing access to the functional devices (90), wherein the collaboration platform is designed to coordinate the simultaneous access of multiple processing devices (160) to the functional devices (90) via the wireless communication network (127).
Citation Information
Patent Citations
Electronic tool and methods for meetings
EP3099009A1
Method and apparatus for wireless image transmission to a projector
US20020196378A1
Systems and methods for transmitting video, network, and USB signals over extension media
US20160125838A1
Enhanced wireless USB protocol and hub
WO2008080107A2
Method and system for USB device virtualization
WO2012128972A1
