Methods and systems for enabling meeting participants to use functional devices.

Abstract

To provide a method, a device, a system, a program, and a storage device that allow a meeting participant to use a functional device.SOLUTION: In a computer system, a parent unit 100 includes a transmitter and a user laptop 160 includes a memory, a display, and an operating system. A peripheral device 90 is coupled to a user laptop via a generic communication protocol and includes a receiver 130 and at least one fixed or configurable endpoint of a functional device (video cam 91, speakerphone 92) exposed on the peripheral device. The parent unit 100 and the peripheral device each transmit and receive data through a communication network from the functional device to the user laptop via at least one fixed or configurable endpoint using the generic communication protocol for communication between the user laptop and the peripheral device.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] The present invention relates to a method, a device, and a system for enabling meeting participants to use functional devices, and software for executing such a method.

Background Art

[0002] Background Enabling the communication of a group of people for a specific purpose is one of the fundamental aspects such as collaboration, problem-solving, negotiation, teaching, and education. There has been a rapid increase in electronic communication tools for supporting communication, such as synchronous and asynchronous meetings, online chats, instant messaging, voice conferences, video conferences, data conferences, application sharing, remote desktop sharing, electronic meeting systems, co-management (coordination) tools, project management systems, knowledge management systems, and social software systems.

[0003] As a typical method, there are lectures or seminars that often involve presentations using presentation software. In many cases, traditional presentations or lectures by a single person have been given to a somewhat passive audience as far as the determination, assembly, enhancement, or modification of the information presented.

[0004] Similar to other business processes, meetings are moving towards digitalization. By people using computer technology alone and in combination with broadband networks, there is an increasing tendency to support the purpose of the meeting before or during the actual meeting. For example, by distributing files using email, people can read them before the meeting.

[0005] However, certain features, such as microphones and speakers, tend to be of lower quality when provided by portable devices like laptops and mobile phones. As meetings grow larger, the need for high-quality audio and video signals becomes increasingly important for meeting participants. [Overview of the project] [Means for solving the problem]

[0006] Summary of the Invention Embodiments of the present invention relate to methods, devices, and systems for enabling meeting participants to use functional devices, as well as software for performing such methods.

[0007] Embodiments of the present invention provide a system for connecting a processing device to a functional device connected to or located within a master unit of a communication network, wherein the processing device comprises memory, a display, and an operating system, and the system is The master unit comprises a first peripheral device configured to be coupled to the processing device via a generic communication protocol, the master unit having a transmitter, and the first peripheral device having a receiver and at least one fixed or configurable endpoint of the functional device exposed on the first peripheral device. The master unit and the first peripheral device described above are, respectively, the processing device and the first peripheral device. The device is configured to transmit and receive data through the communication network from the functional device to the processing device via at least one fixed or configurable endpoint, using the generic communication protocol described above for communication with peripheral devices.

[0008] The above communication network is preferably a wireless network. This would enable "sharing or combining" at least a selection of functions / endpoints of a second peripheral device, i.e., a one-to-many scenario.

[0009] The streaming data that is transmitted or received, i.e., images, audio, etc., may be in its raw or unmodified format.

[0010] There may be two or more first peripheral devices. For example, data can be transmitted from a functional device to at least two processing devices using at least two first peripheral devices.

[0011] Functional devices may operate using only vendor-specific drivers. The above-mentioned functional device may also be a data generating device such as one or more of the following: microphone, speakerphone, speaker, display, touchscreen, projector, camera, video camera, or webcam.

[0012] The at least one fixed or configurable endpoint of the functional device exposed on the first peripheral device may be a human interface device.

[0013] The processing device may be configured to communicate with the at least one fixed or configurable endpoint using a pre-installed generic driver, which is a human interface device driver.

[0014] The at least one fixed or configurable endpoint of the functional device exposed on the first peripheral device may be a mass storage device.

[0015] The processing device may be configured to communicate with the at least one fixed or configurable endpoint using a pre-installed generic driver, which is a mass storage device driver.

[0016] The at least one fixed or configurable endpoint of the functional device exposed on the first peripheral device may be a composite device.

[0017] The processing device may be configured to communicate with the at least one fixed or configurable endpoint using a pre-installed generic driver, which is a composite device driver.

[0018] Embodiments of the present invention include combining endpoints, and the present invention includes a system in which two endpoints are combined and exposed to the processing device as a single endpoint.

[0019] The feature known as "exposure" or "enumeration" refers to the first peripheral device. This determines which endpoint (for example, a button) sees. Regarding "exposure," this can be achieved with a "descriptor field." The ability of a device to expose peripheral devices or other devices means that the particular device comprises one or more endpoints with a specific descriptor field or “descriptive field”. The dot point is defined by multiple descriptive fields.

[0020] The basic form of the descriptive field is "USB over IP," and the actual transfer of unmodified data can be carried out by tunneling.

[0021] Exposure may be realized in a "descriptor field". An endpoint is the capability / function of a "physical device" (a second peripheral device or a functional device). For example, if a telephone is a second peripheral device, it may have several endpoints, such as a microphone or a loudspeaker or an IP connection, etc.

[0022] The "distinction" between an endpoint and a second peripheral device / functional device defines the endpoint.

[0023] "Embodiments of the present invention" "provide" a system that enables the use of the original vendor's driver for each functional device (and related endpoints), for example, in the context of mirroring. Thus, a functional device can operate with only vendor-specific drivers.

[0024] This system may include means for encoding data and optionally encrypting it. Thus, this system may include an encoder for encoding data and optionally encrypting it.

[0025] The processing device may be configured to host unified communication between two or more processing devices. The unified communication may be a Skype (registered trademark) call or a Skype (registered trademark) for Business call. The first peripheral device may be configured to provide a functional device for unified communication between two or more processing devices. For example, the functional device may be a speakerphone, a microphone, a speaker, a video camera, a webcam, a camera, or other audio sources.

[0026] This system may be configured to share data from other processing devices in a meeting in the call. By using a manual action, this sharing performed by the person or host who shares may be enabled via a local view window.

[0027] The system may be configured to select a camera, for example, via a manual action in the UC client. The system may also be configured to synchronize the sharing state of the first peripheral device with the sharing state of the UC client.

[0028] Sharing features / endpoints may also involve sharing a room display. Combining resources may involve using two room loudspeakers (for example, to improve quality), however, in this case, the user can only see "one room loudspeaker." The latter also includes the characteristic that the above combination occurs if there are two or more participants / users in the room. Sharing can also be thought of as selecting features / endpoints for specific feature devices.

[0029] Embodiments of the present invention may use multiple endpoints, at least one of which is a fixed or configurable endpoint, and sharing is possible for multiple processing devices. This is in relation to the `S`, meaning that the use of "multiple first peripheral devices" (e.g., buttons) is also supported.

[0030] This system may be configured to provide a multi-user wireless speakerphone. A first peripheral device can be connected to each processing device so that all processing devices can access this speakerphone during a meeting. No coordination or action is required to access this device.

[0031] The system may be configured to provide separate controls, for example, by providing control to peripheral devices, it may control the volume of the master unit's audio output and mute room microphones. Muting one microphone may also mute all other peripheral devices connected to the processing device in the meeting.

[0032] The audio signal must be picked up by a microphone, or the master unit may be configured to inject the audio signal into a first peripheral device connected to the processing device in any audio feed, such as a microphone signal.

[0033] This system may be configured to expose to the user the same type of functional device connected to the master unit, and to allow the user to use drivers provided by the vendor of the functional device installed on the processing device.

[0034] This system may be configured such that, when multiple processing devices communicate with the master unit, a functional device connected to the master unit is exposed to all of the aforementioned peripheral devices. This functional device may be exposed from the start.

[0035] Exposure may be achieved using a descriptor field. The system may be configured to enable any processing device connected to a communication network via a first peripheral device to view any audiovisual content displayed, provided, or projected on that processing device in a meeting room, referred to as a "local view."

[0036] A processing device that receives a local view may have, but is not limited to, the following functions: 〇A function that opens a window and allows you to view the meeting room content within it. 〇 A feature to zoom in on content. The local view allows users to join a blackboard session or annotation session from their own processing device.

[0037] This system may be configured to provide alternative methods and systems when there are too few primary peripheral devices in a particular meeting room.

[0038] This system may be configured to issue a warning if a first peripheral device is attempting to connect to the wrong master unit.

[0039] In another aspect, the present invention provides a method for connecting a processing device to a functional device connected to or located within a master unit of a communication network, wherein the processing device comprises memory, a display, and an operating system, the master unit comprises a transmitter, the first peripheral device comprises a receiver, and the method is as follows: The first peripheral device is connected to the processing device via a generic communication protocol. , a step of providing at least one fixed or configurable endpoint of the functional device exposed on the first peripheral device, The process includes the steps of transmitting data from the master unit through the communication network from the functional device to the processing device via the at least one fixed or configurable endpoint, using the generic communication protocol for communication between the processing device and the first peripheral device, and receiving the data at the first peripheral device.

[0040] The above communication network is preferably a wireless network. This would enable sharing or combining at least a selection of functions / endpoints of a second peripheral device, i.e., a one-to-many scenario.

[0041] The streaming data that carries the above-mentioned data being transmitted or received, i.e., images, audio, etc., may be in its raw or unmodified format.

[0042] There may be two or more first peripheral devices. Data can be transmitted from a functional device to at least two processing devices using at least two first peripheral devices.

[0043] Functional devices may operate using only vendor-specific drivers. The above-mentioned functional device may provide one or more of the following: microphone, speakerphone, speaker, display, touchscreen, projector, camera, video camera, or webcam.

[0044] The above method may include providing at least one fixed or configurable endpoint of the functional device exposed on the first peripheral device as one of the following: a human interface device, a mass storage device, a composite device, a microphone, a speakerphone, a speaker, a display, a touchscreen, a camera, a video camera, or a webcam.

[0045] The data may be encoded and / or optionally encrypted. Unified communication between two or more other processing devices may be hosted on the processing device.

[0046] The first peripheral device may provide a functional device for the unified communication between two or more processing devices.

[0047] The above method may include the steps of exposing a functional device of the same type as one connected to the master unit to the processing device and using at least one driver for the functional device installed on the processing device.

[0048] Exposure may be achieved using a "descriptor field". In another aspect, a peripheral device is provided that is configured to be coupled to a processing device via a generic communication protocol, the peripheral device having a receiver and at least one fixed or configurable endpoint of a functional device exposed on the first peripheral device, The receiver of the first peripheral device described above is configured to receive data from the functional device through a communication network via the at least one fixed or configurable endpoint and transmit the data to the processing device, using the generic communication protocol described above for communication between the processing device and the peripheral device.

[0049] The above communication network is preferably a wireless network. This would enable "sharing or combining" at least a selection of functions / endpoints of a second peripheral device, i.e., a one-to-many scenario.

[0050] The at least one fixed or configurable endpoint of the functional device exposed on the first peripheral device described above may be one of the following: a human interface device, a mass storage device, a composite device, a microphone, a speakerphone, a speaker, a display, a touchscreen, a projector, a camera, a video camera, or a webcam. The exposure may be implemented in a “descriptor field”.

[0051] Functional devices may operate using only vendor-specific drivers. In another context, a computer program product is provided for executing any of the steps of the method of claim 10 when executed on a processor. The computer program may be stored using non-temporary signal storage means. The non-temporary signal storage means may be an optical disc such as a CD-ROM or DVD-ROM, a magnetic disc such as a hard disk, a solid-state memory such as flash memory, a magnetic tape, or something similar. [Brief explanation of the drawing]

[0052] [Figure 1A] This figure shows one embodiment of the present invention. [Figure 1B] This figure shows one embodiment of the present invention. [Figure 1C] This figure shows one embodiment of the present invention. [Figure 2] This figure shows the message flow of how a peripheral device is connected to a processing device in an embodiment of the present invention. [Figure 3] This figure shows the message flow of how a peripheral device is connected to a processing device in an embodiment of the present invention. [Figure 4] This figure shows the configuration of components usable in embodiments of the present invention, and this drawing is Figure 11 of WO2013 / 037980, titled "Electronic tools and methods with audio for meetings," which is incorporated herein by reference with respect to Figure 11 and in whole. [Figure 5] This figure shows the configuration of a processing device used in an ongoing unified communication meeting, according to one embodiment of the present invention. [Figure 6] This figure shows an embodiment of the present invention in which peripheral devices are coupled to a processing device. [Figure 7] This figure shows an embodiment of the present invention in which peripheral devices are coupled to a processing device. [Figure 8] This figure shows an embodiment of the present invention in which peripheral devices are coupled to a processing device. [Figure 9] This figure shows an embodiment of the present invention in which peripheral devices are coupled to a processing device. [Figure 10] This figure shows an embodiment of the present invention in which peripheral devices are coupled to a processing device. [Figure 11] This figure shows an embodiment of the present invention in which peripheral devices are coupled to a processing device. [Figure 12] This figure shows an embodiment of the present invention in which peripheral devices are not coupled to the processing device. [Figure 13] This figure shows an embodiment of the present invention in which peripheral devices are not coupled to the processing device. [Figure 14]This figure shows an embodiment of the present invention in which peripheral devices are not coupled to the processing device. [Figure 15] This figure shows a message flow used in any of embodiments 1 to 11 and 17 according to one embodiment of the present invention. [Figure 16] This figure shows a message flow used in any of embodiments 12 to 14 according to one embodiment of the present invention. [Figure 17] This figure shows yet another embodiment of the present invention. [Modes for carrying out the invention]

[0053] definition "Plug and play" refers to a computer bus or device specification. Plug-and-play is a term used to describe the characteristics of a system that make hardware components easily discoverable and resolve resource conflicts without requiring physical device configuration or user intervention. Plug-and-play devices can be added to the bus of a computing system (either during operation or shutdown), and the newly added device, and possibly the rest of the computing system, are automatically configured to allow the newly added device to function from both a hardware and software perspective.

[0054] Plug-and-play interfaces include, but are not exhaustive, Firewire (IEEE-1394), PCI, Mini PCI, PCI Express, Mini PCI Express, PCMCIA, PC Card, Universal Serial Port (USB), SDIO Card, HDMI®, DisplayPort, Bluetooth®, etc.

[0055] "Auto-configuration" refers to automatic setup without manual intervention, any switch or jumper settings, and the need for software configuration. An example of an auto-configurable device is a USB device. Examples of auto-configuration protocols include DHCP, Zeroconf, and Bonjour.

[0056] Plug-and-play devices have, by default, auto-configuration software to enable plug-and-play functionality. For example, USB devices become plug-and-play by including the correct auto-configuration software (e.g., host driver, host stack, application software). Auto-configuration can be software-only and is not limited to physical devices.

[0057] "Hot-swapping and hot-plugging" are terms used to describe the ability to replace components of a computer system without shutting down the system. More specifically, hot-swapping describes replacing components without major interruptions to the system, while hot-plugging describes adding components that expand the system without major interruptions to the system's operation. A well-known example of this ability is the Universal Serial Bus (USB), which allows users to add or remove peripheral components such as mice, keyboards, or printers. Other examples include eSATA, PCIe, and FireWire.

[0058] Often called standalone applications, "portable applications" (portable apps) are computer software programs designed to run without being installed on the target machine. These types of applications can be stored on removable media such as CDs, USB flash drives, flash cards, or floppy disks. It is stored on a temporary storage device, which stores its program files, configuration information, and data only on the storage medium. This means it is a program that can be stored on an electronic device such as a USB flash drive, iPod®, memory card, portable hard drive, or other portable electronic device, and runs on a computer or other processing device coupled to the electronic device without making any permanent configuration changes to the host computer. All such programs have zero footprint. This means that all temporary files, registry entries, and any other changes to the machine exist only while the program is running.

[0059] For the purposes of this invention, in order to be considered a portable application, the software program is It does not require any formal installation of any kind on a computer's persistent storage device and must be able to be stored on removable storage devices such as USB flash drives, iPods, memory cards, portable hard drives, or other portable electronic storage devices, and therefore be usable on multiple computers.

[0060] The settings are stored with the software and preferably carried with the software (i.e., they are written to an electronic device such as a USB drive). The settings are not stored in the registry or any other central system database of the computer.

[0061] By making the "footprint" on any PC zero (or nearly zero), this is done after use; that is, all temporary files / registry settings must be avoided, or at least deleted once the program is out. Also, user-created files can be saved directly to the same removable media when the application is stored.

[0062] Portable applications do not leave their files or settings on the host computer on which they run. For example, an application does not write to the Windows registry or store its configuration files (such as INI files) in the user's profile. Instead, it stores those configuration files in the program's directory. Another requirement is that file paths often differ on computers due to differences in Windows drive letter assignments, so applications need to store these in the relevant format. Preferably, such programs do not require a launcher program to copy necessary settings and files to the host computer when the application starts and return them to the application's directory when the application closes, because doing so could leave remnants on the hard drive in the event of a power outage.

[0063] Electronic meeting systems (EMS) need to be distinguished from typical groupware on the one hand, and from web conferencing systems on the other. In fact, there is some overlap in the non-essential features of products in the above categories. The main difference from groupware is the strength of collaboration. EMS must be distinguished from systems that allow multiple users to simultaneously display the content of their individual computer screens on remote displays.

[0064] "Groupware" supports collaboration within a group while maintaining the ability to identify individual contributions. In contrast, EMS supports collaboration within a group as a whole. It enables collaborative work to produce results for which responsibility is due. In business processes, groupware and electronic meeting systems complement each other, with groupware supporting teams in research and documentation before or during EMS sessions.

[0065] Web conferencing systems and electronic meeting systems complement each other in online meetings or workshops. EMS extends web conferencing systems by providing dialogue tools for generating and documenting group results. Conversely, web conferencing systems complement EMS with screen sharing and audio conferencing features, which are necessary for synchronous online meetings but are absent in EMS.

[0066] A "data conference" refers to a real-time communication session between two or more participants who share computer data. Interaction and presentation devices such as screens, keyboards, mice, and cameras can be shared. This term is used to distinguish it from video and audio conferences. The data may include screens, documents, graphics, drawings, and applications that meeting participants can view.

[0067] Application sharing is a remote access element under the umbrella of collaborative software, enabling two or more users to access a shared application or document simultaneously in real time from their respective computers. Generally, the shared application or document runs on the host computer, and remote access to the shared content is provided to other users from the host computer. Application sharing should be distinguished from systems that allow multiple users to simultaneously project the content of their individual computer screens onto a remote display, although collaboration between different users on the application is not possible.

[0068] The term "optional media content" means that users can generate, create, or select any media content that is suitable for display. This is different from a client voting on or selecting media content displayed by others in a meeting or presentation. This term signifies a client-oriented, client-granted right and privilege to display content, rather than the content being provided to meeting members being provided by the main presenter.

[0069] "Unified communication system or tools" means voice or voice-visual content communication such as that provided by "Skype®" or "Skype® for Business." Such software can replace voice and / or visual data provided by a host processing device. Unified communication tools can be described as a collection of tools for performing VoIP, (web) conferencing, shared whiteboards, message exchange (e.g., chat), file transfer, or presence. Unified communication system or tools include Voice-Over-Internet-Protocol (VoIP), text or instant messaging (e.g., AIM, Blauk, eBuddy, Gadu-Gadu, IBM Lotus Sametime, ICQ, iMessage, IMVU, Lync, MXit, Paltalk®, Skype, Tencent QQ, Windows®). Live Messenger (registered trademark) or MSN Messenger (registered trademark), Wireclub, Xfire, and Yahoo! Messenger (registered trademark), email, Twitter (e.g., tweets), Digital Service Protocol (DSP), etc. A unified communication system or tool utilizes a video conferencing cloud service that includes video conferencing nodes to enable video conferencing. This enables communication between one or more users located at the first video conferencing endpoint and one or more users located at the second video conferencing endpoint.

[0070] In the context of this invention, "screen scraping" refers to video This means not only reading the frame buffer and rendering it on the display, but also processing it. Screen scraping for presentations is described in US2002 / 0196378 to Slobodin et al., which is incorporated herein by reference.

[0071] "Automatic composition" or "automatic layout" refers to the automatic nature of multiple graphics / video sources being rendered on the central display in a way that the user intuitively expects, without user intervention.

[0072] "Wireless" or "wireless communication network" can be any network that does not use cable links between nodes, for example, using RF, optical, or infrared such as IrDA, diffuse infrared, WLAN, WiMAX®, WiFi®, WiFi Direct, LiFi, ultrasonic, ZigBee®, or Bluetooth, or any other wireless communication network well known to those skilled in the art for communication. Any communication network disclosed below may preferably be a wireless network. This would enable "sharing or combining" at least a selection of functions / endpoints of a second peripheral device, i.e., a one-to-many scenario.

[0073] The term "computer" generally refers to a processing device, that is, one that has a processing engine capable of various digital processing, such as rendering graphic images for display. Computers may take the form of workstations, personal computers, laptops, palmtops, PDAs, smartphones, tablets, etc. Generally, computers have memory such as volatile RAM. Non-volatile memory such as hard disks, optical discs, or solid-state memory may be included in the computer or are peripheral devices. Currently, most computers are electronic computers, but the term "computer" also includes optical-based computing devices.

[0074] The term "pre-installed generic driver" is intended to mean a driver that is installed as a standard driver on a processing device such as a computer, for example, when an operating system is installed. Such a driver is standard for the operating system and can drive standard class peripheral devices coupled to or connected to the processing device. It is not necessary to install a specific driver for such peripheral devices. Such a generic driver may be a human interface driver (HID) or mass storage device driver, having a predetermined software component configured to drive mass storage, CD-ROMs, keyboards, or combinations thereof. Such devices may be readable and writable computer peripheral memory devices such as USB memory sticks, flash memory, or external hard drives.

[0075] The "host device" is a processing device that desires to share using the first peripheral device or dongle.

[0076] A "functional device" is a second peripheral device connected to the master unit in some way. This system has the function of transparently exposing the second peripheral device connected to the master unit to the first peripheral device as if it were attached to the processing device to which the first peripheral device is connected. This exposure is achieved using a "descriptor field". This is possible. Functional devices can be operated using only vendor-specific drivers.

[0077] An “endpoint” can be described as a data source or sink and is defined for a USB device which may be a physical or virtual device. In this invention, an endpoint must be broadly interpreted as a data source or sink. Thus, data can be stored or released at an endpoint. An endpoint can function as a kind of buffer and can be defined for a physical or virtual device. Data stored at an endpoint may be received from a host such as a USB host, or may be waiting to be sent to such a host. An endpoint is defined by the number of descriptive fields. For example, an endpoint may be configured to support different transfer types, such as four transfer types. These transfers are, in the USB specification, for example, control transfers, interrupt transfers, isochronous transfers, and bulk transfers. These may be defined as follows. Those skilled in the art may use these transfer types as needed in the present invention. Within the limits of the hardware, endpoints may be configured using middleware, such as USB middleware. Endpoints may be limited to specific transfer types.

[0078] A USB host client can send data to endpoint 1, for example. Data from the USB host is sent to OUT endpoint 1. The running software then reads the data when it is ready. When returning data, it must be written to IN endpoint 1 because the USB bus is controlled by the USB host, and the software cannot freely access the USB bus. The data in IN endpoint 1 remains there until the host sends an IN packet to endpoint 1 requesting the data.

[0079] The number of endpoints may be limited. Each endpoint can have only one forwarding direction. A specific endpoint may be designated for, for example, control forwarding only and not assigned to any other function.

[0080] A control transfer may be a bidirectional transfer arranged for a host to send and request configuration information to and from a device using specific IN and OUT endpoints.

[0081] Each control transfer may involve multiple transactions. A control transfer may have several stages. 1. The setup phase involves sending a setup packet, defining the request, and specifying the amount of data to be transferred during the data phase. 2. The DATA stage is optional. If there is a DATA stage, it always begins with a transaction containing a DATA1 packet. Then the transaction types will alternate between DATA0 and DATA1 until all the necessary data has been transferred. 3. The STATUS stage is a transaction containing a DATA1 packet of length zero. If the DATA stage is IN, the STATUS stage is OUT, and vice versa.

[0082] Interrupt transfers have a limited latency to or from a device. In USB, interrupt transfers, or interrupt pipes, have a specified polling rate. Typically, interrupt transfer data consists of event notifications, characters, or coordinates from a pointing device.

[0083] Isochronous transmission is used to transmit real-time information such as voice and video data and can be transmitted at a constant rate. A dedicated portion of the USB bandwidth is allocated to USB isochronous data streams to ensure data is delivered at the desired rate. An isochronous pipe transmits a new data packet in every frame, regardless of whether the last packet was successful or unsuccessful. Error detection is not required for isochronous transmission. Any errors in electrical transmission are not corrected.

[0084] Bulk transfers are used for data of types that are not control, interrupt, or isochronous. Reliable data exchange is ensured at the hardware level using error detection.

[0085] Bulk transfers use all available bandwidth after other transfers have finished. Bulk transfers may experience delays if the bus is very busy.

[0086] "A specific device exposes a peripheral device or other device" means that the specific device configures one or more endpoints with a specific descriptor field.

[0087] Description of Preferred Embodiments Embodiments of the present invention enable wireless connectivity between a user-processing device, such as a laptop, computer, PDA, or smartphone, and a base station of a wireless network. This wireless connectivity may be achieved by an integrated wireless transceiver in the user-processing device, such as a laptop, computer, PDA, or smartphone. Alternatively, in some embodiments of the present invention, the wireless connectivity is achieved via a first peripheral device that is somehow plugged into, coupled to, connected to, or mounted on the user-processing device, or provided by other hardware or software.

[0088] Embodiments of the present invention enable users, for example, participants in a meeting room, to utilize at least one functional device, several functional devices, or all functional devices attached to a master unit, the at least one functional device having an electronic output such as a digital electronic output, and being able to cooperate with other devices plugged into, coupled to, connected to, or attached to the master unit in some way, or provided by other hardware or software. The at least one functional device may be provided by plugging into a second peripheral device, such as a dongle, or by a second wirelessly connected peripheral device, or by connecting to the at least one functional device with some software that can be downloaded or pre-installed. The at least one functional device may have an electronic output provided from a touchscreen, microphone, speakerphone, or camera, and may function with a device having an electronic input such as a display or speaker. Embodiments of the present invention provide a functional device for shared use by participants in a classroom or meeting room.

[0089] For example, via a serial connection such as a serial plug-and-play connection, such as USB, or via a wireless connection such as a Near Field Communication (NFC) connection, such as a Bluetooth connection or an IR connection, at least one of the above functional devices coupled to, connected to, or mounted on the master unit is made accessible and usable by one or more users in a meeting room or classroom. Preferably, this is done in a transparent, standard manner. In various cases, the system can function as a wireless (e.g., USB) hub, and in other cases, the first peripheral device or other hardware or software functions as a combined function of the functional device, for example, a second peripheral device coupled to, connected to, or mounted on the master unit. A virtual device with a single function is exposed.

[0090] Embodiments of the present invention utilize a master unit, which includes at least a transmitting function and typically includes a transceiver function. This master unit is a master unit of a network such as a wireless network and may therefore have a transmitter, receiver, or transceiver. This master unit includes a digital processing engine such as an FPGA, CPU, or GPU, as well as memory and input / output ports, and can be connected, for example, by plugging in, to one or more second peripheral devices that have or do not have visual or audio capabilities for use in a meeting room or classroom. Functional devices such as one or more second peripheral devices used with the master unit may include, for example, one, some, or all of the following: a television or other video provider that provides images such as video via HDMI, DVI, USB, or other connections; an image capture device such as a camera that provides images such as captured video via USB, HDMI, or other connections; an audio device such as a speakerphone that provides sound or music via USB or other connections, or a speaker or one or more microphones using a 3.5mm audio jack, SPDIF, or other connection via USB, or a 3.5mm jack, or USB, or other connection; a display device such as a touchscreen that utilizes USB or other connections, or is integrated via USB; and any other functional devices or first peripheral devices that can be connected to, for example, plugged into, a first device such as the master unit, and have an electronic output such as a digital electronic output.

[0091] Embodiments of the present invention include a device such as a first peripheral device that includes at least a receiving function and typically includes a transmitting and receiving function such as a transceiver function. The first peripheral device (which is at least a receiver device) may be hardware, software, or a combination thereof that has wireless or wired connectivity functions, and can be connected to the base station via a wireless connection such as Wi-Fi, Bluetooth, Wi-Gig, Li-Fi, NFC, or similar connections, or via a wired link such as USB, Ethernet®, etc. Thus, the base station can expose and make available any functional device such as a second peripheral device connected to the base station to one or more first peripheral devices (e.g., receiver devices), and can expose and make available to multiple first peripheral devices (e.g., receiver devices) simultaneously, for example, by one of the following methods. 1. The above-mentioned functional devices (or their respective second peripheral devices, etc.) connected to the master unit are exposed to, for example, one or more first peripheral devices or user processing devices in a one-to-one or one-to-many relationship with respect to the first peripheral devices or user processing devices, preferably by tunneling packets from the master unit to one or more endpoints of the first peripheral devices or user processing devices, without interpreting or processing electronic signals from each functional device, for example, a second peripheral device. 2. Interpret, process, and convert electronic signals from at least one of the above-mentioned functional devices, e.g., a second peripheral device, and mimic the above-mentioned at least one functional device, e.g., a second peripheral device, at the end of the first peripheral device (e.g., the end of the receiver device) using software, hardware, or both. Mimicry means that the same device as the second peripheral device connected to the master is exposed at the endpoint, but the communication is not simply tunneled as in the first and second methods described above. 3. One or more functional devices, such as a second peripheral device or a part of a second peripheral device, are combined and exposed to one or more first peripheral devices, such as a receiver device, through hardware, software, or a custom device of both. 4. Combine one or more functional devices, such as a second peripheral device or a part of a second peripheral device, and expose their functionality to the end user through proprietary software. 5. By combining any of the above methods, the functionality of a functional device, for example, a second peripheral device, is provided to one or more first peripheral devices, such as a receiver device.

[0092] If a functional device, such as a second peripheral device, is originally exposed or exposed as an imitation of the hardware of such a device, an endpoint such as a USB endpoint on the first peripheral device or user-processing device can be used to expose the corresponding second peripheral device, such as a USB device connected to or plugged into the master unit (similar to a hub), and in that case, there is no need to install proprietary software or drivers to support it.

[0093] When a custom device, i.e., a device specifically developed to function in the applications of the present invention, is exposed, these same endpoints of a first peripheral device, such as the receiver hardware, can be utilized, or this can be done in software using a driver.

[0094] By using the first peripheral device, a connection such as a wireless or wired connection between the first peripheral device and the master unit can be set up by the first peripheral device functioning as a hub, such as a wireless USB hub. Regardless of which second peripheral device is connected to or plugged into the master unit, the second peripheral device selected from the non-exclusive list of, for example, TV, video device, audio device, speakerphone, camera, display, etc., is exposed to the first peripheral device in one of the five ways described above.

[0095] The advantages of embodiments of the present invention may be one, some, or all of the following:

[0096] In one implementation example, all second peripheral devices, which may be USB devices, are mirrored one-to-one without requiring any additional drivers or software to be installed on the user's processing device, other than the drivers from the vendor of the second peripheral device. The second peripheral devices can be exposed to the user's processing device either directly by the hardware of the first peripheral device or by using a software virtual hub on the first peripheral device or on the user's processing device.

[0097] In another implementation example, different second peripheral devices (e.g., speakers, microphones, etc., with electronic outputs and / or inputs) can be combined and connected to the same master unit. Additional functions such as echo cancellation, conversion, encoding / decoding can be provided to the master unit. Only one second peripheral device needs to be exposed to the user's processing device through the first peripheral device.

[0098] A combination of the two methods described above can be provided for one or a combination of devices or functions. Depending on whether or not it is necessary for the device or function to be available to multiple users at once, different methods can be used in a single final solution, and technical feasibility such as simple mirroring of devices is possible.

[0099] • By adding some software to the client software running on the user processing device, the configuration of these devices can be made easier. (For example, setting it as the default device). Additionally, you can add several custom controls (mute, pan / tilt / zoom, etc.). Several controls can be added to the first peripheral device. ○Such control can be eliminated by performing several processes on the master unit (for example, automatic gain control for audio signals, participant framing to eliminate the need for PTZ).

[0100] Unlike a regular hub, multiple users can use the functional devices, and only one person can plug it into a single cable. Each user processing device to which the first peripheral device is plugged in can use these functional devices, and the master unit ensures that all streams are correctly captured, played back, processed, combined, and / or enhanced.

[0101] By assigning different priorities (speed, throughput, etc.) to different streams, adjustments can be made to cope with the unpredictability of wireless communication channels. For example, prioritizing voice streams over video streams can improve the call experience.

[0102] • When there are VoIP calls such as Skype® calls in a meeting, the first peripheral device can be connected to the user processing device, for example, plugged in and connected to the master unit, and can use all functional devices, such as a more advanced camera that gives a wider room view, a more advanced microphone, and a more advanced speaker. This is particularly useful when there are multiple people in the room who wish to answer the conference call, such as a VoIP or Skype® call.

[0103] Referring to Figure 1, group 90 is a microprocessor or FPGA-based device that includes one or more peripheral devices 91-93 that can be plugged into, for example, the master unit 100. Peripheral devices 91-93 can be connected to the master unit 100 via standard connections such as Firewire (IEEE-1394), PCI, Mini PCI, PCI Express, Mini PCI Express, PCMCIA, PC Card, Universal Serial Bus (USB), SDIO Card, HDMI, DisplayPort, Bluetooth, and other serial connections.

[0104] By using group 90, for example, peripheral devices • Webcam 91 • Speakerphone 92 • Multiple HID endpoints 93 It can be exposed.

[0105] The master unit 100 has an electronic digital processor such as a microprocessor or FPGA-based device, and includes, for example, the following: • A web interface 110 is an interface for the master unit 100 that allows the system to be configured. - One or more drivers 111, for example, two specific drivers for the webcam 91 and speakerphone 92, which run on the master unit 100 to receive / send data from group 90 and USB peripherals 91-93. Blocks 112, 114, and 115 perform additional processing for encoding, mixing, multiplexing (separation), and other operations on both audio and webcam signals. The data can be encrypted or decrypted by the decrypter / encryptor 113, 116 before being transmitted to the first peripheral device 130 via the wireless link 127. • The control module 117 and the first peripheral device 130 of the master unit 100 are connected to the master unit 1 By using the control channel 128 between 00 and the first peripheral device 130, various functions such as keep-alive, identification, and crypto key exchange are performed. • Unified Protocol Block 118 is a functional block that extracts the system's main functions: screen sharing and other non-media streaming (e.g., mouse pointer). Encryption and decryption modules 119 and 120 are provided for primary communication with the first peripheral device 130 and other applications. The encoder, such as the H264 encoder 121, provides a local view block 123 (which is one or more functional devices that emit signals from the master unit 100 to the user processing device 160), captures and encodes the rendered image on the main display 126, and returns it to the first peripheral device 130 and other clients. • Audio processing block 124 is provided for audio content. A video decoder 125 is provided to decode and render screen content for the display 126.

[0106] The first peripheral device 130 is a microprocessor or FPGA-based device, including, for example, the following: The webcam video from the webcam 91 of the master unit 100 is decoded in block 30 and exposed to the processing device 160 through the USB video endpoint via USB video and drivers such as the H264 driver 31. The audio signal from the speakerphone 92 connected to the master unit 100 is decoded in block 32, processed as a microphone signal from the USB speakerphone endpoint 92, and exposed. The audio signal from the processing device 160, received through the speakerphone USB endpoint, is encrypted in block 35 and transmitted to the base station 100 via the wireless connection 127. The control HID (Human Interface Device) 51 is a mirror of the additional HID USB endpoints exposed to the master unit 100 by group 90 and the second peripheral devices 91-93. A vendor-specific HID endpoint, indicated as data pump 42, is used for channel 52, which may be a high-bandwidth data channel between the master unit 100 and vendor-specific software, such as software 70 running on the processing device 160, via the first peripheral device 130. Screen video and audio signals for display 126 can be transmitted through this channel 52, which is preferable to transmission through the control HID endpoint in the first peripheral device 130. Additionally, screen input from display 126, captured (block 123) and encoded (block 121) in the master unit 100, is also transmitted to vendor-specific software running on the processing device 160 so that multiple users or other users, such as meeting participants, each owning a plugged-in processing device 160 and optionally a first peripheral device 130, can view what is on display 126 on their own processing device 160's display (not shown). Blocks 45-50 contain several basic components such as an operating system (OS) and drivers for controlling light indicators such as LED rings on the first peripheral device 130, a bandwidth measurement module and other Wi-Fi modules such as wireless, an update / upgrade module, a secure boot module, secure key storage, a debug and test module, and more. The audio capture module 43 exposes the speaker endpoint to the processing device 160. Module 43 captures this audio and sends it back to the vendor software 70 running on the processing device 160, where it is timestamped (block 69), aligned with the video data, and then sent to the master unit 100, for example, through the data pump 42. This enables improved time stamping and lip-syncing. Cut. In a less desirable embodiment, the audio can be captured and processed on the first peripheral device 130 and immediately transmitted to the master unit 100.

[0107] Microprocessor or FPGA processing devices 160, such as computers, laptops, smartphones, and personal assistants (PDAs), run proprietary software 60, 70, including, for example, third-party software (e.g., Skype) 60. Third-party software 60 runs on processing device 160 (e.g., communication software such as Skype®), and the processing device uses video endpoints and / or speakerphone endpoints (both speakerphone microphone and speakerphone speaker) from at least one functional device 91-93 (e.g., webcam 91 attached to master unit 100). Block 70 includes dedicated software developed for the present invention, which includes, -This is software for decoding (61) and decoding (62) the pipeline, which then displays the screen data from the display 126 from the master unit 100 in the software window. The data from the final stage (local view 63), following the arrow 71, is captured by communication software such as Skype® client software 60. - Unified protocol block 64 is a module that handles the control layer and (indirectly) controls the media path, the connection to the first peripheral device 130, and the connection to the master unit 100. Pipelines 65-67 are for screen capture (67), encoding (66), and encryption (65) for screen sharing to the master unit 100. - Pipelines 68 and 69 receive the speaker audio signal returning from the first peripheral device 130 (block 68), perform processing (e.g., time stamping, block 69), and send it out through the HID endpoint ("data pump 52"). In this case, the endpoint not used by the software is the control HID from the second peripheral device. If any software is running from the vendor of the second peripheral device, this software can communicate with the relevant second peripheral device through these HID endpoints (e.g., camera pan / tilt / zoom). The speaker endpoint (content audio arrow 53) is used by the operating system (OS) of processing device 160 to transmit all (non-speakerphone) sound / audio signals.

[0108] Connect the first peripheral device to the user processing device. The first peripheral device 130 can be connected to the user processing device 160 by the method 300 shown in Figure 2. This connection may be a serial connection such as a USB connection, such as USB 1, 2, 3, or -C, and the peripheral device 130 is connected, for example, to a plug-and-play serial port provided for the USB connection, for example, by plugging it in. The first peripheral device 130 may include a user input device, such as an actuator. The first peripheral device 130 may be configured as a connection unit, or it may be a physical device in the form of a connector for the plug-and-play interface of the user processing device. It may have a base and flexible data and power connections mounted on the base. The base may have an actuator, for example, a button configured to be an input device that can be activated by the user.

[0109] Flexible data and power connections must be configured such that the base lies flat on the meeting table regardless of the orientation of the connectors that need to be inserted into the plug-and-play interface. Preferably, the base includes electronics and optional It has persistent storage for storing applications such as portable applications. Network setting parameters for WiFi connection to the master unit 100 can also be stored in memory. The first peripheral unit may have a processing engine (e.g., CPU, GPU, FPGA), a wireless transmitter / receiver for WiFi or LiFi, a plug-and-play interface such as a USB interface, and a light ring such as an LED ring or strip as a visual indicator. Portable applications may be stored in the first peripheral device 160, i.e., the base, or may be downloaded from another pluggable device such as a flash drive, or downloaded from the master unit 100, etc. The visual indicator is for enabling user feedback from the connected unit regarding the status of any activity.

[0110] Some examples of actuator activation that can be used in any of the embodiments of the present invention include voice activation (handclap, voice recognition, computer sound, music, etc.), remote control via a wirelessly connected device (IR, Bluetooth, WiFi, LiFi, etc.), light activation, pressure activation, e.g., pressing with a finger or hand, touch activation, proximity (near touch to the actuator or bringing the actuator close to some object), biometric readers such as fingerprint readers, iris scanners, DNA analyzers, and keypads for entering key codes, e.g., passwords.

[0111] Method 300 includes, in step 301, connecting the first peripheral device 130 to the processing device 160, for example by plugging it in. As a result, the initialization scripts in kernel boot steps 302 and 303 are exchanged. The first peripheral device 130 is found to be a USB device in step 306, the USB protocol stack is loaded in step 304, and then the wireless connection protocol stack is loaded in step 305. Parameters for the wireless connection are entered, for example, from memory on the first peripheral device 130 or by the user, and the wireless connection mapping is completed in step 307. Client software that needs to be loaded onto the processing device 160 is then loaded and, in step 308, is started, for example, by a user action. In step 310, the first peripheral device 130 is ready to share data with the display screen 126 mounted on the master unit 100.

[0112] As shown in Figure 3, if a launcher program is available, for example, stored in the first peripheral device 130, or stored in the processing device 160, or can be retrieved from a remote location such as a URL on the internet or from a LAN server, then the client software can be automatically launched in step 328. Steps 321-327, 329, and 330 are identical to steps 301-307, 309, and 310 in Figure 2.

[0113] For example, the client software may be the software 70 shown in Figure 1, which runs on the processing device 160. Communication software 60, such as Skype®, or other software may be pre-installed on the processing device 160. The client software 70 may be for processing video data from the processing device 160, and this client software runs on the processing device 160 using a processing engine such as a CPU, GPU, or FPGA. The client software 60 may be screen scraping software that screen scrapes the screen of the processing device 160. This client software 60 may be stored as a software program in memory such as mass storage on the first peripheral device 130. When this software program is coupled to the processing device 160, it is loaded into the processing device 160 and is automatically loaded as described above or by user action. The software 60 is executed. When the software 60 is running on the processing device 160, it is preferable that the footprint left at termination is zero. When the software 60 is running on the processing device 160, it captures video data available on the processing device 160, for example, from a presentation or video running on the processing device 160. For example, when the software 70 is running on the processing device 160, it is configured to screen scrape and read one or more video frame buffers from, for example, a graphics card on the processing device 160. Generally, the processing device 160 has its own display and therefore has a graphics card or an equivalent of a buffer for storing video data. This read video information is timestamped with clock time (block 69) and can later be synchronized with the associated audio signal. The video signal is then encoded in a video encoder (block 66) for transfer through a plug-and-play interface using a generic driver, for example, through a USB interface using a pre-installed generic driver, for example, a pre-installed generic human interface device (HID) (arrow 53). The data is transmitted to a wireless communication network 127, such as a WiFi or LiFi network, routed through the communication network 127 to the base station 100, and inserted into a suitable composition, such as an OpenGL-based composition in a compositor, for display on the display device 126.

[0114] Data is sent from the processing device to the master unit. Referring to Figure 4, audio data, or voice, on a processing device 160 such as a computer, laptop, smartphone, PDA, or tablet is transmitted via a port using a generic driver, for example, through a USB port 8 using a standard built-in generic voice driver such as a UAC driver 7. In the first peripheral device 130, voice packets are read from a generic port, such as a USB port 11, by a dedicated voice device 14. These packets are then processed by one of the following: a mixer, rate converter, echo canceller, noise canceller, etc. Mixing, rate conversion, echo cancellation, or noise cancellation can be performed using an ALSA driver 18. The ALSA driver provides packets to a dummy voice device 16, such as a voice scraper. In this voice device 16, the voice packets may be time-stamped from a clock 15 that can be synchronized with the clock 9 of the processing device 31. This information is then encoded in an encoder 17, packetized in a packetizer 19, and then transferred to the communication network between the first peripheral device 130 and the master unit 100. The network may be a wireless network such as a WiFi or LiFi network. For this purpose, a suitable transmitter 21 is provided on the first peripheral device 130. At the master unit 100, the audio information stream is restored at a suitable communication interface such as a WiFi access point 22. Next, the audio is unpacked by an unpacking device 23 and decoded by a decoder 24 before being supplied to the audio mixer 28. To synchronize the audio and video streams, they are sent together to a synchronization device 27, where the two streams are synchronized and thus lip-sync is maintained.

[0115] Data is sent from the processing device to the master unit. Embodiments of the present invention allow data such as visual or audio signals to be captured and provided to one or more processing devices 160 so that this captured data can be shared with other participants. These visual or audio signals are captured, provided, and acquired from functional devices such as a webcam 91 and a speakerphone 92, but are also captured from a display 126. For example, the functional device may be a microphone that is placed in an appropriate position in the meeting to receive audio data well. It may be used with a central speaker system that may be of higher quality than the processing device 160 of the meeting participants. Such a speaker system may be, for example, a 4-channel surround speaker system. This can also avoid the difficulties associated with multiple microphones and speakers, which can result in unwanted positive feedback.

[0116] Embodiments of the present invention may include, for example, features described with reference to Figures 1 to 4 for providing additional functionality that can be shared by all participants in a meeting having a processing device 160 together with an optional plug-in first peripheral device 130. In embodiments of the present invention, fixed USB endpoints on the first peripheral device 130 are provided for basic functionality. These are fixed and a combination of vendor-specific endpoints and multiple standard endpoints, and can be interpreted or understood as custom drivers, default OS drivers, and / or host applications described with reference to Figure 4 for screen sharing and audio.

[0117] A configurable USB endpoint is configured when the first peripheral device 130 is paired with the master unit 100, or through the wireless connection 127 between the processing device 160 and the master unit 100. Once configured, the USB endpoint exposes second peripheral devices 91-93, which can be interpreted as custom drivers, vendor-specific drivers, or default OS drivers. These can be used by either a host application or a third-party application. These endpoints may be exact copies of the peripheral devices among the second peripheral devices 91-93 connected to the master unit 100, or they may be dedicated designated endpoints.

[0118] Communication between these USB endpoints and the processing device 160 can be performed through a standard enumeration of USB endpoints when connecting the first peripheral device 130 to the processing device 160, for example, by plugging it in. All information regarding the method of communication with the master unit 100 and the endpoints to be exposed can be stored in the first peripheral device 130 and can be set up by the firmware of the first peripheral device 130.

[0119] The master unit 100 may make a decision regarding which endpoints of the first peripheral device 130 should be exposed, and the master unit may be configured to analyze which second peripheral devices are connected to the master unit 100 (which connections can be achieved via USB connection or other means) and to create settings for the relevant USB endpoints on the first peripheral device 130.

[0120] As described above, in the embodiments of the present invention, during operation, data may flow in both directions, i.e., to or from the processing device 160. 1. Data from a second peripheral device connected to the master unit 100 can flow into the master unit 100, or it can be captured by the master unit 100, such as data for the display 126. Any such data can be interpreted using the firmware of the master unit 100, and processed, enhanced, encoded, and / or encrypted in some cases to generate processed data. In some embodiments, this processed data is then transmitted to the first peripheral device 130 via the wireless link 127. The first peripheral device 130 can, if necessary, decode, decrypt, process, and / or interpret this data and make it available through one or more of the first peripheral device 130's USB endpoints (e.g., fixed or configured endpoints). This data is then captured by the operating system of the processing device 160 and, through custom or standard drivers, to a host application or third-party application running on the processing device 160. The application, or other processing devices 160 of other participants in the meeting, are made available. 2. Data that may originate from a host application and / or one or more third-party applications running on processing device 160 can be transmitted through a handle made available by the system. This data is then transmitted to the appropriate USB endpoint of the first peripheral device 130 via the system (standard generic) driver and USB port. The first peripheral device 130 then receives this data via its firmware, processes, enhances, encodes and / or encrypts it, and transmits it to the master unit 100 via the wireless link 127. The master unit 100 receives this data via its firmware, decrypts, processes, decodes and / or enhances it, and transfers it to an appropriate first peripheral device that can be connected to or mounted on the master unit 100 via a serial connection such as USB, HDMI or other interface.

[0121] A meeting with multiple participants is schematically shown in Figure 6, along with four processing devices 160 (160-1, 160-2, 160-3, 160-4), which may be laptops. Each of the three processing devices (160-1, 160-2, 160-3) is connected to the first peripheral device 130, for example, by inserting a USB connector. One processing device 160-4 is the host for Unified Communication (UC) calls such as Skype calls or Skype for Business calls. It may be present. Two of the processing devices, the first processing devices (160-1, 160-2), have completed a WiFi connection to the base station 100 and are sharing a medium to the main screen 126. Processing devices 160-3 and 160-4 are sharing a Unified Communications (UC) call such as a Skype call or a Skype for Business call.

[0122] One or more of the following will be connected to the main unit 100: • Webcam 91 • 4K touchscreen 94 • Camera 95 connected via USB or HDMI • USB speakerphone 92 • USB microphone 96 and speaker set 97 (connects via audio jack, S / PDIF, USB, HDMI, etc.) The host processing device 160-4 has pre-installed UC client software. This software communicates with a USB endpoint on a first peripheral device 130 connected to the host, for example, a plugged-in device. This type of client software uses a speakerphone USB endpoint as the default audio device (e.g., microphone plus speaker). Embodiments of the present invention are under the control of all audio or visual signals sent from and injected into the processing device 160.

[0123] The speakerphone 92 can be used by any of the users, at least in conjunction with a connected and functioning first peripheral device 130. No action is required to access this device. Separate controls may be available on the first peripheral device, for example, to control the volume of the master unit 100's audio output and, if necessary, to mute the room microphone 96. When muted, the microphone 96 is also muted for all other buttons or apps that use this device.

[0124] UC client software 60 running on host processing device 160-4 It does not inject audio content. Microphone 96 can pick up this signal, or the master unit 100 can inject it into the microphone signal to the first peripheral device 130 connected to host 160-4. Because of the low latency, there is no need to lip-sync with the content video (even if shared).

[0125] A second peripheral device of the same type as the one plugged into the master unit 100 is exposed to the user, and therefore the user can use the driver provided by the vendor. If one of the processing devices 130 is shared within the room, the content audio is also shared within the room.

[0126] A processing device having a functioning first peripheral device can wirelessly access the room camera 96. All other devices connected to the master unit 100 (e.g., the HID control device 93) are essentially exposed.

[0127] For example, any user connected to the system through a first peripheral device can view content projected onto the display of their processing device in the meeting room, i.e., a "local view." A local view receiver may have, but is not limited to, one, some, or all of the following functions:

[0128] The ability to open a window and view the meeting room content within that window, regardless of whether it is the stream itself, annotations, or a whiteboard session.

[0129] A feature that allows you to zoom in on content. An additional feature can be added to ensure image quality when zooming in.

[0130] Local View allows users to start and / or join a whiteboard or annotation session from their own device.

[0131] The ability to take a screenshot and / or pause the local view. Figures 6 to 15 and 17 illustrate methods for enabling participants in a room, for example, a meeting, to utilize various functions. There are four basic methods, each of which is an embodiment of the present invention. a) By using the generic driver of the OS164 of the host processing device 160 and endpoints such as mass storage or HID endpoints, the host processing device 160 can transfer data using client software running on the host processing device 160 without the host processing device 160 realizing that the final device is a functional device such as a camera, touchscreen, microphone, loudspeaker, or speakerphone located in or connected to the master unit 100. b) The endpoint on the first peripheral device of the functional device is connected to or within the master unit 100. This means that the host processing device 160 detects that it is connected to the functional device and interacts with the functional device as if it were such a device. c) Connect the endpoint on the first peripheral device of the functional device to or within the master unit 100. This means that the host processing device 160 detects that it is connected to the functional device, but the client software must enable vendor-specific drivers to interact with the functional device. d) Connect the endpoint on the first peripheral device of the functional device to or within the master unit 100. This means that the host processing device 160 detects that it is connected to the functional device, but the processing device 160 does not interact with the functional device. Preprocessing is performed on the master unit 100 to enable this. One method is to create a common endpoint on the first peripheral device so that the processing device can cooperate with any of the functional devices that are the target of preprocessing on the master unit 100.

[0132] Figure 6 shows a processing device 160 that communicates with a first peripheral device 130 connected to it, and a master unit 100 that performs wireless communication 127 with the first peripheral device 130. Various functional devices such as a display 126, a speakerphone 92 (or a microphone and speaker set 96, 97), and a videocam 91 (or other camera 95) are connected to the master unit 100.

[0133] A generic driver allows the client software 70 to send and receive data from a mass storage device 132 on the first peripheral device 130 via a USB interface 131. The client software 70 also has a USB endpoint 134, which may be a USB HID endpoint or, if necessary, a vendor-specific endpoint. This endpoint 134 is for the client software 70 to send data for the display 126 to the master unit 100, which may undergo image processing (blocks 124 and 125 in Figure 1) before being sent to the group 90, for example, via an HDMI connection. In this way, the HID endpoint 134 makes the HDMI display 126 and its audio-visual signals available to any processing device 160 equipped with the first peripheral device when in a meeting room. However, the display 126 is not exposed to the processing device 160, i.e., it is provided as an HID endpoint rather than as a display. The videocam 91 can be exposed to the first peripheral device 130 as an endpoint (in the manner described for Figure 1), and the video data is encoded in block 122 in Figure 1. Alternatively, as shown in Figure 6, the video data is encoded by the master unit 100 and transmitted via the first peripheral device 130 to the driver 162 of the operating system 164 of the processing device 60. The processing device 60 exposes the encoded data from the videocam 91 to the processing device 160, making the video data available for UC calls, such as Skype®.

[0134] Data from the speakerphone 92 may be processed by the master unit 100 and / or the first peripheral device 130 and simulated for the processing device 160. The microphone of the speakerphone 92 generates a signal that is transmitted to the processing device 160 via the USB speakerphone endpoint 136 on the first peripheral device, thereby enabling noise reduction or echo cancellation in the master unit 100 as described with respect to Figure 1 (block 115). When the processing device 160 recognizes the speakerphone endpoint 136 as a speakerphone, as if the speakerphone were plugged into the processing device 160, the UC call makes the microphone data available. The microphone data is also made available to all other users who have the processing device 160 to which the first peripheral device is connected. In this embodiment, the speakerphone speaker has a USB endpoint 136 on the first peripheral device 130.

[0135] Figure 7 shows a configuration similar to that of Figure 6, except that there are two USB endpoints 136 and 138 for the speakerphone and speaker, respectively. As previously mentioned, the HID endpoint 134 makes the HDMI display 126 available to any processing device 160 equipped with the first peripheral device 130 when in a meeting room, in order to provide audio-visual signals to the display 126. However, the display 126 is not exposed to the processing device 160, i.e., it is provided as an HID endpoint rather than as a display.

[0136] The two endpoints 136 and 138 receive audio signals from two different sources. Audio from the UC call is handled via speakerphone endpoint 136, as described with respect to Figure 6, and can be processed by the master unit. However, if there is additional audio data that is not part of the UC call, this data is transmitted via speaker endpoint 138. Since there is no second peripheral device in group 90 that is a standalone speaker, the master unit 100 mixes the audio signals for the speakerphone and the speaker before sending all the audio data to the speakerphone 92. If there is a separate speaker 97 within group 90, the master unit 100 can send this audio data to this speaker. As a result, a virtual speaker is exposed to the processing device 160.

[0137] Figures 8-11 and 17 illustrate various embodiments of the present invention, along with examples of methods for making a touchscreen 99 available in a meeting room. When the touchscreen is touched, it generates a signal associated with the coordinates of the touch location. In a meeting, this needs to be sent to a processing device 160, so that the actual signal must be interpreted if it is given to different screens on the processing device 160. In a meeting where participants use various processing devices, it is necessary to find a way to provide the touch coordinates to all of these different processing devices. Figures 8-11 disclose four basic methods described above.

[0138] In the embodiment shown in Figure 8, there is a group 90, a master unit 100, a first peripheral device 130, and a processing device 160. There are two functional devices connected to or located within the master unit 100: a display 126 and a touchscreen 99. The touchscreen 99 has a touchscreen endpoint 139 on the first peripheral device 130. The processing device 160 recognizes this endpoint and can interact with it if the OS 164 has the associated driver 162. This driver can be downloaded as needed. In this embodiment, each processing device 160 needs to download or already have the associated driver 162 installed.

[0139] In the embodiment shown in Figure 9, there is a group 90, a master unit 100, a first peripheral device 130, and a processing device 160. There are two functional devices connected to or located within the master unit 100: a display 126 and a touchscreen 99. The touchscreen 99 has a touchscreen endpoint 139 on the first peripheral device 130. The processing device 160 recognizes this endpoint and can interact with it if the master unit 100 is configured to convert touch coordinates to coordinates usable by the processing device 160, i.e., if the OS 164 has the relevant driver 162. This driver can be downloaded as needed. In this embodiment, each processing device 160 would need to download or already have the relevant driver 162 installed in the meeting, but the conversion function of the master unit 100 can reduce the number of different drivers that might be needed.

[0140] In the embodiment shown in Figure 10, there is a group 90, a master unit 100, a first peripheral device 130, and a processing device 160. There are two functional devices connected to or located within the master unit 100: a display 126 and a touchscreen 99. The touchscreen 99 has a touchscreen endpoint 139 on the first peripheral device 130. The processing device 160 recognizes this endpoint, and the master unit 100 is configured to convert the touch coordinates into coordinates usable by the processing device 160. If so, i.e., if OS164 has the associated driver 162, it can communicate with this endpoint. This driver is a standard driver that can be distributed; for example, this driver can be downloaded as needed. In this embodiment, each processing device 160 needs to have downloaded or already installed the associated driver 162 during the meeting, but the conversion function of the master unit 100 can further reduce the number of various drivers that might be needed compared to the embodiment in Figure 9.

[0141] In the embodiment shown in Figure 11, there is a group 90, a master unit 100, a first peripheral device 130, and a processing device 160. There are two functional devices connected to or located within the master unit 100: a display 126 and a touchscreen 99. The touchscreen 99 is handled by client software 70 as previously described; that is, the client software running on the first peripheral device 130 is configured to communicate with the touchscreen 99 via the USB interface 131 and with the OS 164 of the processing device 160. This means that the touchscreen 99 is not exposed to the processing device 160. The client software 70 is configured to simulate touch events and provide the correct coordinates to the OS 164 of the processing device 160. Thus, this is a proprietary solution that requires the client software 70 to be configured to communicate with each type of touchscreen 99 via the USB interface 131.

[0142] Figures 12 to 14 relate to embodiments of the present invention involving multiple users. Firstly, each participant in a meeting can utilize any of the embodiments described in Figures 8 to 11. Thus, the present invention provides a multi-user solution.

[0143] In Figure 12, the processing device 60' does not have a first peripheral device 130, for example, none is available. In such a case, the client software 70' may be configured to communicate directly with the master unit 100 via a wireless connection 127', and the client software 70' is configured to provide the display device 126 to the processing unit 160' and enable communication between them by providing any simulation or conversion as needed. The client software 70' may be configured to simulate any of the embodiments described with respect to Figures 1 to 11 without using the first peripheral device 130. The client software 70' may simulate, for example, a type of UB interface used in any of the embodiments in Figures 1 to 11. For the processing device 160, the method described with respect to any of Figures 1 to 11 can be used.

[0144] In Figure 13, the processing device 60' does not have a first peripheral device 130, for example, none are available. In such a case, the client software 70' may be configured to communicate directly with the master unit 100 via a wireless connection 127', and the client software 70' is configured to provide the display device 126 to the processing unit 160' and enable communication between them by providing any simulation or transformation as needed. This is as described with respect to Figure 12. However, group 90 has a speakerphone 92 or a microphone and speaker combination such as microphone and speaker combination 96, 97. The processing device 160 can utilize any of the embodiments in Figures 1 to 11. The client software 70' of the processing device 160' may be configured to provide video data directly to the master unit 100, where the video data is processed, for example, decoded and synthesized, and provided to the display 126 via a suitable connection such as an HDMI connection.

[0145] The processing device 160' can generate a virtual microphone and virtual speaker by providing the client software 70' and driver 127' with a simulated speakerphone 92 to the OS 164'. Third-party software 60' for unified communications such as Skype® calls is provided along with these virtual devices, which are then used for calls.

[0146] In Figure 14, the processing device 60' does not have a first peripheral device 130, for example, none are available. In such a case, the client software 70' may be configured to communicate directly with the master unit 100 via a wireless connection 127', and the client software 70' is configured to provide the display device 126 to the processing unit 160' and enable communication between them by providing any simulation or conversion as needed. This is as described with respect to Figure 13. However, group 90 has a videocam 91 connected to the master unit 100 via, for example, a USB connection. The processing device 160 can utilize any of the embodiments in Figures 1 to 11. The client software 70' of the processing device 160' may be configured so that the master unit 100 directly receives the video data.

[0147] The processing device 160' can generate a virtual videocam device by providing the simulated videocam 91 to the OS 164' via client software 70' and driver 162'. Third-party software 60' for unified communication such as Skype® calls is provided along with all the virtual devices, which are then used for calls.

[0148] Figure 15 shows a messaging flow diagram applicable to any of the embodiments described with respect to Figures 1 to 11.

[0149] Figure 16 shows a messaging flow applicable to any of the embodiments described with respect to Figures 12 to 14 in which the first peripheral device is not used.

[0150] Figure 17 shows yet another embodiment of the present invention, including a processing device 160, which has a processor and memory and runs and optionally stores client software 70, including a host application, as well as third-party applications 60 that can be configured to perform unified communication calls such as Skype® calls or Skype® for Business calls. The processing device 160 has a serial port, such as a USB port, and a generic or custom driver for communicating with a first peripheral device 130 via this port. The peripheral device 130 has a processor and memory and runs and optionally stores firmware that provides a wireless link 127 to the master unit 100 and provides at least one configurable or fixed endpoint for a functional device 90 connected to or located in the master unit 100. The master unit has a processor and memory for running and optionally storing firmware that provides a connection to the wireless link 127 and also provides a port to the functional device 90. The functional device 90 has a processor and memory, and runs and optionally stores firmware for providing a link and interface to the master unit 100 and for providing data to be sent to at least one configurable or fixed endpoint on the peripheral device 130.

[0151] The method according to the present invention can be executed by a computer system. The present invention can perform functions using a processing engine. The processing engine is preferably one or more microprocessors, FPGAs, or central processing units (CPUs) and / or The processor has digital processing capabilities, such as those provided by a graphics processing unit (GPU), and is configured to perform each function by being programmed with software, i.e., one or more computer programs. The reference to software may encompass any type of program in any language that the processor can execute directly or indirectly via either a compiled or interpretable language. Any implementation of the methods of the present invention can be carried out by logic circuits, electronic hardware, processors, or circuits, which may encompass any type of logic or analog circuit, and can be integrated to any degree, and is not limited to general-purpose processors, digital signal processors, ACIS, FPGAs, discrete components, or transistor logic gates.

[0152] The processing device may have memory (such as non-temporary computer-readable media, RAM and / or ROM), an operating system, a display (optionally such as a fixed-format display), ports for data input devices such as a keyboard, a pointer device such as a mouse, a serial or parallel port for communicating with other devices, a network card, and connections for connecting to a network.

[0153] The software can be implemented in a computer program product configured to perform the functions of any of the methods of the present invention, for example, as listed below, when the software is loaded onto a computer and executed on one or more processing engines such as a microprocessor, ASIC, FPGA, etc. Thus, any embodiment of the present invention can incorporate a computer system capable of running one or more computer applications in the form of computer software.

[0154] The methods previously described with respect to embodiments of the present invention can be executed by one or more computer application programs that run on a computer system by being loaded into memory and running on or in connection with an operating system such as Windows®, Linux®, or Android® provided by Microsoft Corporation in the United States. The computer system may include main memory, preferably random access memory (RAM), and may also include non-temporary hard disk drives and / or removable non-temporary memory and / or non-temporary solid-state memory. Non-temporary removable memory may be optical discs such as compact discs (CD-ROM or DVD-ROM) or magnetic tape that can be read and written by a suitable reader. Removable non-temporary memory may be computer-readable media on which computer software and / or data are stored. Non-volatile storage memory can be used to store persistent information that must not be lost even if the computer system is powered off. Application programs can use the information and store it in non-temporary memory.

[0155] The software implemented in the computer program product is configured to perform the following functions when loaded onto one or more corresponding devices and executed on one or more processing engines such as microprocessors, ASICs, FPGAs, etc., and these functions are: This involves connecting a processing device to a functional device connected to or located within a master unit of a communication network, wherein the processing device has memory, a display, and an operating system, the master unit has a transmitter, and the first peripheral device has a receiver. To connect a first peripheral device to a processing device via a generic communication protocol, and to provide at least one fixed or configurable endpoint of a functional device exposed on the first peripheral device, The process involves transmitting data from a master device to a processing device and receiving the data at the first peripheral device via a communication network from a functional device to a processing device, using a generic communication protocol for communication between the processing device and the first peripheral device, through at least one fixed or configurable endpoint.

[0156] The software implemented in a computer program product is configured to perform the following functions when loaded onto one or more corresponding devices and executed on one or more processing engines such as microprocessors, ASICs, FPGAs, etc., and these functions are: The act of transmitting or receiving streaming data that carries data in its raw or unmodified format, i.e., images, audio, etc. Connecting a functional device to or within a wireless communication network base station. This means running a functional device using only vendor-specific drivers.

[0157] The software implemented in a computer program product is configured to perform the following functions when loaded onto one or more corresponding devices and executed on one or more processing engines such as microprocessors, ASICs, FPGAs, etc., and these functions are: Exposure achieved with descriptor fields, To transmit data from a functional device to at least two processing devices using at least two first peripheral devices, This involves combining two endpoints and exposing them to the processing device as a single endpoint.

[0158] The software implemented in a computer program product is configured to perform the following functions when loaded onto one or more corresponding devices and executed on one or more processing engines such as microprocessors, ASICs, FPGAs, etc., and these functions are: The functional device enables one or more of the following: microphone, speakerphone, speaker, display, touchscreen, projector, camera, video camera, webcam. To indicate at least one fixed or configurable endpoint of a functional device exposed on a first peripheral device as one of the following: a human interface device, a mass storage device, a composite device, a microphone, a speakerphone, a speaker, a display, a touchscreen, a projector, a camera, a video camera, or a webcam.

[0159] The software implemented in a computer program product is configured to perform the following functions when loaded onto one or more corresponding devices and executed on one or more processing engines such as microprocessors, ASICs, FPGAs, etc., and these functions are: Encoding and / or optionally encrypting data, Hosting unified communication between two or more other processing devices on a processing device, The first peripheral device provides a functional device for unified communication between two or more processing devices, or This involves exposing the same type of functional device connected to the master unit to the processing device, and using at least one driver for the functional device installed on the processing device.

[0160] Install any of the above software on one of the network's servers or nodes. It can be implemented as a computer program product compiled for a processing engine in any of these locations. The computer program product can be stored on non-temporary signal storage media such as optical discs (CD-ROM or DVD-ROM), digital magnetic tape, magnetic disks, solid-state memory such as USB flash memory, and ROM.

Claims

1. A system for connecting a processing device to a functional device connected to or located within a wireless base station, wherein the system is The master unit having a transmitter, A first display device connected to or located within the master unit, At least one functional device connected to or located within the master unit, The processing device comprises a memory, a second display device, an operating system, third-party software for unified communication calls, a driver, and client software, wherein the client software, when executed on the processing device, is configured to cause the processing device to communicate directly with the master unit via a wireless connection. The master unit is configured to transmit data to the processing device via the wireless connection through the transmitter. The client software, when executed by the processor of the processing device, is configured to enable communication between the first display device and the processing device by providing any simulation or transformation as needed; the client software and the driver, when executed on the processing device, are configured to provide the operating system of the processing device with the at least one functional device as a virtual functional device; the third-party software for the unified communication call is provided together with the virtual functional device for the unified communication call; and the processing device is configured to host the unified communication call between two or more other processing devices. The at least one functional device is configured to transmit functional data, including video data and / or audio data, to the processing device. The client software is further configured to provide the functional data with data conversion of the video data and / or the audio data.

2. The client software and the driver are configured to generate a virtual microphone and a virtual speaker by providing a simulated speakerphone to the operating system, and / or The system according to claim 1, wherein the client software and the driver are configured to generate a virtual videocam device by providing a simulated videocam to the operating system.

3. The system according to claim 1 or 2, wherein the client software is optionally configured to directly provide the video data obtained by screen scraping to the master unit, and the master unit processes the video data.

4. The system according to any one of claims 1 to 3, wherein the video data and / or audio data are received from the master unit via the wireless connection through the transmitter.

5. The system according to claim 4, wherein the data conversion of the video data and / or the audio data includes decoding and combining or decoding the video data and / or the audio data.

6. The system according to any one of claims 1 to 5, wherein the at least one functional device provided to the operating system as a virtual functional device is one or more of a data capture device, a microphone, a speakerphone, a speaker, another display, a touchscreen, a projector, a camera, a video camera, and a webcam.

7. The system according to any one of claims 1 to 6, further comprising means for encoding and optionally encrypting the data in the master unit or the processing device.

8. The system according to any one of claims 1 to 7, wherein the virtual function device is provided for the unified communication call between the two or more other processing devices.

9. The system according to any one of claims 1 to 8, further comprising at least one fixed or configurable endpoint of the virtual function device, wherein the at least one fixed or configurable endpoint is a data source or data sink configured to store or release data from the virtual function device.

10. A method for connecting a processing device to a functional device connected to or located within a master unit of a communication network, wherein the master unit has a transmitter and a first display device, and the processing device has memory, a second display device, an operating system, third-party software for unified communication calls, a driver, and client software, and the client software is executed by a processor on the processing device. By communicating directly with the master unit via a wireless connection and providing arbitrary simulations or transformations as needed, communication between the first display device and the processing device is enabled, and the master unit transmits data to the processing device via the wireless connection. When the client software and driver are executed on the processing device, they provide the functional device as a virtual functional device to the operating system of the processing device. Third-party software for unified communication calls is provided with the virtual function device, and the virtual function device uses the third-party software for the unified communication calls. The processing device hosts the unified communication call between two or more processing devices, The functional device is configured to transmit functional data, including video data and / or audio data, to the processing device. A method wherein the client software is further configured to provide the functional data with data conversion of the video data and / or the audio data.

11. The client software and the driver generate a virtual microphone and a virtual speaker by providing a simulated speakerphone to the operating system, and / or The method according to claim 10, wherein the client software and the driver generate a virtual videocam device by providing the operating system with a simulated videocam.

12. The method according to claim 10 or 11, wherein the video data is optionally obtained by screen scraping, and the video data is provided directly to the master unit on which the video data is processed.

13. The method according to claim 12, wherein the data conversion includes decoding and combining or decoding the video data and / or the audio data, and the video data and / or the audio data is provided to the first display device.

14. The method according to any one of claims 10 to 13, wherein the functional device provided to the operating system as the virtual functional device is one or more of a data capture device, a microphone, a speakerphone, a speaker, another display, a touchscreen, a projector, a camera, a video camera, and a webcam.

15. The method according to any one of claims 10 to 14, further comprising encoding and optionally encrypting the aforementioned data.

16. The method according to any one of claims 10 to 15, wherein the virtual function device is provided for the unified communication call between the two or more processing devices.

17. The method according to any one of claims 10 to 16, further comprising exposing to the processing device a functional device of the same type as one connected to the master unit, which is provided to the operating system as a virtual functional device.

18. The method according to any one of claims 10 to 17, further comprising providing at least one fixed or configurable endpoint of the virtual function device, wherein the at least one fixed or configurable endpoint is a data source or data sink for storing or releasing data of the virtual function device.

19. The method according to any one of claims 10 to 18, further comprising directly communicating with the master unit to provide the first display device to the processing device.

20. The system according to any one of claims 1 to 9, further comprising directly communicating with the master unit to provide the first display device to the processing device.

21. A computer program that, when executed by a processor, causes the processor to perform the method described in any one of claims 10 to 19.

22. Non-temporary signal storage means for storing the computer program described in claim 21.

Images