Conference system integrating video, voice and remote cooperation and use method thereof

Abstract

The invention discloses a conference system integrating video, voice and remote cooperation and a use method of the conference system, and relates to the technical field of network communication. Conference external equipment in the system comprises video acquisition equipment, audio acquisition equipment and audio playing equipment; the conference external equipment is connected to the receiving unit through a cable or in a wireless manner; the receiving unit communicates with a user computer through a communication network; the client software is used for realizing data sharing, data conversion, conference control and remote cooperation between the receiving unit and the USB external equipment; the USB external equipment is connected to a USB port of the user computer and communicates with the client software through the USB port; during communication, the USB external equipment converts received and sent data streams into UVC and UAC protocols, and then the UVC and UAC protocols are used by a user computer through a UVC interface and a UAC interface respectively. The multi-functional remote conference experience can be realized, and the conference efficiency and quality are improved.

Description

Technical Field

[0001] The present application relates to the field of network communication technology, and in particular to a conference system integrating video, voice and remote collaboration and a method for using the same. Background Art

[0002] Globalization has led to a surge in demand for remote collaboration in various fields. In business, multinational teams need real-time decision-making, education is eager to improve remote interaction, and medical care urgently needs accurate remote consultation. However, early telephone audio communications lacked visualization, and traditional face-to-face meetings were cumbersome, limited, and costly to organize. These limitations highlight the need for efficient communication in remote collaboration, prompting the rise of audio and video conferencing technologies to meet people's needs for communication anytime, anywhere, and promote the development of various industries.

[0003] The advancement of network technology is the key to the development of audio and video conferencing. The popularization of broadband and wireless networks such as Wi-Fi and 5G allows large amounts of audio and video data to be transmitted quickly and stably, reducing latency, improving communication quality, and breaking through the constraints of geographical cables. At the same time, standards such as H.264 and other video encoding, AAC and other audio encoding, and SIP and other communication protocols are constantly being improved to ensure the compatibility of equipment systems, promote the widespread application of audio and video conferencing, and give rise to a variety of solutions to promote industrial development.

[0004] Users have higher and higher requirements for conference experience, and pursue high-definition image quality, stereo audio, low-latency interaction, etc. To meet the needs, technology continues to expand and innovate. Multi-device integration enables users to seamlessly switch to participate in meetings; security and privacy protection measures ensure the security of meeting information; artificial intelligence integration realizes intelligent voice recognition, video analysis and other functions. Fierce market competition has prompted companies to continue to invest in innovation, promote audio and video conferencing technologies to develop in a more efficient, intelligent, convenient and secure direction, adapt to market changes and user expectations, and play a greater role in various fields. Summary of the invention

[0005] The purpose of this application is to provide a conference system integrating video, voice and remote collaboration and a method of using the system, which can achieve a multi-functional remote conference experience and improve conference efficiency and quality.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In a first aspect, the present application provides a conference system integrating video, voice and remote collaboration, the conference system comprising: a receiving unit, a conference external device, a display device, a user computer, a USB external device and client software.

[0008] The conference external device includes a video acquisition device, an audio acquisition device and an audio playback device; the conference external device is connected to the receiving unit via a cable or wirelessly.

[0009] The display device is connected to the receiving unit via an audio and video cable.

[0010] The receiving unit communicates with the user computer via a communication network.

[0011] The client software runs on the user's computer and communicates with the receiving unit; the client software communicates with the USB external device through the USB port; the client software is used to realize data sharing, data conversion, conference control and remote collaboration between the receiving unit and the USB external device.

[0012] The USB external device is connected to the USB port of the user's computer and communicates with the client software through the USB port; when communicating, the USB external device converts the received and sent data streams into UVC and UAC protocols, and provides them to the user's computer through the UVC interface and the UAC interface respectively.

[0013] Optionally, the video acquisition device is connected to the receiving unit via a USB cable or an HDMI cable; the audio acquisition device is connected to the receiving unit via a USB cable, an audio cable, a wireless audio receiver, or a wireless audio transceiver built into the receiving unit; the audio playback device is connected to the receiving unit via a USB cable, an audio cable, a wireless audio receiver, or a wireless audio transceiver built into the receiving unit.

[0014] Optionally, the conference external device includes any one or more of a display, a touch screen, a projector, a microphone, a speaker, a camera, a video camera, a webcam or a still camera.

[0015] Optionally, the USB port includes type interfaces of USB2.0 series, USB3.0 series and USB4.0 series.

[0016] Optionally, the USB external device includes at least one USB large-capacity data communication interface, at least one UVC interface and at least one UAC interface.

[0017] Optionally, the USB large-capacity data communication interface includes a wired network card, NDIS, ECM, NCM, a large-capacity storage device and a human-machine interface device.

[0018] Optionally, the video acquisition device includes a universal UVC device or a video acquisition device with HDMI output.

[0019] Optionally, the audio acquisition device includes a universal UAC device, an audio acquisition device with analog or digital audio input and output, and an audio playback device.

[0020] Optionally, the video acquisition device and the audio acquisition device are integrated devices or separate devices.

[0021] In a second aspect, the present application provides a method for using a conference system integrating video, voice and remote collaboration, including:

[0022] Connect the conference external device and display device to the receiving unit and start the receiving unit.

[0023] Connect the USB external device to the user's computer.

[0024] Open the mass storage device and run the client software on the mass storage device.

[0025] The control client software is connected to the receiving unit based on the communication network.

[0026] The data stream of the conference external device is obtained, and the data stream is transmitted to the USB external device through the USB communication interface; the data stream includes image data and audio data.

[0027] Based on USB external devices, the data stream is sent to the user's computer according to the universal protocol.

[0028] Data sharing, data conversion, conference control and remote collaboration are carried out based on client software.

[0029] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0030] The present application provides a conference system integrating video, voice and remote collaboration and a method for using the same. The conference system includes a receiving unit, a conference external device, a display device, a user computer, a USB external device and a client software; the conference external device includes a video acquisition device, an audio acquisition device and an audio playback device; the conference external device is connected to the receiving unit via a cable or wirelessly; the display device is connected to the receiving unit via an audio and video cable; the receiving unit communicates with the user computer via a communication network; the communication network consists of a wired Ethernet network and a wireless network built into the receiving unit; the client software runs on the user's computer and communicates with the receiving unit; the client software communicates with the USB external device via a USB port; the client software is used to realize data sharing, data conversion, conference control and remote collaboration between the receiving unit and the USB external device; the USB external device is connected to the USB port of the user's computer and communicates with the client software via the USB port; when communicating, the USB external device converts the received and sent data streams into UVC and UAC protocols, and then uses the user computer through the UVC interface and the UAC interface respectively. The present application can significantly improve the efficiency and quality of meetings through a conference system integrating video, voice and remote collaboration. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0032] Figure 1 A schematic diagram of a conference system structure integrating video, voice and remote collaboration provided in an embodiment of the present application Figure 1 .

[0033] Figure 2 A schematic diagram of a conference system structure integrating video, voice and remote collaboration provided in an embodiment of the present application Figure 2 .

[0034] Figure 3 A schematic diagram of signal connections provided in one embodiment of the present application.

[0035] Figure 4 A system signal flow chart provided for an embodiment of the present application.

[0036] Figure 5 An audio signal flow chart provided in one embodiment of the present application.

[0037] Figure 6 A video signal flow chart provided for an embodiment of the present application.

[0038] Figure 7 A schematic diagram of internal signals of a USB external device provided in an embodiment of the present application.

[0039] Figure 8 A schematic diagram of the internal signal connection of the client software provided in one embodiment of the present application.

[0040] Fig. 9 A schematic diagram of a network connection between a user computer, a receiving unit and the Internet provided in one embodiment of the present application.

[0041] Fig.10 A typical application diagram of a user using this conference system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0043] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0044] Embodiment 1

[0045] like Figure 1 and Figure 2 As shown, this embodiment provides a conference system integrating video, voice and remote collaboration, including:

[0046] Receiving unit, conference external device, display device, user computer, USB external device and client software.

[0047] The conference external device includes a video acquisition device, an audio acquisition device and an audio playback device; the conference external device is connected to the receiving unit via a cable or wirelessly.

[0048] The display device is connected to the receiving unit via an audio and video cable.

[0049] The receiving unit communicates with the user computer via a communication network.

[0050] The client software runs on the user's computer and communicates with the receiving unit; the client software communicates with the USB external device through the USB port; the client software is used to realize data sharing, data conversion, conference control and remote collaboration between the receiving unit and the USB external device.

[0051] The USB external device is connected to the USB port of the user's computer and communicates with the client software through the USB port; when communicating, the USB external device converts the received and sent data streams into UVC and UAC protocols, and provides them to the user's computer through the UVC interface and the UAC interface respectively.

[0052] Among them, in some embodiments, the video acquisition device (such as a camera) in the conference external device is connected to the receiving unit via a USB cable or an HDMI cable; the audio acquisition device (such as a microphone) is connected to the receiving unit via a USB cable or an audio cable or a wireless audio receiver (such as a USB Bluetooth adapter, a UHF to USB receiver, etc.) or a wireless audio transceiver built into the receiving unit; the audio playback device (such as a speaker with power amplification) is connected to the receiving unit via a USB cable or an audio cable or a wireless audio receiver (such as a USB Bluetooth adapter, a USB to UHF transceiver, etc.) or a wireless audio transceiver built into the receiving unit. Specific system signal, audio signal and video signal transmission are as follows Figure 4-Figure 6 shown.

[0053] The conference external device can be a data generating device, such as any one or more of a display, a touch screen, a projector, a microphone, a speaker, a camera, a video camera, a webcam, a still camera, etc.

[0054] Video capture equipment: such as Logitech's CC2900ep camera and Aver's CAM520Pro3.

[0055] Audio capture equipment: for example: Jabra's Speak 810UC, Poly's Sync 60, EPOS's EXPAND 80T or video and audio capture integrated equipment: Logitech's CC4000e camera, Jabra's Panacast 50.

[0056] The receiving unit communicates with the user's computer through a communication network, usually a wired Ethernet network, or using the wireless network built into the receiving unit. It is recommended to use Wi-Fi Direct technology to achieve direct communication.

[0057] like Figure 2 As shown, the USB external device is connected to the USB port of the user's computer, which supports Type A interfaces of USB2.0, USB3.0, USB4.0, and USB C interface (also called Type-C).

[0058] USB external devices include devices with USB communication interfaces (such as USB LAN), as well as UVC and UAC interfaces that comply with standards.

[0059] The USB external device is equipped with at least one USB large-capacity data communication interface, at least one UVC (USB Video Class) interface, and at least one UAC (USB Audio Class) interface.

[0060] The USB large-capacity data communication interface is used to exchange data with the user's computer. It can be: wired network card (USB to Ethernet), such as various USB wired network cards, NDIS (Network Driver Interface Specification), ECM (Ethernet Control Model Devices), NCM (Network Control Model Devices), mass storage devices (Mass storage), such as U disk, and human-machine interface devices (HID, Human Interface Device), etc.

[0061] The UVC interface is used to exchange video data with the user's computer, which can be: a camera, a still camera, and other devices.

[0062] The UAC interface is used to exchange audio data with the user's computer, which can be a microphone, speaker, or other device.

[0063] The USB external device includes multiple devices, which can be combined through a USB Hub, or a USB composite device can be used to achieve a combination of multiple functions.

[0064] The mass storage device may be an optical disk, such as a CD-ROM or a DVD-ROM; a magnetic disk, such as a hard disk; a solid-state memory, such as a USB flash drive or a flash memory; a memory card, such as an SD card or a TF card; a magnetic tape or the like.

[0065] like Figure 3 The conference external device is connected to the receiving unit, and the user computer is connected to the receiving unit through the communication network. Fig. 9 As shown, the client software is run on the user's computer and connected to the receiving unit to obtain the audio and video data and control data of the conference external device, and the data is transmitted to the USB external device through the USB port, such as Figure 7 As shown, the USB external device converts these data into standard UVC / UAC device data and transmits them to the user's computer (User Laptop) at the same time. Finally, the third-party instant messaging software (such as Skype) opens the UVC and UAC devices to obtain audio and video data for various communications and exchanges.

[0066] The receiving unit and the user computer respectively send and receive data from the conference external device to the user computer through the communication network, and after the data is converted into a format by the USB external device, the plug-and-play device data is provided to the user computer. Specifically, the receiving unit obtains the data (video data and control data) of the video acquisition device (such as a camera) in the conference external device through the V4L2 interface, encrypts the data, and transmits it to the user computer through the communication network.

[0067] The receiving unit obtains the data (audio data and control data) of the audio acquisition device (such as a microphone) in the conference external device through the Alsa interface, encrypts the data, and transmits it to the user's computer through the communication network.

[0068] The receiving unit obtains data (audio data and control data) from the user's computer through the communication network, decrypts the data, and transmits the data to the audio playback device in the conference external device (such as a speaker with power amplifier) ​​through the Alsa interface to play the sound.

[0069] This embodiment shows a practical application scenario. Fig.10 As shown:

[0070] After the system is set up, the cameras, such as Logitech's CC2900ep and Jabra's Speak 810UC, are connected to the receiving unit via USB and connected to a display, such as SMART's MX065-V2, via an HDMI cable.

[0071] Insert the USB peripheral into the USB port of the user's computer, run the client software in the CD-ROM, connect to the receiving unit, select the corresponding camera (such as meeting Camera), microphone (such as meetingAudioMicrophone) and speaker (such as meetingAudio Speaker) in Skype, and you can start the video conference.

[0072] Embodiment 2

[0073] This embodiment provides a method for using the conference system integrating video, voice and remote collaboration, including:

[0074] Step 1: Connect the conference external device and display device to the receiving unit and start the receiving unit.

[0075] Step 2: Connect the USB external device to the user's computer.

[0076] Step 3: Open the mass storage device and run the client software on the mass storage device.

[0077] Step 4: The control client software connects to the receiving unit based on the communication network.

[0078] Step 5: Acquire the data stream of the conference external device, and transmit the data stream to the USB external device through the USB communication interface; the data stream includes image data and audio data.

[0079] Step 6: Based on the USB external device, the data stream is sent to the user's computer according to the universal protocol.

[0080] Step 7: Perform data sharing, data conversion, conference control and remote collaboration based on client software.

[0081] Specifically, first connect the conference external device and the display device (Display) to the receiving unit, and start the receiving unit, then connect the USB external device to the user's computer. At this time, the user's computer will recognize a variety of USB devices, including but not limited to large-capacity storage devices, USB communication devices (such as Ethernet network cards), UVC (such as cameras), UAC (such as microphones, speakers), HID, etc. These devices are all plug-and-play types, using the operating system's built-in or pre-installed drivers, without the user having to install additional drivers; open the large-capacity storage device and run the client software on it; the client software will automatically or manually connect to the receiving unit through the communication network. These communication networks include Ethernet, WLAN, WiMax, Wi-Fi, Wi-Fi Direct, LiFi, ultrasound, LoRa, ZigBee or Bluetooth. Among them, the communication network gives priority to Wi-Fi Direct in the wireless network; the client software obtains the image, audio and other data streams of the conference external device from the receiving unit, and transmits these data streams to the USB external device through the USB communication interface (such as a network card). The transmitted image, audio and other data streams can be in the original format or in a modified format; then, the USB external device transmits the image data stream (which may include the converted format) to the user's computer through the UVC port in the USB. The third-party communication software (such as Skype) can directly access this UVC device (such as a camera). Similarly, the USB external device transmits the audio data stream (which may include the converted format) from the UAC port of the USB interface to the user's computer, and the client software can also directly access the UAC device for audio input (such as a microphone). Finally, the client software directly accesses the UAC device for audio output (such as a speaker) and sends the audio data stream to the device. The USB external device obtains the corresponding audio data stream from the UAC port of the USB interface and transmits it to the client software in the opposite direction through the USB communication interface (such as a network card). The client software then transmits the audio data stream to the receiving unit, and finally sends the audio data stream to the conference external device (such as a UAC device Speaker), thereby realizing the transmission of audio.

[0082] In summary, this application has the following technical effects:

[0083] 1) In video conferences or audio conferences, USB peripherals greatly improve the convenience of using conference peripherals. In a VoIP call scenario, you only need to plug the USB peripheral into the user laptop to easily enable various conference peripherals connected to the receiving unit. Whether it is a professional camera capturing clear images, a high-quality microphone accurately picking up sound, or an excellent speaker bringing a good listening experience, all can be achieved, thereby significantly improving the quality and efficiency of the meeting and providing great convenience for participants.

[0084] 2) The client software interface integrates a centralized control interface for various peripheral devices: for cameras, it integrates advanced (PTZ) control functions, including precise rotation control in the horizontal (Pan) and vertical (Tilt) directions and zoom (Zoom) operations, allowing users to accurately adjust the shooting angle and picture effect to meet the needs of diverse conference scenarios; for microphones, an intelligent amplification gain adjustment module and a convenient mute function are designed. The amplification gain adjustment can dynamically optimize the sound collection quality according to the acoustic environment of the conference site, and the mute function can quickly interrupt the sound input when necessary, ensuring the orderly progress of the meeting and the flexibility of sound management; for speakers, an intuitive and high-precision volume adjustment mechanism is built, and users can easily and accurately adjust the audio output volume to adapt to the auditory requirements of different conference links such as speeches, discussions, and multimedia playback.

[0085] 3) Through this client software, users can achieve seamless collaborative control of various conference external devices on a single interface without having to switch between multiple control software or remote controls, significantly improving work efficiency and operational convenience, and reshaping the conference interaction process and experience.

[0086] 4) USB external devices are plug-and-play devices. The operating systems of various user computers have built-in or pre-installed drivers, so there is no need to install additional drivers, which not only improves convenience but also ensures good security and stability.

[0087] 5) When multiple people use multiple USB peripheral devices, these conference peripheral devices can be easily switched and shared. Compared with the traditional hub that has only one output port and requires frequent plugging and unplugging, it is extremely convenient.

[0088] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0089] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A conference system integrating video, voice and remote collaboration, characterized in that: The conference system includes: a receiving unit, a conference external device, a display device, a user computer, a USB external device and a client software; The conference external device includes a video acquisition device, an audio acquisition device and an audio playback device; the conference external device is connected to the receiving unit via a cable or wirelessly; The display device is connected to the receiving unit via an audio and video cable; The receiving unit communicates with the user computer via a communication network; The client software runs on the user's computer and communicates with the receiving unit; the client software communicates with the USB external device through the USB port; the client software is used to realize data sharing, data conversion, conference control and remote collaboration between the receiving unit and the USB external device; The USB external device is connected to the USB port of the user's computer and communicates with the client software through the USB port; when communicating, the USB external device converts the received and sent data streams into UVC and UAC protocols, and provides them to the user's computer through the UVC interface and the UAC interface respectively.

2. A conference system integrating video, voice and remote collaboration according to claim 1, characterized in that: The video acquisition device is connected to the receiving unit via a USB cable or an HDMI cable; the audio acquisition device is connected to the receiving unit via a USB cable, an audio cable, a wireless audio receiver, or a wireless audio transceiver built into the receiving unit; the audio playback device is connected to the receiving unit via a USB cable, an audio cable, a wireless audio receiver, or a wireless audio transceiver built into the receiving unit.

3. A conference system integrating video, voice and remote collaboration according to claim 2, characterized in that: The conference external devices include any one or more of a display, a touch screen, a projector, a microphone, a speaker, a camera, a video camera, a webcam or a still camera.

4. A conference system integrating video, voice and remote collaboration according to claim 3, characterized in that: The USB port includes type interfaces of USB2.0 series, USB3.0 series and USB4.0 series.

5. A conference system integrating video, voice and remote collaboration according to claim 4, characterized in that: The USB external device includes at least one USB large-capacity data communication interface, at least one UVC interface and at least one UAC interface.

6. A conference system integrating video, voice and remote collaboration according to claim 5, characterized in that: The USB large-capacity data communication interface includes a wired network card, NDIS, ECM, NCM, a large-capacity storage device and a human-machine interface device.

7. A conference system integrating video, voice and remote collaboration according to claim 6, characterized in that: The video acquisition device includes a general UVC device or a video acquisition device with HDMI output.

8. A conference system integrating video, voice and remote collaboration according to claim 7, characterized in that: The audio acquisition device includes a universal UAC device, an audio acquisition device with analog or digital audio input and output, and an audio playback device.

9. A conference system integrating video, voice and remote collaboration according to claim 8, characterized in that: The video acquisition device and the audio acquisition device are integrated devices or separate devices.

10. A method for using a conference system integrating video, voice and remote collaboration according to any one of claims 1 to 9, characterized in that: include: Connect the conference external device and display device to the receiving unit and start the receiving unit; Connect the USB external device to the user's computer; Open the mass storage device and run the client software on the mass storage device; The control client software is connected to the receiving unit based on the communication network; Acquire a data stream of a conference external device, and transmit the data stream to the USB external device through a USB communication interface; the data stream includes image data and audio data; Based on USB external devices, the data stream is sent to the user's computer according to the universal protocol; Data sharing, data conversion, conference control and remote collaboration are carried out based on client software.

Images