Data interaction method based on multi-protocol multiplexing USB expansion and related device

Abstract

The invention discloses a data interaction method based on multi-protocol multiplexing USB expansion and a related device. The method comprises the following steps: forming a connection state among a POE controller, conference equipment and a host end; the conference equipment collects video data, audio data and coordinate data, and compresses the video data; transmitting packaged data of the compressed video data, the audio data and the coordinate data to a POE (Power Over Ethernet) controller based on a virtual USB (Universal Serial Bus) network card; analyzing the packaged data, determining enumerated USB Gadget composite equipment information, transmitting the analyzed data to a host end, reporting video data and audio data to target software by the host end, responding to a touch event, capturing screen display content, and reversely spreading the screen display content to a POE controller; and coding and packaging the screen display content, and transmitting the target data packet to the conference equipment for analysis and display. According to the invention, low-delay and high-quality synchronous transmission of multiple paths of data streams is realized in a limited bandwidth.

Description

Technical Field

[0001] This invention relates to the field of data interaction technology, and in particular to a data interaction method and related apparatus based on USB extension with multiplexed protocols. Background Technology

[0002] With the increasing prevalence of remote work, smart classrooms, and industrial control, data transmission and interaction are becoming more frequent. In data transmission and interaction, USB devices are typically deployed far from the host. Traditional solutions for data interaction between USB devices and the host include: Solution 1: Using active extension cables or fiber optic USB cables. However, the former has limited range, and the latter is expensive; both require separate power cables, making deployment cumbersome. Solution 2: Transmitting USB data via Ethernet. This solution usually requires a pair of dedicated hardware transceivers and can mostly only extend a single USB device or a USB hub, failing to efficiently and with low latency transmit multiple devices requiring high bandwidth and low latency simultaneously. Solution 3: Installing specific software on both the device and the PC to transmit audio and video via webcams and audio protocols. However, this often cannot seamlessly transmit HID data simultaneously, requiring additional connections and drivers, resulting in low integration and a fragmented user experience. Therefore, existing technologies generally suffer from complex cabling, high costs, limited transmission distances, and the inability to efficiently reuse multiple device protocols simultaneously. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art. This invention provides a data interaction method and related device based on USB extension with multiple protocols, which realizes low-latency, high-quality synchronous transmission of multiple data streams within a limited bandwidth, and overcomes problems such as audio-visual asynchrony and high touch latency.

[0004] To address the aforementioned technical problems, this invention provides a data interaction method based on a multi-protocol multiplexing USB extension, the method comprising: Power conversion is performed using a PoE (Power over Ethernet) decoupling module to obtain the power conversion result, and the connection status between the PoE controller, conferencing equipment, and host is determined based on the power conversion result. The conferencing equipment in a connected state acquires video data, audio data, and coordinate data, and compresses the video data to obtain compressed video data; A virtual Universal Serial Bus (USB) network card is created, and the compressed video data, audio data, and coordinate data are encapsulated to obtain encapsulated data. The encapsulated data is then transmitted to the PoE controller in a connected state based on the virtual USB network card. The PoE controller parses the encapsulation data, obtains the parsed data, and determines the USB Gadget composite device information enumerated by the PoE controller to the host. Based on the USB Gadget composite device information, the parsed data is transmitted to the host. The host obtains video data, audio data, and coordinate data based on the parsed data, and reports the video data and audio data to the target software. In response to the touch event corresponding to the coordinate data, the host captures the screen display content and propagates the screen display content back to the PoE controller. The PoE controller encodes and encapsulates the screen display content on the host side to obtain the target data packet, and transmits the target data packet to the conferencing device. The conferencing device parses the target data packet to obtain the parsed screen display content, and then displays the parsed screen display content.

[0005] Optionally, the power conversion based on the Power over Ethernet (PoE) decoupling module to obtain the power conversion result includes: The bridge rectifier circuit based on the POE separation module determines the corrected DC voltage; The corrected DC voltage is converted from DC to DC to obtain the corrected DC voltage after DC-DC conversion; The corrected DC voltage after DC-DC conversion is filtered to obtain the power conversion result.

[0006] Optionally, the step of compressing the video data to obtain compressed video data includes: Call the video processing unit application programming interface; Based on the video processing unit's application programming interface, the video data is compressed using an H.265 bitstream via a hardware codec library to obtain compressed video data.

[0007] Optionally, the step of creating a virtual Universal Serial Bus (USB) network card and encapsulating the compressed video data, audio data, and coordinate data to obtain encapsulated data includes: Configure the network control model (NCM) based on the USB Gadget driver, and build a virtual USB network card based on the NCM; The compressed video data, audio data, and coordinate data are encapsulated into a Socket data frame to obtain encapsulated data.

[0008] Optionally, the PoE controller parses the encapsulation data to obtain parsed data and determines the USB Gadget composite device information enumerated by the PoE controller to the host, including: The PoE controller obtains the header information from the encapsulated data based on a proprietary protocol, and parses the encapsulated data based on the header information to obtain parsed data; Based on the Linux USB Gadget driver, the OTG interface is used to determine the USB Gadget composite device information enumerated by the PoE controller to the host. The USB Gadget composite device information includes USB video devices, USB audio devices, and USB human-machine interface devices.

[0009] Optionally, the USB video class is used to forward the video stream in the parsed data, the USB audio class is used to forward the audio data in the parsed data, and the USB human-machine interface device is used to pass through the coordinate data in the parsed data.

[0010] Optionally, the PoE controller encodes and encapsulates the screen display content on the host side to obtain a target data packet, transmits the target data packet to the conferencing device, and the conferencing device parses the target data packet to obtain the parsed screen display content, including: The PoE controller encapsulates the screen display content into an IP data packet to obtain the target data packet; The target data packet is transmitted to the conferencing device based on NCM; The conferencing equipment obtains the header information of each data segment in the target data packet based on a proprietary protocol; The target data packet is parsed based on the header information of each data segment to obtain the content to be displayed on the screen.

[0011] In addition, the present invention also provides a data interaction device based on USB extension with multiple protocol reuse, the device comprising: Device connection module: Used to perform power conversion based on the Power over Ethernet (PoE) split module, obtain the power conversion result, and establish the connection status between the PoE controller, conferencing equipment, and host based on the power conversion result; Data processing module: used to collect video data, audio data and coordinate data of the connected conferencing equipment, and to compress the video data to obtain compressed video data; Data transmission module: used to create a virtual universal serial bus (USB) network card, encapsulate the compressed video data, audio data, and coordinate data to obtain encapsulated data, and transmit the encapsulated data to the PoE controller in a connected state based on the virtual USB network card; Device information enumeration module: used by the POE controller to parse the encapsulation data, obtain the parsed data, and determine the USB Gadget composite device information enumerated by the POE controller to the host. The screen information transmission module is used to transmit the parsed data to the host based on the USB Gadget composite device information. The host obtains video data, audio data and coordinate data based on the parsed data, and reports the video data and audio data to the target software. In response to the touch event corresponding to the coordinate data, the host captures the screen display content and propagates the screen display content back to the PoE controller. Screen display module: Used by the PoE controller to encode and encapsulate the screen display content on the host side, obtain the target data packet, transmit the target data packet to the conferencing device, parse the target data packet, obtain the parsed screen display content, and display the parsed screen display content.

[0012] In addition, the present invention also provides an electronic device, which includes a processor and a memory. The memory is used to store instructions, and the processor is used to call the instructions in the memory to cause the electronic device to execute the above-described data interaction method based on USB extension of multiplexed protocol.

[0013] In addition, the present invention provides a computer-readable storage medium that stores computer instructions that, when executed on an electronic device, cause the electronic device to perform the aforementioned data interaction method based on USB extension for multiplexing multiplexing protocols.

[0014] In this embodiment of the invention, power conversion is performed based on a PoE separation module to obtain the power conversion result. Based on the power conversion result, a connection state is established between the PoE controller, the conferencing equipment, and the host, reducing the cabling complexity of the device connections. The conferencing equipment in the connected state acquires video data, audio data, and coordinate data. The video data is compressed to obtain compressed video data. A virtual USB network card is created, and the compressed video data, audio data, and coordinate data are encapsulated to obtain encapsulated data. Based on the virtual USB network card, the encapsulated data is transmitted to the PoE controller in the connected state, achieving smooth and synchronous data transmission while significantly increasing the data transmission distance. The PoE controller parses the encapsulated data to determine the USB Gadget composite device information enumerated by the PoE controller to the host. Enumerating the USB Gadget composite device information enables plug-and-play functionality on the host, requiring no additional drivers. Based on the USB Gadget composite device information, the parsed data is transparently transmitted to the host. The host obtains video data, audio data, and coordinate data based on the parsed data, and reports the video data and audio data to the target software. Responding to the touch event corresponding to the coordinate data, the host captures the screen display content and propagates it back to the PoE controller. The PoE controller encodes and encapsulates the screen display content of the host to obtain the target data packet, and transmits the target data packet to the conferencing device. The conferencing device parses the target data packet to obtain the parsed screen display content and displays it. Within a limited bandwidth, low-latency, high-quality synchronous transmission of multiple data streams is achieved. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart illustrating the data interaction method based on USB extension with multiple protocol reuse in an embodiment of the present invention. Figure 2 This is a flowchart illustrating a data interaction method based on USB extension with multiple protocol reuse in another embodiment of the present invention. Figure 3 This is a schematic diagram of the structural composition of a data interaction device based on USB extension with multiple protocol reuse in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structural composition of the electronic device in an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example 1 Please see Figure 1 , Figure 1 This is a flowchart illustrating a data interaction method based on USB extension with multiple protocol reuse in an embodiment of the present invention. The method includes: S11: Power conversion is performed based on the Power over Ethernet (PoE) separation module to obtain the power conversion result, and the connection status between the PoE controller, conferencing equipment and host is formed based on the power conversion result; In the specific implementation of this invention, the bridge rectifier circuit based on the Power over Ethernet (POE) separation module determines the correction DC voltage; the correction DC voltage is converted from DC to DC to obtain the correction DC voltage after DC-DC conversion; the correction DC voltage after DC-DC conversion is filtered to obtain the power conversion result; the connection status between the POE controller, conference equipment and host is formed based on the power conversion result, which can realize power supply and physical connection with simpler wiring.

[0019] S12: The conferencing device in the connected state acquires video data, audio data, and coordinate data, and compresses the video data to obtain compressed video data; In the specific implementation of this invention, the conferencing device in a connected state collects video data, audio data, and coordinate data, and calls the video processing unit application programming interface; based on the video processing unit application programming interface, the hardware codec library is used to compress the video data into an H.265 bitstream, which can achieve significant compression of the video data.

[0020] S13: Create a virtual Universal Serial Bus (USB) network card, encapsulate the compressed video data, audio data, and coordinate data to obtain encapsulated data, and transmit the encapsulated data to the PoE controller in the connected state based on the virtual USB network card; In the specific implementation of this invention, a network control model (NCM) is configured based on the USB Gadget driver, and a virtual universal serial bus (USB) network card is constructed based on the NCM; the compressed video data, audio data, and coordinate data are encapsulated into a socket data frame to obtain encapsulated data, and the encapsulated data is transmitted to the PoE controller in a connected state based on the virtual USB network card to achieve stable transmission of multiple data streams.

[0021] S14: The POE controller parses the encapsulation data, obtains the parsed data, and determines the USB Gadget composite device information enumerated by the POE controller to the host. In the specific implementation of this invention, the PoE controller obtains the header information in the encapsulated data based on a private protocol, and parses the encapsulated data based on the header information to obtain parsed data; based on the Linux USB Gadget driver, it uses the (On-The-Go, OTG) interface to determine the USB Gadget composite device information enumerated by the PoE controller to the host. The USB Gadget composite device information includes USB video, USB audio, and USB human-machine interface devices, enabling the host to use it plug and play without additional drivers.

[0022] S15: Based on the USB Gadget composite device information, the parsed data is transmitted to the host terminal. The host terminal obtains video data, audio data and coordinate data based on the parsed data, and reports the video data and audio data to the target software. In response to the touch event corresponding to the coordinate data, the host terminal captures the screen display content and propagates the screen display content back to the POE controller. In the specific implementation of this invention, the parsed data is transmitted to the host based on the USB Gadget composite device information. The host obtains video data, audio data, and coordinate data based on the parsed data, and reports the video data and audio data to the target software. In response to the touch event corresponding to the coordinate data, the host captures its own screen display content and propagates the screen display content back to the PoE controller, ensuring efficient bandwidth utilization.

[0023] S16: The PoE controller encodes and encapsulates the screen display content on the host side to obtain the target data packet, transmits the target data packet to the conferencing device, parses the target data packet to obtain the parsed screen display content, and displays the parsed screen display content.

[0024] In the specific implementation of this invention, the PoE controller encapsulates the screen display content into an Internet Protocol (IP) data packet to obtain a target data packet; the target data packet is transmitted to the conferencing device based on NCM; the conferencing device obtains the header information of each data segment in the target data packet based on a private protocol; the target data packet is parsed based on the header information of each data segment to obtain the parsed screen display content, and the parsed screen display content is displayed, thus providing a one-stop solution for the remote extension needs of video, audio, and touch signals, and satisfying the bidirectional transmission of data streams at the conferencing device end.

[0025] In this embodiment of the invention, power conversion is performed based on a PoE separation module to obtain the power conversion result. Based on the power conversion result, a connection state is established between the PoE controller, the conferencing equipment, and the host, reducing the cabling complexity of the device connections. The conferencing equipment in the connected state acquires video data, audio data, and coordinate data. The video data is compressed to obtain compressed video data. A virtual USB network card is created, and the compressed video data, audio data, and coordinate data are encapsulated to obtain encapsulated data. Based on the virtual USB network card, the encapsulated data is transmitted to the PoE controller in the connected state, achieving smooth and synchronous data transmission while significantly increasing the data transmission distance. The PoE controller parses the encapsulated data to determine the USB Gadget composite device information enumerated by the PoE controller to the host. Enumerating the USB Gadget composite device information enables plug-and-play functionality on the host, requiring no additional drivers. Based on the USB Gadget composite device information, the parsed data is transparently transmitted to the host. The host obtains video data, audio data, and coordinate data based on the parsed data, and reports the video data and audio data to the target software. Responding to the touch event corresponding to the coordinate data, the host captures the screen display content and propagates it back to the PoE controller. The PoE controller encodes and encapsulates the screen display content of the host to obtain the target data packet, and transmits the target data packet to the conferencing device. The conferencing device parses the target data packet to obtain the parsed screen display content and displays it. Within a limited bandwidth, low-latency, high-quality synchronous transmission of multiple data streams is achieved.

[0026] Example 2 Please see Figure 2 , Figure 2 This is a flowchart illustrating a data interaction method based on USB extension with multiple protocol reuse according to another embodiment of the present invention. The method includes: S201: Power conversion is performed based on the Power over Ethernet (PoE) separation module to obtain the power conversion result, and the connection status between the PoE controller, conferencing equipment and host is formed based on the power conversion result; In the specific implementation of this invention, the power conversion based on the Power over Ethernet (PoE) separation module to obtain the power conversion result includes: determining the correction DC voltage based on the bridge rectifier circuit of the PoE separation module; performing DC-DC conversion on the correction DC voltage to obtain the correction DC voltage after DC-DC conversion; and filtering the correction DC voltage after DC-DC conversion to obtain the power conversion result.

[0027] Specifically, the PoE controller connects to the conferencing equipment via a network cable. The equipment supplies DC power (e.g., 48V) to the controller through its PoE port. The PoE disconnect module inside the controller converts the power into the operating voltage required by the chips and peripheral devices. The bridge rectifier circuit within the PoE disconnect module determines the calibration DC voltage. This circuit automatically corrects the power supplied by the idle wire pairs of the network cable to a fixed DC voltage polarity, which is the calibration DC voltage. The rectified voltage is typically 44-57V DC (compliant with IEEE 802.3af / at standards).

[0028] Since peripheral devices typically require operating voltages such as 3.3V / 5V / 12V, the corrected DC voltage needs to be converted from DC to DC to obtain a corrected DC voltage after DC-DC conversion. This corrected DC voltage is then filtered to obtain the power conversion result. The DC-DC conversion and filtering processes are performed using corresponding DC-DC conversion and filtering circuits to obtain a clean and stable low-voltage DC power, which is the power conversion result. Based on the power conversion result, a connection is established between the PoE controller, the conference equipment, and the host. The output terminals of the PoE split module provide power to the peripheral devices. The controller, as a device with a display screen and touchscreen, can act as a USB controller to operate the conference equipment. Furthermore, the connection between the PoE controller, the conference equipment, and the host requires only a single standard Ethernet cable. This single cable solves both power supply and data transmission issues for remote devices, significantly reducing cabling costs and complexity, making it particularly suitable for ceiling-mounted or wall-mounted installations.

[0029] S202: The conferencing device in the connected state acquires video data, audio data, and coordinate data, and compresses the video data to obtain compressed video data; In a specific implementation of this invention, the step of compressing the video data to obtain compressed video data includes: calling the video processing unit application programming interface; and using the hardware codec library based on the video processing unit application programming interface to compress the video data into an H.265 bitstream to obtain compressed video data.

[0030] Specifically, the conferencing equipment in a connected state collects video data, audio data, and coordinate data. The video data corresponds to camera data, the audio data corresponds to mic data, and the coordinate data corresponds to touch data. The camera module on the conferencing equipment collects video data, its microphone collects audio data, and the touch screen collects coordinate data.

[0031] The video processing unit (VPU) API is invoked, such as the VPU API on the RK3588 platform. Based on this API, a hardware codec library is used to compress the video data into an H.265 stream, encoding the original NV12 data into H.265 format to obtain compressed video data. This significantly reduces the data volume. The hardware codec library can be from chip platforms such as Rockchip, Qualcomm, or MediaTek.

[0032] S203: Configure the network control model NCM based on the USB Gadget driver, and build a virtual USB network card based on the NCM; In the specific implementation of this invention, the conference device runs an Android / Linux system and loads a USBGadget driver. The USB Gadget driver is a framework in the Linux kernel used to implement USB device software functions. It allows embedded systems to connect to the host as USB slave devices and emulate various USB device classes. It is configured to emulate NCM, a sub-protocol of the USB communication device class, designed for efficient transmission of Ethernet data frames via the USB interface. It allows USB devices to be virtualized as network interface devices on the host operating system, enabling high-speed network connections. A virtual USB network card is constructed based on the NCM, and through the USB gadget driver supported by the Linux kernel, the device's OTG interface is configured to enumerate as a USB Device, where the NCM device is the virtual USB network card.

[0033] S204: Encapsulate the compressed video data, audio data, and coordinate data into a Socket data frame to obtain encapsulated data, and transmit the encapsulated data to the PoE controller in a connected state based on the virtual USB network card; In the specific implementation of this invention, the compressed video data, audio data, and coordinate data are encapsulated into a Socket data frame to obtain encapsulated data. The data encapsulation uses a proprietary protocol, combining each data item with header information (type, format, length, etc.) into a single large packet. The encapsulated data is then transmitted to the PoE controller in a connected state via a virtual USB network card. The high-bandwidth video stream is efficiently compressed using H.265 and then encapsulated together with low-latency audio and touch data into a network data packet established by NCM, achieving stable and synchronous transmission of multiple data streams under limited bandwidth.

[0034] S205: The PoE controller obtains the header information in the encapsulated data based on a proprietary protocol, and parses the encapsulated data based on the header information to obtain parsed data; In the specific implementation of this invention, the PoE controller obtains the packet header information from the encapsulated data based on a proprietary protocol, that is, it obtains the packet header information of each data segment in a large packet of encapsulated data according to the proprietary protocol. Based on the packet header information, the encapsulated data is parsed to obtain parsed data. By extracting and parsing the encapsulated data through the packet header information, the original video, audio, and coordinate data, as well as their corresponding formats and sizes, are obtained, which is the parsed data.

[0035] S206: Based on the Linux USB Gadget driver, the OTG interface is used to determine the USB Gadget composite device information enumerated by the PoE controller to the host. The USB Gadget composite device information includes USB video devices, USB audio devices, and USB human-machine interface devices. In the specific implementation of this invention, the USB video class is used to forward the video bitstream in the parsed data, the USB audio class is used to forward the audio data in the parsed data, and the USB human-machine interface device is used to pass through the coordinate data in the parsed data.

[0036] Specifically, based on the Linux USB Gadget driver, the OTG interface is used to determine the USB Gadget composite device information enumerated by the PoE controller to the host. That is, the Linux USB Gadget driver enumerates composite devices via the OTG port (similar to the driver implementation for conferencing equipment, used to forward various types of data to the host). This USB Gadget composite device information includes USB Video Class (UVC), USB Audio Class (UAC), and USB Human Interface Device (USB HID). The USB Video Class is used to forward and parse the video stream in the data, originating from camera data on the conferencing equipment transmitted via data packets. The USB Audio Class is used to forward and parse the audio data, originating from microphone array data on the conferencing equipment transmitted via data packets. The USB HID is used to transmit coordinate data from the parsed data, and can also transmit touch and button events from the conferencing equipment. A composite USB device integrating UVC, UAC, and HID is simulated through the software driver, enabling plug-and-play functionality on the host without the need for additional drivers.

[0037] S207: Based on the USB Gadget composite device information, the parsed data is transmitted to the host terminal. The host terminal obtains video data, audio data, and coordinate data based on the parsed data, and reports the video data and audio data to the target software. In response to the touch event corresponding to the coordinate data, the host terminal captures the screen display content and propagates the screen display content back to the POE controller. In the specific implementation of this invention, the parsed data is transparently transmitted to the host based on the USB Gadget composite device information. This can be achieved by using a Type-C interface to write the USB Gadget composite device information to the host. The host obtains video data, audio data, and coordinate data based on the parsed data and reports the video data and audio data to the target software, i.e., uploading camera data and mic data to the target app. The host responds to the touch event corresponding to the coordinate data, i.e., responds to the touch event. The host captures its own screen display content based on the video data, audio data, and touch events obtained from the parsed data and transmits the screen display content back to the PoE controller through the DisplayPort (DP) channel.

[0038] S208: The PoE controller encodes and encapsulates the screen display content on the host side to obtain the target data packet, transmits the target data packet to the conferencing device, parses the target data packet to obtain the parsed screen display content, and displays the parsed screen display content.

[0039] In a specific implementation of this invention, the PoE controller encapsulates the screen display content on the host side to obtain a target data packet, and transmits the target data packet to the conferencing device. The conferencing device parses the target data packet to obtain the parsed screen display content. This includes: the PoE controller encapsulating the screen display content into an IP data packet to obtain the target data packet; transmitting the target data packet to the conferencing device based on NCM; the conferencing device obtaining the header information of each data segment in the target data packet based on a proprietary protocol; and parsing the target data packet based on the header information of each data segment to obtain the parsed screen display content.

[0040] Specifically, the PoE controller encapsulates the screen display content into IP data packets, also using the H.265 encoding format, to obtain the target data packet. Based on NCM, the target data packet is transmitted to the conferencing device, which then obtains the header information of each data segment within the target data packet using a proprietary protocol.

[0041] The target data packet is parsed based on the header information of each data segment to obtain the parsed screen display content. The parsed screen information includes video, audio, and coordinate data. The parsed screen display content is then displayed to realize the function of PC screen projection to the conference equipment. Since the data transmission between the controller and the conference equipment is essentially via Ethernet to USB 2.0, the use of H.265 high compression rate bitstream can realize the uptransmission of 4K 30fps video stream via USB 2.0. This provides a one-stop solution for the remote extension needs of video, audio, and touch signals, and provides additional network channels for value-added functions (such as private screen projection). At the same time, it meets the bidirectional transmission of UVC / UAC / HID data streams with the conference equipment and can simultaneously transmit custom screen projection control protocols, realizing the collaborative work of public standards and private protocols.

[0042] In this embodiment of the invention, power conversion is performed based on a PoE separation module to obtain the power conversion result. Based on the power conversion result, a connection state is established between the PoE controller, the conferencing equipment, and the host, reducing the cabling complexity of the device connections. The conferencing equipment in the connected state acquires video data, audio data, and coordinate data. The video data is compressed to obtain compressed video data. A virtual USB network card is created, and the compressed video data, audio data, and coordinate data are encapsulated to obtain encapsulated data. Based on the virtual USB network card, the encapsulated data is transmitted to the PoE controller in the connected state, achieving smooth and synchronous data transmission while significantly increasing the data transmission distance. The PoE controller parses the encapsulated data to determine the USB Gadget composite device information enumerated by the PoE controller to the host. Enumerating the USB Gadget composite device information enables plug-and-play functionality on the host, requiring no additional drivers. Based on the USB Gadget composite device information, the parsed data is transparently transmitted to the host. The host obtains video data, audio data, and coordinate data based on the parsed data, and reports the video data and audio data to the target software. Responding to the touch event corresponding to the coordinate data, the host captures the screen display content and propagates it back to the PoE controller. The PoE controller encodes and encapsulates the screen display content of the host to obtain the target data packet, and transmits the target data packet to the conferencing device. The conferencing device parses the target data packet to obtain the parsed screen display content and displays it. Within a limited bandwidth, low-latency, high-quality synchronous transmission of multiple data streams is achieved.

[0043] Example 3 Please see Figure 3 , Figure 3 This is a schematic diagram of the structural composition of a data interaction device based on USB multiplexing extension according to an embodiment of the present invention. The device includes: Device connection module 31: used to perform power conversion based on the Power over Ethernet (PoE) separation module, obtain the power conversion result, and form the connection status between the PoE controller, the conference equipment and the host based on the power conversion result; Data processing module 32: used to collect video data, audio data and coordinate data of the conferencing equipment in the connected state, and to compress the video data to obtain compressed video data; Data transmission module 33: used to create a virtual universal serial bus (USB) network card, encapsulate the compressed video data, audio data and coordinate data to obtain encapsulated data, and transmit the encapsulated data to the PoE controller in the connected state based on the virtual USB network card; Device information enumeration module 34: used by the POE controller to parse the encapsulation data, obtain the parsed data, and determine the USB Gadget composite device information enumerated by the POE controller to the host. The screen information transmission module 35 is used to transmit the parsed data to the host based on the USB Gadget composite device information. The host obtains video data, audio data and coordinate data based on the parsed data, and reports the video data and audio data to the target software. In response to the touch event corresponding to the coordinate data, the host captures the screen display content and transmits the screen display content back to the POE controller. Screen display module 36: Used by the PoE controller to encode and encapsulate the screen display content on the host side, obtain the target data packet, transmit the target data packet to the conferencing device, parse the target data packet, obtain the parsed screen display content, and display the parsed screen display content.

[0044] In the specific implementation of this invention, the specific implementation of the device item can be referred to the implementation of the method item above, and will not be repeated here.

[0045] In this embodiment of the invention, power conversion is performed based on a PoE separation module to obtain the power conversion result. Based on the power conversion result, a connection state is established between the PoE controller, the conferencing equipment, and the host, reducing the cabling complexity of the device connections. The conferencing equipment in the connected state acquires video data, audio data, and coordinate data. The video data is compressed to obtain compressed video data. A virtual USB network card is created, and the compressed video data, audio data, and coordinate data are encapsulated to obtain encapsulated data. Based on the virtual USB network card, the encapsulated data is transmitted to the PoE controller in the connected state, achieving smooth and synchronous data transmission while significantly increasing the data transmission distance. The PoE controller parses the encapsulated data to determine the USB Gadget composite device information enumerated by the PoE controller to the host. Enumerating the USB Gadget composite device information enables plug-and-play functionality on the host, requiring no additional drivers. Based on the USB Gadget composite device information, the parsed data is transparently transmitted to the host. The host obtains video data, audio data, and coordinate data based on the parsed data, and reports the video data and audio data to the target software. Responding to the touch event corresponding to the coordinate data, the host captures the screen display content and propagates it back to the PoE controller. The PoE controller encodes and encapsulates the screen display content of the host to obtain the target data packet, and transmits the target data packet to the conferencing device. The conferencing device parses the target data packet to obtain the parsed screen display content and displays it. Within a limited bandwidth, low-latency, high-quality synchronous transmission of multiple data streams is achieved.

[0046] This invention provides a computer-readable storage medium storing a computer program. When executed by a processor, this program implements the data interaction method based on USB extension and multiplexing of multiplexing, as described in any of the above embodiments. The computer-readable storage medium includes, but is not limited to, any type of disk (including floppy disks, hard disks, optical disks, CD-ROMs, and magneto-optical disks), ROM (Read-Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, magnetic cards, or optical cards. In other words, the storage device includes any medium that stores or transmits information in a readable form by a device (e.g., a computer, a mobile phone), and can be a read-only memory, a disk, or an optical disk, etc.

[0047] Example 4 Please see Figure 4 , Figure 4 This is a schematic diagram of the structural composition of the electronic device in an embodiment of the present invention.

[0048] This invention also provides an electronic device, such as... Figure 4 As shown, the electronic device includes a memory 41, a processor 43, and a computer program 42 stored in the memory 41 and executable on the processor 43. Those skilled in the art will understand that... Figure 4 The illustrated electronic device does not constitute a limitation on all devices and may include more or fewer components than illustrated, or combine certain components. Memory 41 can be used to store computer program 42 and various functional modules. Processor 43 runs the computer program 42 stored in memory 41, thereby performing various functional applications and data processing of the device. Memory can be internal memory or external memory, or both. Internal memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, or random access memory. External memory may include hard disks, floppy disks, ZIP disks, USB flash drives, magnetic tapes, etc. Processor 43 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor, a single-chip microcomputer, or a processor 43, or any conventional processor, etc. The processors and memories disclosed in this invention include, but are not limited to, these types of processors and memories. The processors and memories disclosed in this invention are merely examples and not intended to be limiting.

[0049] As one embodiment, the electronic device includes: one or more processors 43, a memory 41, and one or more computer programs 42, wherein the one or more computer programs 42 are stored in the memory 41 and configured to be executed by the one or more processors 43, and the one or more computer programs 42 are configured to perform the data interaction method based on USB extension of multi-protocol multiplexing in any of the above embodiments. For specific implementation process, please refer to the above embodiments, which will not be repeated here.

[0050] In this embodiment of the invention, power conversion is performed based on a PoE separation module to obtain the power conversion result. Based on the power conversion result, a connection state is established between the PoE controller, the conferencing equipment, and the host, reducing the cabling complexity of the device connections. The conferencing equipment in the connected state acquires video data, audio data, and coordinate data. The video data is compressed to obtain compressed video data. A virtual USB network card is created, and the compressed video data, audio data, and coordinate data are encapsulated to obtain encapsulated data. Based on the virtual USB network card, the encapsulated data is transmitted to the PoE controller in the connected state, achieving smooth and synchronous data transmission while significantly increasing the data transmission distance. The PoE controller parses the encapsulated data to determine the USB Gadget composite device information enumerated by the PoE controller to the host. Enumerating the USB Gadget composite device information enables plug-and-play functionality on the host, requiring no additional drivers. Based on the USB Gadget composite device information, the parsed data is transparently transmitted to the host. The host obtains video data, audio data, and coordinate data based on the parsed data, and reports the video data and audio data to the target software. Responding to the touch event corresponding to the coordinate data, the host captures the screen display content and propagates it back to the PoE controller. The PoE controller encodes and encapsulates the screen display content of the host to obtain the target data packet, and transmits the target data packet to the conferencing device. The conferencing device parses the target data packet to obtain the parsed screen display content and displays it. Within a limited bandwidth, low-latency, high-quality synchronous transmission of multiple data streams is achieved.

[0051] Furthermore, the above provides a detailed description of a data interaction method and related apparatus based on USB extension with multiple protocol reuse provided by the embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A data interaction method based on USB extension with multiplexed protocols, characterized in that, The method includes: Power conversion is performed using a PoE (Power over Ethernet) decoupling module to obtain the power conversion results, and the connection status between the PoE controller, conferencing equipment, and host is determined based on the power conversion results. The conferencing equipment in a connected state acquires video data, audio data, and coordinate data, and compresses the video data to obtain compressed video data; A virtual Universal Serial Bus (USB) network card is created, and the compressed video data, audio data, and coordinate data are encapsulated to obtain encapsulated data. The encapsulated data is then transmitted to the PoE controller in a connected state based on the virtual USB network card. The PoE controller parses the encapsulation data, obtains the parsed data, and determines the USB Gadget composite device information enumerated by the PoE controller to the host. Based on the USB Gadget composite device information, the parsed data is transmitted to the host. The host obtains video data, audio data, and coordinate data based on the parsed data, and reports the video data and audio data to the target software. In response to the touch event corresponding to the coordinate data, the host captures the screen display content and propagates the screen display content back to the PoE controller. The PoE controller encodes and encapsulates the screen display content on the host side to obtain the target data packet, and transmits the target data packet to the conferencing device. The conferencing device parses the target data packet to obtain the parsed screen display content, and then displays the parsed screen display content.

2. The data interaction method based on USB extension with multiplexed protocols according to claim 1, characterized in that, The power conversion based on the Power over Ethernet (PoE) decoupling module, and the resulting power conversion, include: The bridge rectifier circuit based on the POE separation module determines the corrected DC voltage; The corrected DC voltage is converted from DC to DC to obtain the corrected DC voltage after DC-DC conversion; The corrected DC voltage after DC-DC conversion is filtered to obtain the power conversion result.

3. The data interaction method based on USB extension with multiple protocol reuse as described in claim 1, characterized in that, The step of compressing the video data to obtain compressed video data includes: Call the video processing unit application programming interface; Based on the video processing unit's application programming interface, the video data is compressed using an H.265 bitstream via a hardware codec library to obtain compressed video data.

4. The data interaction method based on USB extension with multiple protocol reuse as described in claim 1, characterized in that, The process of creating a virtual Universal Serial Bus (USB) network card and encapsulating the compressed video data, audio data, and coordinate data to obtain encapsulated data includes: Configure the network control model (NCM) based on the USB Gadget driver, and build a virtual USB network card based on the NCM; The compressed video data, audio data, and coordinate data are encapsulated into a Socket data frame to obtain encapsulated data.

5. The data interaction method based on USB extension with multiplexed protocols according to claim 1, characterized in that, The PoE controller parses the encapsulation data to obtain parsed data and determines the USBGadget composite device information enumerated by the PoE controller to the host, including: The PoE controller obtains the header information from the encapsulated data based on a proprietary protocol, and parses the encapsulated data based on the header information to obtain parsed data; Based on the Linux USB Gadget driver, the OTG interface is used to determine the USB Gadget composite device information enumerated by the PoE controller to the host. The USB Gadget composite device information includes USB video devices, USB audio devices, and USB human-machine interface devices.

6. The data interaction method based on USB extension with multiple protocol reuse as described in claim 5, characterized in that, The USB video class is used to forward the video stream in the parsed data, the USB audio class is used to forward the audio data in the parsed data, and the USB human-machine interface device is used to pass through the coordinate data in the parsed data.

7. The data interaction method based on USB extension with multiplexed protocols according to claim 1, characterized in that, The PoE controller encodes and encapsulates the screen display content on the host side to obtain a target data packet, and transmits the target data packet to the conferencing device. The conferencing device parses the target data packet to obtain the parsed screen display content, including: The PoE controller encapsulates the screen display content into an IP data packet to obtain the target data packet; The target data packet is transmitted to the conferencing device based on NCM; The conferencing equipment obtains the header information of each data segment in the target data packet based on a proprietary protocol; The target data packet is parsed based on the header information of each data segment to obtain the content to be displayed on the screen.

8. A data interaction device based on USB extension with multiplexed protocols, characterized in that, The device includes: Device connection module: Used to perform power conversion based on the Power over Ethernet (PoE) split module, obtain the power conversion result, and establish the connection status between the PoE controller, conferencing equipment, and host based on the power conversion result; Data processing module: used to collect video data, audio data and coordinate data of the connected conferencing equipment, and to compress the video data to obtain compressed video data; Data transmission module: used to create a virtual universal serial bus (USB) network card, encapsulate the compressed video data, audio data, and coordinate data to obtain encapsulated data, and transmit the encapsulated data to the PoE controller in a connected state based on the virtual USB network card; Device information enumeration module: used by the POE controller to parse the encapsulation data, obtain the parsed data, and determine the USB Gadget composite device information enumerated by the POE controller to the host. The screen information transmission module is used to transmit the parsed data to the host based on the USB Gadget composite device information. The host obtains video data, audio data and coordinate data based on the parsed data, and reports the video data and audio data to the target software. It responds to the touch event corresponding to the coordinate data, and the host captures the screen display content and propagates the screen display content back to the PoE controller. Screen display module: Used by the PoE controller to encode and encapsulate the screen display content on the host side, obtain target data packets, transmit the target data packets to the conferencing device, parse the target data packets, obtain the parsed screen display content, and display the parsed screen display content.

9. An electronic device, the electronic device comprising a processor and a memory, characterized in that, The memory is used to store instructions, and the processor is used to call the instructions in the memory to cause the electronic device to execute the data interaction method based on the USB extension of multiplexed multiprotocol as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed on an electronic device, cause the electronic device to perform the data interaction method based on USB extension of multiplexed protocol as claimed in any one of claims 1 to 7.

Images