USB tunnel management method and apparatus

Abstract

Disclosed are a USB tunnel management method and apparatus, which relate to the technical field of multimedia. Various stages of virtual paths on a virtual channel for a USB tunnel service flow are established between an initiating device and a target device for the USB tunnel service flow by means of processing an adapter list request, an adapter list response, a bandwidth application request and a bandwidth application response, and a transmission route corresponding to the virtual channel is flexibly determined, thereby preventing the transmission route from relying on fixed physical line connection; and the USB tunnel service flow is transmitted between two ports by means of the virtual paths, such that the transmission of the USB tunnel service flow is decoupled from the mode of physical line connection between the ports, thereby further improving the flexibility of change in the transmission route of the USB tunnel service flow.

Description

USB tunnel management method and device Technical Field

[0001] The present application relates to the field of multimedia technology, and in particular to a USB tunnel management method and device. Background Art

[0002] In traditional Universal Serial Bus (USB) networks, whether a host device with USB host functionality can access USB resources depends on the physical link between the host device and the USB resources. Therefore, switching USB resources between host devices requires changing the physical link between the host device and the USB resources, resulting in limited USB networking flexibility.

[0003] Summary of the Invention

[0004] The present application provides a USB tunnel management method and device, which solves the problem of poor flexibility in USB networking.

[0005] In a first aspect, the present application provides a USB tunnel management method. The USB tunnel management method is applied to an initiating device or a processor of an initiating device, and the initiating device is connected to an intermediate device or a target device through a unified multimedia interconnection interface (UMI) bus. The USB tunnel management method includes: first, the initiating device sends an adapter list request to the intermediate device and the target device on the audio and video service flow path, and then receives an adapter list response sent by the intermediate device or the target device, wherein the adapter list response includes the identifiers of multiple adapters of the device, and the multiple adapters include a USB downstream adapter or a USB upstream adapter. The initiating device then allocates bandwidth to the first virtual channel of the virtual path according to the outgoing bandwidth value required by the USB tunnel service flow. Finally, the initiating device sends a first bandwidth application request to the next-level device in the virtual path, and the initiating device receives the first bandwidth application response, and the first bandwidth application request is used to indicate the establishment of a first virtual channel.

[0006] The virtual channel includes multiple virtual channels cascaded between the USB downstream adapter of the initiator device of the USB tunnel service flow and the target USB upstream adapter among the multiple adapters. The virtual channel is used to transmit the USB tunnel service flow on the link between the two ports.

[0007] Based on the above-mentioned USB tunnel management method, the initiating device can determine the multiple adapters contained in the target device based on the adapter list response, thereby identifying the virtual path from the initiating device to the target device. In this way, the devices passing through the virtual path can be selected according to demand, that is, the transmission path corresponding to the virtual path can be flexibly determined, thereby avoiding the transmission path from relying on fixed physical connections. At the same time, through the bandwidth application request, each device on the virtual path is instructed to complete the bandwidth allocation of the corresponding virtual channel, realizing the hierarchical creation of designated virtual channels. The USB tunnel service flow is transmitted between the two ports through the virtual channel, thereby decoupling the transmission of the USB tunnel service flow from the physical connection method between the ports, further improving the flexibility of changing the transmission path of the USB tunnel service flow.

[0008] In a second aspect, the present application provides a USB tunnel management method. This USB tunnel management method is applied to an intermediate device or its processor, and a target device or its processor. The intermediate device is connected to the initiator device and the target device, respectively, via a UMI bus. This USB tunnel management method includes: first, the intermediate device receives an adapter list request and returns an adapter list response. The adapter list request includes the identifier of the adapter of the intermediate device. Then, the intermediate device receives a first bandwidth request and returns a first bandwidth request response to complete the establishment of a first virtual channel. The first bandwidth request includes a requested bandwidth value and / or an inbound bandwidth value.

[0009] Based on the above-mentioned USB tunnel management method, the intermediate device can complete the establishment of a specified virtual channel based on the transmission and reception of bandwidth application requests and bandwidth application responses, thereby decoupling the transmission of the USB tunnel service flow from the physical connection method between the ports, avoiding the transmission path from relying on fixed physical connections, and improving the flexibility of changing the transmission path of the USB tunnel service flow.

[0010] In combination with the USB tunnel management method provided in the first and second aspects, as a possible implementation method, after receiving the adapter binding list, the initiating device determines the target USB upstream adapter of the target device from multiple adapters, and then determines a virtual path from the USB downstream adapter of the initiating device to the target USB upstream adapter from multiple topological paths between the initiating device and the target device.

[0011] In conjunction with the USB tunnel management methods provided in the first and second aspects, as a possible implementation, after receiving the adapter binding list, the initiating device sends a first adapter binding request and receives a first adapter binding response. The first adapter binding request is used to instruct the USB downstream adapter of a next-level device in the virtual path to bind with the USB upstream adapter of the next-level device. The first adapter binding request includes a first outbound bandwidth value and / or a first inbound bandwidth value. The first outbound bandwidth value is used to indicate the outbound bandwidth value of the USB host corresponding to the USB downstream adapter, and the second inbound bandwidth value is used to indicate the inbound bandwidth value of the USB host corresponding to the USB downstream adapter.

[0012] In combination with the USB tunnel management method provided in the first and second aspects, as a possible implementation method, an intermediate device receives a first adapter binding request and sends a first adapter binding response. The first adapter binding request is used to instruct the USB downstream adapter of this device to bind to the USB upstream adapter of the next-level device in the virtual path. Then, the intermediate device allocates bandwidth to the second virtual channel of the virtual path based on the outbound bandwidth value required by the USB tunnel service flow, and the second virtual channel is the next-level virtual channel of the first virtual channel in the virtual path. The initiating device then sends a second bandwidth application request to the next-level device in the virtual path, and the second bandwidth application request is used to instruct the establishment of a second virtual channel, and receives a second bandwidth application response.

[0013] In conjunction with the USB tunnel management methods provided in the first and second aspects, as a possible implementation, the initiating device may also adjust the bandwidth of a virtual channel to which a bandwidth value has been allocated. The initiating device generates a first bandwidth adjustment request, which includes a first target bandwidth value. Then, when the first target bandwidth value is greater than the outbound bandwidth value of the virtual channel, the initiating device adjusts the outbound bandwidth value of the virtual channel on the device upward, then sends the first bandwidth adjustment request to the next-level device in the virtual channel and receives a first bandwidth adjustment response.

[0014] In conjunction with the USB tunnel management methods provided in the first and second aspects, as a possible implementation, the initiating device may also bind the next-level device and the next-next-level device before adjusting the bandwidth. The initiating device sends a second adapter binding request and receives a second adapter binding response. The second adapter binding request is used to instruct the USB downstream adapter of the next-level device in the virtual path to bind to the USB upstream adapter of the next-next-level device. The second adapter binding request includes a second outbound bandwidth value and / or a second inbound bandwidth value. The second outbound bandwidth value is used to indicate the outbound bandwidth value of the USB host after the bandwidth adjustment, and the second inbound bandwidth value is used to indicate the inbound bandwidth value of the USB host after the bandwidth adjustment.

[0015] In combination with the USB tunnel management methods provided in the first and second aspects, as a possible implementation, an intermediate device receives a second adapter binding request and returns a second adapter binding response. The second adapter binding request is used to instruct the USB downstream adapter of the device to bind to the USB upstream adapter of the next-level device in the virtual path. The intermediate device then generates a second bandwidth adjustment request, which includes a second target bandwidth value. When the second target bandwidth value is greater than the outbound bandwidth value of the virtual path, the outbound bandwidth value of the virtual path at the device is adjusted upward. Finally, the second bandwidth adjustment request is sent to the next-level device in the virtual path, and a second bandwidth adjustment response is received.

[0016] In conjunction with the USB tunnel management methods provided in the first and second aspects, as one possible implementation, when a virtual channel needs to be dismantled, the initiating device initiates an adapter unbinding request. The initiating device sends the adapter unbinding request and receives an adapter unbinding response. The adapter unbinding request instructs the USB downstream adapter of the next-lower device in the virtual channel to unbind from the USB upstream adapter of the next-lower device.

[0017] In conjunction with the USB tunnel management methods provided in the first and second aspects, as one possible implementation, an intermediate device receives an adapter unbinding request, which instructs the intermediate device to unbind its USB downstream adapter from the USB downstream adapter of a next-level device in the virtual path. The intermediate device then sends an adapter unbinding response, then sends a bandwidth release request, and receives a bandwidth release response.

[0018] In combination with the USB tunnel management method provided in the first aspect and the second aspect, as a possible implementation manner, upon receiving the first bandwidth application request, the target device returns a first bandwidth application response to complete the establishment of the first virtual channel.

[0019] In a third aspect, the present application provides a USB tunnel management device, which includes a module for executing the method of any one of the implementations of the first aspect or the second aspect.

[0020] In a fourth aspect, the present application provides a multimedia device. The multimedia device includes a processor and a transceiver. Exemplarily, the processor is configured to process a USB tunnel traffic flow, and the transceiver is configured to transmit and receive the USB tunnel traffic flow. The processor and the transceiver collaborate to perform the method of any optional implementation of the first aspect or the second aspect.

[0021] In a fifth aspect, the present application provides a multimedia data transmission system. This multimedia data transmission system includes multiple multimedia devices provided in the fourth aspect, wherein the multimedia device for sending USB tunnel service streams is an initiating device, the multimedia device for receiving USB tunnel service streams is a target device, and the multimedia device for forwarding USB tunnel service streams is an intermediate device. The initiating device can be used to implement the functions of the initiating device in the first aspect, the intermediate device can be used to implement the functions of the intermediate device in the second aspect, and the target device can be used to implement the functions of the target device in the second aspect. Therefore, this multimedia data transmission system can also achieve the beneficial effects of the methods described in the first or second aspects above, which will not be elaborated here.

[0022] In a sixth aspect, the present application provides a computer-readable storage medium. The computer-readable storage medium includes computer software instructions. When the computer software instructions are executed in a computing device, the computing device executes the operating steps of the method described in the first aspect or any possible implementation of the first aspect, as well as the operating steps of the method described in the second aspect or any possible implementation of the second aspect. For example, the computing device is the aforementioned initiating device, intermediate device, or target device.

[0023] In a seventh aspect, the present application provides a computer program product. When the computer program product is executed on a computer, the computer program product causes the computing device to perform the steps of the method described in the first aspect or any possible implementation of the first aspect, as well as the steps of the method described in the second aspect or any possible implementation of the second aspect. For example, the computer is the aforementioned initiating device, intermediate device, or target device.

[0024] Regarding the beneficial effects of the third to seventh aspects, reference may be made to the description of any implementation in the first or second aspects, and no further details will be given here. Based on the implementations provided in the above aspects, this application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] FIG1 is a schematic diagram of a video transmission system provided by the present application;

[0026] FIG2 is a schematic diagram of an audio and video encoding and decoding system provided by the present application;

[0027] FIG3 is a flow chart of a USB tunnel management method provided by the present application;

[0028] FIG4 is a schematic diagram of a virtual path provided by the present application;

[0029] FIG5 is a schematic diagram of multicast of a virtual channel provided by the present application;

[0030] FIG6 is a schematic diagram of a flow chart of a bandwidth adjustment operation provided by the present application;

[0031] FIG7 is a flowchart of a USB tunnel removal operation provided by the present application;

[0032] FIG8 is a flowchart of another USB tunnel removal operation provided by the present application;

[0033] FIG9 is a schematic structural diagram of a USB tunnel management device provided by the present application;

[0034] FIG10 is a schematic structural diagram of a multimedia device provided in this application. DETAILED DESCRIPTION

[0035] The present application provides a USB tunnel management method, in which an initiating device first sends an adapter list request to all intermediate devices and target devices on the audio and video service flow path, and receives an adapter list response returned by each intermediate device or target device. The adapter list response includes the identifiers of multiple adapters of the device, thereby determining a virtual path between the source adapter of the initiating device and the sink adapter among the multiple adapters of the target device. The initiating device then allocates bandwidth to the first virtual channel of the virtual path based on the outbound bandwidth value required by the USB tunnel service flow, and sends a first bandwidth request request to the next-level device to establish the first virtual channel. Finally, after receiving the first bandwidth request request, the next-level device returns a first bandwidth request response. The initiating device receives the first bandwidth request response, completing the establishment of the first-level virtual channel of the virtual path.

[0036] In this way, the initiating device can determine the multiple adapters contained in the target device based on the adapter list response, thereby identifying the virtual path from the initiating device to the target device. The devices that the virtual path passes through can be selected according to demand, that is, the transmission path corresponding to the virtual path can be flexibly determined, thereby avoiding the transmission path from relying on fixed physical connections. At the same time, a bandwidth application request is used to instruct each device on the virtual path to complete the bandwidth allocation of the corresponding virtual channel, thereby realizing the hierarchical creation of designated virtual channels, and transmitting the USB tunnel service flow between the two ports through the virtual channel, thereby decoupling the transmission of the USB tunnel service flow from the physical connection method between the ports, further improving the flexibility of changing the transmission path of the USB tunnel service flow.

[0037] The technical solutions involved in this application may be applied not only to current audio and video transmission technologies or audio and video standards, but also to future audio and video transmission technologies or audio and video standards. The terms used in the embodiments of this application are only used to explain the specific embodiments of this application and are not intended to limit this application. The following is a brief introduction to some concepts that may be involved in this application.

[0038] Audio and video service flows include audio streams and / or video streams. An audio stream is a data stream used to transmit audio data in real time. A video stream is the transmission of video data. For example, a video stream can be processed as a stable and continuous flow over a network. A video stream consists of multiple video frames, each of which corresponds to an image.

[0039] In this embodiment, "video" is a general term that refers to a sequence of multiple consecutive frames, with one frame corresponding to one image. "Audio and video" is an information application technology term that refers to video, audio, or multimedia content that includes both video and audio. Furthermore, this application does not limit the type of service flow. For example, the service flow in this application can be an audio and video service flow or a USB service flow. The following description will primarily use audio and video service flows as an example.

[0040] USB tunneling refers to the management adapter establishing a point-to-point, bidirectional logical path between the USB downstream adapter in one UMI device and the USB upstream adapter in another UMI device, while allocating the maximum downstream data bandwidth and maximum upstream data bandwidth of the logical path. This process is called establishing a USB "logical path".

[0041] USB may refer to Universal Serial Bus 3.0, or USB3. In this application, a logical path is also referred to as a virtual path, and data transmitted in a USB tunnel is referred to as a USB service flow. USB3 data or messages encapsulated by a USB adapter are called USB3 tunnel messages. USB adapters, such as USB upstream adapters, USB downstream adapters, and USB hub adapters, convert USB link layer data and messages into USB tunnel messages specified by the UMI protocol and send them to the UMI transport layer. Furthermore, the UMI transport layer converts USB tunnel messages received into USB data or messages and reports them to the USB link layer.

[0042] The initiator device, intermediate device or target device in this application can be collectively referred to as a USB device. The USB device includes a USB port, which is divided into a standard USB port, an internal USB port and an external USB port.

[0043] A standard USB port fully complies with the USB 3.2 protocol specification, encompassing the mechanical layer, physical layer, link layer, protocol layer, and application layer. Depending on the role played by the USB device, USB ports are categorized as either USB downstream ports or USB upstream ports.

[0044] An internal USB port is a USB port that directly connects to a USB adapter. It includes the link layer, protocol layer, and application layer of the USB 3.2 protocol specification, but does not include the mechanical layer and physical layer of the USB 3.2 protocol specification. Based on their position within the USB link, internal USB ports are categorized as internal USB downstream ports and internal USB upstream ports. Based on the role played by the USB device hosting the port, internal USB ports are categorized as internal USB host downstream ports, internal USB device upstream ports, internal USB hub upstream ports, and internal USB hub downstream ports.

[0045] To enable USB data transmission over UMI, the UMI protocol makes minor modifications to the link layer, protocol layer, and application layer of the USB 3.2 protocol. In the UMI topology, only internal USB ports can be connected to each other, not to standard USB ports.

[0046] The UMI protocol places additional requirements on standard USB downstream ports. When no USB device is plugged into the USB downstream port, the insertion of the USB device must be detected in real time, and the local receiving-end matching impedance remains in the off-state. When a USB device is detected, the USB downstream port does not initiate link establishment and reports the device insertion event to the management adapter. The management adapter establishes a connection and completes enumeration for the internal USB Hub where the USB3 downstream port is located. The receiving-end matching impedance of the external USB downstream port is then switched to the on-state and link establishment is initiated. The UMI protocol refers to this type of USB downstream port as an external USB downstream port. An external USB downstream port can only exist as a downstream port on the internal USB Hub of a UMI-end device. The UMI protocol does not have a special definition for an external USB upstream port.

[0047] In this application, according to the definition of the above ports, USB devices are also divided accordingly.

[0048] A device that has only internal USB downstream ports and implements the USB host function in the USB topology is called an internal USB host device; a device that has one internal USB upstream port and several internal USB downstream ports and implements the USB hub function in the USB topology is called an internal USB hub device; a device that has only internal USB upstream ports and implements the USB peripheral function in the USB topology is called an internal USB device.

[0049] Source and sink devices in the UMI protocol are collectively referred to as UMI-side devices to distinguish them from UMI routers. Internal USB hosts or devices can only exist in UMI-side devices, and internal USB hubs can only exist in UMI routers. A UMI-side device that includes an internal USB host is called a UMI-USB host; a UMI-side device that includes an internal USB device is called a UMI-USB device.

[0050] In order to make the description of the following embodiments clear and concise, the video transmission system to which the USB tunnel management method of the present application is applicable is first introduced.

[0051] Figure 1 is a schematic diagram of a video transmission system provided by the present application. The video processing process may include but is not limited to: video acquisition, video encoding, video transmission, video decoding and playback.

[0052] The video transmission system in Figure 1 includes a set-top box 110, a smart TV 120, multiple audio and video playback devices, and a server 130. The set-top box 110 is connected to the operator's network via a network cable and can receive audio and video streams from the server 130. The network can implement the function of audio and video transmission, and the network can include one or more network devices, such as a network device 131, which can be a router or a switch. In some optional implementations, the set-top box 110 and the server 130 can also communicate via wireless communication, which is not limited in this application.

[0053] The video transmission system in Figure 1 includes a set-top box 110, a smart TV 120, multiple audio and video playback devices, and a server 130. The set-top box 110 is connected to the operator's network via a network cable and can receive audio and video streams from the server 130. The network can implement the function of audio and video transmission, and the network can include one or more network devices, such as a network device 131, which can be a router or a switch. In some optional implementations, the set-top box 110 and the server 130 can also communicate via wireless communication, which is not limited in this application.

[0054] Smart TV 120 is a display device with audio and video processing capabilities that implements functions such as receiving, processing, pushing, and playing video streams or audio and video streams. In some possible scenarios, the smart TV 120 may refer to an audio and video device such as a conference tablet, a smart TV, or a projector, but this application is not limited to this.

[0055] The plurality of audio and video playback devices include audio and video playback devices 121 to 124. For example, these audio and video playback devices may include, but are not limited to, multimedia control platforms or other devices supporting audio and video playback functions, such as virtual reality (VR) terminal devices or augmented reality (AR) terminal devices.

[0056] In this embodiment, the set-top box 110 and the smart TV 120 can be connected via a bus 125. The set-top box 110 and various audio and video playback devices can also be connected via the bus 125. The smart TV 120 and various audio and video playback devices can also be connected via the bus 125. Exemplarily, the bus 125 can be a data bus that supports video and audio and video transmission. For example, the bus 125 is a UMI bus. A UMI bus is a bus connected based on the UMI interface provided by the source device and the sink device. The UMI bus can be used to connect a charger to charge an electronic device (such as the smart TV, set-top box, or audio and video playback device described above) and can also be used to transmit data between the electronic device and peripheral devices. It can also be used to connect headphones to play audio through the headphones. This interface can also be used to connect other electronic devices, such as augmented reality devices. When the UMI bus is used to implement data communication between devices, it can support both uncompressed and compressed video transmission, as well as various advanced features such as Quick Video Transport (QVT), Auto Low Latency Mode (ALLM), and Dynamic Frame Rate Refresh (DFR). Furthermore, the UMI bus can support LPCM audio and video formats defined by IEC 60958, as well as various HDR protocols, such as HDR Vivid (HDR Vivid) and those specified in T / UWA005.1-2022. The UMI also supports encryption control and protection for data transmission, such as audio and video. In some optional implementations, the bus 125 can also refer to other types of buses that can implement the functions supported by the aforementioned UMI bus.

[0057] Server 130 can be an application server or an authentication and authorization server. Server 130 can provide video services, game services, messaging services, music services, authentication and authorization services, and the like. In one example, the functions of multiple services can be integrated on server 130. For example, a game service and a music service can be deployed on server 130. In another example, the functions of some services can be integrated on server 130. For example, server 130 can deploy some game services and some video services. Server 130 can also utilize virtualization technology to provide multiple virtual machines, which provide various services. The embodiments of this application do not limit the deployment of the server. Network device 131 is connected to server 130 via wireless or wired connections. Figure 1 is merely a schematic diagram; the network may also include other devices, not shown in Figure 1. It will be understood that the aforementioned audio and video is a general term. Audio and video include multiple video frames, each of which corresponds to a group of packets carrying the audio and video data to be parsed and played. Figure 1 is merely a schematic diagram; the video transmission system may also include other devices, not shown in Figure 1. The embodiments of the present application do not limit the number and type of each device included in the system.

[0058] Based on the video transmission system shown in FIG1 , FIG2 is a schematic diagram of an audio and video codec system provided by the present application. The audio and video codec system includes a source device 210 and a sink device 220. The source device 210 establishes a communication connection with the sink device 220 via a UMI bus.

[0059] The above-mentioned source device 210 can implement the function of audio and video encoding. As shown in Figure 1, the source device 210 can be a set-top box 110 or a smart TV 120. The source device 210 can also be an audio and video control center with audio and video encoding capabilities. For example, the audio and video control center includes one or more servers.

[0060] The source device 210 may include a data source 211 , a pre-processing module 212 , an audio and video transmission adapter 213 , and a communication interface 214 .

[0061] The data source 211 may include or may be any type of electronic device for collecting audio and video, and / or any type of source audio and video generating device, such as a computer graphics processor for generating computer animation scenes or any type of device for obtaining and / or providing source audio and video, or computer-generated source audio and video. The data source 211 may be any type of memory or storage for storing the above-mentioned source audio and video. The above-mentioned source audio and video may include multiple audio and video streams or images obtained by multiple audio and video acquisition devices (such as cameras), such as ultra high definition (UHD) video, high definition (HD) video, 4K video, etc.

[0062] The pre-processing module 212 is configured to receive source audio and video and pre-process the source audio and video to obtain audio and video or multiple frames of images. For example, the pre-processing performed by the pre-processing module 212 may include color format conversion (e.g., from RGB to YCbCr), octree structuring, audio and video splicing, audio track merging and deletion, or channel number adjustment.

[0063] The audio and video transmission adapter 213 is also referred to as a source adapter in this application, and is used to receive audio and video or images, and encode the audio and video, images or images to obtain coded data. In some optional situations, the code stream (coded data) obtained by encoding can also be called a bit stream. If the coded data is obtained by encoding audio and video data, then the bit stream refers to the audio and video stream.

[0064] The communication interface 214 in the source device 210 may be configured to receive encoded data (e.g., a video stream or an audio / video stream) and send the encoded data (or a version of the encoded data after any other processing) to another device such as the sink device 220 or any other device via a UMI bus for storage, display, playback, or image reconstruction.

[0065] Optionally, the source device 210 includes a bitstream buffer, which is used to store bitstreams corresponding to one or more coding units.

[0066] The sink device 220 can implement the audio and video decoding function. As shown in FIG1 , the sink device 220 can be any one of the smart TV 120 or the audio and video playback device shown in FIG1 .

[0067] The sink device 220 may include an audio and video playback unit 221 , a post-processing module 222 , an audio and video receiving adapter 223 , and a communication interface 224 .

[0068] The communication interface 224 in the sink device 220 is configured to receive the encoded data (or a version of the encoded data after any other processing has been performed on the encoded data) from the source device 210 or any other source device such as a storage device.

[0069] Communication interface 214 and communication interface 224 may be used to communicate via a direct communication link between source device 210 and sink device 220, such as a direct wired connection, as shown in the UMI bus in FIG2 . For details about the UMI bus, please refer to the description of FIG1 , and will not be repeated here.

[0070] The communication interface 224 corresponds to the communication interface 214 and can be used, for example, to receive transmission data and process the transmission data using any type of corresponding transmission decoding or processing and / or decapsulation to obtain encoded data (such as a video stream or an audio and video stream).

[0071] Both the communication interface 224 and the communication interface 214 may be configured as a unidirectional communication interface, as indicated by the arrow pointing from the source device 210 to the corresponding UMI bus of the sink device 220 in FIG2 , or a bidirectional communication interface, and may be used to send and receive messages, etc., to establish a connection, confirm and exchange any other information related to the communication link or data transmission, such as encoded compressed data transmission, etc.

[0072] The audio and video receiving adapter 223 is also referred to as a sink adapter in this application, and is used to receive encoded data and decode the encoded data to obtain decoded data (video or audio and video, etc.).

[0073] The post-processing module 222 is used to post-process the decoded data to obtain post-processed data (such as an image to be displayed or audio and video to be played). The post-processing performed by the post-processing module 222 may include, for example, color format conversion (such as from YCbCr to RGB), octree reconstruction, audio and video splitting and fusion, or any other processing for generating data for output by the audio and video playback unit 221.

[0074] The audio and video playback unit 221 is used to receive post-processed data for display or playback to a user or viewer, etc. The audio and video playback unit 221 can be or include any type of display for representing the reconstructed image, such as an integrated or external display screen or display. For example, the display screen may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS) display, a digital light processor (DLP), or any other type of display screen. The audio and video playback unit 221 can also include one or more audio and video playback modules, each of which can refer to a speaker, a smart speaker, an amplifier, etc.

[0075] As an optional implementation, the source device 210 and the sink device 220 can transmit encoded data through a data forwarding device. For example, the data forwarding device can be a router or a switch. It is worth noting that the data forwarding device needs to support the UMI interface. The UMI interface of the two devices consists of a main link (ML), a sideban link (SL), a power bus link (PL), and a cable information link (CL). This application does not involve other links other than the main link, and only the main link will be described below. The main link is mainly used to transmit audio and video service streams (such as ultra-high-definition audio and video signals) and high-speed data (such as USB3 data). The main link includes multiple pairs of differential lines, each pair of differential lines forming a differential channel (lane). The virtual channel between the audio and video sending adapter 213 and the audio and video receiving adapter 223 can use the differential channel to transmit audio and video service streams.

[0076] The following describes in detail the implementation of the USB tunnel management method provided in the embodiment of the present application with reference to the accompanying drawings.

[0077] The USB tunnel service is an enhanced application of the UMI audio and video service, which supports the simultaneous existence of multiple USB hosts in the UMI network. In the case of UMI network resources, the USB Device can theoretically be enumerated by any USB Host in the UMI network.

[0078] In a UMI network, there is at least one source and one sink device, and the sink device generally has a display. The source device defaults to the USB Host function. Scenarios without a Host are not supported by this protocol.

[0079] Users can view all USB devices on a device with a USB Host through topology management messages (topology changes and obtaining adapter lists), and can manually select the USB devices required for enumeration.

[0080] All USB ports on this device can only be enumerated by the local USB host and are not related to the UMI network. This means that USB devices enumerated by the source device's USB host cannot be enumerated by other USB hosts over the UMI network. Typical source devices include computers, portable mobile devices (such as mobile phones and tablets), and set-top boxes.

[0081] If the local device does not have a USB host, it can only be enumerated by the source device's USB host. If there is no source device on the UMI network, it will not be enumerated. If the local device has a USB host, it will be enumerated by the local standard USB host first. When switching to an external audio or video input source, it will switch to the external audio or video source device's USB host enumeration, and the management adapter will control the switching management. Typical sink devices include TVs, monitors, and other products.

[0082] All local USB ports of the routing device are only used by local devices, that is, they are only enumerated by the local standard USB Host and do not enter the UMI network.

[0083] Here, the USB tunnel management method according to an embodiment of the present application is described as being executed by the source device 210 and the sink device 220 shown in FIG. FIG. 3 is a flow chart illustrating a USB tunnel management method provided by the present application. Initiator device 31 is used to implement the functions of source device 210, and target device 32 is used to implement the functions of sink device 220. In this embodiment, initiator device 31 may also be referred to as a source device, an audio and video transmitter, or an audio and video transmitter, and target device 32 may also be referred to as a sink device, a display device, an audio and video receiver, or an audio and video player. In this embodiment, initiator device 31 and target device 32 are connected via one or more intermediate devices 33 (only one intermediate device 33 is shown in FIG. 3 , but the number of intermediate devices is not limited). Initiator device 31, intermediate device 33, and target device 32 are connected via a bus 34, which may be a UMI bus.

[0084] In a first possible application scenario, the initiating device 31 may be the set-top box 110 in Figure 1, and the target device 32 may be the smart TV 120 in Figure 1. For example, the set-top box pushes audio and video data to the smart TV.

[0085] In a second possible application scenario, the initiating device 31 may be the set-top box 110 in FIG1 , and the target device 32 may be any audio and video playback device in FIG1 , such as any one of the audio and video playback devices 121 to 124. For example, the set-top box pushes audio and video data to the audio and video playback device.

[0086] In a third possible application scenario, the initiating device 31 may be the smart TV 120 in FIG1 , and the target device 32 may be any audio and video playback device in FIG1 , such as any one of the audio and video playback devices 121 to 124. For example, the smart TV pushes audio and video data to the audio and video playback device.

[0087] The above three possible application scenarios are merely examples provided in this embodiment and should not be construed as limiting the present application. In other possible examples, the initiating device 31 may be any of the audio and video playback devices in FIG1 (e.g., audio and video playback device 121), and the target device 32 may be another audio and video playback device different from the aforementioned audio and video playback device (e.g., audio and video playback device 122).

[0088] In the embodiment of the present application, the rules for establishing a USB tunnel, i.e., a virtual channel, between the initiator device 31, the intermediate device 33, and the target device 32 include: (1) one upstream adapter can only establish a virtual channel with one downstream adapter; (2) one downstream adapter can only establish a virtual channel with one upstream adapter; (3) when the USB of the routing device When the hub resources are insufficient, a multi-hop virtual path is allowed to be established across routing devices; (4) For routing devices, a single-hop virtual path can only pass through one UMI port; (5) For routing devices, a multi-hop virtual path passes through two UMI ports, and data in the downstream or upstream direction enters from one UMI port and is output from the other UMI port; (6) A UMI port can pass through multiple virtual paths; (7) For routing devices, internal USB adapters are not allowed to establish virtual paths with each other; (8) When two UMI end devices are directly connected, or when a UMI end device is connected to a routing device, a virtual path is allowed to be established between the upstream adapter and the downstream adapter at will; (9) When two UMI end devices pass through a routing device, a virtual path is allowed to be established between the upstream adapter and the downstream adapter in the two UMI end devices at will, but the number of routing devices that can be crossed is limited to one; (10) If the adapter to which the virtual path is to be established is already occupied, that is, a virtual path has been established with other adapters, the management adapter can choose to remove the virtual path first, provided that the impact of the removal of the virtual path on the USB tunnel service is properly handled.

[0089] Referring to FIG. 3 , the USB tunnel management provided by this embodiment includes the following steps 301 to 324 .

[0090] Step 301: The initiator device 31 sends an adapter list request.

[0091] When the initiator device 31 needs to obtain the adapter list information before establishing a virtual path, it generates an adapter list request and sends the adapter list request according to the device addressing information.

[0092] As a possible implementation, the message structure of the adapter list request is shown in Table 1, and the message field description of the adapter list request is shown in Table 2.

[0093] Table 1

[0094] Table 2

[0095] Here, Command=6 means that the device control command number corresponding to the adapter list request is 6.

[0096] Optionally, the message structure of the message header is as shown in Table 3.

[0097] Table 3

[0098] In the message header of the adapter list request, the Type value is 27, indicating that the message type is an adapter list request. The Response value is 0, indicating that it is a request message.

[0099] Optionally, the message structure of the general field is as shown in Table 4.

[0100] Table 4

[0101] Optionally, the addressing information requested by the bandwidth device is device address addressing information, or device address addressing, or addressing information. Each device in the UMI network has a unique 8-byte (64-bit) identifier that can uniquely identify a device, so this identifier is called a device address. The device address consists of a vendor identifier (Vendor ID) and a device identifier (Device ID). The Vendor ID consists of 16 bits and must be registered with the UMI organization. The Device ID is assigned by the device manufacturer when manufacturing the UMI device and must be guaranteed to be unique.

[0102] The message structure of the device address addressing information is shown in Table 5.

[0103] Table 5

[0104] The destination address is the device address of the device that the message is intended to receive, and the source address is the device address of the device that sends the message.

[0105] In the present application, the addressing method based on the above-mentioned device address addressing information includes: when a device receives a message carrying device address addressing information, if the target address of the message is its own device address, then the device is the target device of the message. Otherwise, it is necessary to query the address table based on the target address of the message and process it according to the query result: (1) If the target address of the message is in the address table, and the corresponding port is different from the port receiving the message, then the message is forwarded from the corresponding port. The address table includes the correspondence between the device address and the forwarding port. (2) If the target address of the message is in the address table, but the corresponding port is the same as the port receiving the message. It is considered that the target address of the message is in the device connected to the port, so the target device should have received the message, and the device should discard the message. (3) If the target address of the message is not found in the address table, then the device does not know whether there is a port connected to the target device, and the device should flood the message from all ports except the receiving port.

[0106] Step 302: The intermediate device 33 receives an adapter list request.

[0107] Step 303: The intermediate device 33 returns an adapter list response.

[0108] Step 304: The target device 32 receives the adapter list request.

[0109] Step 305: The target device 32 returns an adapter list response.

[0110] After receiving the adapter list request, the intermediate device 33 or the target device 32 generates an adapter list response, fills the information of all adapters in the online state of the device (including the adapter identification) into the adapter list response, and then sends the adapter list response according to the address table.

[0111] In this application, adapter list request and adapter list response can be collectively referred to as adapter list message, which is used to obtain the adapter list of a specific device in the UMI network, and is often used to obtain detailed information of the device adapter when establishing a virtual path.

[0112] As a possible implementation, the message structure of the adapter list response is shown in Table 6.

[0113] Table 6

[0114] The multiple AdapterIDs are identifiers of multiple adapters carried in the adapter list response.

[0115] Optionally, descriptions of the fields in the message structure of the adapter list response are shown in Table 7.

[0116] Table 7

[0117] Step 306 : The initiator device 31 receives the adapter list response from the intermediate device 33 .

[0118] Step 307 : The initiator device 31 receives the adapter list response from the target device 32 .

[0119] As a possible implementation method, the adapter list response of the target device 32 received by the initiator device 31 can be forwarded by the intermediate device 33. Please refer to the addressing method corresponding to the above-mentioned device addressing information for the way in which the intermediate device 33 forwards the adapter list response, which will not be repeated here.

[0120] After receiving the adapter list response, the initiator device 31 may determine the sink adapter from the plurality of adapters, and determine a virtual path from the source adapter to the sink adapter from the plurality of paths included in the topology structure between the source adapter and the sink adapter.

[0121] As one possible implementation, the initiating device 31 selects a virtual path based on network quality indicators of the transmission path corresponding to the virtual path. Network quality indicators may include path length, latency, etc. For example, the initiating device 31 selects based on path length, preferably the shortest path to establish the virtual path.

[0122] Optionally, the virtual path includes multiple virtual channels cascaded between the source adapter of the initiator device 31 and the sink adapter among the multiple adapters of the target device 32. For example, the virtual path includes a first virtual channel between the initiator device 31 and the intermediate device 33, and a second virtual channel between the intermediate device 33 and the target device 32.

[0123] Step 308: The initiating device 31 generates a first bandwidth application request.

[0124] As a possible implementation manner, the message structure of the first bandwidth application request is shown in Table 8.

[0125] Table 8

[0126] The initiating device 31 assigns the OutStreamBW in the message body to the outgoing bandwidth value required by the service, assigns the In StreamBW to 0xFFFFFFFF, and updates the source device address, source adapter ID, target device address, target adapter ID, priority (service priority) and flow control mechanism information to the message body.

[0127] Optionally, descriptions of the fields of the first bandwidth application request are shown in Table 9.

[0128] Table 9

[0129] Step 309 : The initiating device 31 allocates bandwidth to the first virtual channel of the virtual path according to the outgoing bandwidth value required by the USB tunnel service flow.

[0130] As a possible implementation, step 309 may include the following sub-steps S1 to S7.

[0131] S1. The initiating device 31 determines the output port number OutPortID of the first bandwidth application request, and checks whether the output port number exists in the router forwarding table corresponding to the USB tunnel service flow according to the routing table.

[0132] When the forwarding list addressing information of the initiating device 31 does not include the port number of the output port from which the initiating device 31 sends the first bandwidth application request, the initiating device 31 determines the port number of the output port for sending the bandwidth application based on the device's internal addressing, i.e., the outgoing port number. For example, the initiating device 31 maintains service flow information, which includes the input port number, source device address, source adapter identifier, output port number, target device address, target adapter identifier, bandwidth value, and priority. The initiating device 31 uses the source device address, target device address, etc. of the bandwidth application request to query the output port number, i.e., the outgoing port number, in the service flow information. At the same time, the outgoing port number is also the output port of the virtual path of the USB tunnel service flow on this device.

[0133] Alternatively, a router forwarding table is used to indicate the forwarding rule of message between different virtual channels in this device, and a router forwarding table comprises one or more forwarding entries (or router forwarding table items). As shown in Table 10, the router forwarding table is composed of inflow information and outflow information, and a router forwarding table item can be represented by {inflow information|outflow information}. Wherein the inflow information is composed of the information of a single inflow node, and the outflow information is composed of the information of one or more outflow nodes. The information of the inflow node mainly describes the port and the virtual channel (such as port number and virtual channel mark) corresponding to the inflow, and the information of the outflow node mainly describes the relevant information such as the target port and the target virtual channel (such as port number and virtual channel mark) of the forwarding of the message. For example, the first router forwarding table entry in Table 10 can be represented as {[0,4]|[2,4]}. The inbound flow information contains only one inbound node, [0,4], indicating that the flow receives packets from the receive buffer of adapter 4 on this router. The outbound flow information contains one outbound node, [2,4], indicating that the flow packet needs to be forwarded to virtual channel 4 on port 2. The virtual channel ID in the inbound node information of the inbound flow information is equivalent to the adapter ID (AdapterID). An inbound node can also be called an inbound node, and an outbound node can also be called an outbound node.

[0134] Table 10

[0135] In a multicast scenario, if one or more devices on the same destination port receive the stream corresponding to the same virtual channel, a multicast count (ReceiverCount) is added to the outbound node to facilitate router forwarding table management. The outbound node is represented by a triplet of [port number (Port), virtual channel identifier (ShuttleID), multicast count (ReceiverCount)]. The multicast count indicates the number of destination devices receiving the audio and video service stream on the output port (also called the outbound port).

[0136] S2. If the output port number is in the router forwarding table, the initiating device 31 uses the existing virtual channel identifier and increases the corresponding multicast count by one, and then executes S5.

[0137] S3. If the output port number is not in the router forwarding table, the initiating device 31 allocates a new virtual channel identifier to the output port and sets the corresponding multicast count value to 1.

[0138] S4: Determine whether the currently available bandwidth meets the requirement. If so, the initiating device 31 allocates bandwidth as required. Otherwise, the initiating device 31 starts the bandwidth management process.

[0139] The initiating device 31 allocates bandwidth as required by comparing the current actual available bandwidth of the output port with the outgoing bandwidth value required by the audio and video service flow, ie, the outgoing bandwidth value. If the current actual available bandwidth is larger, the requirement is met.

[0140] S5. The initiating device 31 records the service flow information and updates the router forwarding table information.

[0141] S6. The initiating device 31 compares the actually allocated bandwidth value with the OutStreamBW in the bandwidth application request. If the actually allocated bandwidth value is smaller than the OutStreamBW value, OutStreamBW is assigned the actually allocated bandwidth value; otherwise, the assigned value of OutStreamBW remains unchanged.

[0142] The OutStreamBW value is called the outflow bandwidth value.

[0143] S7. The initiating device 31 configures the transport layer.

[0144] The initiating device 31 sends the information shown in Table 11 to the transport layer.

[0145] Table 11

[0146] In a possible implementation of the present application, the serial numbers of the above steps do not limit the execution steps of the steps. For example, the above step 309 can be executed after step 308 or before step 308. That is, when the initiating device 31 does not generate the first bandwidth application request, it executes step 309 using the data related to the first bandwidth application request.

[0147] Step 310: The initiating device 31 sends a first bandwidth application request.

[0148] The OutStreamBW value in the first bandwidth application request is the outbound bandwidth value required by the USB tunnel service flow.

[0149] As a possible implementation manner, the initiating device 31 forwards the first bandwidth application request according to the forwarding list addressing information of the first bandwidth application request.

[0150] Optionally, the addressing information of the first bandwidth application request is forwarding list addressing information, or referred to as forwarding list addressing, or addressing information. The message structure of the forwarding list addressing information is shown in Table 12.

[0151] Table 12

[0152] The Forwarding List Level of the first bandwidth application request is used to indicate the current level, and the corresponding port number in the port number of the level is the port number used by the current device to send the bandwidth application request.

[0153] Step 311: The intermediate device 33 receives a first bandwidth application request.

[0154] When the intermediate device 33 receives the first bandwidth application request, it records the service flow information.

[0155] Step 312: The intermediate device 33 sends a first bandwidth request response.

[0156] After receiving the first bandwidth application request, the intermediate device 33 records the service flow information and determines that the intermediate device is the target device of the first bandwidth application request based on the forwarding list addressing information. It then uses the input port of the first bandwidth application request as the output port of the first bandwidth application response and sends the bandwidth application response based on the forwarding list addressing method.

[0157] As a possible implementation manner, the message structure of the first bandwidth request response is shown in Table 13.

[0158] Table 13

[0159] Optionally, descriptions of the fields in the bandwidth request response are shown in Table 14.

[0160] Table 14

[0161] Step 313: The initiating device 31 receives the first bandwidth request response.

[0162] Step 314 : The initiating device 31 allocates bandwidth to the first virtual channel in response to the first bandwidth request response.

[0163] As a possible implementation, the manner in which the initiating device 31 allocates bandwidth in response to the first bandwidth request response includes the following sub-steps S1-S4.

[0164] S1. The initiating device 31 identifies the corresponding virtual channel according to the Channel Shuttle ID and tag in the first bandwidth request response.

[0165] The Channel ShuttleID and tag (message identifier) ​​in the bandwidth request response are used to indicate the first virtual channel of the virtual path.

[0166] S2. The initiating device 31 determines whether the multicast count value of the flow node is greater than 1. If so, execute S5; otherwise, execute S3.

[0167] S3. The initiating device 31 compares the bandwidth value (OutStreamBW) in the first bandwidth request response with the allocated bandwidth value. If the allocated bandwidth value is larger, the excess bandwidth value is released, the weight and flow control cache are recalculated, and the updated weight and flow control cache information is sent down to the transport layer.

[0168] Optionally, the allocated bandwidth value refers to the outbound bandwidth value allocated to the first virtual channel by the initiating device 31 according to the first bandwidth application request. The excess bandwidth value refers to the difference between the allocated bandwidth value and the bandwidth value in the first bandwidth application response.

[0169] S4. The initiating device 31 updates the record of the allocated bandwidth value.

[0170] In a possible embodiment of the present application, when the initiating device 31 receives the first bandwidth response, the ErrCode is non-zero, and the first bandwidth request can be resent based on the original tag. When the initiating device 31 receives the first bandwidth response, the ErrCode is non-zero, and the first bandwidth release process can be invoked before sending the first bandwidth request using a new tag.

[0171] Step 315: The initiator device 31 sends a first adapter binding request.

[0172] As a possible implementation manner, the message structure of the first adapter binding request is shown in Table 15.

[0173] Table 15

[0174] Optionally, descriptions of message fields of the first adapter binding request are as shown in Table 16.

[0175] Table 16

[0176] As a possible implementation manner, the initiating device 31 sends the first adapter binding request based on a device address addressing mode.

[0177] Step 316: The intermediate device 33 receives the first adapter binding request.

[0178] The first adapter binding request is used to instruct the USB downstream adapter of the next-level device in the virtual path to bind to the USB upstream adapter of the next-level device. For the intermediate device 33, it instructs the USB downstream adapter of the intermediate device 33 to bind to the USB downstream adapter of the target device.

[0179] Step 317: The intermediate device 33 sends a first adapter binding response.

[0180] As a possible implementation, in addition to sending the first adapter binding response, the target device 32 also synchronously records the adapter binding information.

[0181] In a possible embodiment of the present application, if the adapter is currently in a bound state, the ErrCode value is assigned to 0x5 (request rejected), and the first adapter binding response message is directly returned.

[0182] As a possible implementation manner, the message structure of the first adapter binding response is shown in Table 17.

[0183] Table 17

[0184] Optionally, the field description of the message structure of the first adapter binding response is as shown in Table 18.

[0185] Table 18

[0186] Step 318: The initiator device 31 receives the first adapter binding response.

[0187] As a possible implementation manner, the initiator device 31 synchronously records the adapter binding information.

[0188] Step 319: The intermediate device 33 allocates bandwidth to the second virtual channel of the virtual path according to the outgoing bandwidth value required by the USB tunnel service flow.

[0189] As a possible implementation manner, the processing manner of the intermediate device 33 allocating bandwidth to the second virtual channel is described in step 309 and will not be repeated here.

[0190] Step 320: The intermediate device 33 sends a second bandwidth application request to the next-level device in the virtual path.

[0191] As a possible implementation, the next-level device of the intermediate device 33 in the virtual path is the device corresponding to the upstream adapter bound in the first adapter binding response, such as the target device 32 .

[0192] For the message structure and field description of the second bandwidth application request, please refer to the first bandwidth application request in step 310, which will not be repeated here.

[0193] Step 321: The target device 32 receives a second bandwidth application request.

[0194] As a possible implementation manner, when the target device 32 receives the second bandwidth application request, it records the service flow information.

[0195] Step 322: The target device 32 sends a second bandwidth request response.

[0196] After receiving the second bandwidth application request, the target device 32 records the service flow information and determines that the device is the target device of the second bandwidth application request based on the forwarding list addressing information. It then uses the input port of the second bandwidth application request as the output port of the second bandwidth application response and sends the bandwidth application response based on the forwarding list addressing method.

[0197] As a possible implementation manner, for the message structure and field description of the second bandwidth application response, please refer to the first bandwidth application response in step 312, which will not be repeated here.

[0198] Step 323: The intermediate device 33 receives the second bandwidth request response.

[0199] Step 324 : The intermediate device 33 allocates bandwidth to the second virtual channel in response to the second bandwidth request response.

[0200] As a possible implementation manner, please refer to step 314 for the process of the intermediate device 33 allocating bandwidth to the second virtual channel, which will not be described in detail here.

[0201] In a possible embodiment of the present application, the initiator device 31 and the target device 32 complete the establishment of a virtual path of the USB tunnel through a USB tunnel management method. FIG4 is a schematic diagram of a virtual path provided by the present application.

[0202] The communication link between communication interface 214 and communication interface 224, through multiple communication links between intermediate devices, forms the communication link between source device 210 and sink device 220. For example, taking source device 210 as device A, sink device 220 as device D, and intermediate devices including devices B and C, a virtual path is formed by cascading multiple virtual channels between adapter 4 of device A and adapter 5 of device D. This virtual path includes three virtual channels: a virtual channel with a virtual channel ID (shuttle ID) of 7 from device A to device B, a virtual channel with a virtual channel ID of 4 from device B to device C, and a virtual channel with a virtual channel ID of 13 between devices C and D. Port 2 of each device is a main downstream port (MDP), and port 1 of each device is a main upstream port (MUP). The downstream and upstream ports are connected via connectors and cables.

[0203] A virtual path can be represented by a quadruple. For example, the virtual path shown in FIG4 can be represented by (device A, adapter 4, device D, adapter 5).

[0204] The UMI bus supports multicast functionality. Data, audio and video streams, etc. generated by the same adapter can be transmitted to multiple adapters. As shown in Figure 5, the stream sent by adapter 4 of device A is received by adapter 7 of device B, adapter 6 of device C, and adapter 5 of device D at the same time. The corresponding three virtual paths are (device A, adapter 4, device B, adapter 7), (device A, adapter 4, device C, adapter 6), and (device A, adapter 4, device D, adapter 5).

[0205] In a possible embodiment of the present application, when an audio or video service requires both outbound and inbound bandwidth, such as a USB tunnel service, the intermediate device must allocate both outbound bandwidth to the output port and the input port, corresponding to the InStreamBW value in the bandwidth request. The target device must also allocate the outbound bandwidth value to the input port, corresponding to the InStreamBW value in the bandwidth request. The bandwidth allocation rules and process are the same as above.

[0206] The overall process of the USB tunnel management method has been described above with reference to Figures 3-5 . To further enhance the flexibility of USB tunnels in complex UMI networks, this USB tunnel management method can also adjust the bandwidth value of the virtual path after establishing the corresponding virtual path. The following detailed description of the bandwidth adjustment operation provided in the embodiments of this application will be provided with reference to the accompanying figures.

[0207] Bandwidth adjustment messages include bandwidth adjustment requests and bandwidth adjustment responses, which are used to adjust the bandwidth of each device on the virtual path of the service flow. They can support adjustment in both upward (bandwidth increase) and downward (bandwidth reduction) directions to achieve the goals of full bandwidth utilization and low power consumption.

[0208] When the service flow changes, for example, when the bandwidth demand increases, a bandwidth adjustment message (upward adjustment) can be initiated. Alternatively, when the bandwidth demand decreases, a bandwidth adjustment message (downward adjustment) can be initiated. The released bandwidth can allow some main link channels (differential channels) to enter a low-power state or be used by other services.

[0209] When adjusting upward, the process of increasing bandwidth is completed in the direction of bandwidth adjustment request, which is similar to bandwidth application. When adjusting downward, the process of decreasing bandwidth is completed in the direction of bandwidth adjustment response.

[0210] The bandwidth adjustment request must be initiated by the source device of the corresponding service flow. For example, the video stream must be initiated by the device where the audio and video sending adapter that sends the video is located, and the USB service stream must be initiated by the device where the USB tunnel adapter connected to the USB host is located.

[0211] Here, the bandwidth adjustment operation of an embodiment of the present application is performed by the initiating device 210 and the target device 220 shown in Figure 2 as an example. Figure 6 is a schematic flow chart of a bandwidth adjustment operation provided by the present application. In this embodiment, the initiating device 61 is used to implement the functions of the source device 210, and the target device 62 is used to implement the functions of the sink device 220. In this embodiment, the initiating device 61 may also be referred to as a source device, an audio and video transmitter, or an audio and video transmitter, and the target device 62 may also be referred to as a sink device, a display device, an audio and video receiver, or an audio and video player. In this embodiment, the initiating device 61 and the target device 62 are connected via one or more intermediate devices 63 (only one intermediate device 63 is shown in Figure 6, but the number of intermediate devices is not limited). The initiating device 61, the intermediate device 63, and the target device 62 are connected via a bus 64, which may be a UMI bus. The virtual channels between the initiating device 61 and the intermediate device 63, and the virtual channels between the intermediate device 63 and the target device 62 constitute a virtual path.

[0212] Referring to FIG. 6 , the bandwidth adjustment operation provided in this embodiment includes the following steps 601 to 618 .

[0213] Step 601: The initiating device 61 generates a first bandwidth adjustment request.

[0214] The initiating device 61 determines an adjusted first target bandwidth value according to the change in the service flow, and generates a first bandwidth adjustment request.

[0215] The first target bandwidth value may be an adjusted outbound target bandwidth value. In a possible embodiment, the first bandwidth adjustment request may further include an inbound bandwidth value. The initiating device 61 may further determine an adjusted third target bandwidth value based on changes in the service flow. The third target bandwidth value may be an adjusted inbound target bandwidth value.

[0216] As a possible implementation manner, the message structure of the first bandwidth adjustment request is shown in Table 19.

[0217] Table 19

[0218] Optionally, the message field description of the first bandwidth adjustment request is as shown in Table 20.

[0219] Table 20

[0220] The initiating device 61 assigns the OutStreamBW of the first bandwidth adjustment request to the first target bandwidth value and the InStreamBW to the third target bandwidth value. In a possible embodiment, if the service flow does not have an inbound direction, the third target bandwidth value is 0xFFFFFFFF.

[0221] Step 602: The initiating device 61 adjusts the outbound bandwidth value of the outbound port according to the target bandwidth value.

[0222] As a possible implementation, the step of the initiating device 61 adjusting the outbound bandwidth value may include the following sub-steps S1-S6.

[0223] S1. The initiating device 61 compares the target bandwidth value with the bandwidth value of the current path. If the target bandwidth value is greater than the bandwidth value of the current path, it is adjusted upward and S5 is executed. If the target bandwidth value is less than the bandwidth value of the current path, it is adjusted downward. If the target bandwidth value is equal to the bandwidth value of the current path, no adjustment is made.

[0224] The target bandwidth value of the initiating device 61 includes a first target bandwidth value, and the bandwidth value of the current path refers to the outbound bandwidth value allocated to the virtual path of the service flow at the output port of the device.

[0225] S2. The initiating device 61 determines whether the current available bandwidth meets the requirement. If yes, execute S3; otherwise, execute S4.

[0226] The current available bandwidth refers to the idle bandwidth value that the initiating device 61 can allocate to the output port. When the idle bandwidth value is greater than or equal to the first target bandwidth value, the current available bandwidth meets the requirement; otherwise, the current available bandwidth does not meet the requirement.

[0227] S3. The initiating device 61 allocates bandwidth to the output port.

[0228] The initiating device 61 allocates a bandwidth value to the output port, where the bandwidth value is equal to the difference between the first target bandwidth value and the outbound bandwidth value allocated to the output port.

[0229] As a possible implementation manner, when the available bandwidth meets the first target bandwidth value, the initiating device 61 adjusts the outbound bandwidth value of the virtual path on the initiating device to the first target bandwidth value.

[0230] S4. The initiating device 61 starts the bandwidth management upgrade process. If the available bandwidth of the bandwidth management upgrade meets the requirement, S5 is executed. Otherwise, a bandwidth adjustment response indicating that the bandwidth adjustment fails is directly returned.

[0231] Among them, the bandwidth improvement management process refers to the initiating device 61 allocating the idle differential channels in the differential channels of the main link to the virtual path of the business flow at the output port of the local device at the link layer to increase the available bandwidth of the output port, or switching the direction of the reverse differential channel in the virtual channel, that is, the differential channel for sending the business flow from the target device 62 to the initiating device 61, to increase the available bandwidth of the output port.

[0232] S5. The initiating device 61 refreshes the service flow information.

[0233] The service flow information includes the input port number, source device address, source adapter identifier, output port number, target device address, target adapter identifier, bandwidth value, and priority.

[0234] S6. The initiating device 61 configures the transport layer.

[0235] The initiating device 61 sends the information shown in Table 21 to the transport layer.

[0236] Table 21

[0237] Step 603: The initiating device 61 sends a first bandwidth adjustment request.

[0238] As a possible implementation manner, the initiating device 61 forwards the first bandwidth adjustment request according to the forwarding list addressing information of the first bandwidth adjustment request.

[0239] Step 604: The intermediate device 63 receives a first bandwidth adjustment request.

[0240] Step 605: The intermediate device 63 adjusts the outbound bandwidth value of the outbound port according to the target bandwidth value.

[0241] For the processing manner in which the intermediate device 63 adjusts the outbound bandwidth value of the outbound port according to the first target bandwidth value, please refer to step 602 and will not be described in detail here.

[0242] In a possible embodiment of the present application, when the intermediate device 63 carries the inflow bandwidth value in the first bandwidth adjustment request, it is also necessary to adjust the outflow bandwidth value of the inflow port according to the third target bandwidth value. The processing method is similar to the bandwidth value adjustment method of the outflow port, and will not be repeated here.

[0243] If the available bandwidth of the bandwidth management process cannot meet the requirement of the target bandwidth value, the intermediate device 63 directly returns a bandwidth adjustment response carrying an ErrCode of 0x6.

[0244] Step 606: The intermediate device 63 sends a first bandwidth adjustment response.

[0245] As a possible implementation manner, the message structure of the first bandwidth adjustment response is shown in Table 22.

[0246] Table 22

[0247] Optionally, the message field description of the first bandwidth adjustment response is as shown in Table 23.

[0248] Table 23

[0249] OutStreamBW is assigned to the second target bandwidth value, and InStreamBW is assigned to the fourth target bandwidth value.

[0250] As a possible implementation manner, the intermediate device 63 forwards the first bandwidth adjustment response according to the forwarding list addressing information of the first bandwidth adjustment response.

[0251] Step 607: The initiating device 61 receives a first bandwidth adjustment response.

[0252] As a possible implementation manner, the processing manner of the initiating device 61 after receiving the first bandwidth adjustment response includes the following steps S1-S5.

[0253] S1. The initiating device 61 identifies a path according to the Channel Shuttle ID and tag in the first bandwidth adjustment response.

[0254] The initiating device 61 queries the input port of the corresponding bandwidth adjustment request based on the Channel Shuttle ID and tag in the bandwidth adjustment response to complete the path identification. Requests and responses from the same source device have the same tag value on the same device.

[0255] S2. The initiating device 61 compares the target bandwidth value with the bandwidth value of the current path. If the target bandwidth value is greater than the bandwidth value of the current path, the bandwidth is adjusted upward and step 609 is executed. If the target bandwidth value is less than the bandwidth value of the current path, the bandwidth is adjusted downward and step S3 is executed. If the target bandwidth value is equal to the bandwidth value of the current path, no adjustment is made.

[0256] The target bandwidth values ​​include a second target bandwidth value, and the bandwidth value of the current path compared with the second target bandwidth value is the outbound bandwidth value of the virtual path at the output port of the local device. The target bandwidth values ​​include a fourth target bandwidth value, and the bandwidth value of the current path compared with the fourth target bandwidth value is the outbound bandwidth value of the virtual path at the input port of the local device.

[0257] S3. The initiating device 61 newly calculates the flow control buffer and weight, and sends them to the transport layer.

[0258] S4. The initiating device 61 waits for the transport layer down-flow control buffer to be released, and then releases the excess bandwidth value.

[0259] Excess bandwidth refers to the difference between the current path's bandwidth and the target bandwidth. For output ports, excess bandwidth is the difference between the output port's allocated outbound bandwidth and the outbound bandwidth value carried in the response. For input ports, excess bandwidth is the difference between the input port's allocated outbound bandwidth and the inbound bandwidth value carried in the response.

[0260] S5. The initiating device 61 updates and records the actual adjusted bandwidth value.

[0261] Step 608: The initiator device 61 sends a second adapter binding request.

[0262] Step 609: The intermediate device 63 receives the second adapter binding request.

[0263] Step 610: The intermediate device 63 sends a second adapter binding response.

[0264] Step 611: The initiator device 61 receives a second adapter binding response.

[0265] As a possible implementation manner, the difference between the second adapter binding request and the first adapter binding request in step 315 is that the first adapter binding request includes a first outbound bandwidth value and / or a first inbound bandwidth value, the first outbound bandwidth value is used to indicate the outbound bandwidth value of the USB downstream adapter corresponding to the USB host, and the second inbound bandwidth value is used to indicate the inbound bandwidth value of the USB downstream adapter corresponding to the USB host, and the second adapter binding request includes a second outbound bandwidth value and / or a second inbound bandwidth value, the second outbound bandwidth value is used to indicate the outbound bandwidth value of the USB host after the bandwidth is adjusted, and the second inbound bandwidth value is used to indicate the inbound bandwidth value of the USB host after the bandwidth is adjusted.

[0266] Step 612: The intermediate device 63 generates a second bandwidth adjustment request.

[0267] The intermediate device 63 determines an adjusted second target bandwidth value according to the second outbound bandwidth value and / or the second inbound bandwidth value in the second adapter binding request, and generates a first bandwidth adjustment request.

[0268] As a possible implementation manner, the second bandwidth adjustment request may refer to the first bandwidth adjustment request in step 601 and will not be described in detail here.

[0269] Step 613: The intermediate device 63 adjusts the outbound bandwidth value of the outbound port according to the target bandwidth value.

[0270] Step 614: The intermediate device 63 sends a second bandwidth adjustment request.

[0271] Step 615: The target device 62 receives the second bandwidth adjustment request.

[0272] Step 616: The target device 62 adjusts the outbound bandwidth value of the outbound port according to the target bandwidth value.

[0273] Step 617: The target device 62 sends a second bandwidth adjustment response.

[0274] Step 618: The target device 62 receives the second bandwidth adjustment response.

[0275] As a possible implementation method, for the specific processing methods of the above steps 613 to 618, please refer to steps 602 to 607, which will not be repeated here.

[0276] 3 to 6 , the virtual path establishment and bandwidth adjustment operations in the USB tunnel management method are described in detail. Next, the virtual path removal operation in the USB tunnel management method is described in detail with reference to FIG. 7 .

[0277] Adapter unbinding involves unbinding an adapter from its original binding and dismantling the virtual path from the source adapter to the target adapter. Dismantling the virtual path requires a bandwidth release message. The unbinding is considered successful only after the bandwidth of the virtual path has been released. The adapter unbinding operation consists of an adapter unbinding request and an adapter unbinding response.

[0278] Here, the bandwidth adjustment operation of an embodiment of the present application is performed by the initiator device 210 and the target device 220 shown in Figure 2 as an example. Figure 7 is a schematic flow chart of a USB tunnel removal operation provided by the present application. In this embodiment, the initiator device 71 is used to implement the functions of the source device 210, and the target device 72 is used to implement the functions of the sink device 220. In this embodiment, the initiator device 71 can also be referred to as a source device, an audio and video transmitter, or an audio and video transmitter, and the target device 72 can also be referred to as a sink device, a display device, an audio and video receiver, or an audio and video player. In this embodiment, the initiator device 71 and the target device 72 are connected via one or more intermediate devices 73 (only one intermediate device 73 is shown in Figure 7, but the number of intermediate devices is not limited). The initiator device 71, the intermediate device 73, and the target device 72 are connected via a bus 74, which can be a UMI bus. The virtual channels between the initiator device 71 and the intermediate device 73, and the virtual channels between the intermediate device 73 and the target device 72 constitute a virtual path.

[0279] Referring to FIG. 7 , the USB tunnel removal operation provided in this embodiment includes the following steps 701 to 710 .

[0280] Step 701: The initiating device 71 generates an adapter unbinding request.

[0281] When the initiator device 71 is equipped with a USB Host, an adapter unbinding request is generated due to a topology change detected by the user or detected, and the USB downstream adapter of the designated intermediate device 73 is unbound from the USB upstream adapter of the target device 72.

[0282] As a possible implementation method, the message structure of the adapter unbinding request is shown in Table 24.

[0283] Table 24

[0284] Optionally, the message field description of the adapter unbinding request is as shown in Table 25.

[0285] Table 25

[0286] Step 702: The initiating device 71 sends an adapter unbinding request.

[0287] Step 703: The intermediate device 73 receives the adapter unbinding request.

[0288] The adapter unbinding request is used to instruct the USB downstream adapter of the current device to unbind from the USB downstream adapter of the next-level device in the virtual path, that is, to unbind the intermediate device 73 from the target device 72 .

[0289] Step 704: The intermediate device 73 sends an adapter unbinding response.

[0290] When the intermediate device 73 receives the adapter unbinding request, it determines whether to accept the request based on the business situation and returns the adapter binding response with the accurate ErrCode.

[0291] As a possible implementation method, the message structure of the adapter unbinding response is shown in Table 26.

[0292] Table 26

[0293] Optionally, the message field description of the adapter unbinding response is shown in Table 27.

[0294] Table 27

[0295] Step 705: The initiating device 71 receives the adapter unbinding response.

[0296] Step 706: The intermediate device 73 sends a bandwidth release request.

[0297] The management adapter of the intermediate device 73 first refreshes the binding mark between the designated downstream adapter and the upstream adapter of the target device 72 to unbinding, and then generates a bandwidth release request and completes the bandwidth release.

[0298] The intermediate device 73 generates a bandwidth release request according to the adapter unbinding request.

[0299] As a possible implementation, the message structure of the bandwidth release request is shown in Table 28.

[0300] Table 28

[0301] Optionally, the message field description of the bandwidth release request is shown in Table 29.

[0302] Table 29

[0303] The steps of the intermediate device 73 releasing bandwidth according to the bandwidth release request include: configuring the identifier of the forwarding entry of the outgoing flow node and sending the configuration information to the transport layer, which will not be described in detail here.

[0304] Step 707: The target device 72 receives the bandwidth release request.

[0305] Step 708: The target device 72 sends a bandwidth release response.

[0306] As a possible implementation, the message structure of the bandwidth release response is shown in Table 30 below.

[0307] Table 30

[0308] The structures of the fields in Table 29 are similar to those in the bandwidth release request. The difference is that in the bandwidth release response header, the value of Response is 1, indicating that it is a request message. In the general field, the value of Channel ShuttleID is assigned to the virtual channel identifier. These are all adaptive content adjustments between requests and responses and are not detailed here.

[0309] Optionally, the bandwidth release response also includes a message identifier (tag). For the same management message, the tags of the request and response are consistent. Therefore, when the bandwidth release response is forwarded in the virtual path, the device can perform path identification based on the virtual channel identifier and message identifier carried by the bandwidth release response to determine the output port of the bandwidth release response.

[0310] Step 709: The intermediate device 73 receives a bandwidth release response.

[0311] Step 710: In response to the bandwidth release response, the intermediate device 73 removes the virtual channel corresponding to the virtual channel identifier.

[0312] As a possible implementation method, the intermediate device 73 determines whether the multicast count value of the flow node is equal to zero. If so, it dismantles the virtual channel corresponding to the virtual channel identifier, including: starting to release the transport layer flow control cache, and waiting for the release to complete the bandwidth value release.

[0313] Optionally, the bandwidth value release performed by the intermediate device 73 is to release the bandwidth value allocated to the virtual channel corresponding to the virtual channel identifier, that is, to release the outbound bandwidth value on the output port corresponding to the virtual channel identifier.

[0314] As a possible implementation, when the multicast count value of the outgoing node is not zero (greater than zero), the intermediate device removes the virtual channel corresponding to the virtual channel identifier, including clearing adapter binding information.

[0315] Optionally, the adapter binding information cleared by the intermediate device 73 is binding information between the adapter of the intermediate device and the adapter of the next-level device in the virtual path, or binding information between the adapter of the target device 72 and the adapter of the initiator device 71 .

[0316] In a possible embodiment of the present application, for USB business flows, it is necessary to release both outgoing and incoming bandwidth at the same time. In addition to releasing outgoing bandwidth for the output port, the intermediate device also needs to release outgoing bandwidth for the input port. The rules and processes for bandwidth release are the same as above.

[0317] In a possible embodiment of the present application, if the USB upstream adapter corresponding to the USB Hub receiving the bandwidth release request has been released, all downstream adapters of the Hub (already bound to other USB upstream adapters) initiate bandwidth release requests. As shown in FIG8 , the intermediate device has a Hub resource, which has two USB downstream adapters, both of which are bound to downstream USB upstream adapters. After the USB upstream adapter of the intermediate device Hub completes bandwidth release and unbinding, the two downstream adapters corresponding to the Hub, target devices 1 and 2, respectively, generate bandwidth release requests.

[0318] The USB tunnel management method provided in accordance with the present embodiment is described in detail above with reference to FIG. 1 to FIG. 8 . The USB tunnel management device provided in accordance with the present embodiment will be described below with reference to FIG. 9 .

[0319] Figure 9 is a schematic diagram of the structure of a USB tunnel management device provided by this application. This USB tunnel management device 900 can be used to implement the functions of any device in the above-mentioned method embodiments, thereby also achieving the beneficial effects of the above-mentioned method embodiments. In this embodiment, the USB tunnel management device 900 can be the set-top box 110, smart TV 120, or any display device as shown in Figure 1, or the source device 210 or sink device 220 as shown in Figure 2, or a multimedia device or display device provided in subsequent embodiments. It should be understood that the USB tunnel management device 900 can also be a module (such as a chip) applied to any of the aforementioned devices.

[0320] As shown in Figure 9 , the USB tunnel management device 900 includes a transceiver module 910 and a processing module 920. Transceiver module 910 and processing module 920 can collaboratively implement the various steps in the aforementioned method embodiment. A more detailed description of transceiver module 910 and processing module 920 can be directly obtained by referring to the description of the devices in the method embodiment shown in the aforementioned figures, and is not further elaborated here.

[0321] When the USB tunnel management device 900 implements the USB tunnel management method shown in any of the aforementioned figures through software, the USB tunnel management device 900 and its various units may also be software modules. The USB tunnel management method is implemented by a processor calling the software module. The processor may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0322] It can be understood that the USB tunnel management device shown in Figure 9 is only an example provided in this embodiment. The USB tunnel management device may include more or fewer units according to different audio and video service stream transmission processes, and this application is not limited to this.

[0323] When the USB tunnel management device 900 is implemented via hardware, the hardware may be implemented via a processor or a chip system. The chip system includes one or more chips, each of which includes an interface circuit and a control circuit. The interface circuit is used to receive data from devices outside the chip and transmit it to the control circuit, or to send data from the control circuit to devices outside the chip. The control circuit and the interface circuit implement the method of any possible implementation method in the above embodiments through logic circuits or executing code instructions. The beneficial effects can be found in the description of any aspect of the above embodiments and will not be repeated here.

[0324] It is understood that the processor in the embodiments of the present application may be a CPU, or other general-purpose processor, digital signal processor (DSP), ASIC, FPGA or other programmable logic device, transistor logic device, hardware component or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0325] In addition, the USB tunnel management device 900 shown in FIG9 can also be implemented by a multimedia device, as shown in FIG10 , which is a schematic structural diagram of the multimedia device provided by the present application. The multimedia device includes: a processor 1010, an external memory interface 1020, an internal memory 1021, a universal serial bus (USB) interface 1030, a UMI interface 1031, antenna 1, antenna 2, a mobile communication module 1050, a wireless communication module 1060, an audio module 1070, a speaker 1070A, a receiver 1070B, a microphone 1070C, a sensor module 1080, a button 1090, an indicator 1092, a camera 1093, a display screen 1094, and subscriber identification module (SIM) card interfaces 1-N 1095.

[0326] Among them, the above-mentioned sensor module 1080 may include sensors such as pressure sensor, gyroscope sensor, air pressure sensor, magnetic sensor, acceleration sensor, distance sensor, proximity light sensor, fingerprint sensor, temperature sensor, touch sensor, ambient light sensor and bone conduction sensor.

[0327] It is understood that the structure illustrated in this embodiment does not constitute a specific limitation on the multimedia device. In other embodiments, the multimedia device may include more or fewer components than shown, or combine or separate certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0328] The multimedia device shown in FIG10 may be any device in FIG1 , or a source device or a sink device in subsequent embodiments.

[0329] The processor 1010 may include one or more processing units. For example, the processor 1010 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors.

[0330] The controller can be the nerve center and command center of the multimedia device. The controller can generate operation control signals based on instruction opcodes and timing signals to complete the control of instruction fetching and execution.

[0331] Processor 1010 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 1010 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 1010. If processor 1010 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 1010 latency, and thus improves system efficiency.

[0332] In some embodiments, the processor 1010 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a USB interface, a UMI interface, etc.

[0333] It is understood that the interface connection relationship between the modules illustrated in this embodiment is merely a schematic illustration and does not constitute a structural limitation on the multimedia device. In other embodiments, the multimedia device may also adopt a different interface connection method from the above embodiment, or a combination of multiple interface connection methods.

[0334] The wireless communication function of the multimedia device can be implemented through antenna 1, antenna 2, mobile communication module 1050, wireless communication module 1060, modem processor, and baseband processor. In some embodiments, antenna 1 of the multimedia device is coupled to mobile communication module 1050, and antenna 2 is coupled to wireless communication module 1060, so that the multimedia device can communicate with the network and other devices through wireless communication technology.

[0335] The wired communication function of the multimedia device can be implemented through the USB interface 1030 or the UMI interface 1031. For example, the multimedia device receives or sends video streams and AVP messages through the bus connected to the UMI interface 1031.

[0336] The multimedia device implements display functions through a GPU, display screen 1094, and an application processor. A GPU is a microprocessor for image processing that connects display screen 1094 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 1010 may include one or more GPUs that execute program instructions to generate or modify display information.

[0337] The display screen 1094 is used to display images, videos, etc. The display screen 1094 includes a display panel.

[0338] The multimedia device can implement a camera function through an ISP, a camera 1093, a video codec, a GPU, a display 1094, and an application processor. The ISP processes data fed back by the camera 1093. The camera 1093 is used to capture still images or videos. In some embodiments, the multimedia device may include one or N cameras 1093, where N is a positive integer greater than 1.

[0339] In this embodiment, the above display screen 1094, video codec, GPU, display screen 1094 and application processor can also be collectively referred to as the display unit of the multimedia device, which is used to process and display the received multimedia data stream (such as video stream).

[0340] External memory interface 1020 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the multimedia device. The external memory card communicates with processor 1010 via external memory interface 1020 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0341] The internal memory 1021 can be used to store computer executable program code, which includes instructions. The processor 1010 executes various functional applications and data processing of the multimedia device by running the instructions stored in the internal memory 10101. For example, in an embodiment of the present application, the processor 210 can execute instructions stored in the internal memory 1021, and the internal memory 1021 can include a program storage area and a data storage area.

[0342] The program storage area may store an operating system and at least one application required for a function (such as a sound playback function, an image playback function, etc.). The data storage area may store data created during the use of the multimedia device (such as audio and video data, a phone book, etc.). In addition, the internal memory 1021 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, a flash memory device, or a universal flash storage (UFS).

[0343] The multimedia device can implement audio functions such as music playback and recording through the audio module 1070, the speaker 1070A, the receiver 1070B, the microphone 1070C, and the application processor.

[0344] Buttons 1090 include a power button, a volume button, and the like. Buttons 1090 may be mechanical buttons or touch buttons. Indicator 1092 may be an indicator light that can be used to indicate charging status, battery level changes, messages, missed calls, notifications, and the like.

[0345] The method steps in the embodiments of the present application can also be implemented by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and storage medium can be located in an ASIC. In addition, the ASIC can be located in a network device or a terminal device. Of course, the processor and storage medium can also exist as discrete components in a video processing device and a multimedia device.

[0346] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the process or function described in the embodiments of the present application is performed in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program or instruction can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired or wireless means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, a hard disk, or a tape; it can also be an optical medium, such as a digital video disc (DVD); it can also be a semiconductor medium, such as a solid state drive (SSD).

[0347] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A USB tunnel management method, characterized in that, the method includes: Sending an adapter list request to intermediate devices and target devices on the audio and video service flow path; Receiving an adapter list response sent by an intermediate device or a target device; the adapter list response includes identifiers of multiple adapters of the device, and the multiple adapters include a USB downstream adapter or a USB upstream adapter; Allocating bandwidth to a first virtual channel of a virtual path according to the out-flow bandwidth value required by the USB tunnel service flow; the virtual path includes multiple virtual channels cascaded between the USB downstream adapter of the initiating device of the USB tunnel service flow and the target USB upstream adapter among the multiple adapters, and the virtual channels are used to transmit the USB tunnel service flow on the link between two ports; Sending a first bandwidth application request to the next-level device in the virtual path; the first bandwidth application request is used to indicate the establishment of the first virtual channel; Receiving a first bandwidth application response.

2. The method according to claim 1, characterized in that, the method further includes: Determining the target USB upstream adapter of the target device among the multiple adapters; Determining the virtual path from the USB downstream adapter of the initiating device to the target USB upstream adapter among multiple topological paths between the initiating device and the target device.

3. The method according to claim 1 or 2, characterized in that, the method further includes: Sending a first adapter binding request; the first adapter binding request is used to indicate the binding of the USB downstream adapter of the next-level device in the virtual path to the USB upstream adapter of the next-next-level device, and the first adapter binding request includes a first out-flow bandwidth value and / or a first in-flow bandwidth value, the first out-flow bandwidth value is used to indicate the out-flow bandwidth value of the USB host corresponding to the USB downstream adapter, and the second in-flow bandwidth value is used to indicate the in-flow bandwidth value of the USB host corresponding to the USB downstream adapter; Receiving a first adapter binding response.

4. The method according to any one of claims 1-3, characterized in that, the method further includes: Generating a first bandwidth adjustment request; the first bandwidth adjustment request includes a first target bandwidth value; When the first target bandwidth value is greater than the out-flow bandwidth value of the virtual path, upward-adjusting the out-flow bandwidth value of the virtual path at the device; Sending the first bandwidth adjustment request to the next-level device in the virtual path; Receiving a first bandwidth adjustment response.

5. The method according to claim 4, characterized in that, the method further includes: Sending a second adapter binding request; the second adapter binding request is used to indicate the binding of the USB downstream adapter of the next-level device in the virtual path to the USB upstream adapter of the next-next-level device, and the second adapter binding request includes a second out-flow bandwidth value and / or a second in-flow bandwidth value, the second out-flow bandwidth value is used to indicate the out-flow bandwidth value of the USB host after bandwidth adjustment, and the second in-flow bandwidth value is used to indicate the in-flow bandwidth value of the USB host after bandwidth adjustment; Receive a second adapter binding response.

6. The method according to any one of claims 1-5, wherein, the method further includes: Sending an adapter unbinding request; the adapter unbinding request is used to instruct the USB downstream adapter of the next-level device in the virtual path to unbind from the USB upstream adapter of the next-next-level device; Receiving an adapter unbinding response.

7. A USB tunnel management method, wherein, the method includes: Receiving an adapter list request; Sending an adapter list response; the adapter list request includes the identifier of the adapter of this device; Receiving a first bandwidth application request; the first bandwidth application request includes the requested outgoing bandwidth value and / or incoming bandwidth value; Sending a first bandwidth application response.

8. The method according to claim 7, wherein, the method further includes: Receiving a first adapter binding request; the first adapter binding request is used to instruct the USB downstream adapter of this device to bind to the USB upstream adapter of the next-level device in the virtual path; Sending a first adapter binding response; Allocating bandwidth for the second virtual channel of the virtual path according to the outgoing bandwidth value required by the USB tunnel service flow, and the second virtual channel is the next-level virtual channel of the first virtual channel in the virtual path; Sending a second bandwidth application request to the next-level device in the virtual path; the second bandwidth application request is used to instruct to establish the second virtual channel; Receiving a second bandwidth application response.

9. The method according to claim 7 or 8, wherein, the method further includes: Receiving a first bandwidth adjustment request; Sending a first bandwidth adjustment response.

10. The method according to claim 9, wherein, the method further includes: Receiving a second adapter binding request; the second adapter binding request is used to instruct the USB downstream adapter of this device to bind to the USB upstream adapter of the next-level device in the virtual path; Sending a second adapter binding response; Generating a second bandwidth adjustment request; the second bandwidth adjustment request includes a second target bandwidth value; When the second target bandwidth value is greater than the outgoing bandwidth value of the virtual path, upward-adjust the outgoing bandwidth value of the virtual path at the device; Sending the second bandwidth adjustment request to the next-level device in the virtual path; Receiving a second bandwidth adjustment response.

11. The method according to any one of claims 7-10, wherein, the method further includes: Receiving an adapter unbinding request; the adapter unbinding request is used to instruct the USB downstream adapter of this device to unbind from the USB downstream adapter of the next-level device in the virtual path; Sending an adapter unbinding response; Sending a bandwidth release request; Receiving a bandwidth release response.

12. A USB tunnel management device, wherein, includes: A transceiver module, configured to send an adapter list request to an intermediate device and a target device on the audio and video service flow path; The transceiver module is further configured to receive an adapter list response sent by an intermediate device or a target device; the adapter list response includes identifiers of multiple adapters of a device, and the multiple adapters include a USB downstream adapter or a USB upstream adapter; The processing module is configured to allocate bandwidth for a first virtual channel of a virtual path according to an outflow bandwidth value required by a USB tunnel service flow; The virtual path includes multiple virtual channels cascaded between a USB downstream adapter of an initiating device of the USB tunnel service flow and a target USB upstream adapter among the multiple adapters, and the virtual channels are used to transmit the USB tunnel service flow on a link between two ports; The transceiver module is further configured to send a first bandwidth application request to a next-level device in the virtual path; the first bandwidth application request is used to indicate the establishment of the first virtual channel; The transceiver module is further configured to receive a first bandwidth application response.

13. A USB tunnel management device, Characterized in that, It includes: A transceiver module, configured to receive an adapter list request; The transceiver module is further configured to send an adapter list response; the adapter list request includes an identifier of an adapter of this device; The transceiver module is further configured to receive a first bandwidth application request; the first bandwidth application request includes an applied outflow bandwidth value and / or an inflow bandwidth value; The transceiver module is further configured to send a first bandwidth application response.

14. A multimedia device, Characterized in that, It includes: A transceiver and a processor; The processor is configured to process a USB tunnel service flow; The transceiver is configured to transmit and receive the USB tunnel service flow; The transceiver and the processor are configured to cooperate to execute the method according to any one of claims 1-6, or the method according to any one of claims 7-11.

15. A readable storage medium, Characterized in that, The readable storage medium includes a computer program or instruction, and when the computer program or instruction runs on a computer, the computer is caused to execute the operation steps of the method according to any one of claims 1-6, or the operation steps of the method according to any one of claims 7-11.

16. A computer program product, Characterized in that, The computer program product includes a computer program or instruction, and when the computer program or instruction runs on a computer, the computer is caused to execute the operation steps of the method according to any one of claims 1-6, or the operation steps of the method according to any one of claims 7-11.

Images