Virtual channel establishment method and apparatus

Abstract

Disclosed are a virtual channel establishment method and apparatus, which relate to the technical field of multimedia. By means of processing an adapter list request, an adapter list response, a bandwidth application request and a bandwidth application response, various stages of virtual paths on a virtual channel of an audio / video service flow are established between an initiating device of the audio / video service flow and a target device thereof, and a transmission route corresponding to the virtual channel is flexibly determined, such that the transmission route is prevented from relying on a fixed physical connection; and the audio / video service flow is transmitted between two ports by means of the virtual paths, such that the transmission of the audio / video service flow is decoupled from a physical connection manner between the ports, thereby further improving the flexibility of changing the transmission route of the audio / video service flow.

Description

Virtual path establishment method and device Technical Field

[0001] The present application relates to the field of multimedia technology, and in particular to a method and device for establishing a virtual path. Background Art

[0002] Existing audio and video transmission networks are typically tree-like networks, such as those based on High Definition Multimedia Interface (HDMI). In a tree-like network, the transmission path of audio and video traffic is determined by the physical connections between the ports of each device in the audio and video transmission network.

[0003] However, when the source device of the audio and video service flow needs to specify a destination device to transmit the audio and video service flow, or when the source device of the audio and video service flow needs to switch a destination device to transmit the audio and video service flow, it is necessary to connect or change the physical connection of the audio and video transmission network, resulting in poor flexibility of the transmission path of the audio and video service flow.

[0004] Summary of the Invention

[0005] The present application provides a method and apparatus for establishing a virtual path, which solves the problem that the transmission path establishment of an audio and video transmission network depends on physical connections, and improves the flexibility of the transmission path of audio and video service flows.

[0006] In a first aspect, the present application provides a method for establishing a virtual path. The method for establishing a virtual path is applied to an initiating device or a processor of an initiating device, and the initiating device is connected to an intermediate device or a target device through a unified multimedia interconnection interface (UMI) bus. The method for establishing a virtual path includes: first, the initiating device sends an adapter list and receives a returned adapter list response, where the adapter list response includes the identifiers of multiple adapters contained in the target device. Then, the initiating device allocates bandwidth to the first virtual channel of the virtual path between the adapters according to the outgoing bandwidth value required by the audio and video service flow, and sends a bandwidth application request to instruct the next-level device in the virtual path to allocate bandwidth to the second virtual channel according to the first outgoing bandwidth value, where the second virtual channel is the next-level virtual channel of the first virtual channel. Finally, the initiating device receives the bandwidth application response and completes the establishment of the virtual path.

[0007] The virtual channel includes multiple virtual channels cascaded between a source adapter of an initiating device of an audio or video service flow and a sink adapter among multiple adapters of a target device of the audio or video service flow. The virtual channel is used to transmit the audio or video service flow on a link between two ports.

[0008] Based on the above-mentioned virtual path establishment method, the initiating device can determine the multiple adapters contained in the target device according to the adapter list response, thereby identifying the virtual path from the initiating device to the target device. In this way, the devices through which the virtual path passes can be selected according to demand, that is, the transmission path corresponding to the virtual path can be flexibly determined, thereby avoiding the transmission path from relying on fixed physical connections. At the same time, through the bandwidth application request, each device on the virtual path is instructed to complete the bandwidth allocation of the corresponding virtual channel, thereby realizing the hierarchical creation of designated virtual channels, and transmitting audio and video service streams between two ports through the virtual channel, thereby decoupling the transmission of audio and video service streams from the physical connection method between the ports, further improving the flexibility of changing the transmission path of audio and video service streams.

[0009] In a second aspect, the present application provides a method for establishing a virtual path. This method is applied to an intermediate device or a processor of the intermediate device, where the intermediate device is connected to an initiator device and a target device via a UMI bus. The method includes the following steps: first, the intermediate device receives a bandwidth request, where the bandwidth request response includes a first outbound bandwidth value. Then, in response to the bandwidth request request, the intermediate device allocates bandwidth to a second virtual channel based on the first outbound bandwidth value, and forwards the bandwidth request request carrying a fourth outbound bandwidth value to instruct the next-level device in the virtual path to allocate bandwidth to a fourth virtual channel, the next-level virtual channel of the second virtual channel, based on the fourth outbound bandwidth value. Finally, upon receiving the bandwidth request response, the intermediate device forwards the bandwidth request response, completing the establishment of the virtual path.

[0010] Based on the above-mentioned virtual channel establishment method, the intermediate device can complete the establishment of the specified virtual channel according to the transmission and reception of bandwidth application requests and bandwidth application responses, thereby decoupling the transmission of audio and video service flows from the physical connection method between ports, avoiding the transmission path from relying on fixed physical connections, and improving the flexibility of changing the transmission path of audio and video service flows.

[0011] In a third aspect, the present application provides a method for establishing a virtual path. This method is applied to a target device or a processor of the target device, where the target device is connected to an intermediate device or an initiator device via a UMI bus. The method includes: first, the target device receives an adapter list request and returns an adapter list response, where the adapter list response includes identifiers of multiple adapters included in the target device. Then, the target device receives a bandwidth request including a fifth outbound bandwidth value and returns a bandwidth request response including a sixth outbound bandwidth value, where the sixth outbound bandwidth value indicates the requested outbound bandwidth value.

[0012] Based on the above-mentioned virtual path establishment method, the target device returns a bandwidth application response after receiving the bandwidth application request to indicate that the initiating device on the virtual path has applied for the outbound bandwidth value and started the transmission of the audio and video service flow, thereby realizing the establishment or switching of the transmission path of the audio and video service flow without changing the physical connection of the audio and video transmission network, thereby improving the flexibility of changing the transmission path of the audio and video service flow.

[0013] In combination with the virtual path establishment method provided in the first to third aspects, in one possible implementation, before allocating bandwidth to the first virtual channel, the initiating device determines a sink adapter from a plurality of adapters, and determines a virtual path from the source adapter to the sink adapter from a plurality of paths included in the topology structure between the source adapter and the sink adapter.

[0014] In combination with the virtual path establishment method provided by the first to third aspects, in a possible implementation, in a complex networking scenario, the initiating device first performs adapter binding before sending a bandwidth application request to lock resources (such as bandwidth and other resources) to prevent other devices from preempting the adapter and its bandwidth, resulting in the failure of virtual path establishment. Optionally, the initiating device sends an adapter binding request, which includes the identifier of the host adapter. The target device receives the adapter binding request and returns an adapter binding response, which is used to indicate that the host adapter has been bound. The initiating device receives the adapter binding response and completes the binding of the source adapter and the host adapter. Among them, the adapter binding request or the adapter binding response can be forwarded through an intermediate device.

[0015] In combination with the virtual path establishment method provided in the first to third aspects, in a possible implementation, before sending a bandwidth application request, the initiating device may also query the available outbound bandwidth values ​​of multiple paths included in the topological structure between the source adapter and the sink adapter to select a suitable virtual path. Optionally, the initiating device sends a bandwidth query request, the bandwidth query request includes a second outbound bandwidth value, and the second outbound bandwidth value is used to indicate the available outbound bandwidth value of the output port of the initiating device. The target device receives the bandwidth query request and returns a bandwidth query response, the bandwidth query response includes a third outbound bandwidth value, and the third outbound bandwidth value is used to indicate the minimum outbound bandwidth value available for each port through which the virtual path passes. After receiving the bandwidth query response, the initiating device completes the bandwidth query of each port of the topological path between the source adapter and the sink adapter.

[0016] Optionally, the initiating device sends the bandwidth application request when the third outbound bandwidth value in the bandwidth query response is greater than or equal to the required outbound bandwidth value.

[0017] Optionally, when the third outbound bandwidth value of the bandwidth query response is less than the required outbound bandwidth value, the initiator device switches the virtual path between the source adapter and the sink adapter, that is, selects another virtual path from multiple paths included in the topology structure between the source adapter and the sink adapter.

[0018] Optionally, if the third outbound bandwidth value in the bandwidth query response is less than the required outbound bandwidth value, the initiating device may determine whether to reduce the specifications of the audio and video service flows based on service flow requirements before sending a bandwidth request to establish the virtual path. For example, if the third outbound bandwidth value is less than the required outbound bandwidth value, the specifications of the audio and video service flows may be reduced, including resolution, color space, and / or bit depth.

[0019] In a fourth aspect, the present application provides a virtual path establishment device, which includes a module for executing the method of any one of the implementations of the first aspect, the second aspect, or the third aspect.

[0020] In a fifth aspect, the present application provides a multimedia device. The multimedia device includes a processor and a transceiver. Exemplarily, the processor is configured to process audio and video service streams, and the transceiver is configured to receive and transmit audio and video service streams. The processor and transceiver collaborate to perform the method of any optional implementation of the first, second, or third aspects.

[0021] In a sixth aspect, the present application provides a multimedia data transmission system. This multimedia data transmission system includes multiple multimedia devices provided in the fifth aspect, wherein the multimedia device used to send audio and video service streams is an initiating device, the multimedia device used to receive audio and video service streams is a target device, and the multimedia device used to forward audio and video service streams is an intermediate device. The initiating device can be used to implement the function of the initiating device in the first aspect, the intermediate device can be used to implement the function of the intermediate device in the second aspect, and the target device can be used to implement the function of the target device in the third aspect. Therefore, this multimedia data transmission system can also achieve the beneficial effects of the methods in the first to third aspects above, which will not be elaborated here.

[0022] In a seventh aspect, the present application provides a computer-readable storage medium. The computer-readable storage medium includes computer software instructions. When the computer software instructions are executed in a computing device, the computing device executes the operating steps of the method described in the first aspect or any possible implementation of the first aspect, as well as the operating steps of the method described in the second aspect or any possible implementation of the second aspect, and as well as the operating steps of the method described in the third aspect or any possible implementation of the third aspect. For example, the computing device is the aforementioned initiating device or target device.

[0023] In an eighth aspect, the present application provides a computer program product. When the computer program product is executed on a computer, the computer program product causes the computing device to perform the steps of the method described in the first aspect or any possible implementation of the first aspect, as well as the steps of the method described in the second aspect or any possible implementation of the second aspect, and the steps of the method described in the third aspect or any possible implementation of the third aspect. For example, the computer is the aforementioned initiating device or target device.

[0024] Regarding the beneficial effects of the fourth to eighth aspects, reference may be made to the description of any implementation method in the first to third aspects, and no further details will be given here. Based on the implementation methods provided in the above aspects, this application can also be further combined to provide more implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] FIG1 is a schematic diagram of a video transmission system provided by the present application;

[0026] FIG2 is a schematic diagram of an audio and video encoding and decoding system provided by the present application;

[0027] FIG3 is a flow chart of a method for establishing a virtual path provided by the present application;

[0028] FIG4 is a schematic diagram of a virtual path provided by the present application;

[0029] FIG5 is a schematic diagram of multicast of a virtual channel provided by the present application;

[0030] FIG6 is a flow chart of an adapter binding operation provided by the present application;

[0031] FIG7 is a schematic diagram of a flow chart of a bandwidth query operation provided by the present application;

[0032] FIG8 is a schematic structural diagram of a virtual path establishment device provided by the present application;

[0033] FIG9 is a schematic structural diagram of a multimedia device provided by this application. DETAILED DESCRIPTION

[0034] The present application provides a method for establishing a virtual path, in which an initiating device first sends an adapter list request to a target device to receive an adapter list response returned by the target device. The adapter list response includes the identifiers of multiple adapters contained in the target device, thereby determining the virtual path between the source adapter of the initiating device and the host device among the multiple adapters of the target device. Then, the initiating device allocates bandwidth to the first virtual channel of the virtual path according to the outgoing bandwidth value required by the audio and video service flow, and sends a bandwidth application request to instruct the next-level device in the virtual path to allocate bandwidth to the second virtual channel according to the first outgoing bandwidth value. The second virtual channel is the next-level virtual channel of the first virtual channel. Finally, after receiving the bandwidth application request, the target device returns a bandwidth application response. The initiating device receives the bandwidth application response and completes the establishment of the virtual path. The above-mentioned request or response can be forwarded between the initiating device and the target device through one or more intermediate devices.

[0035] In this way, the initiating device can determine the multiple adapters contained in the target device based on the adapter list response, thereby identifying the virtual path from the initiating device to the target device. The devices that the virtual path passes through can be selected according to demand, that is, the transmission path corresponding to the virtual path can be flexibly determined, thereby avoiding the transmission path from relying on fixed physical connections. At the same time, a bandwidth application request is used to instruct each device on the virtual path to complete the bandwidth allocation of the corresponding virtual channel, thereby realizing the hierarchical creation of designated virtual channels, and transmitting audio and video service streams between two ports through the virtual channel, thereby decoupling the transmission of audio and video service streams from the physical connection method between the ports, further improving the flexibility of changing the transmission path of audio and video service streams.

[0036] The technical solutions involved in this application may be applied not only to current audio and video transmission technologies or audio and video standards, but also to future audio and video transmission technologies or audio and video standards. The terms used in the embodiments of this application are only used to explain the specific embodiments of this application and are not intended to limit this application. The following is a brief introduction to some concepts that may be involved in this application.

[0037] Audio and video service flows include audio streams and / or video streams. An audio stream is a data stream used to transmit audio data in real time. A video stream is the transmission of video data. For example, a video stream can be processed as a stable and continuous flow over a network. A video stream consists of multiple video frames, each of which corresponds to an image.

[0038] In this embodiment, "video" is a general term that refers to a sequence of multiple consecutive frames, with one frame corresponding to one image. "Audio and video" is an information application technology term that refers to video, audio, or multimedia content that includes both video and audio. Furthermore, this application does not limit the type of service flow. For example, the service flow in this application can be an audio and video service flow or a USB service flow. The following description will primarily use audio and video service flows as an example.

[0039] A router's forwarding table, also known as a forwarding table or forwarding information base (FIB), is a table located on the router's data plane. The entries in the router's forwarding table are called forwarding entries or forwarding entries. Each forwarding entry specifies a destination, the outgoing interface to be reached, the next-hop IP address, and other information.

[0040] In order to make the description of the following embodiments clear and concise, the video transmission system to which the virtual path establishment method of the present application is applicable is first introduced.

[0041] Figure 1 is a schematic diagram of a video transmission system provided by the present application. The video processing process may include but is not limited to: video acquisition, video encoding, video transmission, video decoding and playback.

[0042] The video transmission system in Figure 1 includes a set-top box 110, a smart TV 120, multiple audio and video playback devices, and a server 130. The set-top box 110 is connected to the operator's network via a network cable and can receive audio and video streams from the server 130. The network can implement the function of audio and video transmission, and the network can include one or more network devices, such as a network device 131, which can be a router or a switch. In some optional implementations, the set-top box 110 and the server 130 can also communicate via wireless communication, which is not limited in this application.

[0043] The video transmission system in Figure 1 includes a set-top box 110, a smart TV 120, multiple audio and video playback devices, and a server 130. The set-top box 110 is connected to the operator's network via a network cable and can receive audio and video streams from the server 130. The network can implement the function of audio and video transmission, and the network can include one or more network devices, such as a network device 131, which can be a router or a switch. In some optional implementations, the set-top box 110 and the server 130 can also communicate via wireless communication, which is not limited in this application.

[0044] Smart TV 120 is a display device with audio and video processing capabilities that implements functions such as receiving, processing, pushing, and playing video streams or audio and video streams. In some possible scenarios, the smart TV 120 may refer to an audio and video device such as a conference tablet, a smart TV, or a projector, but this application is not limited to this.

[0045] The plurality of audio and video playback devices include audio and video playback devices 121 to 124. For example, these audio and video playback devices may include, but are not limited to, multimedia control platforms or other devices supporting audio and video playback functions, such as virtual reality (VR) terminal devices or augmented reality (AR) terminal devices.

[0046] In this embodiment, the set-top box 110 and the smart TV 120 can be connected via a bus 125. The set-top box 110 and various audio and video playback devices can also be connected via the bus 125. The smart TV 120 and various audio and video playback devices can also be connected via the bus 125. Exemplarily, the bus 125 can be a data bus that supports video and audio and video transmission. For example, the bus 125 is a UMI bus. A UMI bus is a bus connected based on the UMI interface provided by the source device and the sink device. The UMI bus can be used to connect a charger to charge an electronic device (such as the smart TV, set-top box, or audio and video playback device described above) and can also be used to transmit data between the electronic device and peripheral devices. It can also be used to connect headphones to play audio through the headphones. This interface can also be used to connect other electronic devices, such as augmented reality devices. When the UMI bus is used to implement data communication between devices, it can support both uncompressed and compressed video transmission, as well as various advanced features such as Quick Video Transport (QVT), Auto Low Latency Mode (ALLM), and Dynamic Frame Rate Refresh (DFR). Furthermore, the UMI bus can support LPCM audio and video formats defined by IEC 60958, as well as various HDR protocols, such as HDR Vivid (HDR Vivid) specified in T / UWA005.1-2022. The UMI also supports encryption control and protection for data transmission, such as audio and video. In some optional implementations, the bus 125 can also refer to other types of buses that can implement the functions supported by the aforementioned UMI bus.

[0047] Server 130 can be an application server or an authentication and authorization server. Server 130 can provide video services, game services, messaging services, music services, authentication and authorization services, and the like. In one example, the functions of multiple services can be integrated on server 130. For example, a game service and a music service can be deployed on server 130. In another example, the functions of some services can be integrated on server 130. For example, server 130 can deploy some game services and some video services. Server 130 can also utilize virtualization technology to provide multiple virtual machines, which provide various services. The embodiments of this application do not limit the deployment of the server. Network device 131 is connected to server 130 via wireless or wired connections. Figure 1 is merely a schematic diagram; the network may also include other devices, not shown in Figure 1. It will be understood that the aforementioned audio and video is a general term. Audio and video include multiple video frames, each of which corresponds to a group of packets carrying the audio and video data to be parsed and played. Figure 1 is merely a schematic diagram; the video transmission system may also include other devices, not shown in Figure 1. The embodiments of the present application do not limit the number and type of each device included in the system.

[0048] Based on the video transmission system shown in FIG1 , FIG2 is a schematic diagram of an audio and video codec system provided by the present application. The audio and video codec system includes a source device 210 and a sink device 220. The source device 210 establishes a communication connection with the sink device 220 via a UMI bus.

[0049] The above-mentioned source device 210 can implement the function of audio and video encoding. As shown in Figure 1, the source device 210 can be a set-top box 110 or a smart TV 120. The source device 210 can also be an audio and video control center with audio and video encoding capabilities. For example, the audio and video control center includes one or more servers.

[0050] The source device 210 may include a data source 211 , a pre-processing module 212 , an audio and video transmission adapter 213 , and a communication interface 214 .

[0051] The data source 211 may include or may be any type of electronic device for collecting audio and video, and / or any type of source audio and video generating device, such as a computer graphics processor for generating computer animation scenes or any type of device for obtaining and / or providing source audio and video, or computer-generated source audio and video. The data source 211 may be any type of memory or storage for storing the above-mentioned source audio and video. The above-mentioned source audio and video may include multiple audio and video streams or images obtained by multiple audio and video acquisition devices (such as cameras), such as ultra high definition (UHD) video, high definition (HD) video, 4K video, etc.

[0052] The pre-processing module 212 is configured to receive source audio and video and pre-process the source audio and video to obtain audio and video or multiple frames of images. For example, the pre-processing performed by the pre-processing module 212 may include color format conversion (e.g., from RGB to YCbCr), octree structuring, audio and video splicing, audio track merging and deletion, or channel number adjustment.

[0053] The audio and video transmission adapter 213 is also referred to as a source adapter in this application, and is used to receive audio and video or images, and encode the audio and video, images or images to obtain coded data. In some optional situations, the code stream (coded data) obtained by encoding can also be called a bit stream. If the coded data is obtained by encoding audio and video data, then the bit stream refers to the audio and video stream.

[0054] The communication interface 214 in the source device 210 may be configured to receive encoded data (e.g., a video stream or an audio / video stream) and send the encoded data (or a version of the encoded data after any other processing) to another device such as the sink device 220 or any other device via a UMI bus for storage, display, playback, or image reconstruction.

[0055] Optionally, the source device 210 includes a bitstream buffer, which is used to store bitstreams corresponding to one or more coding units.

[0056] The sink device 220 can implement the audio and video decoding function. As shown in FIG1 , the sink device 220 can be any one of the smart TV 120 or the audio and video playback device shown in FIG1 .

[0057] The sink device 220 may include an audio and video playback unit 221 , a post-processing module 222 , an audio and video receiving adapter 223 , and a communication interface 224 .

[0058] The communication interface 224 in the sink device 220 is configured to receive the encoded data (or a version of the encoded data after any other processing has been performed on the encoded data) from the source device 210 or any other source device such as a storage device.

[0059] Communication interface 214 and communication interface 224 may be used to communicate via a direct communication link between source device 210 and sink device 220, such as a direct wired connection, as shown in the UMI bus in FIG2 . For details about the UMI bus, please refer to the description of FIG1 , and will not be repeated here.

[0060] The communication interface 224 corresponds to the communication interface 214 and can be used, for example, to receive transmission data and process the transmission data using any type of corresponding transmission decoding or processing and / or decapsulation to obtain encoded data (such as a video stream or an audio and video stream).

[0061] Both the communication interface 224 and the communication interface 214 may be configured as a unidirectional communication interface, as indicated by the arrow pointing from the source device 210 to the corresponding UMI bus of the sink device 220 in FIG2 , or a bidirectional communication interface, and may be used to send and receive messages, etc., to establish a connection, confirm and exchange any other information related to the communication link or data transmission, such as encoded compressed data transmission, etc.

[0062] The audio and video receiving adapter 223 is also referred to as a sink adapter in this application, and is used to receive encoded data and decode the encoded data to obtain decoded data (video or audio and video, etc.).

[0063] The post-processing module 222 is used to post-process the decoded data to obtain post-processed data (such as an image to be displayed or audio and video to be played). The post-processing performed by the post-processing module 222 may include, for example, color format conversion (such as from YCbCr to RGB), octree reconstruction, audio and video splitting and fusion, or any other processing for generating data for output by the audio and video playback unit 221.

[0064] The audio and video playback unit 221 is used to receive post-processed data for display or playback to a user or viewer. The audio and video playback unit 221 can be or include any type of display for representing the reconstructed image, such as an integrated or external display screen or display. For example, the display screen may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS) display, a digital light processor (DLP), or any other type of display screen. The audio and video playback unit 221 can also include one or more audio and video playback modules, each of which can refer to a speaker, a smart speaker, an amplifier, or the like.

[0065] As an optional implementation, the source device 210 and the sink device 220 can transmit encoded data through a data forwarding device. For example, the data forwarding device can be a router or a switch. It is worth noting that the data forwarding device needs to support the UMI interface. The UMI interface of the two devices consists of a main link (ML), a sideban link (SL), a power bus link (PL), and a cable information link (CL). This application does not involve other links other than the main link. Only the main link will be described below. The main link is mainly used to transmit audio and video service streams (such as ultra-high-definition audio and video signals) and high-speed data (such as USB3 data). The main link includes multiple pairs of differential lines, each pair of differential lines forming a differential channel (lane). The virtual channel between the audio and video sending adapter 213 and the audio and video receiving adapter 223 can use the differential channel to transmit audio and video service streams.

[0066] The implementation of the virtual path establishment method provided by the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0067] Here, the virtual path establishment method of an embodiment of the present application is described using the source device 210 and the sink device 220 shown in FIG2 as an example. FIG3 is a flow chart of a virtual path establishment method provided by the present application. In this embodiment, the initiating device 31 is used to implement the functions of the source device 210, and the target device 32 is used to implement the functions of the sink device 220. In this embodiment, the initiating device 31 may also be referred to as a source device, an audio and video transmitter, or an audio and video transmitter, and the target device 32 may also be referred to as a sink device, a display device, an audio and video receiver, or an audio and video player. In this embodiment, the initiating device 31 and the target device 32 are connected via one or more intermediate devices 33 (only one intermediate device 33 is shown in FIG3 , but the number of intermediate devices is not limited). The initiating device 31, the intermediate device 33, and the target device 32 are connected via a bus 34, which may be a UMI bus.

[0068] In a first possible application scenario, the initiating device 31 may be the set-top box 110 in Figure 1, and the target device 32 may be the smart TV 120 in Figure 1. For example, the set-top box pushes audio and video data to the smart TV.

[0069] In a second possible application scenario, the initiating device 31 may be the set-top box 110 in FIG1 , and the target device 32 may be any audio and video playback device in FIG1 , such as any one of the audio and video playback devices 121 to 124. For example, the set-top box pushes audio and video data to the audio and video playback device.

[0070] In a third possible application scenario, the initiating device 31 may be the smart TV 120 in FIG1 , and the target device 32 may be any audio and video playback device in FIG1 , such as any one of the audio and video playback devices 121 to 124. For example, the smart TV pushes audio and video data to the audio and video playback device.

[0071] The above three possible application scenarios are merely examples provided in this embodiment and should not be construed as limiting the present application. In other possible examples, the initiating device 31 may be any of the audio and video playback devices in FIG1 (e.g., audio and video playback device 121), and the target device 32 may be another audio and video playback device different from the aforementioned audio and video playback device (e.g., audio and video playback device 122).

[0072] Referring to FIG. 3 , the virtual path establishment method provided in this embodiment includes the following steps 301 to 321 .

[0073] Step 301: The initiator device 31 sends an adapter list request.

[0074] When the initiator device 31 needs to obtain the adapter list information before establishing a virtual path, it generates an adapter list request and sends the adapter list request according to the device addressing information.

[0075] As a possible implementation, the message structure of the adapter list request is shown in Table 1, and the message field description of the adapter list request is shown in Table 2.

[0076] Table 1

[0077] Table 2

[0078] Here, Command=6 means that the device control command number corresponding to the adapter list request is 6.

[0079] Optionally, the message structure of the message header is as shown in Table 3.

[0080] Table 3

[0081] In the message header of the adapter list request, the Type value is 27, indicating that the message type is an adapter list request. The Response value is 0, indicating that it is a request message.

[0082] Optionally, the message structure of the general field is as shown in Table 4.

[0083] Table 4

[0084] Optionally, the addressing information requested by the bandwidth device is device address addressing information, or device address addressing, or addressing information. Each device in the UMI network has a unique 8-byte (64-bit) identifier that can uniquely identify a device, so this identifier is called a device address. The device address consists of a vendor identifier (Vendor ID) and a device identifier (Device ID). The Vendor ID consists of 16 bits and must be registered with the UMI organization. The Device ID is assigned by the device manufacturer when manufacturing the UMI device and must be guaranteed to be unique.

[0085] The message structure of the device address addressing information is shown in Table 5.

[0086] Table 5

[0087] The destination address is the device address of the device that the message is intended to receive, and the source address is the device address of the device that sends the message.

[0088] In the present application, the addressing method based on the above-mentioned device address addressing information includes: when a device receives a message carrying device address addressing information, if the target address of the message is its own device address, then the device is the target device of the message. Otherwise, it is necessary to query the address table based on the target address of the message and process it according to the query result: (1) If the target address of the message is in the address table, and the corresponding port is different from the port receiving the message, then the message is forwarded from the corresponding port. The address table includes the correspondence between the device address and the forwarding port. (2) If the target address of the message is in the address table, but the corresponding port is the same as the port receiving the message. It is considered that the target address of the message is in the device connected to the port, so the target device should have received the message, and the device should discard the message. (3) If the target address of the message is not found in the address table, then the device does not know whether there is a port connected to the target device, and the device should flood the message from all ports except the receiving port.

[0089] Step 302: The intermediate device 33 receives an adapter list request.

[0090] Step 303: The intermediate device 33 sends an adapter list request.

[0091] In the above steps 302 and 303 , the manner in which the intermediate device 33 forwards the adapter list request can be found in the addressing manner corresponding to the above device address information, which will not be described in detail here.

[0092] Step 304: The target device 32 receives the adapter list request.

[0093] Step 305: The target device 32 sends an adapter list response.

[0094] After receiving the adapter list request, the target device 32 generates an adapter list response, fills the information of all adapters in the online state (including the identifiers of the adapters) of the device into the adapter list response, and then sends the adapter list response according to the address table.

[0095] In this application, adapter list request and adapter list response can be collectively referred to as adapter list message, which is used to obtain the adapter list of a specific device in the UMI network, and is often used to obtain detailed information of the device adapter when establishing a virtual path.

[0096] As a possible implementation, the message structure of the adapter list response is shown in Table 6.

[0097] Table 6

[0098] The multiple AdapterIDs are identifiers of multiple adapters carried in the adapter list response.

[0099] Optionally, descriptions of the fields in the message structure of the adapter list response are shown in Table 7.

[0100] Table 7

[0101] Step 306: The intermediate device 33 receives the adapter list response.

[0102] Step 307: The intermediate device 33 sends an adapter list response.

[0103] In the above steps 306 and 307 , the manner in which the intermediate device 33 forwards the adapter list response can be found in the addressing manner corresponding to the above device address addressing information, which will not be described in detail here.

[0104] Step 308 : The initiator device 31 receives the adapter list response.

[0105] After receiving the adapter list response, the initiator device 31 may determine the sink adapter from the plurality of adapters, and determine a virtual path from the source adapter to the sink adapter from the plurality of paths included in the topology structure between the source adapter and the sink adapter.

[0106] As one possible implementation, the initiating device 31 selects a virtual path based on network quality indicators of the transmission path corresponding to the virtual path. Network quality indicators may include path length, latency, etc. For example, the initiating device 31 selects based on path length, preferably the shortest path to establish the virtual path.

[0107] Optionally, the virtual path includes multiple virtual channels cascaded between the source adapter of the initiator device 31 and the sink adapter among the multiple adapters of the target device 32. For example, the virtual path includes a first virtual channel between the initiator device 31 and the intermediate device 33, and a second virtual channel between the intermediate device 33 and the target device 32.

[0108] Step 309: The initiating device 31 generates a bandwidth application request.

[0109] As a possible implementation, the message structure of the bandwidth application request is shown in Table 8.

[0110] Table 8

[0111] The initiating device 31 assigns the OutStreamBW in the message body to the outgoing bandwidth value required by the service, assigns the In StreamBW to 0xFFFFFFFF, and updates the source device address, source adapter ID, target device address, target adapter ID, priority (service priority) and flow control mechanism information to the message body.

[0112] Optionally, descriptions of the fields of the bandwidth application request are shown in Table 9.

[0113] Table 9

[0114] Step 310: The initiating device 31 allocates bandwidth to the first virtual channel of the virtual path according to the outgoing bandwidth value required by the audio and video service flow.

[0115] As a possible implementation, step 310 may include the following sub-steps S1 to S7.

[0116] S1. The initiating device 31 determines the output port number OutPortID of the bandwidth application request and checks whether the output port number exists in the router forwarding table corresponding to the audio and video service flow according to the routing table.

[0117] When the forwarding list addressing information of the initiating device 31 does not include the port number of the output port from which the initiating device 31 sends the bandwidth request, the initiating device 31 determines the port number of the output port from which the bandwidth request is sent, i.e., the first outbound port number, based on the device's internal addressing. For example, the initiating device 31 maintains service flow information, which includes the input port number, source device address, source adapter identifier, output port number, target device address, target adapter identifier, bandwidth value, and priority. The initiating device 31 uses the source device address, target device address, etc. of the bandwidth request to query the service flow information for the output port number, i.e., the first outbound port number. Furthermore, the first outbound port number is also the output port of the virtual path of the audio and video service flow on this device.

[0118] Alternatively, a router forwarding table is used to indicate the forwarding rule of message between different virtual channels in this device, and a router forwarding table comprises one or more forwarding entries (or router forwarding table items). As shown in Table 10, the router forwarding table is composed of inflow information and outflow information, and a router forwarding table item can be represented by {inflow information|outflow information}. Wherein the inflow information is composed of the information of a single inflow node, and the outflow information is composed of the information of one or more outflow nodes. The information of the inflow node mainly describes the port and the virtual channel (such as port number and virtual channel mark) corresponding to the inflow, and the information of the outflow node mainly describes the relevant information such as the target port and the target virtual channel (such as port number and virtual channel mark) of the forwarding of the message. For example, the first router forwarding table entry in Table 10 can be represented as {[0,4]|[2,4]}. The inbound flow information contains only one inbound node, [0,4], indicating that the flow receives packets from the receive buffer of adapter 4 on this router. The outbound flow information contains one outbound node, [2,4], indicating that the flow packet needs to be forwarded to virtual channel 4 on port 2. The virtual channel ID in the inbound node information of the inbound flow information is equivalent to the adapter ID (AdapterID). An inbound node can also be called an inbound node, and an outbound node can also be called an outbound node.

[0119] Table 10

[0120] In a multicast scenario, if one or more devices on the same destination port receive the stream corresponding to the same virtual channel, a multicast count (ReceiverCount) is added to the outbound node to facilitate router forwarding table management. The outbound node is represented by a triplet of [port number (Port), virtual channel identifier (ShuttleID), multicast count (ReceiverCount)]. The multicast count indicates the number of destination devices receiving the audio and video service stream on the output port (also called the outbound port).

[0121] S2. If the output port number is in the router forwarding table, the initiating device 31 uses the existing virtual channel identifier and increases the corresponding multicast count by one, and then executes S5.

[0122] S3. If the output port number is not in the router forwarding table, the initiating device 31 allocates a new virtual channel identifier to the output port and sets the corresponding multicast count value to 1.

[0123] S4: Determine whether the currently available bandwidth meets the requirement. If so, the initiating device 31 allocates bandwidth as required. Otherwise, the initiating device 31 starts the bandwidth management process.

[0124] The initiating device 31 allocates bandwidth as required by comparing the current actual available bandwidth of the output port with the outgoing bandwidth value required by the audio and video service flow, that is, the first outgoing bandwidth value. If the current actual available bandwidth is larger, the requirement is met.

[0125] S5. The initiating device 31 records the service flow information and updates the router forwarding table information.

[0126] S6. The initiating device 31 compares the actually allocated bandwidth value with the OutStreamBW in the bandwidth application request. If the actually allocated bandwidth value is smaller than the OutStreamBW value, OutStreamBW is assigned the actually allocated bandwidth value; otherwise, the assigned value of OutStreamBW remains unchanged.

[0127] The OutStreamBW value is called the first outbound bandwidth value.

[0128] S7. The initiating device 31 configures the transport layer.

[0129] The initiating device 31 sends the information shown in Table 11 to the transport layer.

[0130] Table 11

[0131] In a possible implementation of the present application, the serial numbers of the above steps do not limit the execution steps of the steps. For example, the above step 310 can be executed after step 309 or before step 309, that is, when the initiating device 31 does not generate a bandwidth application request, it uses the data related to the bandwidth application request to execute step 310.

[0132] Step 311: The initiating device 31 sends a bandwidth application request.

[0133] The bandwidth application request includes the first outbound bandwidth value, that is, the value OutStreamBW in the bandwidth application request is set to the first outbound bandwidth value.

[0134] As a possible implementation manner, the initiating device 31 forwards the bandwidth application request according to the forwarding list addressing information of the bandwidth application request.

[0135] Optionally, the addressing information of the bandwidth application request is forwarding list addressing information, or referred to as forwarding list addressing, or addressing information. The message structure of the forwarding list addressing information is shown in Table 12.

[0136] Table 12

[0137] The Forwarding List Level of the bandwidth application request is used to indicate the current level, and the corresponding port number in the port number of the level is the port number used by the current device to send the bandwidth application request.

[0138] Step 312: The intermediate device 33 receives the bandwidth application request.

[0139] When the intermediate device 33 receives the bandwidth application request, it records the service flow information.

[0140] Step 313: The intermediate device 33 allocates bandwidth to the second virtual channel according to the first outbound bandwidth value in response to the bandwidth application request.

[0141] If the intermediate device 33 receives a bandwidth application request with a repeated message tag from the same source device and the bandwidth value has been allocated, it directly forwards the request without processing.

[0142] Please refer to step 310 for the processing method of allocating bandwidth by the intermediate device 33, which will not be described in detail here.

[0143] Step 314: The intermediate device 33 sends a bandwidth application request.

[0144] The bandwidth request includes a fourth outbound bandwidth value. The fourth outbound bandwidth value is used to indicate the value assigned to OutStreamBW by the intermediate device 33 based on the current actual available bandwidth. The fourth outbound bandwidth value may be the same as or different from the first outbound bandwidth value.

[0145] As a possible implementation manner, the intermediate device 33 forwards the bandwidth application request according to the forwarding list addressing information of the bandwidth application request.

[0146] Optionally, the intermediate device 33 updates the forwarding list addressing information, and sends a bandwidth application request according to the updated forwarding list addressing information.

[0147] Step 315: The target device 32 receives the bandwidth application request.

[0148] The bandwidth application request includes a fifth outbound bandwidth value. The fifth outbound bandwidth value is used to indicate the outbound bandwidth value to be applied for obtained by the upper-level device in the virtual path based on the value assigned to OutStreamBW.

[0149] Step 316: The target device 33 sends a bandwidth request response.

[0150] After receiving the bandwidth application request, the target device 32 records the service flow information and determines that the device is the target device of the bandwidth application request based on the forwarding list addressing information. It then uses the input port of the bandwidth application request as the output port of the bandwidth application response and sends the bandwidth application response according to the forwarding list addressing method.

[0151] As a possible implementation, the message structure of the bandwidth request response is shown in Table 13.

[0152] Table 13

[0153] Optionally, descriptions of the fields in the bandwidth request response are shown in Table 14.

[0154] Table 14

[0155] Step 317: The intermediate device 33 receives the bandwidth request response.

[0156] Step 318: The intermediate device 33 allocates bandwidth to the second virtual channel in response to the bandwidth request response.

[0157] As a possible implementation, the way in which the intermediate device 33 allocates bandwidth in response to the bandwidth application response includes the following sub-steps S1-S4.

[0158] S1. The intermediate device 33 identifies the corresponding virtual channel according to the Channel ShuttleID and tag in the bandwidth request response.

[0159] The Channel ShuttleID and tag in the bandwidth request response are used to indicate the second virtual channel of the virtual path.

[0160] S2. The intermediate device 33 determines whether the multicast count value of the flow node is greater than 1. If so, execute S4; otherwise, execute S3.

[0161] S3. The intermediate device 33 compares the bandwidth value (OutStreamBW) in the bandwidth request response, i.e., the sixth outbound bandwidth value, with the allocated bandwidth value. If the allocated bandwidth value is larger, the excess bandwidth value is released, the weight and flow control cache are recalculated, and the updated weight and flow control cache information is sent to the transport layer.

[0162] Optionally, the allocated bandwidth value refers to the outbound bandwidth value allocated to the second virtual channel by the intermediate device 33 according to the bandwidth application request. The excess bandwidth value refers to the difference between the allocated bandwidth value and the bandwidth value in the bandwidth application response.

[0163] S4. The intermediate device 33 updates the record of the allocated bandwidth value.

[0164] In a possible embodiment of the present application, if the intermediate device 33 receives a bandwidth request response with a repeated tag from the same device and has already been allocated a bandwidth value, it directly forwards the request without processing.

[0165] Step 319: The intermediate device 33 sends a bandwidth request response.

[0166] The tag, Channel ShuttleID, etc. in the bandwidth request response sent by the intermediate device 33 are updated values. For example, the updated Channel ShuttleID in the bandwidth request response is assigned a value of the first virtual channel identifier.

[0167] Step 320: The initiating device 31 receives the bandwidth request response.

[0168] Step 321 : The initiating device 31 allocates bandwidth to the first virtual channel in response to the bandwidth request response.

[0169] As a possible implementation, the method in which the initiating device 31 allocates bandwidth in response to the bandwidth application response includes the following sub-steps S1-S4.

[0170] S1. The initiating device 31 identifies the corresponding virtual channel according to the Channel ShuttleID and tag in the bandwidth request response.

[0171] The Channel ShuttleID and tag in the bandwidth request response are used to indicate the first virtual channel of the virtual path.

[0172] S2. The initiating device 31 determines whether the multicast count value of the flow node is greater than 1. If so, execute S5; otherwise, execute S3.

[0173] S3. The initiating device 31 compares the bandwidth value (OutStreamBW) in the bandwidth request response, i.e., the sixth outbound bandwidth value, with the allocated bandwidth value. If the allocated bandwidth value is larger, the excess bandwidth value is released, the weight and flow control cache are recalculated, and the updated weight and flow control cache information is sent to the transport layer.

[0174] Optionally, the allocated bandwidth value refers to the outbound bandwidth value allocated to the first virtual channel by the initiating device 31 according to the bandwidth application request. The excess bandwidth value refers to the difference between the allocated bandwidth value and the bandwidth value in the bandwidth application response.

[0175] S4. The initiating device 31 updates the record of the allocated bandwidth value.

[0176] In a possible embodiment of the present application, after the initiating device 31 executes step 321, it may also decide the next operation based on the bandwidth request response. The decision strategy corresponding to the decision may be as follows:

[0177] 1) If the bandwidth value in the bandwidth application response (i.e., OutStreamBW is assigned, the sixth outbound bandwidth value) is consistent with the service application value (i.e., OutStreamBW in the bandwidth application request is assigned, the first outbound bandwidth value), audio and video service stream transmission can be started.

[0178] 2) If the bandwidth value in the bandwidth request response is less than the service request value and the service is acceptable, for example, the service requirements can be continued to be met by adjusting the resolution, color space, bit depth, etc., then service transmission can be started.

[0179] 3) If the bandwidth value in the response message is smaller than the service application value, but the service is unacceptable, the bandwidth release process is initiated, the next shortest path is selected, and the bandwidth application process is restarted.

[0180] 4) If no path in the UMI network can meet the bandwidth requirements of the current audio and video service flow, an exception is reported.

[0181] In a possible embodiment of the present application, when the initiating device 31 receives a bandwidth response, the ErrCode is non-zero, and the bandwidth request can be resent based on the original tag. When the initiating device 31 receives a bandwidth response, the ErrCode is non-zero, and the bandwidth release process can be called first, and then the bandwidth request can be sent using a new tag.

[0182] In a possible embodiment of the present application, the initiating device 31 and the target device 32 complete the establishment of a virtual path through a virtual path establishment method. FIG4 is a schematic diagram of a virtual path provided by the present application.

[0183] The communication link between communication interface 214 and communication interface 224, through multiple communication links between intermediate devices, forms the communication link between source device 210 and sink device 220. For example, taking source device 210 as device A, sink device 220 as device D, and intermediate devices including devices B and C, a virtual path is formed by cascading multiple virtual channels between adapter 4 of device A and adapter 5 of device D. This virtual path includes three virtual channels: a virtual channel with a virtual channel ID (shuttle ID) of 7 from device A to device B, a virtual channel with a virtual channel ID of 4 from device B to device C, and a virtual channel with a virtual channel ID of 13 between devices C and D. Port 2 of each device is a main downstream port (MDP), and port 1 of each device is a main upstream port (MUP). The downstream and upstream ports are connected via connectors and cables.

[0184] A virtual path can be represented by a four-tuple. For example, the virtual path in FIG4 can be represented by (device A, adapter 4, device D, adapter 5).

[0185] The UMI bus supports multicast functionality. Data, audio and video streams, etc. generated by the same adapter can be transmitted to multiple adapters. As shown in Figure 5, the stream sent by adapter 4 of device A is received by adapter 7 of device B, adapter 6 of device C, and adapter 5 of device D at the same time. The corresponding three virtual paths are (device A, adapter 4, device B, adapter 7), (device A, adapter 4, device C, adapter 6), and (device A, adapter 4, device D, adapter 5).

[0186] In a possible embodiment of the present application, when an audio or video service requires both outbound and inbound bandwidth, such as a USB tunnel service, the intermediate device must allocate both outbound bandwidth to the output port and the input port, corresponding to the InStreamBW value in the bandwidth request. The target device must also allocate the outbound bandwidth value to the input port, corresponding to the InStreamBW value in the bandwidth request. The bandwidth allocation rules and process are the same as above.

[0187] Based on steps 301 to 321 of the above-mentioned virtual path establishment method, the initiating device can determine the multiple adapters contained in the target device according to the adapter list response, thereby identifying the virtual path from the initiating device to the target device. The devices that the virtual path passes through can be selected according to demand, that is, the transmission path corresponding to the virtual path can be flexibly determined, thereby avoiding the transmission path from relying on fixed physical connections. At the same time, a bandwidth application request is used to instruct each device on the virtual path to complete the bandwidth allocation of the corresponding virtual channel, thereby realizing the hierarchical creation of designated virtual channels, and transmitting audio and video service streams between two ports through the virtual channel, thereby decoupling the transmission of audio and video service streams from the physical connection method between the ports, further improving the flexibility of changing the transmission path of audio and video service streams.

[0188] The overall process of the virtual path establishment method has been described above with reference to Figures 3-5 . In complex UMI networks, to avoid multiple devices competing for the bandwidth of a virtual channel corresponding to an adapter, which could result in the initiating device sending a bandwidth request but failing to apply for it, in embodiments of the present application, the virtual path establishment method may further include an adapter binding operation. The following describes in detail the implementation of the adapter binding operation provided in embodiments of the present application with reference to the accompanying figures.

[0189] Adapter bonding is the process of pairing two adapters to establish a virtual path between them. The two adapters that need to be bonded follow the following rules: audio transmitter adapter and audio / video receiver adapter, USB3 downstream adapter and USB3 upstream adapter.

[0190] The bandwidth application operation, i.e., the generation and sending of the bandwidth application request, must be initiated by the source device of the audio and video service flow or the USB host device. Therefore, when the sink device of the target device or the USB3 upstream device establishes a virtual path, an adapter binding request must be initiated to the corresponding source device so that the source device of the audio and video service flow or the USB host device receives the adapter binding request and then initiates a bandwidth application request to establish the corresponding virtual path.

[0191] In another scenario, the source device of the audio and video service stream or the USB host device can also actively initiate an adapter binding operation to lock the bandwidth resources of the target device.

[0192] Here, the adapter binding operation of an embodiment of the present application is performed by the source device 210 and the host device 220 shown in Figure 2 as an example for explanation, and Figure 6 is a flow chart of an adapter binding operation provided by the present application. Among them, when the adapter binding operation is initiated by the source device 210 after step 308 shown in Figure 3, the initiating device 41 is used to implement the function of the source device 210. When the adapter binding operation is initiated by the host device 210 after step 308 shown in Figure 3, the initiating device 41 is used to implement the function of the host device 220. In this embodiment, the initiating device 41 can also be referred to as a source device, an audio and video sending end or an audio and video sending device, as well as a host device, a display device, an audio and video receiving device or an audio and video playing device, and the target device 42 can also be referred to as a host device, a display device, an audio and video receiving device or an audio and video playing device, as well as a source device, an audio and video sending end or an audio and video sending device. In this embodiment, the initiating device 41 and the target device 42 are connected through one or more intermediate devices 43 (only one intermediate device 43 is shown in Figure 6 but the number of intermediate devices is not limited). The initiator device 41 , the intermediate device 43 , and the target device 42 are connected via a bus 44 , which may be a UMI bus.

[0193] Referring to FIG. 6 , the virtual path establishment method provided in this embodiment includes the following steps 401 to 408 .

[0194] Step 401: The initiator device 41 sends an adapter binding request.

[0195] As a possible implementation method, the message structure of the adapter binding request is shown in Table 15.

[0196] Table 15

[0197] Optionally, descriptions of the message fields of the adapter binding request are shown in Table 16.

[0198] Table 16

[0199] As a possible implementation manner, the initiating device 41 sends the adapter binding request based on a device address addressing mode.

[0200] Step 402: The intermediate device 43 receives an adapter binding request.

[0201] Step 403: The intermediate device 43 sends an adapter binding request.

[0202] Step 404: The target device 42 receives the adapter binding request.

[0203] Step 405: The target device 42 sends an adapter binding response.

[0204] As a possible implementation, in addition to sending the adapter binding response, the target device 42 also synchronously records the adapter binding information, ie, the device address of the bound initiating device and the identifier of the source adapter.

[0205] In a possible embodiment of the present application, if the adapter is currently in a bound state, the ErrCode value is assigned to 0x5 (request rejected), and an adapter binding response message is directly returned.

[0206] Step 406: The intermediate device 43 receives the adapter binding response.

[0207] Step 407: The intermediate device 43 sends an adapter binding response.

[0208] Step 408: The initiator device 41 receives the adapter binding response.

[0209] As a possible implementation manner, the initiating device 41 synchronously records the adapter binding information, that is, the device address of the bound target device and the identifier of the sink adapter.

[0210] The overall process of the virtual path establishment method has been described above with reference to Figures 3-6 . In complex UMI networks, to avoid invalid bandwidth requests resulting from sending a bandwidth request request and specifying a virtual path whose available bandwidth cannot meet the bandwidth requirements of the audio and video service flow, the virtual path establishment method, in embodiments of the present application, may further include a bandwidth query operation. The implementation of the bandwidth query operation provided in embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0211] The bandwidth query operation is used to query the minimum available bandwidth for each port on the virtual path between two devices in a UMI network. The bandwidth query operation, which generates and sends a bandwidth request, must be initiated by the device with the audio and video transmission adapter that sends audio and video service flows, or by the device with the USB tunnel adapter connected to the USB host in a USB service flow.

[0212] Here, the bandwidth query operation performed by the source device 210 and the sink device 220 shown in FIG. 2 is used as an example to illustrate an embodiment of the present application. FIG. 7 is a flow chart illustrating a bandwidth query operation provided by the present application. In this embodiment, the initiating device 51 is used to implement the functions of the source device 210, and the target device 52 is used to implement the functions of the sink device 220. In this embodiment, the initiating device 51 may also be referred to as a source device, an audio and video transmitter, or an audio and video transmitter, and the target device 52 may also be referred to as a sink device, a display device, an audio and video receiver, or an audio and video player. In this embodiment, the initiating device 51 and the target device 52 are connected via one or more intermediate devices 53 (only one intermediate device 53 is shown in FIG. 7 , but the number of intermediate devices is not limited). The initiating device 51, the intermediate device 53, and the target device 52 are connected via a bus 54, which may be a UMI bus.

[0213] Referring to FIG. 7 , the bandwidth query operation provided in this embodiment includes the following steps 501 to 508 .

[0214] Step 501: The initiating device 51 sends a bandwidth query request.

[0215] The bandwidth query request includes a second outbound bandwidth value, where the second outbound bandwidth value is used to indicate an available outbound bandwidth value of an output port of the initiating device 51 .

[0216] As a possible implementation, the message structure of the bandwidth query request is shown in Table 17.

[0217] Table 17

[0218] Optionally, the message fields of the bandwidth query request are described as shown in Table 18.

[0219] Table 18

[0220] As a possible implementation manner, the initiating device 51 sends a bandwidth query request in a forwarding list addressing manner, and OutStreamBW in the sent bandwidth query request is assigned the second outbound bandwidth value.

[0221] Step 502: The intermediate device 53 receives a bandwidth query request.

[0222] As a possible implementation, after receiving the bandwidth query request, the intermediate device 53 executes the following steps S1-S3 to update the bandwidth query request.

[0223] S1. The intermediate device 53 compares the available outbound bandwidth value of the input port with the InStremBW in the message, and updates the smaller value to the InStremBW field in the message body.

[0224] The input port refers to the port that receives the bandwidth query request by the intermediate device 53. Taking the smaller value to update the InStremBW field in the message body means assigning the smaller value to the InStremBW field of the bandwidth query request.

[0225] S2. The intermediate device 53 extracts the corresponding output port number from the forwarding list addressing information.

[0226] The intermediate device 53 extracts the corresponding output port number OutPortID from the forwarding list addressing information according to the FL Level value, where FL Level represents the level of the current device with the source device as the origin.

[0227] S3. The intermediate device 53 compares the available outbound bandwidth value of the output port with the OutStremBW in the message, and updates the smaller value to the OutStremBW field of the message body.

[0228] The output port corresponds to the output port number OutPortID.

[0229] Step 503: The intermediate device 53 sends a bandwidth query request.

[0230] As a possible implementation manner, the intermediate device 53 forwards the bandwidth query request according to a forwarding list addressing manner.

[0231] Step 504: The target device 52 receives the bandwidth query request.

[0232] As a possible implementation manner, after receiving the bandwidth query request, the target device 52 executes the following steps S1-S3.

[0233] S1. The target device 52 compares the available outbound bandwidth value of the input port with the InStremBW in the message, takes the smaller value and updates it to the InStremBW field in the message body (bandwidth query response).

[0234] S2. The target device 52 determines that the device is the target device according to FL Level=FL Length+1.

[0235] S3. The target device 52 uses the bandwidth query request as an output port of the bandwidth query response.

[0236] In a possible embodiment of the present application, when the target device 52 receives a bandwidth query request message, if the port is in the link configuration or link training state, it should set ErrCode to 0x1 (Busy) and directly return a bandwidth query response. At the same time, when an abnormal error response is returned, it is necessary to set the PortID in the forwarding list corresponding to the FL Level to 0.

[0237] Step 505: The target device 52 sends a bandwidth query response.

[0238] The bandwidth query response includes a third outbound bandwidth value, which is used to indicate the minimum available outbound bandwidth value of each port through which the virtual path passes. That is, OutStremBW in the bandwidth query response is assigned the third outbound bandwidth value.

[0239] As a possible implementation, the message structure of the bandwidth query response is shown in Table 19.

[0240] Table 19

[0241] Optionally, please refer to Table 20 for a description of each field in the bandwidth query response.

[0242] Table 20

[0243] Step 506: The intermediate device 53 receives the bandwidth query response.

[0244] Step 507: The intermediate device 53 sends a bandwidth query response.

[0245] Step 508: The initiating device 51 receives the bandwidth query response.

[0246] The bandwidth query response includes the third outbound bandwidth value.

[0247] In a possible embodiment of the present application, after receiving the bandwidth query response, the initiating device 51 makes a comprehensive decision based on the bandwidth query response. The decision strategy may be as follows:

[0248] 1) If the available bandwidth is greater than or equal to the expected bandwidth, the bandwidth application process is initiated and a bandwidth application request is sent. The expected bandwidth is the bandwidth required for the audio and video service flows.

[0249] 2) Available bandwidth < expected bandwidth:

[0250] a) If the service is acceptable, for example, if the resolution, color space, bit depth, or other changes can continue to meet the audio and video service flow requirements, the bandwidth application process is initiated.

[0251] b) If the service is unacceptable, select another path and start the bandwidth query operation again.

[0252] c) If no path in the UMI network can meet the bandwidth requirements of the current audio and video service flow, an exception is reported.

[0253] It is understood that in order to implement the functions in the above embodiments, the source device and the sink device include hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily appreciate that, in combination with the units and method steps of the various examples described in the embodiments disclosed in this application, this application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in hardware or in a manner driven by computer software depends on the specific application scenario and design constraints of the technical solution.

[0254] The above description is in detail of the method for establishing a virtual path provided by the present embodiment in conjunction with FIG. 1 to FIG. 7 . The following description is in conjunction with FIG. 8 , of the apparatus for establishing a virtual path provided by the present embodiment.

[0255] FIG8 is a schematic structural diagram of a virtual path establishment device provided by the present application. The virtual path establishment device 800 can be used to implement the functions of any one of the devices in the above-mentioned method embodiments, and thus can also achieve the beneficial effects possessed by the above-mentioned method embodiments. In this embodiment, the virtual path establishment device 800 can be the set-top box 110, smart TV 120, or any display device as shown in FIG1 , or the source device 210 or sink device 220 as shown in FIG2 , or the multimedia device or display device provided in subsequent embodiments. It should be understood that the virtual path establishment device 800 can also be a module (such as a chip) applied to any of the aforementioned devices.

[0256] As shown in Figure 8, the virtual path establishment device 800 includes a transceiver module 810 and a processing module 820. Transceiver module 810 and processing module 820 can collaborate to implement the various steps in the above-described method embodiment. A more detailed description of transceiver module 810 and processing module 820 can be directly obtained by referring to the relevant description of the device in the method embodiment shown in the above figures, and will not be repeated here.

[0257] When the virtual path establishment device 800 implements any of the virtual path establishment methods shown in the aforementioned figures through software, the virtual path establishment device 800 and its various units may also be software modules. The aforementioned virtual path establishment method is implemented by invoking the software module via a processor. The processor may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0258] It can be understood that the virtual path establishment device shown in FIG8 is only an example provided in this embodiment. The virtual path establishment device may include more or fewer units according to different audio and video service flow transmission processes, and this application does not limit this.

[0259] When the virtual path establishment device 800 is implemented via hardware, the hardware can be implemented via a processor or a chip system. The chip system includes one or more chips, each of which includes an interface circuit and a control circuit. The interface circuit is used to receive data from other devices outside the chip and transmit it to the control circuit, or to send data from the control circuit to other devices outside the chip. The control circuit and interface circuit are used to implement the method of any possible implementation method in the above embodiments through logic circuits or executing code instructions. The beneficial effects can be found in the description of any aspect of the above embodiments and will not be repeated here.

[0260] It is understood that the processor in the embodiments of the present application may be a CPU, or other general-purpose processor, digital signal processor (DSP), ASIC, FPGA or other programmable logic device, transistor logic device, hardware component or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0261] In addition, the virtual path establishment apparatus 800 shown in FIG8 can also be implemented by a multimedia device, as shown in FIG9 , which is a schematic structural diagram of the multimedia device provided by the present application. The multimedia device includes: a processor 910, an external memory interface 920, an internal memory 921, a universal serial bus (USB) interface 930, a UMI interface 931, antenna 1, antenna 2, a mobile communication module 950, a wireless communication module 960, an audio module 970, a speaker 970A, a receiver 970B, a microphone 970C, a sensor module 980, a button 990, an indicator 992, a camera 993, a display screen 994, and subscriber identification module (SIM) card interfaces 1-N 995.

[0262] Among them, the above-mentioned sensor module 980 may include sensors such as pressure sensor, gyroscope sensor, air pressure sensor, magnetic sensor, acceleration sensor, distance sensor, proximity light sensor, fingerprint sensor, temperature sensor, touch sensor, ambient light sensor and bone conduction sensor.

[0263] It is understood that the structure illustrated in this embodiment does not constitute a specific limitation on the multimedia device. In other embodiments, the multimedia device may include more or fewer components than shown, or combine or separate certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0264] The multimedia device shown in FIG. 9 may be any device in FIG. 1 , or a source device or a sink device in subsequent embodiments.

[0265] The processor 910 may include one or more processing units. For example, the processor 910 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0266] The controller can be the nerve center and command center of the multimedia device. The controller can generate operation control signals based on instruction opcodes and timing signals to complete the control of instruction fetching and execution.

[0267] Processor 910 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 910 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 910. If processor 910 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 910 latency, and thus improves system efficiency.

[0268] In some embodiments, the processor 910 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a USB interface, a UMI interface, etc.

[0269] It is understood that the interface connection relationship between the modules illustrated in this embodiment is merely a schematic illustration and does not constitute a structural limitation on the multimedia device. In other embodiments, the multimedia device may also adopt a different interface connection method from the above embodiment, or a combination of multiple interface connection methods.

[0270] The wireless communication function of the multimedia device can be implemented through antenna 1, antenna 2, mobile communication module 950, wireless communication module 960, modem processor, and baseband processor. In some embodiments, antenna 1 of the multimedia device is coupled to mobile communication module 950, and antenna 2 is coupled to wireless communication module 960, so that the multimedia device can communicate with the network and other devices through wireless communication technology.

[0271] The wired communication function of the multimedia device can be implemented through the USB interface 930 or the UMI interface 931. For example, the multimedia device receives or sends video streams and AVP messages through the bus connected to the UMI interface 931.

[0272] The multimedia device implements display functions through a GPU, display screen 994, and an application processor. The GPU is a microprocessor for image processing that connects the display screen 994 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 910 may include one or more GPUs that execute program instructions to generate or modify display information.

[0273] The display screen 994 is used to display images, videos, etc. The display screen 994 includes a display panel.

[0274] The multimedia device can implement a camera function using an ISP, a camera 993, a video codec, a GPU, a display 994, and an application processor. The ISP processes data fed back by the camera 993. The camera 993 is used to capture still images or video. In some embodiments, the multimedia device may include one or N cameras 993, where N is a positive integer greater than one.

[0275] In this embodiment, the above display screen 994, video codec, GPU, display screen 994 and application processor can also be collectively referred to as the display unit of the multimedia device, which is used to process and display the received multimedia data stream (such as video stream).

[0276] The external memory interface 920 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the multimedia device. The external memory card communicates with the processor 910 via the external memory interface 920 to implement data storage functions. For example, files such as music and videos can be stored on the external memory card.

[0277] The internal memory 921 can be used to store computer executable program code, which includes instructions. The processor 910 executes various functional applications and data processing of the multimedia device by running the instructions stored in the internal memory 921. For example, in an embodiment of the present application, the processor 910 can execute instructions stored in the internal memory 921, and the internal memory 921 can include a program storage area and a data storage area.

[0278] The program storage area may store an operating system, at least one application required for a function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the multimedia device (such as audio and video data, a phone book, etc.). In addition, the internal memory 921 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, a flash memory device, or a universal flash storage (UFS).

[0279] The multimedia device can implement audio functions such as music playback and recording through the audio module 970, the speaker 970A, the receiver 970B, the microphone 970C, and the application processor.

[0280] Buttons 990 include a power button, a volume button, and the like. Buttons 990 may be mechanical buttons or touch buttons. Indicator 992 may be an indicator light that can be used to indicate charging status, battery level changes, messages, missed calls, notifications, and the like.

[0281] The method steps in the embodiments of the present application can also be implemented by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and storage medium can be located in an ASIC. In addition, the ASIC can be located in a network device or a terminal device. Of course, the processor and storage medium can also exist as discrete components in a video processing device and a multimedia device.

[0282] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the process or function described in the embodiments of the present application is performed in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program or instruction can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired or wireless means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, a hard disk, or a tape; it can also be an optical medium, such as a digital video disc (DVD); it can also be a semiconductor medium, such as a solid state drive (SSD).

[0283] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for establishing a virtual path, characterized in that, the method includes: sending an adapter list request; receiving an adapter list response; the adapter list response includes the identifiers of multiple adapters included in the target device; allocating bandwidth for a first virtual channel of the virtual path according to the outgoing bandwidth value required by the audio - video service flow; the virtual path includes multiple cascaded virtual channels between the source adapter of the initiating device and the sink adapter among the multiple adapters of the target device, and the virtual channel is used to transmit an audio - video service flow on the link between two ports; sending a bandwidth application request; the bandwidth application request includes a first outgoing bandwidth value, and the bandwidth application request is used to instruct the next - level device in the virtual path to allocate bandwidth for a second virtual channel according to the first outgoing bandwidth value, and the second virtual channel is the next - level virtual channel of the first virtual channel; receiving a bandwidth application response.

2. The method according to claim 1, characterized in that, the method further includes: determining the sink adapter among the multiple adapters; determining the virtual path from the source adapter to the sink adapter among multiple paths included in the topological structure between the source adapter and the sink adapter.

3. The method according to claim 1 or 2, characterized in that, after receiving the adapter list response, the method further includes: sending an adapter binding request; the adapter binding request includes the identifier of the sink adapter; receiving an adapter binding response; the adapter binding response is used to indicate that the sink adapter has been bound.

4. The method according to any one of claims 1 - 3, characterized in that, the method further includes: sending a bandwidth query request; the bandwidth query request includes a second outgoing bandwidth value, and the second outgoing bandwidth value is used to indicate the available outgoing bandwidth value of the output port of the initiating device; receiving a bandwidth query response; the bandwidth query response includes a third outgoing bandwidth value, and the third outgoing bandwidth value is used to indicate the minimum available outgoing bandwidth value of each port through which the virtual path passes.

5. The method according to claim 4, characterized in that, the sending of the bandwidth application request includes: when the third outgoing bandwidth value is greater than or equal to the required outgoing bandwidth value, sending the bandwidth application request.

6. The method according to claim 4, characterized in that, the method further includes: when the third outgoing bandwidth value is less than the required outgoing bandwidth value, switching the virtual path between the source adapter and the sink adapter.

7. The method according to claim 6, characterized in that, the method further includes: when the third outgoing bandwidth value is less than the required outgoing bandwidth value, reducing the specification of the audio - video service flow; the specification includes resolution, color space, and / or bit depth.

8. A method for establishing a virtual path, characterized in that, the method includes: receiving a bandwidth application request; the bandwidth application request includes a first outgoing bandwidth value; Allocating bandwidth for a second virtual channel according to the first outgoing bandwidth value in response to the bandwidth application request; the second virtual channel is the next-level virtual channel of the first virtual channel corresponding to the first virtual channel identifier in the virtual path, and the virtual path includes a plurality of virtual channels cascaded between the source adapter of the initiating device and the destination adapter among the plurality of adapters of the destination device, and the virtual channel is used to transmit an audio-video service flow on the link between two ports; Sending the bandwidth application request; the bandwidth application request includes a fourth outgoing bandwidth value, and the bandwidth application request is used to instruct the next-level device in the virtual path to allocate bandwidth for a third virtual channel according to the fourth outgoing bandwidth value, and the third virtual channel is the next-level virtual channel of the second virtual channel; Receiving a bandwidth application response; Sending the bandwidth application response.

9. A method for establishing a virtual path, characterized in that, the method includes: Receiving an adapter list request; Sending an adapter list response; the adapter list response includes the identifiers of a plurality of adapters included in the destination device; Receiving a bandwidth application request; the bandwidth application request includes a fifth outgoing bandwidth value; Returning a bandwidth application response; the bandwidth application response includes a sixth outgoing bandwidth value, and the sixth outgoing bandwidth value is used to indicate the applied outgoing bandwidth value.

10. The method according to claim 9, characterized in that, the method further includes: Receiving an adapter binding request; the adapter binding request includes the identifier of the destination adapter among the plurality of adapters; Sending an adapter binding response; the adapter binding response is used to indicate that the destination adapter has been bound.

11. The method according to claim 9 or 10, characterized in that, the method further includes: Receiving a bandwidth query request; Sending a bandwidth query response; the bandwidth query response includes a third outgoing bandwidth value, and the third outgoing bandwidth value is used to indicate the minimum available outgoing bandwidth value of each port passed by the virtual path.

12. The method according to claim 9, characterized in that, before receiving the bandwidth application request, the method further includes: Sending an adapter binding request; the adapter binding request includes the identifier of the source adapter; Receiving an adapter binding response; the adapter binding response is used to indicate that the source device sends the bandwidth application request to the destination device.

13. A virtual path establishment device, characterized in that, includes: A transceiver module, configured to send an adapter list request; The transceiver module is further configured to receive an adapter list response; the adapter list response includes the identifiers of a plurality of adapters included in the destination device; A processing module, configured to allocate bandwidth for a first virtual channel of a virtual path according to the outgoing bandwidth value required by an audio-video service flow; The virtual path includes a plurality of virtual channels cascaded between the source adapter of the initiating device and the destination adapter among the plurality of adapters of the destination device, and the virtual channel is used to transmit the audio-video service flow on the link between two ports; The transceiver module is further configured to send a bandwidth application request; the bandwidth application request includes a first outgoing bandwidth value, and the bandwidth application request is used to instruct a downstream device in the virtual path to allocate bandwidth for a second virtual channel according to the first outgoing bandwidth value, where the second virtual channel is a downstream virtual channel of the first virtual channel; The transceiver module is further configured to receive a bandwidth application response.

14. A virtual path establishment device, characterized in that, it includes: a transceiver module, configured to receive a bandwidth application request; the bandwidth application request includes a first outgoing bandwidth value; a processing module, configured to allocate bandwidth for a second virtual channel according to the first outgoing bandwidth value in response to the bandwidth application request; the second virtual channel is a downstream virtual channel of the first virtual channel corresponding to the first virtual channel identifier in the virtual path, and the virtual path includes a plurality of cascaded virtual channels between a source adapter of an initiating device and a destination adapter among a plurality of adapters of a destination device, and the virtual channel is used to transmit an audio and video service flow on a link between two ports; The transceiver module is further configured to send the bandwidth application request; the bandwidth application request includes a fourth outgoing bandwidth value, and the bandwidth application request is used to instruct a downstream device in the virtual path to allocate bandwidth for a third virtual channel according to the fourth outgoing bandwidth value, where the third virtual channel is a downstream virtual channel of the second virtual channel; The transceiver module is further configured to receive a bandwidth application response; The transceiver module is further configured to send the bandwidth application response.

15. A virtual path establishment device, characterized in that, it includes: a transceiver module, configured to receive an adapter list request; The transceiver module is further configured to send an adapter list response; the adapter list response includes identifiers of a plurality of adapters included in a destination device; The transceiver module is further configured to receive a bandwidth application request; the bandwidth application request includes a fifth outgoing bandwidth value; The transceiver module is further configured to return a bandwidth application response; the bandwidth application response includes a sixth outgoing bandwidth value, and the sixth outgoing bandwidth value is used to indicate the applied outgoing bandwidth value.

16. A multimedia device, characterized in that, it includes: a transceiver and a processor; the processor is configured to process an audio and video service flow; the processor is configured to transmit and receive the audio and video service flow; the transceiver and the processor are configured to cooperate to execute the method according to any one of claims 1-7, or the method according to claim 8.

17. A multimedia device, characterized in that, it includes: a transceiver and a processor; the processor is configured to process an audio and video service flow; the processor is configured to transmit and receive the audio and video service flow; the display unit, the memory, the transceiver and the processor are configured to cooperate to execute the method according to any one of claims 9-12.

18. A readable storage medium, characterized in that, The readable storage medium includes a computer program or instructions, which, when run on a computer, cause the computer to perform the operation steps of the method described in any one of claims 1-7 above, or the operation steps of the method described in claim 8, or the operation steps of the method described in any one of claims 9-12.

19. A computer program product, characterized in that the computer program product includes a computer program or instructions, which, when run on a computer, cause the computer to perform the operation steps of the method described in any one of claims 1-7 above, or the operation steps of the method described in claim 8, or the operation steps of the method described in any one of claims 9-12.

Images