Adaptive Real-Time Communication Plugin for Virtual Desktop Infrastructure Solutions

Abstract

To provide a method, system, and program for improving performance of a remote desktop client that is executed on a client computing device to present a user interface of a communications application that is remotely executed in a cloud computing environment.SOLUTION: In a client computing device, a plugin works with a remote desktop client that is executed on the client computing device to present a user interface of a communications application that is executed in a cloud computing environment. The plugin enables the remote desktop client to conduct audio and / or video communication with a remote computing device in a peer-to-peer manner rather than via the communications application. The plugin also enables the remote desktop client to determine a hardware-based media processing capability of the client computing device and leverage such capability in conducting the peer-to-peer audio and / or video communication with the remote computing device.SELECTED DRAWING: Figure 4

Description

Background Art

[0001] Background

[0001] Virtualization technology abstracts a desktop operating system (OS) and related applications from the client computing devices used to access them. For example, virtual desktop infrastructure (VDI) may include hosting a desktop on a server within a data center and delivering an image of the desktop over a network to a client computing device. The desktop image can then be rendered on the client computing device, and the user of the client computing device can interact directly with the image as if the desktop and its applications were being executed locally on the client computing device.

[0002]

[0002] This approach enables customers to rationalize management and costs by integrating and centralizing user desktops. The advantages of centralization include optimization of hardware resources, reduction of software maintenance, and improvement of security. For example, in one instance, software patching and OS migration can be applied and tested for all users. Additionally, software assets are centralized, thereby facilitating monitoring and protection, and ensuring that the security of confidential data is not compromised in the event that a desktop is lost or stolen. Moreover, desktop virtualization enhances user mobility and the freedom to access a virtual desktop from any location and any device.

[0003]

[0003] Despite the many advantages provided by virtualization technology, some applications may not be optimized for or may not be adequately supported within a VDI environment. One such example is communication applications that run within a VDI environment and enable real-time communication (RTC) between users (e.g., Microsoft® Teams, Slack®). RTC refers to the exchange of information (e.g., voice, instant messaging, video, etc.) over a network with near real-time latency from sender to receiver. [Overview of the Initiative]

[0004] overview

[0004] This summary is provided to introduce in a simplified form a series of concepts that will be further described in the following detailed description. This summary is not intended to identify any important or essential features of the claims, nor to be used to limit the scope of the claims.

[0005]

[0005] This specification describes methods, systems, and computer program products for improving the performance of remote desktop clients running on client computing devices to present user interfaces (UIs) of communication applications running remotely within a cloud computing environment. In embodiments, these methods, systems, and computer program products enable remote desktop clients to communicate with remote computing devices in a peer-to-peer manner rather than through remotely running communication applications. Further according to such embodiments, these methods, systems, and computer program products enable remote desktop clients to determine one or more hardware-based media processing capabilities of the client computing device and utilize such one or more hardware-based media processing capabilities when communicating with remote computing devices in a peer-to-peer manner, thereby improving the quality of such audio and / or video communication and reducing the processing load on the client computing device. One or more hardware-based media processing capabilities can be used, for example, to process media received from a remote computer for rendering by a client computer, to process media captured from a client computer's media source for transmission to a remote computer, or as a basis for negotiating media communication parameters with a remote computer.

[0006]

[0006] Further features and advantages of the present invention, as well as the structure and operation of various embodiments, will be described in detail below with reference to the accompanying drawings. It should be noted that embodiments are not limited to the specific embodiments described herein. Such embodiments are shown herein for illustrative purposes only. Further embodiments will become apparent to those skilled in the art based on the teachings contained herein.

[0007] Brief explanation of the drawing

[0007] The accompanying drawings incorporated herein and forming part thereof illustrate embodiments of the present application, describe the principles of the embodiments together with the description, and further enable those skilled in the art to create and utilize the embodiments. [Brief explanation of the drawing]

[0008] [Figure 1]

[0008] A block diagram of an example of a system for enabling video and / or voice communication by a communication application running in a cloud computing environment is shown, according to an example of an embodiment. [Figure 2]

[0009] In contrast to voice and / or video communication between a client computer and a remote computer via a communication application, as described in one embodiment, a block diagram of an example system for enabling peer-to-peer voice and / or video communication between a client computer and a remote computer is shown. [Figure 3]

[0010] A block diagram of an example system, according to one embodiment, is shown, which includes a plugin that enables redirection of communications from a communication application in order to enable peer-to-peer voice and / or video communication between a client computer and a remote computer. [Figure 4]

[0011] Another example of the embodiment shows a block diagram of a client computer that includes determining the hardware-based media processing capability of the client computer and plugging it into a remote desktop client that uses the detected hardware-based media processing capability to process media. [Figure 5]

[0012] A flowchart illustrating a method for determining the hardware-based media processing capability of a client computer and using the detected hardware-based media processing capability to process media, according to one embodiment, is shown. [Figure 6]

[0013] A flowchart illustrating a method for negotiating media communication parameters with a remote computing device based on the determined hardware-based media processing capabilities of a client computing device is shown, according to one embodiment. [Figure 7]

[0014] A flowchart illustrating an example of an embodiment shows a method for determining the hardware-based media processing capability of a client computer and using that capability to process media captured from a media source on the client computer. [Figure 8]

[0015] This is a block diagram of an example of a processor-based computer system that can be used to implement various embodiments. [Modes for carrying out the invention]

[0009]

[0016] The features and advantages of the present invention will become more apparent from the detailed description below when understood in conjunction with the drawings, in which similar reference letters are used throughout the drawings to identify corresponding elements. In the drawings, similar reference numbers generally indicate identical, functionally similar, and / or structurally similar elements. The drawing in which a certain element first appears is indicated by the most significant digit of the corresponding reference number.

[0010] Detailed explanation I. Introduction

[0017] This specification and accompanying drawings disclose one or more embodiments incorporating features of the present invention. The scope of the present invention is not limited to the disclosed embodiments. The disclosed embodiments are merely illustrative of the present invention, and modifications of the disclosed embodiments are also included in the present invention. Embodiments of the present invention are defined by the appended claims.

[0011]

[0018] References in this specification to “one embodiment,” “a certain embodiment,” “example of an embodiment,” etc., mean that the embodiments described may include certain features, structures, or characteristics, but not all embodiments may necessarily include those specific features, structures, or characteristics. Furthermore, such phrases do not necessarily refer to the same embodiment. Moreover, if certain features, structures, or characteristics are described in relation to a certain embodiment, it is considered that the influence of such features, structures, or characteristics on other embodiments, whether explicitly stated or not, falls within the scope of knowledge of those skilled in the art.

[0012]

[0019] In this commentary, unless otherwise specified, adjectives such as “approximately” and “about” that modify the condition or relationship of a feature of an embodiment of the present disclosure are understood to mean that the condition or characteristic is defined within an acceptable tolerance range with respect to the operation of the embodiment of the application in question.

[0013]

[0020] Numerous exemplary embodiments are described below. Note that the headings of any section / subsection shown herein are not intended to be limiting. Embodiments are described throughout this specification, and any type of embodiment may be contained within any section / subsection. Furthermore, embodiments disclosed within any section / subsection may be combined in any way with any other embodiments described within the same section / subsection and / or different sections / subsections.

[0014] II. Examples of Embodiments

[0021] Virtualization technology abstracts a desktop's operating system (OS) and associated applications from the client computers used to access them. For example, a virtual desktop infrastructure solution (VDI) may involve hosting desktops on servers in a data center and delivering images of those desktops to client computers over a network. The desktop images can then be rendered on the client computers, allowing users of the client computers to interact directly with the images as if the desktop and its applications were running locally on the client computers.

[0015]

[0022] This approach allows customers to streamline management and reduce costs by consolidating and centralizing user desktops. The benefits of centralization include optimized hardware resources, reduced software maintenance, and improved security. For example, in one scenario, software patching and OS migration can be applied and tested for all users. Furthermore, software assets are centralized, making them easier to monitor and protect, and ensuring that confidential data security is not compromised if a desktop is lost or stolen. In addition, desktop virtualization enhances user mobility and the freedom to access virtual desktops from any location and device.

[0016]

[0023] Despite the many advantages offered by virtualization technology, some applications may not be optimized for or adequately supported within a VDI environment. One such example is communication applications running within a VDI environment that enable real-time communication (RTC) between users (e.g., Microsoft® Teams, Slack®). As used herein, RTC refers to the near-simultaneous exchange of information (e.g., voice, instant messaging, video, etc.) over a network with minimal latency.

[0017]

[0024] For example, FIG. 1 shows a block diagram of an example of a system 100 for enabling video and / or voice communication by a communication application executed within a cloud computing environment. As shown in FIG. 1, the system 100 includes a cloud computing environment 102 including a virtual desktop host 104, a client device 112, and a remote computing device 114.

[0018]

[0025] For purposes of explanation, the cloud computing environment 102 is shown as including only one virtual desktop host 104, but can include any number of resources. For example, the cloud computing environment 102 can be composed of resources (such as servers) executed within one or more cloud data centers and / or on-premises with respect to an enterprise or organization. Additionally, in embodiments, the cloud computing environment 102 can include resources (such as network switches, networks, etc.) that assist in communication with servers and communication between servers, storage areas by servers (such as storage devices, etc.), resources that manage other resources (such as a hypervisor that manages virtual machines to present a virtual operating platform for tenants of a multi-tenant cloud, etc.), and / or any other resources of any type and any number including additional types of resources. In some embodiments, the virtual desktop host 104 can include a remote server or virtual machine accessible within a data center or on-premises server.

[0019]

[0026] Additionally, the client computing device 112 and the remote computing device 114 are illustrated as a laptop and a smartphone, respectively, but can be any type of mobile computing device or fixed computing device. Examples of mobile computing devices include, but are not limited to, Microsoft® Surface® devices, personal digital assistants (PDAs), thin clients, laptop computers, notebook computers, tablet computers such as Apple iPad™, netbooks, mobile phones, mobile gaming consoles, or wearable computing devices. Examples of fixed computing devices include, but are not limited to, desktop computers or personal computers (PCs), gaming consoles, or servers.

[0020]

[0027] As further shown in FIG. 1, client computing device 112 is communicatively coupled to cloud computing environment 102 via network 108, and remote computing device 114 is communicatively coupled to cloud computing environment 102 via network 118. Networks 108 and 118 can include one or more networks such as a local area network (LAN), wide area network (WAN), enterprise network, the Internet, etc., and can include one or more of wired and / or wireless portions. In certain scenarios, networks 108 and network 118 can include the same network.

[0021]

[0028] Client computing device 112, remote computing device 114, and virtual desktop host 104 can include at least one network interface to enable communication on networks 108 and 118. Examples of such wired or wireless network interfaces include IEEE 802.11 wireless local area network (WLAN) wireless interfaces, WiMAX (Worldwide Interoperability for Microwave Access) interfaces, Ethernet interfaces, Universal Serial Bus (USB) interfaces, cellular network interfaces, Bluetooth (trademark) interfaces, Near Field Communication (NFC) interfaces, etc. Further examples of network interfaces are described elsewhere in this specification.

[0022]

[0029] The client computer 112 also includes a remote desktop client configured to enable video and / or audio communication between users of a communication application and to present the user interface 106 of the communication application running on the virtual desktop host 104 within the user interface of the client computer 112. For example, data related to the user interface 106 of the communication application can be transferred to the client computer 112 and rendered as the user interface within the screen display of the client computer 112. Any interaction (e.g., mouse and / or keyboard input) that user 110 may have with the user interface on the client computer 112 presenting the user interface 106 of the communication application can also be tracked and transmitted to the virtual desktop host 104.

[0023]

[0030] More specifically, user 110 can initiate video and / or voice communication with user 116 on remote computing device 114 by interacting with the user interface of client computing device 112 (for example, by clicking the "phone icon" next to the contacts displayed in the user interface). This interaction can be provided to virtual desktop host 104, and a communication application running on virtual desktop host 104 can initiate and execute video and / or voice communication with user 116.

[0024]

[0031] In some embodiments, during video and / or audio communication between user 110 and user 116, device sources (such as video cameras) on client computing device 112 and remote computing device 114 can capture images appearing in front of the device sources. As shown in Figure 1, the captured videos of users 110 and 116 can be transferred to the other user's computing device via the virtual desktop host 104 for presentation on the display screen of the other user's computing device. For example, the video of user 116 captured by the device source on remote computing device 114 can be transferred to the virtual desktop host 104, processed (e.g., decoded) on the virtual desktop host 104, and rendered locally as part of the user interface 106 of the communication application. In some embodiments, screen scraping can be used to collect screen display data related to the video of user 116 rendered within the user interface 106 of client computing device 112, so that user 110 can simultaneously view the video of user 116 as it is captured by the device source.

[0025]

[0032] Regarding Figure 1, the video and / or audio communication method described above can lead to latency issues (especially in high-resolution video and audio experiences) because the video and / or audio communication between the client computer 112 and the remote computer 114 is transmitted via network 108, virtual desktop host 104, and network 118. Furthermore, this method can increase the amount of time and resources spent processing the media. For example, when the virtual desktop host 104 receives media, it may have to decode the media, and when the virtual desktop host 104 transmits media, it may have to encode the media.

[0026]

[0033] Implementing a peer-to-peer video and / or voice communication method between the client computing device 112 and the remote computing device 114 can not only mitigate the latency issues mentioned above, but also reduce the processing workload on the virtual desktop host 104. For example, Figure 2 shows a block diagram of an example of a system 200 for enabling peer-to-peer voice and / or video communication between a client computing device and a remote computing device that interact with a communication application in the cloud.

[0027]

[0034] As explained above with respect to Figure 1, user 110 can initiate video and / or audio communication with user 116 on remote computing device 114 by interacting with the user interface of client computing device 112. This interaction can be given to the virtual desktop host 104, which can prompt a communication application running on the virtual desktop host 104 to initiate media communication with remote computing device 114. However, in Figure 2, the media communication call can be intercepted and redirected to client computing device 112 to enable peer-to-peer media communication between client computing device 112 and remote computing device 114. The remote desktop client on client computing device 112 is configured to receive the redirected communication from the communication application to enable peer-to-peer audio and / or video communication between client computing device 112 and remote computing device 114.

[0028]

[0035] Once the client computer 112 and the remote computer 114 are connected for voice and / or video communication (a process described in more detail below with respect to Figures 4 and 5), the client computer 112 and the remote computer 114 can communicate video and / or voice to each other via network 202 (which may be the same as, similar to, or completely different from, networks 108 and 118 in Figure 1).

[0029]

[0036] By enabling peer-to-peer audio and / or video communication between the client computer 112 and the remote computer 114, media communication via the virtual desktop host 104 is avoided, thereby reducing latency associated with such audio and / or video communication and the processing load on the virtual desktop host 104. Furthermore, as part of moving the media communication connection between the client computer 112 and the remote computer 114 to a peer-to-peer channel, the embodiments described herein also allow the remote desktop client to discover and utilize the local hardware-based media processing capabilities (e.g., hardware-based video or audio codecs) of the client computer 112 to perform such communication more effectively. As used herein, the term media should be understood to include at least video, audio, or a combination thereof. Since such embodiments can utilize the hardware-based media processing capabilities of the client computer 112, the quality and efficiency of media processing by the client computer 112 can be improved compared to software-based media processing methods. Furthermore, utilizing the hardware-based media processing capabilities of the client computer 112 reduces the load on the processor of the client computer 112, eliminating the need for the processor to perform the corresponding software-based media processing instead.

[0030]

[0037] To help illustrate the process of redirecting communication from a communication application running on a virtual desktop host 104 to a client computer 112, Figure 3 will be described, continuing with reference to Figures 1 and 2. Figure 3 shows a block diagram of an example of a system 300 that enables redirection of communication from a communication application to enable peer-to-peer voice and / or video communication between a client computer and a remote computer.

[0031]

[0038] As shown in Figure 3, the cloud computing environment 102 includes a virtual desktop host 104 and a plugin 302. Further shown in Figure 3, the plugin 302 includes a redirection sim 304. The redirection sim 304 is configured to intercept communications (e.g., API calls) initiated by communication applications running on the virtual desktop host 104 and redirect those calls to the client computing device 112 via the network 108. For example, if a communication application initiates video and / or voice communication with the remote computing device 114, the redirection sim 304 intercepts the video and / or voice communication request, modifies the request as necessary (e.g., replaces a specific script object with a different implementation), and redirects it to the client computing device 112 over the communication channel 306 (via the network 108).

[0032]

[0039] In one embodiment, if certain conditions are met (for example, if the communication application is running on a specific virtual desktop environment and multi-session users are enabled), the communication application can load plugin 302 at runtime. Once loaded, plugin 302 can attempt to establish a communication channel 306 with the client computer 112 (e.g., establish a dynamic virtual channel (DVC)) to enable communication between the virtual desktop host 104 and the client computer 112. In this embodiment, plugin 302 can make a remote procedure call (RPC) to the client computer 112 using a WebSocket server process.

[0033]

[0040] To provide a more detailed overview of the client computer 112, Figure 4 is described next. Figure 4 shows the client computer 112 including a plug-in 404 that enables peer-to-peer communication between the client computer 112 and the remote computer, determines the hardware-based media processing capability of the client computer 112, and uses the detected hardware-based media processing capability to process media to a remote desktop client 402. The plug-in 404 includes a set of components 432 that are non-OS specific and adaptable to any platform, and a set of components 430 that are OS specific and need to be reimplemented for each platform. The set of non-OS specific components 432 includes an RTC listener 406, an RTC manager 408, a window manager 410, and an RTC component 412.

[0034]

[0041] As further shown in Figure 4, the client computing device 112 also includes a remote desktop client 402. As previously described, the remote desktop client 402 of the client computing device 112 is configured to receive communications redirected from a communication application (i.e., via the plug-in 302 of the cloud computing environment 102 in Figure 3) on the communication channel 306 in order to enable peer-to-peer voice and / or video communication between the client computing device 112 and a remote computing device (e.g., the remote computing device 114 in Figures 1 and 2).

[0035]

[0042] In the process of enabling peer-to-peer communication between client computer 112 and remote computer 114, the RTC listener 406 is configured to listen for redirected communications on communication channel 306 and provide the RTC manager 408 with a stream of redirected communications upon reception. The RTC manager 408 is configured to receive the stream of redirected communications (e.g., serialized RPC call information) and convert the communications into a format compatible with a framework for enabling real-time communication (e.g., WebRTC). The RTC manager 408 then provides the converted communications (e.g., WebRTC API calls) to the RTC component 412. The RTC component 412 is configured to connect client computer 112 and remote computer 114 for voice and / or video communication based on the converted communications (e.g., WebRTC API calls) provided by the RTC manager 408. The RTC component 412 can establish a communication channel 436 between client computer 112 and remote computer 114 for voice and / or video communication.

[0036]

[0043] A set of components 430 of the OS-specific plugin 404 includes a media capture component 414, a media renderer component 416, and a window manager component 418. After the client computer 112 and the remote computer 114 are connected for voice and / or video communication, the media capture component 414 and the media renderer component 416 can be used to detect the hardware-based media processing capabilities of the client computer 112, and these hardware-based media processing capabilities can be used to optimize voice and / or video communication between the client computer 112 and the remote computer 114.

[0037]

[0044] For example, the media capture component 414 can determine the hardware-based media processing capabilities of the client computer 112 by using OS-specific APIs to determine what is available on the client computer 112. After determining what is available on the client computer 112, the media capture component 414 is configured to inform the source reader 420 of any information related to processing the captured media (such as formatting) and any hardware configuration necessary to process the captured media using the hardware-based media processing capabilities. The source reader 420 is configured to capture media from the video camera 426 and other device sources (e.g., microphones) of the client computer 112.

[0038]

[0045] To aid in the explanation, let us assume that the media capture component 414 determines that the client computing device 112 has hardware-based media processing capabilities to transcode the captured video to a low resolution / quality. The media capture component 414 can inform the source reader 420 that the original captured media needs to be decoded into an intermediate uncompressed format (e.g., PCM for audio, or YUV for video) and then encoded into a target format. The source reader 420 can then perform the transcoding and provide the processed video to the media capture component 414. When capturing video, the source reader 420 can function like a frame server and stream the captured video to the media capture component 414.

[0039]

[0046] The media capture component 414 is further configured to receive processed and / or unprocessed media captured from the video camera 426 from the source reader 420 and to provide the captured media to the RTC component 412. The RTC component 412 is configured to prepare the captured media for transmission to the remote computing device 114. As shown in Figure 4, the RTC component 412 can transmit the captured media to the remote computing device 114 over the communication channel 436 via the socket API 424.

[0040]

[0047] The media renderer component 416 can determine the hardware-based media processing capabilities of the client computer 112 by using OS-specific APIs to determine what is available on the client computer 112. After determining what is available on the client computer 112, the media renderer component 416 is configured to inform the media engine 422 of any information related to the processing of the received media (such as formatting) and any hardware configuration necessary to process the received media using the hardware-based media processing capabilities.

[0041]

[0048] The media renderer component 416 is further configured to provide the media engine 422 with a specific source of media received from the RTC component 412. The media engine 422 can then build a media pipeline and set it up for rendering. The media engine 422 is configured to provide the media stream source 428 with a specific source (compressed or decompressed). The media stream source 428 is configured to provide the media source to the hardware necessary to process the received media, leveraging hardware-based media processing capabilities. In one embodiment, the media stream source 428 can use a frame server technique to create video media.

[0042]

[0049] As shown in Figure 4, the media stream source 428 is connected to the hardware decoder 438. In some embodiments, if the received media is compressed, the hardware decoder 438 can decode the received media. In addition, the media stream source 428 can provide the received media to other hardware-based media processing capabilities, such as hardware-accelerated video processing (e.g., frame resizing, color conversion, etc.) or hardware-accelerated audio processing. As shown in Figure 4, the processed media can be provided to the remote desktop client 402 for rendering within the user interface 434 of the remote desktop client 402. Alternatively, the processed media can be rendered on the display 440. The window manager 410 and window manager component 418 are configured to monitor and track any changes that need to be made to the video being rendered within the user interface 434 (e.g., moving windows, resizing windows, pop-up menus, etc.).

[0043]

[0050] The client computing device 112 will be described in more detail below with respect to Figures 5 to 7. Figure 5 shows a flowchart 500 of a method for enabling peer-to-peer communication between the client computing device and the remote computing device, determining the hardware-based media processing capability of the client computing device, and using the detected hardware-based media processing capability to process the media.

[0044]

[0051] Flowchart 500 begins at step 502. In step 502, the remote desktop client 402 receives redirected communications from the communication application to enable peer-to-peer voice and / or video communication between the client computer and the remote computer, as opposed to voice and / or video communication by the communication application. For example, and continuing to refer to Figure 4, the remote desktop client 402 receives redirected communications from the communication application on communication channel 306 to enable peer-to-peer voice and / or video communication between the client computer 112 and the remote computer 114, as opposed to voice and / or video communication by the communication application.

[0045]

[0052] In step 504, a framework for enabling real-time communication is used to connect the client computer and the remote computer for voice and / or video communication based on redirected communication. For example, continuing to Figure 4, the RTC component 412 uses a framework for enabling real-time communication to connect the client computer 112 and the remote computer 114 for voice and / or video communication based on redirected communication. In one embodiment, the redirected communication includes WebRTC API calls intercepted from a communication application, and the WebRTC API and protocol is the framework used to connect the client computer 112 and the remote computer 114.

[0046]

[0053] In step 506, the hardware-based media processing capabilities of the client computer are determined. For example, continuing to Figure 4, the media renderer component 416 determines the hardware-based media processing capabilities of the client computer 112 by using OS-specific APIs to determine what is available on the client computer 112. Such hardware-based media processing capabilities of the client computer 112 may include, for example, video codecs (including video decoders), audio codecs (including audio decoders), hardware-accelerated video processing, or hardware-accelerated audio processing, without limitation.

[0047]

[0054] In step 508, the media transmitted from the remote computing device is received. For example, continuing to Figure 4, the media renderer component 416 receives the media transmitted from the remote computing device 114 from the RTC component 412.

[0048]

[0055] In step 510, the incoming media from the remote computer is processed using hardware-based media processing capabilities for rendering by the client computer. For example, continuing to Figure 4, the media stream source 428 processes the incoming media from the remote computer 114 for rendering by the client computer 112 using hardware-based media processing capabilities. For illustrative purposes, the media stream source 428 is connected to a hardware decoder 438. In some embodiments, if the incoming media is compressed, the hardware decoder 438 can decode the incoming media. In addition, the media stream source 428 can provide the incoming media to other hardware-based media processing capabilities, such as hardware-accelerated video processing (e.g., frame resizing, color conversion, etc.) or hardware-accelerated audio processing.

[0049]

[0056] Figure 6 shows a flowchart 600 of a method for negotiating media communication parameters with a remote computer based on the determined hardware-based media processing capabilities of a client computer. Flowchart 600 begins in step 602. In step 602, another hardware-based video processing capability of the client computer is determined. For example, continuing to Figure 4, the RTC manager 408 can determine another hardware-based video processing capability of the client computer 112. The RTC manager 408 can determine other hardware-based media processing capabilities by APIs of a framework for enabling real-time communication.

[0050]

[0057] In step 604, media communication parameters are negotiated with the remote computing device based on other hardware-based media processing capabilities. For example, continuing to Figure 4, the RTC manager 408 negotiates media communication parameters with the remote computing device 114 based on other hardware-based media processing capabilities. For example, the RTC manager 408 may determine that the client computing device 112 has a particular decoder (e.g., a VP9 decoder) that the remote computing device 114 does not have. The RTC manager 408 can negotiate with the remote computing device 114 to provide the media in a format compatible with the remote computing device 114's particular decoder.

[0051]

[0058] Figure 7 shows a flowchart 700 of a method for determining the hardware-based media processing capability of a client computer and using that capability to process media captured from a media source on the client computer. Flowchart 700 begins in step 702. In step 702, another hardware-based media processing capability of the client computer is determined. For example, continuing to Figure 4, the media capture component 414 determines another hardware-based media processing capability of the client computer 112. The media capture component 414 can determine the available hardware-based media processing capability of the client computer 112 by using an API specific to the client computer 112's OS. Such hardware-based media processing capability of the client computer 112 may include, for example, video codecs (including video encoders), audio codecs (including audio encoders), hardware-accelerated video processing, or hardware-accelerated audio processing, without limitation.

[0052]

[0059] In step 704, the media captured from the media source of the client computer is processed using other hardware-based media processing capabilities. For example, a hardware-based video or audio encoder can be used to encode the video or audio captured from the media source of the client computer 112, respectively. Alternatively, hardware-accelerated video or audio processing can be used to accelerate the processing of the video or audio captured from the media source of the client computer 112, respectively. In some embodiments, and continuing to Figure 4, the media capture component 414 is configured to inform the source reader 420 of any information relating to the processing of the captured media (such as formatting) and any hardware configuration necessary to process the captured media by leveraging hardware-based media processing capabilities.

[0053]

[0060] In step 706, the processed media is transmitted to the remote computing device. For example, continuing to Figure 4, the RTC component 412 transmits the processed media to the remote computing device 114 over the communication channel 436.

[0054]

[0061] Although the embodiments described above refer to communication applications running on a virtual desktop host, it should be understood that the techniques described herein can be used in combination with any communication application running remotely from the client computing device 112, even if such communication application runs on a virtual desktop or any other type of platform. In other words, the embodiments described herein are not limited to communication applications running on a virtual desktop, but broadly encompass any type of communication application that can run on a server and enable voice and / or video communication between remotely located users and user devices.

[0055]

[0062] Furthermore, while the above embodiments refer to remote desktops and associated plugins running on the client computer 112, it should be understood that the functionality provided by these components can be encapsulated within any type of software component having any type of structure or architecture. For example, the functionality and features of plugin 404 may be contained within a software component that is not necessarily a plugin, but rather some other type of software component capable of operating in conjunction with the remote desktop client 402. Moreover, the functionality and features of plugin 404 can be directly integrated into the remote desktop client 402 so as not to require additional software components beyond the remote desktop client 402. Furthermore, the functionality and features of the remote desktop client 402 and plugin 404 may be contained within an application or operating system component of the client computer 112 that provides similar functionality to the components described herein, even if they are not referred to as remote desktop clients or plugins.

[0056] III. Examples of Computer System Implementations

[0063] The cloud computing environment 102, virtual desktop host 104, plug-in 302, client computing device 112, remote desktop client 402, plug-in 404, flowchart 500, flowchart 600, and / or flowchart 700 can be implemented by hardware, or by hardware combined with either or both software and / or firmware. For example, the cloud computing environment 102, virtual desktop host 104, plug-in 302, client computing device 112, remote desktop client 402, plug-in 404, flowchart 500, flowchart 600, and / or flowchart 700 can be implemented as computer program code / instructions configured to run in one or more processors and be stored in a computer-readable storage medium. In another embodiment, the cloud computing environment 102, virtual desktop host 104, plug-in 302, client computing device 112, remote desktop client 402, plug-in 404, flowchart 500, flowchart 600, and / or flowchart 700 can also be implemented by hardware that operates Software as a Service (SaaS) or Platform as a Service (PaaS). Alternatively, the cloud computing environment 102, virtual desktop host 104, plug-in 302, client computing device 112, remote desktop client 402, plug-in 404, flowchart 500, flowchart 600, and / or flowchart 700 can be implemented as hardware logic / electrical circuits.

[0057]

[0064] For example, in one embodiment, one or more of any combination of the cloud computing environment 102, virtual desktop host 104, plug-in 302, client computing device 112, remote desktop client 402, plug-in 404, flowchart 500, flowchart 600, and / or flowchart 700 can be implemented together by a system-on-a-chip (SoC). The SoC may include an integrated circuit chip that includes a processor (e.g., a central processing unit (CPU), microcontroller, microprocessor, digital signal processor (DSP), etc.), memory, one or more communication interfaces, and / or one or more further circuits, and may optionally include embedded firmware for executing received program code and / or performing functions.

[0058]

[0065] Figure 8 shows an exemplary implementation of a computing device 800 that can implement the embodiment. For example, the components of the cloud computing environment 102, virtual desktop host 104, plug-in 302, client computing device 112, remote desktop client 402, and plug-in 404 can each be implemented by one or more computing devices similar to computing device 800, including one or more features and / or alternative features of computing device 800, in embodiments of a fixed computer or mobile computer. The description of computing device 800 provided herein is for illustrative purposes only and is not intended to be limiting. As those skilled in the art will see, the embodiment can be implemented by further types of computer systems.

[0059]

[0066] As shown in Figure 8, the computing device 800 includes one or more processors called processor circuits 802, system memory 804, and a bus 806 that connects various system components, including the system memory 804, to the processor circuits 802. The processor circuits 802 are electrical and / or optical circuits implemented by one or more physical hardware electrical circuit elements and / or integrated circuit devices (semiconductor material chips or dies) as central processing units (CPUs), microcontrollers, microprocessors, and / or other physical hardware processor circuits. The processor circuits 802 can execute program code stored in a computer-readable medium, such as program code for the operating system 830, application programs 832, and other programs 834. The bus 806 represents one or more of several types of bus structures, including a memory bus or memory controller using any of various bus architectures, peripheral buses, accelerated graphics ports, and processor or local buses. The system memory 804 includes read-only memory (ROM) 808 and random access memory (RAM) 810. The Basic Input / Output System 812 (BIOS) is stored in ROM 808.

[0060]

[0067] The computer unit 800 also has one or more of the following drives: a hard disk drive 814 for reading and writing to and from a hard disk, a magnetic disk drive 816 for reading and writing to and from a removable magnetic disk 818, and an optical disk drive 820 for reading and writing to and from a removable optical disk 822 such as a CD-ROM, DVD-ROM, or other optical medium. The hard disk drive 814, the magnetic disk drive 816, and the optical disk drive 820 are connected to the bus 806 by hard disk drive interface 824, magnetic disk drive interface 826, and optical drive interface 828, respectively. The drives and their associated computer-readable media provide non-volatile storage for computer-readable instructions, data structures, program modules, and other data for the computer. Although hard disks, removable magnetic disks, and removable optical disks have been described, data can be stored using other types of hardware-based computer-readable storage media such as flash memory cards, digital video discs, RAM, ROM, and other hardware storage media.

[0061]

[0068] Several program modules can be stored on a hard disk, magnetic disk, optical disk, ROM, or RAM. These programs include an operating system 830, one or more application programs 832, other programs 834, and program data 836. The application programs 832 or other programs 834 may include, for example, a virtual desktop host 104, a plug-in 302, a remote desktop client 402, a plug-in 404, flowchart 500, flowchart 600, and / or flowchart 700 (including any appropriate steps of flowcharts 500, 600, and 700), and / or computer program logic (e.g., computer program code or instructions) for implementing further embodiments described herein.

[0062]

[0069] The user can input commands and information into the computing device 800 using input devices such as the keyboard 838 and the pointing device 840. Other input devices (not shown) may include a microphone, joystick, gamepad, satellite receiving antenna, scanner, touchscreen and / or touchpad, voice recognition system for receiving voice input, gesture recognition system for receiving gesture input, etc. These and other input devices are often connected to the processor circuit 802 by a serial port interface 842 coupled to bus 806, but may also be connected by other interfaces such as a parallel port, game port, or Universal Serial Bus (USB).

[0063]

[0070] The display screen 844 is also connected to the bus 806 via an interface such as the video adapter 846. The display screen 844 may be located outside the computer 800 or incorporated into the computer 800. The display screen 844 can not only display information but can also serve as a user interface for receiving user commands and / or other information (e.g., by touch, finger gestures, virtual keyboard, etc.). In addition to the display screen 844, the computer 800 may include other peripheral output devices (not shown), such as speakers and printers. The display screen 844 and / or any other peripheral output devices (not shown) may be used to implement user interfaces 106 and 434 and / or any further embodiments described herein.

[0064]

[0071] The computing device 800 is connected to the network 848 (e.g., the Internet) by an adapter or network interface 850, a modem 852, or other means for establishing communication over the network. The modem 852, which may be built-in or external, can be connected to the bus 806 via the serial port interface 842 as shown in Figure 8, or it can be connected to the bus 806 using another type of interface, including a parallel interface.

[0065]

[0072] As used herein, the terms “computer program medium,” “computer-readable medium,” and “computer-readable storage medium” are used to refer to hard disks associated with the hard disk drive 814, removable magnetic disks 818, removable optical disks 822, RAM, ROM, flash memory cards, digital video disks, zip disks, MEM, nanotechnology-based storage devices, and other physical hardware media, and further types of physical / tangible hardware storage media. Such computer-readable storage media are distinct from and do not overlap with (and do not include) communication media. Communication media incorporate computer-readable instructions, data structures, program modules, or other data within modulated data signals, such as carrier waves. The term “modulated data signal” means a signal in which one or more of its properties are set or modified to encode information within the signal. By example, but not limited to, communication media include wireless media such as acoustic, RF, and infrared, and other wireless media, as well as wired media. Embodiments also cover such communication media, which are separate and non-overlapping with embodiments that cover computer-readable storage media.

[0066]

[0073] As described above, computer programs and modules (including application program 832 and other programs 834) can be stored on hard disks, magnetic disks, optical disks, ROMs, RAMs, or other hardware storage media. Such computer programs can also be received via network interface 850, serial port interface 842, or any other type of interface. When such computer programs are executed or loaded by an application, they enable the computer 800 to implement features of the embodiments discussed herein. Thus, such computer programs represent the controller of the computer 800.

[0067]

[0074] The embodiments also cover computer program products that include computer code or instructions stored on any computer-readable medium. Such computer program products include hard disk drives, optical disk drives, memory device packages, portable memory sticks, memory cards, and other types of physical storage hardware.

[0068] IV. Further Embodiments

[0075] In a first embodiment, the client computing device includes one or more processors and one or more memory devices for storing computer program logic to be executed by one or more processors, the computer program logic being a remote desktop client configured to present a user interface of a communication application running in a cloud computing environment within the user interface of the client computing device, and further configured to receive redirected communications from a communication application to enable peer-to-peer voice and / or video communication between the client computing device and the remote computing device, as opposed to voice and / or video communication by the communication application; a real-time communication manager configured to receive redirected communications from a remote desktop channel component and convert the redirected communications into a format that conforms to a framework for enabling real-time communication; a real-time communication component configured to connect the client computing device and the remote computing device for voice and / or video communication based on the converted communications; and a plug-in including a media capture component configured to determine the hardware-based media processing capability of the client computing device, process media captured from media sources of the client computing device using the hardware-based media processing capability, and provide the processed media to the real-time communication component for transmission to the remote computing device.

[0069]

[0076] In one embodiment, the media includes either video or audio.

[0070]

[0077] In one embodiment, the real-time communication manager is further configured to determine another hardware-based video processing capability of the client computer and to negotiate media communication parameters with the remote computer based on the other hardware-based media processing capability.

[0071]

[0078] In one embodiment, a media capture component determines hardware-based media processing capabilities using the application programming interface (API) of the client computer's operating system, and a real-time communication manager determines other hardware-based media processing capabilities using the API of a framework for enabling real-time communication.

[0072]

[0079] In one embodiment, the plugin further includes a media renderer component configured to receive media transmitted from a remote computing device by a real-time communication component and provide the received media to the client computing device's media engine for rendering within the client computing device's user interface, the video renderer component further configured to determine another hardware-based media processing capability of the client computing device and use that other media processing capability to process the received media from the remote computing device.

[0073]

[0080] In one embodiment, the hardware-based media processing capability includes one of either a video codec or an audio codec.

[0074]

[0081] In one embodiment, the hardware-based media processing capability includes either hardware-accelerated video processing or hardware-accelerated audio processing.

[0075]

[0082] In another embodiment, a method performed by one or more processors of a client computing device that presents a user interface for a communication application running in a cloud computing environment includes receiving redirected communications from a communication application to enable peer-to-peer voice and / or video communications between the client computing device and a remote computing device, in contrast to voice and / or video communications by the communication application; using a framework to enable real-time communications; connecting the client computing device and the remote computing device for voice and / or video communications based on the redirected communications; determining the hardware-based media processing capabilities of the client computing device; receiving media transmitted from the remote computing device; and processing the received media from the remote computing device for rendering by the client computing device using the hardware-based media processing capabilities.

[0076]

[0083] In one embodiment, the media includes either video or audio.

[0077]

[0084] In one embodiment, the method further includes determining another hardware-based video processing capability of the client computer and negotiating media communication parameters with the remote computer based on the other hardware-based media processing capability.

[0078]

[0085] In one embodiment, determining hardware-based media processing capability includes determining hardware-based media processing capability using an application programming interface (API) of the client computer's operating system, and determining other hardware-based media processing capability includes determining other hardware-based media processing capability using an API of a framework for enabling real-time communication.

[0079]

[0086] In one embodiment, the method further includes determining another hardware-based media processing capability of the client computer, using the other hardware-based media processing capability to process media captured from a media source of the client computer, and transmitting the processed media to a remote computer.

[0080]

[0087] In one embodiment, the hardware-based media processing capability includes one of either a video codec or an audio codec.

[0081]

[0088] In one embodiment, the hardware-based media processing capability includes either hardware-accelerated video processing or hardware-accelerated audio processing.

[0082]

[0089] In another embodiment, a computer-readable storage medium recording program instructions causing at least one processor to execute a method, when executed by at least one processor of a client computer presenting a user interface for a communication application running in a cloud computing environment, to enable peer-to-peer voice and / or video communication between the client computer and a remote computer, as opposed to voice and / or video communication by the communication application, including receiving redirected communications from a communication application, using a framework for enabling real-time communications, connecting the client computer and the remote computer for voice and / or video communication based on the redirected communications, determining the hardware-based media processing capabilities of the client computer, and negotiating media communication parameters with the remote computer based on the hardware-based media processing capabilities.

[0083]

[0090] In one embodiment, the media includes either video or audio.

[0084]

[0091] In one embodiment, the method further includes determining another hardware-based video processing capability of the client computer, receiving media transmitted from the remote computer, and processing the received media from the remote computer using the other hardware-based media processing capability for rendering by the client computer.

[0085]

[0092] In one embodiment, the method further includes determining another hardware-based media processing capability of the client computer, using the other hardware-based media processing capability to process media captured from a media source of the client computer, and transmitting the processed media to a remote computer.

[0086]

[0093] In one embodiment, the hardware-based media processing capability includes one of either a video codec or an audio codec.

[0087]

[0094] In one embodiment, the hardware-based media processing capability includes either hardware-accelerated video processing or hardware-accelerated audio processing.

[0088] V. Conclusion

[0095] While various embodiments of the present invention have been described above, it should be understood that these are merely examples and not limitations. Those skilled in the art will understand that various modifications in form and detail can be made without departing from the spirit and scope of the invention as defined in the appended claims. Accordingly, the breadth and scope of the present invention should not be limited by any of the above exemplary embodiments, but should be determined solely in accordance with the appended claims and their equivalents.

Claims

1. At least one processor, At least one memory device for storing computer program logic and A first computing device including, where the computer program logic, when executed by the at least one processor, the first computing device, Receiving a redirected request from a communication application running within a network computing environment, wherein the redirected request relates to voice and / or video communication between the first computing device and the second computing device, Based on the redirected request, identify the hardware-based media processing capability of the first computer, wherein the hardware-based media processing capability includes at least one of a video codec, an audio codec, hardware-accelerated video processing, or hardware-accelerated audio processing, and the hardware-based media processing capability of the first computer has the corresponding software-based media processing capability of the communication application. Instead of the corresponding software-based media processing capabilities of the communication application, the hardware-based media processing capabilities of the first computer are used to process media as part of peer-to-peer audio and / or video communication between the first computer and the second computer. A first computing device that executes the following.

2. The first computing device according to claim 1, wherein the communication application is prompted to initiate communication with the second computing device through interaction with a user interface.

3. The first computing device according to claim 1, wherein the computer program logic further causes the first computing device to negotiate media communication parameters with the second computing device based on the hardware-based media processing capability.

4. The first computer according to claim 1, wherein the hardware-based media processing capability is identified by an application programming interface (API) of the operating system of the first computer.

5. The first computing device according to claim 3, wherein the media communication parameters include a type of decoder that can be used to determine the media format of the peer-to-peer audio and / or video communication.

6. The first computing device according to claim 1, wherein the redirected request includes a web real-time communication (WebRTC) API call intercepted from the communication application.

7. The first computing device according to claim 1, wherein the computer program logic further causes the first computing device to load a plug-in that intercepts the redirected requests from the communication application.

8. A method performed by a first computing device, Receiving a redirected request from a communication application running within a network computing environment, wherein the redirected request relates to voice and / or video communication between the first computing device and the second computing device, At least one processor identifies the hardware-based media processing capability of the first computer based on the redirected request, wherein the hardware-based media processing capability includes at least one of a video codec, an audio codec, hardware-accelerated video processing, or hardware-accelerated audio processing, and the hardware-based media processing capability of the first computer has the corresponding software-based media processing capability of the communication application. Instead of the corresponding software-based media processing capabilities of the communication application, the hardware-based media processing capabilities of the first computer are used to process media as part of peer-to-peer audio and / or video communication between the first computer and the second computer. Methods that include...

9. The method according to claim 8, wherein the communication application is prompted to initiate communication with the second computing device through interaction with a user interface.

10. Negotiate media communication parameters with the second computing device based on the hardware-based media processing capabilities. The method according to claim 8, further comprising:

11. The method according to claim 10, wherein the media communication parameters include a type of decoder that can be used to determine the media format of the peer-to-peer audio and / or video communication.

12. The method according to claim 8, wherein the hardware-based media processing capability is identified by an application programming interface (API) of the operating system of the first computer.

13. The method according to claim 8, wherein the redirected request includes a web real-time communication (WebRTC) API call that is intercepted from the communication application.

14. Loading a plugin that intercepts the redirected request from the aforementioned communication application. The method according to claim 8, further comprising:

15. When executed by at least one processor of the first computing device, the first computing device: Receiving a redirected request from a communication application running within a network computing environment, wherein the redirected request relates to voice and / or video communication between the first computing device and the second computing device, Based on the redirected request, identify the hardware-based media processing capability of the first computer, wherein the hardware-based media processing capability includes at least one of a video codec, an audio codec, hardware-accelerated video processing, or hardware-accelerated audio processing, and the hardware-based media processing capability of the first computer has the corresponding software-based media processing capability of the communication application. Instead of the corresponding software-based media processing capabilities of the communication application, the hardware-based media processing capabilities of the first computer are used to process media as part of peer-to-peer audio and / or video communication between the first computer and the second computer. A computer-readable storage medium on which program instructions for executing a program are recorded.

16. The computer-readable storage medium according to claim 15, wherein interaction with a user interface prompts the communication application to initiate communication with the second computing device.

17. The computer-readable storage medium according to claim 15, wherein the program instructions further cause the first computing device to negotiate media communication parameters with the second computing device based on the hardware-based media processing capability.

18. The computer-readable storage medium according to claim 17, wherein the media communication parameters include a type of decoder that can be used to determine the media format of the peer-to-peer audio and / or video communication.

19. The computer-readable storage medium according to claim 15, wherein the hardware-based media processing capability is identified by an application programming interface (API) of the operating system of the first computer.

20. The computer-readable storage medium according to claim 15, wherein the redirected request includes a web real-time communication (WebRTC) API call intercepted from the communication application.

21. One or more processors, One or more memory devices for storing computer program logic to be executed by the one or more processors mentioned above, A client computing device including, wherein the computer program logic is A remote desktop client configured to present the user interface of a communication application running in a cloud computing environment within the user interface of the client computing device, further configured to receive redirected requests from the communication application to enable peer-to-peer voice and / or video communication between the client computing device and the remote computing device, separate from voice and / or video communication by the communication application, wherein the redirected requests relate to voice and / or video communication between the client computing device and the remote computing device, Based on the redirected request, the hardware-based media processing capability of the client computer is determined, and without using the software media processing capability of the communication application, the media is processed using the hardware-based media processing capability as part of peer-to-peer audio and / or video communication between the client computer and the remote computer, and the processed media is provided to the real-time communication component for transmission to the remote computer, wherein the hardware-based media processing capability includes one of a video codec, an audio codec, hardware-accelerated video processing, or hardware-accelerated audio processing. To inform the source reader of information relating to the use of one of the video codec, audio codec, hardware-accelerated video processing, or hardware-accelerated audio processing in order to encode the media as part of the peer-to-peer audio and / or video communication. Media capture component configured to perform A client computing device, including one.

22. The system further includes a real-time communications manager configured to receive the redirected requests from the remote desktop client and translate the redirected requests into a format compatible with a framework for enabling real-time communications, The real-time communication component is configured to connect the client computer and the remote computer for audio and / or video communication of the media based on the converted request. The client computing device according to claim 21.

23. The aforementioned real-time communication manager Determine the other hardware-based video processing capabilities of the client computing device. The remote computing device negotiates media communication parameters based on the aforementioned hardware-based media processing capabilities. The client computing device according to claim 22, further configured as follows.

24. The client computing device according to claim 23, wherein the real-time communication manager determines the other hardware-based media processing capabilities by APIs of the framework for enabling real-time communication.

25. The media renderer component further includes a media renderer component configured to receive remote media transmitted from the remote computing device by the real-time communication component and to provide the received remote media to the media engine of the client computing device for rendering within the user interface of the client computing device, wherein the media renderer component Determine the other hardware-based media processing capabilities of the client computing device. The other media processing capabilities are used to process the remote media received from the remote computing device. The client computing device according to claim 22, further configured as follows.

26. The client computing device according to claim 21, wherein the redirected request includes a web real-time communication (WebRTC) API call intercepted from the communication application.

27. The client computing device according to claim 21, further comprising an operating system plugin, wherein the plugin provides the media capture component.

28. A method performed by one or more processors of a client computing device that presents a user interface for a communication application running in a cloud computing environment, In addition to the voice and / or video communication by the aforementioned communication application, the system receives a redirected request from the communication application to enable peer-to-peer voice and / or video communication between the client computer and the remote computer, wherein the redirected request relates to voice and / or video communication between the client computer and the remote computer. Based on receiving the redirected request, determine the hardware-based media processing capability of the client computer to be used when processing media as part of peer-to-peer audio and / or video communication between the client computer and the remote computer, without using the software media processing capability of the communication application, wherein the hardware-based media processing capability includes one of a video codec, an audio codec, hardware-accelerated video processing, or hardware-accelerated audio processing. To inform the source reader of information relating to the use of one of the video codec, audio codec, hardware-accelerated video processing, or hardware-accelerated audio processing in order to decode or process the media as part of the peer-to-peer audio and / or video communication, Receiving the media from the remote computing device as peer-to-peer audio and / or video communication, To process the media from the remote computer for rendering by the client computer using the aforementioned hardware-based media processing capabilities. Methods that include...

29. Connecting the client computer and the remote computer for voice and / or video communication based on the redirected request using a framework for enabling real-time communication. The method according to claim 28, further comprising:

30. To determine the other hardware-based video processing capabilities of the client computing device, Negotiating media communication parameters with the remote computing device based on the aforementioned hardware-based media processing capabilities. The method according to claim 29, further comprising:

31. Determining the other hardware-based media processing capabilities includes determining the other hardware-based media processing capabilities using the API of the framework for enabling real-time communication. The method according to claim 30.

32. To determine the other hardware-based media processing capabilities of the client computing device, Using the other hardware-based media processing capabilities to process the source media captured from the media source of the client computing device, Transmitting the processed source media to the remote computing device. The method according to claim 29, further comprising:

33. To determine the hardware-based media processing capability of the client computer, the operating system is plugged in. The method according to claim 29, further comprising:

34. When executed by at least one processor of a client computing device that presents a user interface for a communication application running in a cloud computing environment, the at least one processor: In addition to the voice and / or video communication by the aforementioned communication application, the system receives a redirected request from the communication application to enable peer-to-peer voice and / or video communication between the client computer and the remote computer, wherein the redirected request relates to voice and / or video communication between the client computer and the remote computer. Based on receiving the redirected request, determine the hardware-based media processing capability of the client computer to be used when processing media as part of peer-to-peer audio and / or video communication between the client computer and the remote computer, without using the software media processing capability of the communication application, wherein the hardware-based media processing capability includes one of a video codec, an audio codec, hardware-accelerated video processing, or hardware-accelerated audio processing. To inform the source reader of information relating to the use of one of the video codec, audio codec, hardware-accelerated video processing, or hardware-accelerated audio processing in order to decode or process the media as part of the peer-to-peer audio and / or video communication, Negotiating media communication parameters with the remote computing device based on the aforementioned hardware-based media processing capabilities. A non-temporary, computer-readable storage medium on which program instructions are recorded that cause a method to be executed, including the method described.

35. The method further comprises connecting the client computer and the remote computer for voice and / or video communication based on the redirected request using a framework for enabling real-time communication, according to claim 34, for a non-temporary computer-readable storage medium.

36. The aforementioned method, To determine the other hardware-based video processing capabilities of the client computing device, Receiving remote media transmitted from the aforementioned remote computing device, Using the aforementioned other hardware-based media processing capabilities, the remote media received from the remote computing device for rendering by the client computing device is to be processed by the client computing device. A non-temporary computer-readable storage medium according to claim 35, further comprising:

Images