Method for sharing cloud desktop to conference television system, cloud desktop terminal and cloud desktop

Abstract

The application discloses a cloud desktop sharing method to a conference television system, a cloud desktop terminal and the cloud desktop. The cloud desktop sharing method to the conference television system comprises the following steps: establishing the connection between the cloud desktop client and the conference television system; converting the desktop stream of the cloud desktop client into the auxiliary stream supported by the conference television system through the cloud desktop server; and sending the converted auxiliary stream supported by the conference television system to the conference television system through the cloud desktop client. The conference processing module is introduced into the cloud desktop client, the conference processing module is responsible for registering and initiating the call to the conference television system, and the cloud desktop is accessed to the conference television system. One auxiliary stream is encoded and sent to the conference television system through the cloud desktop server, so that the resource advantage of the cloud desktop server is fully utilized for encoding, the sharing desktop is realized to the conference television terminal, and the unclear and lagging desktop stream caused by the poor cloud desktop terminal is avoided.

Description

Technical Field

[0001] This invention relates to the fields of cloud desktops and video conferencing. By combining cloud desktops and video conferencing, the cloud desktop stream can be shared with the conferencing television terminal, enabling users to join the meeting and share the desktop anytime, anywhere. Background Technology

[0002] Cloud desktops, with their anytime, anywhere accessibility and unified, convenient maintenance and management, are gaining increasingly widespread adoption. Traditional video conferencing systems, however, are limited by terminal conditions. They typically deploy terminals in fixed meeting rooms, requiring users to be in or join meetings from within the room. Furthermore, sharing personal computers or cloud desktops requires connecting the computer to the conferencing terminal and sharing the desktop via dual-stream. While some video conferencing client software now allows users to join meetings and share desktops within cloud desktops, this method generally involves taking screenshots within the cloud desktop and then encoding and sending them to the conferencing system. This approach has several drawbacks: first, encoding screenshots consumes significant resources, potentially affecting smoothness if the virtual machine or terminal has poor performance; second, the clarity of the screenshots after encoding is compromised, especially for densely packed images or graphic design software; and third, screenshots may leave temporary files locally, posing security risks. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a method, cloud desktop terminal and cloud desktop that fully utilizes cloud desktop server resources, shares a desktop stream to a video conferencing terminal through server encoding, and achieves clear and smooth sharing of cloud desktops anytime and anywhere.

[0004] The present invention solves the above-mentioned technical problems by adopting the following technical solution: A method for sharing a cloud desktop to a video conferencing system, characterized by comprising: Establish a connection between the cloud desktop client and the video conferencing system; The cloud desktop client's desktop stream is converted into an auxiliary stream supported by the video conferencing system via a cloud desktop server. The cloud desktop client sends the converted auxiliary stream, which is supported by the video conferencing system, to the video conferencing system.

[0005] A method for sharing a cloud desktop to a video conferencing system includes: Establish a connection between the video conferencing system and the cloud desktop client; The cloud desktop server receives the desktop stream from the cloud desktop client and converts it into an auxiliary stream supported by the video conferencing terminal.

[0006] A cloud desktop client, characterized in that it includes: The meeting processing module is used for: Registration of the video conferencing system and connection of the registered cloud desktop client to the video conferencing system; Submit a sharing request to the video conferencing system or accept a sharing invitation from the video conferencing system; Initiate a desktop stream conversion request to the cloud desktop server and receive the converted auxiliary stream; Send the converted auxiliary stream to the video conferencing system.

[0007] A cloud desktop, comprising a cloud desktop server and N cloud desktop clients, characterized in that: The cloud desktop client includes: The meeting processing module is used for: Registration of the video conferencing system and connection of the registered cloud desktop client to the video conferencing system; Submit a sharing request to the video conferencing system or accept a sharing invitation from the video conferencing system; Initiate a desktop stream conversion request to the cloud desktop server and receive the converted auxiliary stream; Send the converted auxiliary stream to the video conferencing system; The cloud desktop server includes: The conversion module converts the desktop stream of the cloud desktop client into an auxiliary stream supported by the video conferencing system, and sends the converted auxiliary stream to the cloud desktop client.

[0008] Compared with the prior art, the beneficial effects of the present invention are: This invention introduces a conference processing module into the cloud desktop client and modifies the cloud desktop server. The conference processing module is responsible for registering with the video conferencing system and initiating a call to connect the cloud desktop to the system. Since the conference terminal does not support the cloud desktop's desktop stream, the RAP / ICE server needs modification. Upon receiving the desktop sharing message from the conference processing module, the cloud desktop server, in addition to sending the main desktop stream, also encodes an auxiliary desktop stream and sends it to the video conferencing system. Since the cloud desktop itself requires server-side calls to virtualization interfaces for encoding and decoding the desktop stream, combining the cloud desktop and the video conferencing system allows the cloud desktop server to encode an additional desktop stream for the video conferencing system. This enables desktop sharing with the conference room's video conferencing terminals without being limited by the cloud desktop client's terminal resources. If the cloud desktop server has a GPU, it can also utilize the GPU for hardware encoding, fully utilizing the cloud desktop server's resources while maintaining clarity. Furthermore, all encoding processes are completed on the server side, eliminating security concerns. Attached Figure Description

[0009] Figure 1 This is the overall system framework diagram; Figure 2 This is a structural diagram of the meeting processing module; Figure 3 This refers to the process from the meeting processing module to the meeting AS registration process; Figure 4 The meeting processing module is added to the meeting workflow; Figure 5 It is the process of sharing the desktop to the video conferencing terminal. Detailed Implementation

[0010] To make the objectives and technical advantages of this solution clearer, the following detailed description is provided in conjunction with the accompanying drawings and examples. It should be understood that all other embodiments obtained by those skilled in the art based on the examples in this invention without inventive effort are within the scope of protection of this invention.

[0011] In the attached diagram of the instruction manual: The meeting processing module is the module in the cloud desktop client that handles meetings, and is responsible for processing meeting-related workflows.

[0012] The cloud desktop server (also known as the cloud desktop server-side) is responsible for processing the desktop stream, encoding the desktop display image frames into a desktop stream and sending it to the cloud desktop client.

[0013] A graphics processing unit (GPU) is a hardware graphics card device on a cloud desktop server. Some servers do not have one, and in such cases, the server uses software encoding. If the GPU is present, the cloud desktop server uses the GPU for hardware encoding, improving encoding efficiency.

[0014] The cloud desktop client is responsible for decoding the desktop stream sent by the cloud desktop server and presenting it to the client.

[0015] The conference server is responsible for organizing and managing audio and video conferences, as well as negotiating the media streams for the conference.

[0016] The media control unit is responsible for audio and video conferencing mixing, screen mixing, and the receiving and sending of auxiliary streams.

[0017] A video conferencing terminal is a hardware terminal used to connect to a conference room. Unless otherwise specified, the meaning of each module follows this definition.

[0018] The following is in conjunction with the appendix Figure 1 The invention will be described in detail below: This invention relates to a method for sharing a cloud desktop to a video conferencing system, which is used by a cloud desktop client to share its desktop to the video conferencing system. The structural block diagram of the cloud desktop client is shown below. Figure 1 As shown, it includes: The meeting processing module is used for: Registration of the video conferencing system and connection of the registered cloud desktop client to the video conferencing system; Submit a sharing request to the video conferencing system or accept a sharing invitation from the video conferencing system; Initiate a desktop stream request to the cloud desktop server and receive the shared desktop stream; Send the converted auxiliary stream to the video conferencing system.

[0019] The decoding module is responsible for decoding the desktop stream from the cloud desktop and sending the decoded image to the client for display.

[0020] The video processing module is responsible for capturing and encoding images from the local camera and sending them to the video conferencing system, as well as decoding and displaying the video stream sent by the video conferencing system to the cloud desktop terminal.

[0021] The audio processing module is responsible for capturing and encoding the sound from the local microphone and sending it to the video conferencing system, as well as decoding and playing the audio stream sent from the video conferencing system to the terminal.

[0022] The connection module is responsible for establishing and releasing the connection between the cloud desktop client and the cloud desktop server.

[0023] The peripheral processing module is responsible for redirecting external devices to the desktop, such as USB flash drives, printers, and other interface devices.

[0024] The meeting processing module is composed of the following: Figure 2 As shown, it includes the following modules: Control module: Responsible for controlling the overall process, including initiating cloud desktop client user registration, initiating the meeting joining process, and notifying the cloud desktop server to send auxiliary streams; SIP (Session Initiation Protocol) module: responsible for sending and receiving SIP protocol packets, establishing a media channel between the conference processing module of the cloud desktop client and the media processing unit of the video conferencing system; HTTP (Hypertext Transfer Protocol) module protocol processing block: responsible for the sending and receiving of HTTP messages, and returns the audio and video media addresses and auxiliary stream sending media addresses of the media processing unit to the conference processing module from the conference server; Audio and video processing module: After joining an audio and video conference, it is responsible for encoding and decoding audio and video streams; Dual-stream processing module: processes requests for and releases of auxiliary streams to the cloud desktop server; upon receiving the auxiliary stream request, the cloud desktop server encodes an H264 CIF desktop stream and sends it to the cloud desktop terminal. Stream processing module: Responsible for processing desktop streams. After receiving the H264 CIF desktop stream from the cloud desktop server, it sends the desktop stream to the auxiliary stream channel of the media processing unit.

[0025] The video conferencing system includes: Conference server; responsible for organizing and managing audio and video conferences, and negotiating conference media streams.

[0026] The media control unit is responsible for audio and video conferencing mixing, screen mixing, and the receiving and sending of auxiliary streams.

[0027] as well as Several video conferencing terminals, the hardware terminals for connecting to the conference in the conference room.

[0028] The method for sharing a cloud desktop to a video conferencing system according to the present invention is as follows: Establishing a connection between the cloud desktop client and the video conferencing system includes: Step 1: First, create an account for the cloud desktop user on the conference server of the video conferencing system; the conference processing module initiates the registration process to the conference server; after successful registration, a conference is started or the user joins. Step 2: The conference processing module initiates or joins a conference process, the conference server responds to the request, and the two network elements complete media negotiation through the SIP protocol; after the media negotiation is completed, the conference server will return the audio and video media addresses and auxiliary stream transmission media addresses of the media processing unit; after the addresses are sent, the cloud desktop client and the media processing unit interact to transmit audio and video streams. Step 3: When it is necessary to share the desktop of the cloud desktop client to the conferencing terminal of the conferencing system, the conference processing module initiates an auxiliary stream request to the conference server. After the request is successful, the conference processing module notifies the cloud desktop server to encode a separate H264 CIF desktop stream and send it to the cloud desktop terminal. After receiving the desktop stream, the conference processing module forwards it directly to the auxiliary stream channel of the media processing unit without decoding. The media processing unit then forwards the auxiliary stream to the conferencing terminal, and the conferencing terminal displays the desktop to the user.

[0029] The registration process from the meeting processing module to the meeting server, such as... Figure 3 As shown, the specific steps are as follows: Step 1: The conference processing module sends a message to the conference server to obtain user information and SIP registration address. After successful authentication, the conference server returns the conference server's SIP registration address.

[0030] Step 2: The conference processing module sends a SIP registration message to the conference server to register. After successful authentication, the conference server returns a registration success response. Step 3: SIP registration is successful. The conference processing module sends a message to retrieve the address book so that frequently used terminals can be invited to join the conference in subsequent meetings.

[0031] The meeting processing module incorporates the meeting workflow, such as... Figure 4 As shown, the specific steps are as follows: Step 1: The conference processing module initiates a SIP call request to start or join a conference. The SIP call request carries the local media communication address. After receiving the call request, the conference server determines whether the user is a registered user and returns intermediate responses such as ringing and connection response. The response message carries the audio and video media addresses of the media processing unit and the media addresses of the conference auxiliary stream. Step 2: After media negotiation is completed, the conference processing module sends audio and video streams to the media processing unit. At the same time, the media processing unit also sends the mixed audio and video streams to the terminal. After decoding, the terminal can display the conference sound and images normally.

[0032] The process of sharing the desktop to the video conferencing terminal, such as... Figure 5 As shown, the steps are as follows: Step 1: When a cloud desktop client user initiates sharing the desktop to the video conferencing terminal, the conference processing module first sends an auxiliary stream request to the media processing unit. When the media processing unit determines that no other video conferencing terminal is currently sending an auxiliary stream, it returns a successful request message to the conference processing module. Step 2: After receiving the successful auxiliary stream request message, the conference processing module sends a notification message to the cloud desktop server. Upon receiving the message, the cloud desktop server, in addition to the normal desktop stream, separately encodes an H264 CIF format desktop stream supported by the video conferencing terminal and sends it to the cloud desktop client.

[0033] If the service has a GPU, the server uses the GPU for hardware encoding to improve encoding efficiency. If the server does not have GPU resources, it directly uses the CPU of the cloud desktop server for software encoding.

[0034] Step 3: After receiving the desktop stream from Step 2, the conference processing module sends the auxiliary stream to the media processing unit through the auxiliary stream channel. Step 4: After receiving the auxiliary stream, the media processing unit forwards the auxiliary stream to the video conferencing terminal; after receiving the auxiliary stream, the video conferencing terminal decodes the desktop stream, at which point the video conferencing terminal can display the image and operation of the entire desktop.

Claims

1. A method for sharing a cloud desktop to a video conferencing system, characterized in that, include: Establish a connection between the cloud desktop client and the video conferencing system; Send a notification message to the cloud desktop server, receive the desktop stream sent by the cloud desktop server, and an auxiliary stream supported by the video conferencing system that is separately encoded by the cloud desktop server; The auxiliary stream is sent directly to the video conferencing system without decoding via the cloud desktop client.

2. The method according to claim 1, characterized in that, Establishing a connection between a cloud desktop client and a video conferencing system includes: a connection request sent by the cloud desktop client or an invitation connection sent by the video conferencing system; The connection requests sent by the cloud desktop client include: The cloud desktop client initiates a connection request to the video conferencing system; The cloud desktop client receives the response from the video conferencing system to the connection request; The cloud desktop client initiates a sharing request to the video conferencing system; The cloud desktop client receives the response from the video conferencing system to the sharing request; The invitation links sent by the video conferencing system include: The cloud desktop client accepts the connection invitation initiated by the video conferencing system; The cloud desktop client accepts the sharing invitation initiated by the video conferencing system.

3. The method according to claim 2, characterized in that, After the cloud desktop client successfully connects to the video conferencing system, the cloud desktop client and the video conferencing system initiate a sharing request; after the sharing request is successful, a notification message is sent to the cloud desktop server.

4. The method according to claim 3, characterized in that, The auxiliary stream is a desktop stream of the H.264 CIF digital video coding standard.

5. The method according to any one of claims 1-4, characterized in that, After receiving the auxiliary stream sent by the cloud desktop client, the video conferencing system forwards it to the video conferencing terminal through the media channel.

6. A cloud desktop client, characterized in that, include: The meeting processing module is used for: Registration of the video conferencing system and connection of the registered cloud desktop client to the video conferencing system; Submit a sharing request to the video conferencing system or accept a sharing invitation from the video conferencing system; Send notification messages to the cloud desktop server and receive desktop streams sent by the cloud desktop server, as well as an auxiliary stream supported by a video conferencing system that is separately encoded by the cloud desktop server; The auxiliary stream is sent directly to the video conferencing system without being decoded.

7. The cloud desktop client according to claim 6, characterized in that, The conference processing module includes: Process control module: Used for controlling the initiation of sessions between the cloud desktop client and the video conferencing system. Notification to the cloud desktop server and control of sending auxiliary streams, Session initiation includes registration initiation and membership application initiation; Protocol packet sending and receiving processing module: used for sending and receiving protocol packets initiated by the process control module session, and connecting the cloud desktop client to the video conferencing system; Hypertext transmission module: used for sending and receiving hypertext information, and sending the address of the auxiliary stream received by the video conferencing system to the cloud desktop terminal; Dual-stream control module: used to request and release dual-stream tokens from the video conferencing system. After requesting the token, it can send auxiliary streams to the video conferencing system. Stream processing module: Used to send the auxiliary stream received from the cloud desktop server to the auxiliary stream channel of the video conferencing system.

8. The cloud desktop client according to claim 7, characterized in that, The conference processing module also includes an audio / video module: after joining an audio / video conference, it is used to encode and decode audio and video streams.

9. The cloud desktop client according to claim 7, characterized in that, The auxiliary stream received by the stream processing module is an H264 CIF desktop stream.

10. The cloud desktop client according to claim 7, characterized in that, Protocol packet transmission and reception processing module: Establishes a media channel between the conference processing module of the cloud desktop client and the multimedia control unit of the video conferencing system, and connects the cloud desktop client to the video conferencing system.

11. The cloud desktop client according to claim 7, characterized in that, The address for receiving the auxiliary stream is the audio and video media address of the multimedia control unit and the auxiliary stream sending media address returned by the conference server to the conference processing module.

12. A cloud desktop, comprising a cloud desktop server and N cloud desktop clients, characterized in that: The cloud desktop client includes: The meeting processing module is used for: Registration of the video conferencing system and connection of the registered cloud desktop client to the video conferencing system; Submit a sharing request to the video conferencing system or accept a sharing invitation from the video conferencing system; Send notification messages to the cloud desktop server and receive desktop streams sent by the cloud desktop server, as well as an auxiliary stream supported by a video conferencing system that is separately encoded by the cloud desktop server; The auxiliary stream is sent directly to the video conferencing system without being decoded; The cloud desktop server includes: The conversion module sends the desktop stream to the cloud desktop client and sends a separately encoded auxiliary stream supported by the video conferencing system to the cloud desktop client.

Images