Method, device and storage medium for implementing video conference

Abstract

The application discloses a video conference implementation method and device and a storage medium. The method comprises the following steps: acquiring video conference main window information in video conference window information; the video conference window information comprises the video conference main window information and video conference auxiliary window information; and the video conference main window information is synchronously displayed with the video conference auxiliary window information, wherein the video conference auxiliary window information is displayed on a cloud desktop server. The technical scheme of the application can optimize the display effect of a video image, and reduce the occupation of network bandwidth resources and cloud server resources.

Description

Technical Field

[0001] This invention relates to the field of cloud computing technology, and in particular to a method, apparatus and storage medium for implementing video conferencing. Background Technology

[0002] Cloud desktop access supports various terminals such as PCs, thin clients, mobile phones, and tablets, and is compatible with multiple operating systems such as Windows and Linux. Therefore, cloud desktops can be used for remote work and video conferencing.

[0003] When using cloud desktops for video conferencing, the acquisition and playback of video images and sound not only consume a large amount of network bandwidth and cloud server resources, easily causing the cloud server to be overloaded, but also result in poor video display quality. Summary of the Invention

[0004] The main objective of this invention is to provide a method, apparatus, and storage medium for implementing video conferencing, aiming to reduce the consumption of network bandwidth and cloud server resources and improve video display effects when using cloud desktops for video conferencing.

[0005] In a first aspect, embodiments of the present invention provide a method for implementing video conferencing, comprising the following steps:

[0006] Retrieve the main video conference window information from the video conference window information; the video conference window information includes the main video conference window information and the auxiliary video conference window information.

[0007] The main video conferencing window information and the auxiliary video conferencing window information are displayed synchronously, wherein the auxiliary video conferencing window information is displayed on the cloud desktop server.

[0008] Secondly, embodiments of the present invention provide a method for implementing video conferencing, the method comprising the following steps:

[0009] In response to a video conference join message, the main video conference window information in the video conference window information is forwarded to the cloud desktop client so that the main video conference window information and the auxiliary video conference window information in the video conference window information are displayed synchronously; wherein, the join message is a message sent by the participant through the cloud desktop client to confirm joining the video conference, the main video conference window information is displayed on the cloud desktop client, and the auxiliary video conference window information is displayed on the cloud desktop server.

[0010] Thirdly, embodiments of the present invention provide a video conferencing implementation apparatus, comprising:

[0011] The client acquisition module is used to acquire the main video conferencing window information from the video conferencing window information; the video conferencing window information includes the main video conferencing window information and the auxiliary video conferencing window information.

[0012] The display module is used to synchronize the display of the main video conferencing window information and the auxiliary video conferencing window information, wherein the auxiliary video conferencing window information is displayed on the cloud desktop server.

[0013] Fourthly, embodiments of the present invention provide a video conferencing implementation apparatus, comprising:

[0014] The sending module is used to join the video conference by forwarding the main video conference window information from the video conference window information to the cloud desktop client, so that the main video conference window information and the auxiliary video conference window information are displayed synchronously; wherein, the main video conference window information is displayed on the cloud desktop client, and the auxiliary video conference window information is displayed on the cloud desktop server.

[0015] Fifthly, embodiments of the present invention provide a storage medium for computer-readable storage, the storage medium storing one or more programs, the one or more programs being executable by one or more processors to implement the steps of the aforementioned method.

[0016] The video conferencing implementation method and apparatus provided in this embodiment of the invention, after obtaining the main video conferencing window information from the video conferencing window information, displays the main video conferencing window information on the cloud desktop client and displays the auxiliary video conferencing window information on the cloud desktop server. Moreover, the main video conferencing window information and the auxiliary video conferencing window information are displayed synchronously, thereby optimizing the display effect of video images and reducing the occupation of network bandwidth resources and cloud server resources. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of a video conferencing system provided in an embodiment of the present invention.

[0018] Figure 2 This is a flowchart illustrating the video conferencing implementation method according to an embodiment of the present invention;

[0019] Figure 3 A flowchart illustrating another method for implementing video conferencing as provided in an embodiment of the present invention;

[0020] Figure 4 A flowchart illustrating yet another method for implementing video conferencing as provided in an embodiment of the present invention;

[0021] Figure 5 A flowchart illustrating another method for implementing video conferencing as provided in an embodiment of the present invention;

[0022] Figure 6 A flowchart illustrating another method for implementing video conferencing as provided in an embodiment of the present invention;

[0023] Figure 7 This is a flowchart illustrating the video conferencing implementation method according to an embodiment of the present invention;

[0024] Figure 8 A flowchart illustrating another method for implementing video conferencing as provided in an embodiment of the present invention;

[0025] Figure 9 A flowchart illustrating another method for implementing video conferencing as provided in an embodiment of the present invention;

[0026] Figure 10 This is a schematic block diagram of a video conferencing implementation device according to an embodiment of the present invention;

[0027] Figure 11 A schematic block diagram of a video conferencing implementation device provided in an embodiment of the present invention;

[0028] Figure 12 This is a flowchart illustrating the interaction between the video conferencing module, the cloud desktop client, and the cloud desktop server in an embodiment of the present invention.

[0029] Figure 13 This is a flowchart illustrating the process of inviting participants to a video conference in an embodiment of the present invention;

[0030] Figure 14 This is a flowchart illustrating desktop sharing during collaborative work, as described in an embodiment of the present invention.

[0031] Figure 15 This is a flowchart illustrating the process of obtaining the desktop stream from the media resource center in an embodiment of the present invention;

[0032] Figure 16 This is a flowchart illustrating the application of sharing in an embodiment of the present invention;

[0033] Figure 17 A flowchart illustrating the shared licensing of embodiments of the present invention;

[0034] Figure 18 A schematic block diagram of an electronic device is provided for an embodiment of the present invention. Detailed Implementation

[0035] To enable those skilled in the art to better understand the technical solutions of the present invention, the implementation method, equipment and storage medium of the video conferencing provided by the present invention will be described in detail below with reference to the accompanying drawings.

[0036] Exemplary embodiments will be described more fully below with reference to the accompanying drawings; however, these exemplary embodiments may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that the invention will be thorough and complete, and that those skilled in the art will fully understand the scope of the invention.

[0037] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0038] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used herein, the singular forms “a” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded.

[0039] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having the meaning consistent with their meaning in the context of the relevant art and the invention, and will not be interpreted as having an idealized or overly formal meaning unless expressly so defined herein.

[0040] Traditional desktop sharing methods involve first taking a screenshot of the shared desktop content, then encoding the screenshot and sending it to the member's device, which then decodes and displays it. In a cloud desktop scenario, using the same screenshot, encoding, and decoding method for desktop sharing not only consumes cloud server system resources and increases server load, but also results in poor video display quality.

[0041] The video conferencing implementation method provided in this embodiment is to decouple the display part of the video conferencing system to the terminal, that is, the video acquisition and playback are both implemented on the terminal, so as to reduce the number of times the desktop stream travels back and forth between the terminal and the cloud desktop, thereby reducing the consumption of network bandwidth resources and reducing the consumption of cloud server resources.

[0042] Figure 1 This is a block diagram illustrating the principle of a video conferencing system provided in an embodiment of the present invention. In the diagram, solid lines represent desktop streams and media, and dashed lines represent signaling. Figure 1 As shown, the video conferencing system includes a cloud desktop client 11 and a cloud desktop server 12. The cloud desktop client 11 is set on a terminal 10, such as a mobile phone, tablet, computer, or other smart device. When a user needs to conduct a video conference, they can start the cloud desktop client 11 on the terminal 10.

[0043] In this embodiment, the cloud desktop client 11 is mainly used for initiating video conferences and managing video conference participants. The cloud desktop client 11 includes a video conferencing module 111, which comprises a media display unit (client conference interface), a message center unit, and a main control unit. The client conference interface only includes the main window of the video conference and does not include auxiliary windows. In this embodiment, auxiliary windows include, but are not limited to, the video conference menu bar. It is easy to understand that the cloud desktop client 11 only displays information from the main window excluding the menu bar.

[0044] In this embodiment, the cloud desktop server 12 includes a server interface 121, a cloud media proxy 122, and a virtual machine component 123. The server interface 121 is used to display video conferencing auxiliary window information, such as the video conferencing menu bar, and the video conferencing auxiliary window information is displayed synchronously with the video conferencing main window information displayed on the cloud desktop client 11. The cloud media proxy 122 can transmit information with the video conferencing module 111 of the cloud desktop client 11, such as transmitting desktop streams and receiving video conferencing invitations from the video conferencing module 111. The virtual machine component 123 is used to transmit connection parameters when the cloud media proxy 122 and the video conferencing module 111 establish a connection, notify the cloud media proxy 122 when the connection is lost, and initiate a reconnection request when a disconnection with the video conferencing module 11 is detected.

[0045] In this embodiment, the video conferencing system also includes a media resource center 13 and a video conferencing server (MCU) 14. The MCU 14 sends media-related parameters, such as the MCU address, port number, and media reception format, to the video conferencing module 11. The cloud media agent 122 sends main window messages to the video conferencing module 11, including window maximization, minimization, movement, dragging, and obstruction position information for the main window. During desktop sharing, the cloud media agent 122 also requests the sharer's desktop stream from the cloud desktop server and forwards the desktop stream to the media resource center 13. The media resource center 13 is used to send the shared desktop stream to the cloud desktop clients of other participants.

[0046] In a first aspect, embodiments of the present invention provide a method for implementing video conferencing, which can be applied to cloud desktop clients. Figure 2 This is a flowchart illustrating a video conferencing implementation method according to an embodiment of the present invention. (Refer to...) Figure 2 The implementation of video conferencing includes the following steps:

[0047] Step 201: Obtain the main video conference window information from the video conference window information.

[0048] In this embodiment, the video conferencing window information includes main video conferencing window information and auxiliary video conferencing window information; wherein, the main video conferencing window information includes the specific content of the video conferencing, such as the materials displayed in the video conferencing. The auxiliary video conferencing window information includes the menu bar of the video conferencing.

[0049] Step 202: Synchronize the display of the main video conferencing window information and the auxiliary video conferencing window information, wherein the auxiliary video conferencing window information is displayed on the cloud desktop server.

[0050] In this embodiment, the main video conference window information is displayed on the cloud desktop client, while the auxiliary video conference window information is displayed on the cloud desktop server. In other words, the cloud desktop client only displays the main video conference window information and does not display the auxiliary video conference window information, while the cloud desktop server only displays the auxiliary video conference window information. This embodiment offloads the video conference display portion to the terminal, reducing the amount of data transmission between the cloud desktop client and the cloud server, saving network bandwidth resources, reducing the load on the cloud server, and improving the display effect of the cloud desktop client.

[0051] In some embodiments, the main video conferencing window information and the auxiliary video conferencing window information are simultaneously sent to the cloud desktop client and the cloud desktop server, respectively. This enables the cloud desktop client to display the acquired main video conferencing window information in real time, while the cloud desktop server displays the acquired auxiliary video conferencing window information in real time, thereby ensuring that the main video conferencing window information and the auxiliary video conferencing window information are displayed synchronously.

[0052] In some embodiments, the main window information of the video conference includes main window position information and main window size information. This main window information is sent to the cloud desktop client via a cloud media proxy and displayed by the media display unit in the video conferencing module. When the position and size of the main window of the video conference change, the window of the video conferencing module changes accordingly; for example, when the main window moves from the left to the right, the window of the video conferencing module changes accordingly. When the main window of the video conference is minimized, the window of the video conferencing module moves off-screen.

[0053] In some embodiments, when the main window of a video conference is obscured by other windows, the cloud desktop client obtains the window obscuration information simultaneously with the main video conference window information. When the main video conference window information and the auxiliary video conference window information are displayed synchronously, the obscured portion of the main video conference window is not displayed.

[0054] For example, when the main window of the video conference is obscured by other windows, the video conference module window will crop the corresponding obscured area; when the main window of the video conference is completely obscured by other windows, the video conference module window will be moved out of the terminal's screen range.

[0055] Figure 3 A flowchart illustrating another method for implementing video conferencing as provided in an embodiment of the present invention. Figure 3 As shown, the methods for implementing video conferencing include:

[0056] Step 301: Create a Transmission Control Protocol (TCP) server based on the video conferencing server's address and the current user's account.

[0057] The video conferencing server address is the address of the video conferencing server located in the cloud, and the current user's account is the account of the user who initiated the video conference.

[0058] In some embodiments, when a user initiates a video conference on the cloud desktop client, the video conferencing module and cloud media proxy set up on the terminal are launched. The address of the video conferencing server and the username (account) of the user currently logged into the video conferencing server are passed to the video conferencing module. The video conferencing module creates a TCP server based on the address of the video conferencing server and the current user's account, and returns the video conferencing module's listening port to the cloud desktop client.

[0059] The video conferencing server address can be entered directly by the user on the terminal or selected through options. The IP address and listening port of the cloud desktop server, the IP address and listening port of the media resources, and the listening port of the virtual machine component are passed to the cloud media agent as startup parameters. The cloud media agent establishes a connection with the virtual machine component through these startup parameters.

[0060] Step 302: Establish a connection with the video conferencing server via the TCP server.

[0061] In step 302, the cloud desktop client sends the video conferencing module's listening port and the terminal's IP address to the virtual machine component via the desktop protocol. The virtual machine component then sends the video conferencing module's listening port and the terminal's IP address to the cloud media agent via a TCP link. The cloud media agent establishes a connection with the video conferencing module based on the terminal's IP address and the video conferencing module's listening port. The video conferencing module connects to the video conferencing server and logs in to the video conferencing server using the user account.

[0062] In some embodiments, when the initiator of a video conference initiates the video conference, the cloud desktop client on which the initiator resides further includes:

[0063] Step 303: Send the video conference type and participants to the video conferencing server (MCU) so that the video conferencing server can create the video conference and invite participants.

[0064] The video conferencing types include point-to-point meetings, video conferencing, and audio conferencing. The video conferencing server creates a video conference according to the type of video conference the initiator expects, and then invites meeting members based on the participants provided by the initiator.

[0065] In some embodiments, the initiator (chairperson) of a video conference selects invited participants by querying the address book or recent meeting records. If a video conference does not exist, the initiator can create one on the video conferencing server. For example, the initiator sends the video conference type and participant information to the video conferencing server, which then creates the video conference and invites participants.

[0066] In this embodiment, when creating a video conference, if the video conference creation fails, the video conferencing server returns a failure (ERROR) message to the initiator. If the video conference is created successfully, it returns a success message and the newly created video conference ID to the initiator. The newly created video conference ID can be included in the returned success message; that is, the newly created video conference ID is sent to the initiator along with the successful video conference creation message. Alternatively, the newly created video conference ID and the successful video conference creation message can be sent to the initiator separately.

[0067] After a video conferencing server creates a video conference, the video conferencing module can connect to the server and log in using its username. The server verifies the logged-in user. For example, it first checks if the invited participant is in another meeting. If so, it sends a message to the initiator indicating that the participant cannot attend. If the invited participant is offline or unable to connect for other reasons, it also sends a message to the initiator indicating that the participant cannot attend. Once the video conferencing server identifies eligible participants, it sends them a meeting invitation. Upon receiving the invitation, the member's video conferencing module forwards it to the cloud media agent. The cloud media agent, upon receiving the invitation, can display a pop-up notification. Participants can choose to accept or decline the invitation and send a response message to their own video conferencing module. This module then forwards the response message to the server, which in turn forwards it to the initiator's corresponding video conferencing module.

[0068] Step 304: Receive video conference response messages from meeting members.

[0069] The video conference response messages include messages from invited meeting members indicating whether they wish to attend the meeting. For example, if an invited meeting member chooses to attend, a response message indicating attendance is returned; if an invited meeting member chooses not to attend, a response message indicating refusal to attend is returned.

[0070] In step 304, the initiator of the video conference receives the video conference response messages from the participants through the video conferencing server. Once the initiator receives the video conference response messages from the participants, the video conference can proceed.

[0071] In some embodiments, after logging into the video conferencing system via the cloud desktop client, if the user is not using the video conferencing system, the system will be minimized to the lower right corner of the cloud desktop. When the user requests to join the video conference, a prompt message will pop up in the cloud desktop client.

[0072] In this embodiment, participants in the video conference use a cloud desktop client to send a response message indicating whether they wish to participate in the video conference to the video conference server via the video conferencing module. For example, when a participant receives a video conference invitation message, the video conferencing module returns a video conference response message, which includes whether they accept or decline the video conference request.

[0073] Step 305: Obtain the main video conference window information from the video conference window information.

[0074] Step 306: Synchronize the display of the main video conferencing window information and the auxiliary video conferencing window information, wherein the auxiliary video conferencing window information is displayed on the cloud desktop server.

[0075] Steps 305 and 306 are the same as steps 201 and 202, and will not be repeated here.

[0076] In some embodiments, video conference participants (including the initiator and participants of the video conference) can also collaborate using cloud desktop clients to achieve desktop or application sharing.

[0077] Figure 4 A flowchart illustrating another method for implementing video conferencing as provided in an embodiment of the present invention. Figure 4 As shown, the methods for implementing video conferencing include:

[0078] Step 401: The initiator and participants of the video conference join the video conference.

[0079] In some embodiments, the initiator and participants of a video conference can join the video conference through steps 301 to 304.

[0080] Step 402: Initiate desktop sharing message.

[0081] The desktop sharing message includes the video conference ID and the initiator's account.

[0082] In step 402, both the initiator and participants of the video conference can initiate a desktop sharing request. When a desktop sharing request is initiated, the initiator sends the video conference number and the initiator's account to other participants.

[0083] In some embodiments, the desktop sharing initiator sends the video conference ID and the initiator's account to the video conferencing server. The video conferencing server then sends a desktop sharing request to other members of the video conference. If other members have already enabled desktop sharing, they return an error message to the video conferencing server. If other members have not enabled desktop sharing, it indicates that desktop sharing is possible, and they return an OK message to the video conferencing server.

[0084] Step 403: Establish a connection with the Media Resource Center based on the desktop sharing connection parameters and the account of the desktop sharing initiator.

[0085] The desktop sharing connection parameters are generated by the Media Resource Center based on the video conference ID and the initiator's account.

[0086] The video conferencing server sends the conference ID and initiator account to the media resource center. The media resource center then generates desktop sharing connection parameters and the initiator account, and sends these parameters and the initiator account to the video conferencing server. The video conferencing server then sends these parameters and the initiator account to all members in the same conference room. When the video conferencing module corresponding to a non-initiator receives the desktop sharing connection parameters and the initiator account, the non-initiator's cloud desktop client is launched. The cloud desktop client can then obtain the shared desktop stream from the media resource center based on the desktop sharing connection parameters and the initiator account.

[0087] It should be noted that, in this embodiment, the desktop sharing connection parameters and the initiator's account are sent to the cloud desktop client via the video conferencing server, and the cloud desktop client receives the shared desktop stream. In other embodiments, the desktop sharing connection parameters and the initiator's account are sent to the shared desktop client via the video conferencing server. The shared desktop client is set up on the terminal and communicates with the video conferencing module. The shared desktop client obtains the shared desktop stream from the media resource center based on the desktop sharing connection parameters and the initiator's account.

[0088] In some embodiments, the Media Resource Center obtains the desktop stream of the sharer through various methods. Only one method of desktop stream acquisition is described below. For example, when participants in a video conference wish to share their desktop, they initiate a desktop sharing message to the cloud media agent. After receiving the message, the cloud media agent sends the initiator's account to the cloud desktop server. The cloud desktop server then forwards the desktop stream of the initiator to the cloud media agent, which in turn forwards it to the Media Resource Center. The participants in the video conference can then obtain the desktop stream of the initiator from the Media Resource Center.

[0089] Step 404: Receive the desktop stream from the desktop sharing initiator forwarded by the Media Resource Center.

[0090] When the desktop sharer ends desktop sharing, the desktop sharing initiator sends a message to the video conferencing server to stop desktop sharing. The video conferencing server then forwards the message to the video conferencing modules of the participants in the video conference. The participants' video conferencing modules then send the message to the shared desktop client, which is then closed or minimized.

[0091] When conducting video conferences, it is not desirable to share the entire desktop, but rather to share only a specific application, meaning that participants can only see the video from that application. Figure 5 A flowchart illustrating another method for implementing video conferencing as provided in an embodiment of the present invention. Figure 5 As shown, the methods for implementing video conferencing include:

[0092] Step 501: The initiator and participants of the video conference join the video conference.

[0093] In some embodiments, the initiator and participants of a video conference can join the video conference through steps 301 to 304.

[0094] Step 502: Initiate an application sharing message so that the cloud media agent can encapsulate application window information and occlusion window information.

[0095] Among them, application window information is the information displayed in the application window, such as the content displayed in the application window; obscuring window information is the information about obscuring the application window, such as when the application window is obscured by other windows, in which case only the unobscured part of the application window is displayed.

[0096] In step 502, all participants in the video conference can initiate an application sharing message. The application sharing initiator sends the application sharing message to the cloud media agent, which opens the shared application and sets the window of the shared application as a child window of the cloud media agent. The shared application can be an application already opened by the cloud media agent (e.g., selecting an application from existing applications), or a newly opened application.

[0097] In the application sharing message, the initiator's account is carried. The cloud media agent sends the initiator's account to the cloud desktop server. The cloud desktop server sends the desktop stream corresponding to the initiator's account to the cloud media agent. After receiving the desktop stream, the cloud media agent encapsulates the application window information and the obscuring window information, and sends the encapsulated information to the media resource center. The media resource center sends the encapsulated information to other participants in the video conference, that is, the shared desktop clients of other participants.

[0098] Step 503: Based on the occlusion window information, crop the application window information to obtain the application window display information.

[0099] In step 503, after receiving the encapsulation message, the shared desktop client first decodes the encapsulation message and then trims the application window information according to the occlusion window information to obtain the application window display information, wherein the application window display information is the information of the application window that is not occluded.

[0100] Step 504: Simultaneously display the application window information in the main video conference window.

[0101] In step 504, the shared desktop client displays application window information and moves and resizes the application window based on this information. The main video conferencing window information is displayed on the cloud desktop client, while the auxiliary video conferencing window information is displayed on the cloud desktop server.

[0102] When the application sharing initiator ends the application sharing, it sends an application sharing end message to the video conferencing server. The video conferencing server forwards the application sharing end message to the video conferencing modules of the participants in the video conference, and the participants' video conferencing modules then send it to the shared desktop client.

[0103] When sharing desktops or applications, if participants cannot understand each other's voice communication, such as when participants have different native languages, they can be authorized to modify the documents. Figure 6 A flowchart illustrating another method for implementing video conferencing as provided in an embodiment of the present invention. Figure 6 As shown, the methods for implementing video conferencing include:

[0104] Step 601: The initiator and participants of the video conference join the video conference.

[0105] In step 601, the initiator and participants of the video conference can join the video conference through steps 301 to 304.

[0106] Step 602: Send a document modification authorization message and the authorized member account to verify the authorized member account.

[0107] In this context, an authorized member account is an account granted by the initiator of desktop sharing or application sharing to one or more attendees who can modify documents. The initiator can select authorized members from the list of attendees, and then send the authorized member account and document modification authorization message to the cloud media agent. The cloud media agent forwards the authorized member account and document modification authorization message to the media resource center, which verifies the authorized member account to determine whether authorization is granted to that member.

[0108] Step 603: Modify the authorization information in the response document and provide authentication information so that the Media Resource Center can authenticate the authorized member account.

[0109] In some embodiments, after receiving an authorized member account, the media resource center sends an authorization verification message to the shared desktop client corresponding to that account. The shared desktop client provides authentication information, such as the authorized member's language and identity verification information provided through mouse and keyboard operations. The media resource center then authenticates the authorized member account based on this language and identity verification information. The media resource center sends the authentication information of authenticated members to the cloud desktop server. For member accounts that fail authentication, the media resource center takes no action.

[0110] Step 604: Send the desktop stream to the desktop sharing initiator and the media resource center.

[0111] In step 604, the cloud desktop server sends the desktop stream to the desktop sharing initiator and the media resource center in the video conference. The media resource center then sends the desktop stream to other participants in the video conference. Authorized participants can modify the documents in the desktop stream.

[0112] Step 605: Obtain the main video conference window information from the video conference window information.

[0113] In step 605, the other members of the meeting obtain a desktop stream from the media resource center, which contains information about the main video conference window.

[0114] Step 606: Synchronize the display of information in the main video conference window and the auxiliary video conference window.

[0115] In step 606, the main video conference window information is displayed on the cloud desktop client, and the auxiliary video conference window information is displayed on the cloud desktop server.

[0116] The main window information for a video conference includes its position and size. This information is sent to the cloud desktop client via a cloud media proxy and displayed by the media display unit within the video conferencing module. When the position and size of the main window change, the video conferencing module's window changes accordingly; for example, if the main window moves from the left to the right, the video conferencing module's window will follow suit. When the main window is minimized, the video conferencing module's window moves off-screen.

[0117] The video conferencing implementation method provided in this invention obtains the main video conferencing window information from the video conferencing window information, displays the main video conferencing window information on the cloud desktop client, and displays the auxiliary video conferencing window information on the cloud desktop server. Furthermore, the main and auxiliary video conferencing window information are displayed synchronously, thereby optimizing the display effect of the video image and reducing the consumption of network bandwidth and cloud server resources. In addition, this video conferencing implementation method can offload the display portion of the video conferencing system to the terminal, improving the utilization rate of terminal resources and simplifying additional deployment and configuration in the cloud.

[0118] Secondly, embodiments of the present invention provide a method for implementing video conferencing, which can be applied to cloud media proxies. Figure 7 This is a flowchart illustrating a video conferencing implementation method according to an embodiment of the present invention. (Refer to...) Figure 7 The implementation of video conferencing includes the following steps:

[0119] Step 701: Respond to the video conference join message and receive video conference window information.

[0120] The video conferencing window information includes the main video conferencing window and auxiliary video conferencing windows. The main video conferencing window includes the specific content of the video conferencing, such as the materials being presented. The auxiliary video conferencing windows include the menu bar.

[0121] In step 701, the cloud desktop server sends the video conferencing window information to the cloud media agent, which then sends the video conferencing window information to the media resource center.

[0122] Step 702: Forward the video conference main window information to the cloud desktop client.

[0123] In step 702, the cloud media agent sends the main video conference window information to the media resource center, which then forwards it to the cloud desktop client. The cloud desktop client displays the main video conference window information, while the cloud desktop server displays the auxiliary video conference window information. Furthermore, the main video conference window information and the auxiliary video conference window information are displayed synchronously. In other words, the cloud desktop client only displays the main video conference window information and not the auxiliary window information, while the cloud desktop server only displays the auxiliary window information. This embodiment offloads the video conference display to the terminal, reducing the amount of data transmission between the cloud desktop client and the cloud server, saving network bandwidth resources, reducing the load on the cloud server, and improving the display effect of the cloud desktop client.

[0124] In some embodiments, the main window information of the video conference includes main window position information and main window size information. This main window information is sent to the cloud desktop client via a cloud media proxy and displayed by the media display unit in the video conferencing module. When the position and size of the main window of the video conference change, the window of the video conferencing module changes accordingly; for example, when the main window moves from the left to the right, the window of the video conferencing module changes accordingly. When the main window of the video conference is minimized, the window of the video conferencing module moves off-screen.

[0125] In some embodiments, when the main window of a video conference is obscured by other windows, the cloud desktop client obtains both the main video conference window information and the window obscuration information. When the main video conference window information and the auxiliary video conference window information are displayed synchronously, the main video conference window information, combined with the window obscuration information, is displayed within the video conference module window.

[0126] For example, when the main window of the video conference is obscured by other windows, the video conference module window will crop the corresponding obscured area; when the main window of the video conference is completely obscured by other windows, the video conference module window will be moved out of the terminal's screen range.

[0127] Figure 8 A flowchart illustrating another method for implementing video conferencing as provided in an embodiment of the present invention. Figure 8 As shown, the methods for implementing video conferencing include:

[0128] Step 801: Establish a connection with the video conferencing module based on the IP address and listening port of the cloud desktop client.

[0129] In step 801, when the client initiates a video conference on the cloud desktop client, the video conferencing module and the cloud media agent of the cloud desktop will be launched. The address of the video conferencing server and the account of the user currently logged into the video conferencing server are passed to the video conferencing module. The video conferencing module creates a TCP server based on the address of the video conferencing server and the current user's account, and returns the listening port of the video conferencing module to the cloud desktop client.

[0130] The cloud desktop client passes its IP address and listening port, the media resource center's IP address and listening port, and the virtual machine component's listening port as startup parameters to the cloud media proxy. The cloud media proxy then establishes a connection with the virtual machine component. The cloud desktop client sends the video conferencing module's listening port and the terminal's IP address to the virtual machine component via the desktop protocol. The virtual machine component then sends the video conferencing module's listening port and the terminal's IP address to the cloud media proxy via a TCP link. The cloud media proxy establishes a connection with the video conferencing module based on the terminal's IP address and the video conferencing module's listening port.

[0131] Once the cloud media agent establishes a connection with the video conferencing module's listening port, the video conferencing initiator sends the video conferencing type and participants to the video conferencing server (MCU). The video conferencing server then creates the video conference and invites the participants.

[0132] The video conferencing types include point-to-point meetings, video conferencing, and audio conferencing. The video conferencing server creates a video conference according to the type of video conference the initiator expects, and then invites meeting members based on the participants provided by the initiator.

[0133] In some embodiments, the initiator (chairperson) of a video conference selects invited participants by querying the address book or recent meeting records. If a video conference does not exist, the initiator can create one on the video conferencing server. For example, the initiator sends the video conference type and participant information to the video conferencing server, which then creates the video conference and invites participants.

[0134] In this embodiment, when creating a video conference, if the video conference creation fails, the video conferencing server returns a failure (ERROR) message to the initiator. If the video conference is created successfully, it returns a success message and the newly created video conference ID to the initiator. The newly created video conference ID can be included in the returned success message; that is, the newly created video conference ID is sent to the initiator along with the successful video conference creation message. Alternatively, the newly created video conference ID and the successful video conference creation message can be sent to the initiator separately.

[0135] Step 802: Receive video conference invitation message.

[0136] The video conference invitation message includes the video conference ID and the account of the video conference initiator.

[0137] After receiving the list of participants invited by the video conference initiator, the cloud-based media agent sends participation invitations to the participants to invite them to join the video conference.

[0138] Step 803: Send a video conference invitation message to the participants, who can then join or decline the video conference.

[0139] The participants are those selected by the organizer of the video conference.

[0140] In some embodiments, after receiving a meeting invitation message, the member-side video conferencing module forwards the invitation message to the cloud media agent. Upon receiving the invitation message, the cloud media agent can issue a pop-up notification. Participants can choose to accept or decline the meeting invitation and send a response message to their member-side video conferencing module. The video conferencing module then forwards the response message to the video conferencing server, which in turn forwards it to the video conferencing module corresponding to the initiator.

[0141] Step 804: Send the video conferencing main window information from the video conferencing window information to the video conferencing module.

[0142] In step 804, the cloud media agent sends the main video conference window information from the video conference window information to the video conference module, and the video conference module displays the main video conference window information.

[0143] In this embodiment, the main video conference window information is displayed on the cloud desktop client, while the auxiliary video conference window information is displayed on the cloud desktop server. In other words, the cloud desktop client only displays the main video conference window information and does not display the auxiliary video conference window information, while the cloud desktop server only displays the auxiliary video conference window information. This embodiment offloads the video conference display portion to the terminal, reducing the amount of data transmission between the cloud desktop client and the cloud server, saving network bandwidth resources, reducing the load on the cloud server, and improving the display effect of the cloud desktop client.

[0144] Figure 9 A flowchart illustrating another method for implementing video conferencing as provided in an embodiment of the present invention. For example... Figure 9 As shown, the methods for implementing video conferencing include:

[0145] Step 901: Respond to the desktop sharing request and establish a connection with the cloud desktop server.

[0146] After the initiator and participants of a video conference join, both can initiate a desktop sharing request. When a desktop sharing request is initiated, the initiator sends the video conference ID and the initiator's account to the other participants.

[0147] In some embodiments, the desktop sharing initiator sends the video conference ID and the initiator's account to the video conferencing server. The video conferencing server then sends a desktop sharing request to other members of the video conference. If other members have already enabled desktop sharing, they return an error message to the video conferencing server. If other members have not enabled desktop sharing, it indicates that desktop sharing is possible, and they return an OK message to the video conferencing server.

[0148] The video conferencing server sends the conference ID and initiator account to the media resource center. The media resource center then generates desktop sharing connection parameters and the initiator account, and sends these parameters and the initiator account to the video conferencing server. The video conferencing server then sends these parameters and the initiator account to all members in the same conference room. When the video conferencing module corresponding to a non-initiator receives the desktop sharing connection parameters and the initiator account, the non-initiator's cloud desktop client is launched. The cloud desktop client can then obtain the shared desktop stream from the media resource center based on the desktop sharing connection parameters and the initiator account.

[0149] It should be noted that, in this embodiment, the desktop sharing connection parameters and the initiator's account are sent to the cloud desktop client via the video conferencing server, and the cloud desktop client receives the shared desktop stream. In other embodiments, the desktop sharing connection parameters and the initiator's account are sent to the shared desktop client via the video conferencing server. The shared desktop client is set up on the terminal and communicates with the video conferencing module. The shared desktop client obtains the shared desktop stream from the media resource center based on the desktop sharing connection parameters and the initiator's account.

[0150] Step 902: Receive desktop stream from cloud desktop server.

[0151] The desktop stream is initiated by the desktop sharing provider, and other participants obtain it from the media resource center. This desktop stream contains information about the main video conference window.

[0152] Step 903: Extract application window information and occlusion window information from the desktop stream; where application window information is the information that displays the application window, and occlusion window information is the information that occludes the application window.

[0153] In a video conference, all participants can initiate an application sharing message. The initiator sends the message to the cloud media agent, which then opens the shared application and sets its window as a child window of the agent. The shared application can be an application already opened by the agent (e.g., selecting an application from existing ones) or a newly opened application.

[0154] In the application sharing message, the initiator's account is carried. The cloud media agent sends the initiator's account to the cloud desktop server. The cloud desktop server sends the desktop stream corresponding to the initiator's account to the cloud media agent. After receiving the desktop stream, the cloud media agent encapsulates the application window information and the obscuring window information, and sends the encapsulated information to the media resource center. The media resource center sends the encapsulated information to other participants in the video conference, that is, the shared desktop clients of other participants.

[0155] Step 904: Encapsulate the application window message and the occlusion window message to obtain the application encapsulation information;

[0156] In step 904, the cloud media agent encapsulates the application window message and the occlusion window message to obtain the application encapsulation information.

[0157] Step 905: Send the application encapsulation information to the cloud desktop client.

[0158] In step 905, the cloud media agent sends the application encapsulation information to the cloud desktop client. After receiving the encapsulation message, the cloud desktop client first decodes the encapsulation message and then trims the application window information according to the occlusion window information to obtain the application window display information. The application window display information is the information of the application window that is not occluded.

[0159] When sharing desktops or applications, if participants cannot understand each other's voice communication, such as due to differences in native languages, they can be authorized to modify documents. In some embodiments, after step 901, the process further includes forwarding the document modification authorization message and the authorized member account to the media resource center for verification of the authorized member account.

[0160] It should be noted that authorized member accounts are accounts granted by the initiator of desktop sharing or application sharing to one or more attendees who can modify documents. The initiator can select authorized members from the list of attendees, and then send the authorized member account and document modification authorization message to the cloud media agent. The cloud media agent forwards the authorized member account and document modification authorization message to the media resource center, which verifies the authorized member account, i.e., determines whether to authorize the member based on the authorized member account.

[0161] In some embodiments, after receiving an authorized member account, the media resource center sends an authorization verification message to the shared desktop client corresponding to that account. The shared desktop client provides authentication information, such as the authorized member's language and identity verification information provided through mouse and keyboard operations. The media resource center then authenticates the authorized member account based on this language and identity verification information. The media resource center sends the authentication information of authenticated members to the cloud desktop server. For member accounts that fail authentication, the media resource center takes no action.

[0162] The video conferencing implementation method provided in this embodiment involves a cloud media agent sending the main video conferencing window information to a media resource center, which then forwards the main video conferencing window information to a cloud desktop client. The main video conferencing window information is displayed on the cloud desktop client, while auxiliary video conferencing window information is displayed on the cloud desktop server. Furthermore, the main video conferencing window information and the auxiliary video conferencing window information are displayed synchronously, thereby optimizing the display effect of the video image and reducing the consumption of network bandwidth resources and cloud server resources.

[0163] Thirdly, embodiments of the present invention provide a video conferencing implementation device for a cloud desktop client. This video conferencing implementation device offloads the display portion of the video conferencing system to the terminal, reducing the resource consumption of the video conferencing system on the network and server, and improving the display effect of the cloud desktop.

[0164] Figure 10 This is a schematic block diagram of a video conferencing implementation device according to an embodiment of the present invention. (Refer to...) Figure 10 The video conferencing device includes:

[0165] The client acquisition module 1001 is used to acquire the main video conference window information from the video conference window information.

[0166] In this embodiment, the client acquisition module 1001 can be set within the video conferencing module, which is part of the video conferencing system set in the cloud desktop client.

[0167] In some embodiments, the video conferencing window information includes the main video conferencing window information and the auxiliary video conferencing window information, and the client acquisition module 1001 acquires the main video conferencing window information.

[0168] Display module 1002 is used to synchronize the display of information in the main video conferencing window with information in the auxiliary video conferencing window.

[0169] In this embodiment, the main video conference window information is displayed on the cloud desktop client, while the auxiliary video conference window information is displayed on the cloud desktop server. That is, the display part of the video conference is separated to the terminal, which reduces the amount of data transmission between the cloud desktop client and the cloud server, saves network bandwidth resources, reduces the load on the cloud server, and improves the display effect of the cloud desktop client.

[0170] In some embodiments, the client acquisition module 1001 acquires window occlusion information along with the main video conference window information. When the main video conference window information and the auxiliary video conference window information are displayed synchronously, the occluded portion of the main video conference window is not displayed.

[0171] The display module 1002 clips the corresponding obscured area when the main window of the video conference is obscured by other windows; when the main window of the video conference is completely obscured by other windows, the video conference window is moved out of the terminal's screen range.

[0172] The video conferencing implementation device provided in this embodiment decouples the display portion of the video conferencing system to the terminal. The client acquisition module 1001 acquires the main video conferencing window information from the video conferencing window information, and the display module 1002 only displays the main video conferencing window information, excluding the auxiliary video conferencing window information. Moreover, when the display module 1002 displays the main video conferencing window information, it synchronizes with the display of the auxiliary video conferencing window information on the cloud desktop server. This optimizes the display effect of the video image and reduces the consumption of network bandwidth resources and cloud server resources. In addition, by decoupling the display portion of the video conferencing system to the terminal, this video conferencing implementation device improves the utilization rate of terminal resources and simplifies the additional deployment and configuration in the cloud.

[0173] Fourthly, embodiments of the present invention provide a video conferencing implementation device for cloud media proxy. This video conferencing implementation device offloads the display portion of the video conferencing system to the terminal, reducing the resource consumption of the video conferencing system on the network and server, and improving the display effect of the cloud desktop.

[0174] Figure 11 This is a schematic block diagram of a video conferencing implementation device provided in an embodiment of the present invention. (Refer to...) Figure 11 The video conferencing device includes:

[0175] The receiving module 1101 is used to receive video conference window information from the video conference initiator.

[0176] In this embodiment, when the video conference initiator shares the desktop or application, the recipient receives the video conference window information.

[0177] The sending module 1102 is used to forward the video conferencing main window information in the video conferencing window information to the cloud desktop client.

[0178] In this embodiment, the video conferencing window information includes main video conferencing window information and auxiliary video conferencing window information. The main video conferencing window information is displayed on the cloud desktop client, and the auxiliary video conferencing window information is displayed on the cloud desktop server.

[0179] In this embodiment, the cloud media agent only sends the main video conference window information to the cloud desktop client; the auxiliary video conference window information is not sent to the cloud desktop client, thus saving network resources. The cloud desktop client interacts with the cloud desktop server to synchronize the main video conference window information displayed on the cloud desktop client with the auxiliary video conference window information displayed on the cloud desktop server.

[0180] In some embodiments, the cloud media agent sends the main video conference window information to the media resource center, which then sends the main video conference window information to the cloud desktop client.

[0181] In some embodiments, the sending module 1102 sends window occlusion information while sending the main video conference window information, so that the cloud desktop client can crop the corresponding occlusion position when displaying the main video conference window; when the main video conference window is completely occluded by other windows, the video conference window is moved out of the terminal's screen range.

[0182] The video conferencing device provided in this embodiment receives the video conferencing window information from the video conferencing initiator, and only forwards the main video conferencing window information in the video conferencing window information to the cloud desktop client. This allows the cloud desktop client to display the main video conferencing window information in sync with the auxiliary video conferencing window information displayed on the cloud desktop server. This optimizes the display effect of the video image and reduces the consumption of network bandwidth resources and cloud server resources.

[0183] To better understand the various embodiments of the present invention, the video conferencing system of this embodiment will be further described below in conjunction with the video conferencing implementation apparatus and method provided in the above embodiments.

[0184] Figure 12 This is a flowchart illustrating the interaction between the video conferencing module, the cloud desktop client, and the cloud desktop server in an embodiment of the present invention. Figure 12 The process of logging into the video conferencing module using the cloud desktop client includes:

[0185] Step 1201: The cloud desktop client sends the video conferencing server address and the username of the video conferencing initiator to the video conferencing module.

[0186] In step 1201, the video conference initiator inputs the video conference server address and username into the video conferencing module via a terminal. For example, the video conference initiator inputs the video conference server address and username into a cloud desktop client via the terminal's input module, and then the cloud desktop client sends it to the video conferencing module.

[0187] Step 1202: The video conferencing module returns the listening port of the video conferencing module to the cloud desktop client.

[0188] In step 1202, the video conferencing module creates a TCP server and returns the listening port of the video conferencing module to the cloud desktop client.

[0189] Step 1203: The cloud desktop client sends its IP address and the listening port of the video conferencing module to the virtual machine component.

[0190] In step 1203, the cloud desktop client sends the listening port of the video conferencing module and the IP address of the cloud desktop client (the IP address of the terminal) to the virtual machine component via the desktop protocol.

[0191] Step 1204: The virtual machine component sends the IP address and listening port of the cloud desktop client, the IP address and listening port of the media resource center, and the listening port of the virtual machine component to the cloud media agent.

[0192] In step 1204, after receiving the IP address and listening port of the cloud desktop client, the virtual machine component sends the IP address and listening port of the cloud desktop client, the IP address and listening port of the media resource center, and its own listening port to the cloud media agent.

[0193] Step 1205: Send the main video conference window information and window occlusion information to the video conference module.

[0194] In step 1205, after receiving the IP address and listening port of the cloud desktop client, the IP address and listening port of the media resource center, and the listening port of the virtual machine component, the cloud media agent establishes a connection with the video conferencing module based on the IP address and listening port of the cloud desktop client, and then sends the video conferencing main window information and window occlusion information to the video conferencing module in real time.

[0195] Step 1206: The video conferencing module adjusts the window based on the main video conferencing window information and window occlusion information.

[0196] When the position and size of the main video conference window change, the video conferencing module window changes accordingly; for example, when the main video conference window moves from the left to the right, the video conferencing module window changes accordingly. When the main video conference window is minimized, the video conferencing module window moves off-screen.

[0197] Step 1207: The video conferencing module connects to the video conferencing server, and participants log in to the video conferencing server.

[0198] In step 1207, the video conferencing module connects to the video conferencing server, and participants log in to the video conferencing server through a cloud desktop client. The server can verify the logged-in participants.

[0199] Figure 13 This is a flowchart illustrating the process of inviting participants to a video conference, as described in an embodiment of the present invention. Figure 13 As shown, the steps for inviting participants to a video conference include:

[0200] Step 1301: The first video conferencing module of the video conferencing initiator sends the initiator's account, video conferencing type, and the accounts of the invited participants to the video conferencing server.

[0201] The video conference initiator can query the address book or recent meeting records to select participants. When initiating a video conference, the initiator can also choose the type of video conference. Video conference types include point-to-point meetings, video conferences, and audio conferences. The video conference initiator sends the initiator's account, the video conference type, and the accounts of the invited participants to the video conference to the video conference server through the video conference module.

[0202] In some embodiments, when the first video conferencing module does not have a video conference, the video conference initiator can also create a video conference through the first video conferencing module.

[0203] Step 1302: The video conferencing server creates a video conference.

[0204] In step 1302, the video conferencing server creates a video conference based on the video conference type selected by the video conference initiator.

[0205] Step 1303: Send the conference ID of the created video conference to the video conference initiator.

[0206] In step 1303, if the video conferencing server successfully creates a video conference, it sends the conference ID of the created video conference to the video conference initiator. If the video conferencing server fails to create a video conference, it returns a creation failure message to the video conference initiator.

[0207] Step 1304: The video conferencing server checks whether the participants are present.

[0208] In step 1304, the video conferencing server checks each invited participant against the list provided by the video conferencing initiator to see if they are already in other video conferences. If an invited participant is in another video conference, it means they cannot accept the invitation. If an invited participant is not in another video conference, it means they can accept the invitation.

[0209] Step 1305: The video conferencing server returns a response message to the first video conferencing module of the video conferencing initiator.

[0210] In step 1305, if the video conferencing server matches an invited participant who is in another video conference, it returns a response message indicating that the participant cannot attend. If the video conferencing server matches an invited participant who is not in another video conference, it returns an attendance response message to the video conference initiator.

[0211] Step 1306: The video conferencing server sends the meeting number and initiator account to the second video conferencing module corresponding to the eligible video conferencing members.

[0212] In step 1306, the video conferencing server sends the conference number of the created video conference and the initiator account of the video conference to the video conferencing module corresponding to the eligible video conferencing members.

[0213] Step 1307: The video conferencing modules of the video conferencing members forward the meeting number and the initiator's account to the cloud media agent.

[0214] In step 1307, the cloud media agent refers to the cloud media agent on the side of the invited member.

[0215] Step 1308: The cloud media agent prompts a video conference invitation.

[0216] In step 1308, when the cloud media agent receives the video conferencing module from the member side, it can display a meeting invitation via a pop-up window.

[0217] Step 1309: The cloud media agent returns the participation information of the invited members to the video conferencing modules of the video conferencing members. The video conferencing modules return the participation information to the video conferencing server, and the video conferencing server then returns the participation message to the video conferencing module corresponding to the video conferencing initiator of the video conferencing member.

[0218] In step 1309, the meeting information includes information on whether the meeting invitation is accepted or declined; that is, invited members can refuse to attend or agree to attend.

[0219] Figure 14 This is a flowchart illustrating desktop sharing during collaborative work, as described in an embodiment of the present invention. The initiator of desktop sharing can be any member of the meeting, such as the initiator of the video conference or another member of the video conference. For ease of description, the following example uses the initiator of the video conference as the initiator of desktop sharing. Figure 14 As shown, the steps for desktop sharing include:

[0220] Step 1401: The first video conferencing module sends a desktop sharing request to the video conferencing server.

[0221] The first video conferencing module is owned by the user who shares the video conferencing data. The user sends desktop sharing requests through the first video conferencing module.

[0222] Step 1402: The video conferencing server returns a confirmation message to the first video conferencing module confirming receipt of the desktop sharing request.

[0223] Step 1403: The video conferencing server sends the meeting number and initiator account to the media resource center.

[0224] Step 1404: The Media Resource Center generates connection parameters based on the conference number and the initiator account, and returns the connection parameters and the initiator account to the video conferencing server.

[0225] Step 1405: The Media Resource Center sends the connection parameters and the initiator's account to all participants in the video conference.

[0226] Step 1406: The first video conferencing module and the second video conferencing module return a connection completed message to the video conferencing server.

[0227] Step 1407: After receiving the connection parameters and the initiator's account, the second video conferencing module starts the shared desktop client and passes the connection parameters and the initiator's account as startup parameters to the shared desktop client.

[0228] The second video conferencing module is the video conferencing module corresponding to the participants in the video conference. It starts the shared desktop client and passes the connection parameters and the initiator's account as startup parameters to the shared desktop client.

[0229] In this video conference, participants are those other than the initiator of the desktop sharing. The shared desktop client can be an external module independent of the cloud desktop client, or it can be a built-in module integrated into the cloud desktop client. The shared desktop client and the cloud desktop client can communicate with each other.

[0230] Step 1408: After receiving the connection parameters and the initiator's account, the video conference initiator and participants establish a connection with the media resource center based on the connection parameters and their own account, and send a message to the video conference server agreeing to receive the desktop stream.

[0231] Step 1409: The shared desktop client receives the desktop stream from the media resource center.

[0232] Step 1410: When desktop sharing ends, the first video conferencing module sends a message to the video conferencing server to end sharing.

[0233] In step 1411, the video conferencing server forwards the end-of-share message to the second video conferencing module.

[0234] In step 1412, the second video conferencing module forwards the end-of-share message to the shared desktop client.

[0235] In some embodiments, the Media Resource Center obtains desktop streaming via a cloud-based media proxy. Figure 15 This is a flowchart illustrating the process of obtaining the desktop stream from the media resource center in an embodiment of the present invention. For example... Figure 15 As shown, the steps for obtaining a desktop stream from the Media Resource Center include:

[0236] Step 1501: The first video conferencing module sends a desktop sharing request message to the cloud media agent.

[0237] The first video conferencing module is the video conferencing module corresponding to the sharer. The first video conferencing module sends a desktop sharing request message to the cloud media agent.

[0238] When a desktop sharer successfully requests desktop sharing from the video conferencing server, they send a desktop sharing request message and the initiator's account to the cloud media agent.

[0239] Step 1502: The cloud media agent sends the initiator's account to the cloud desktop server.

[0240] The cloud-based media agent will establish a connection with the cloud desktop server based on the IP address and port obtained during startup, and send desktop sharing messages and the initiator's account to the cloud desktop server.

[0241] Step 1503: The cloud desktop server forwards the desktop stream corresponding to the initiator's account to the cloud media agent.

[0242] Step 1504: The cloud media agent forwards the desktop stream to the media resource center.

[0243] In some embodiments, the desktop sharing initiator does not want to share the entire desktop content, but only wants to share a specific application with the participants of the video conference. Figure 16 This is a flowchart illustrating the application of sharing in an embodiment of the present invention. For example... Figure 16 As shown, the steps for application sharing include:

[0244] Step 1601: The sharer in the video conference initiates an application sharing message to the cloud media agent.

[0245] Step 1602: The cloud media agent opens the application.

[0246] In some embodiments, the cloud media proxy opens the application on behalf of the cloud, or selects the application that is already open and sets the application's window as a child window of the cloud shared proxy.

[0247] Step 1603: The cloud media agent sends the application sharing initiator's account to the cloud desktop server.

[0248] Step 1604: The cloud desktop server sends the desktop stream to the cloud media agent.

[0249] In some embodiments, the desktop stream includes application window information and occlusion window information.

[0250] Step 1605: The cloud media proxy encapsulates the application window's position information and window occlusion information, and then forwards the encapsulated desktop stream to the media resource center.

[0251] Step 1606: The Media Resource Center sends the encapsulated desktop stream to the shared desktop client.

[0252] Step 1607: The shared desktop client decapsulates the desktop stream, clips the application window position information based on the occlusion window information, and scales the clipped application window information for display.

[0253] In some embodiments, when desktop or application sharing occurs, if participants do not speak the same language, there may be a situation where they cannot understand each other's documents or speech. This embodiment addresses this situation by authorizing participants to modify documents, i.e., granting sharing authorization to the participants.

[0254] Figure 17 This is a flowchart illustrating the shared licensing process in an embodiment of the present invention. For example... Figure 17 As shown, the shared licensing process includes:

[0255] Step 1701: The desktop sharer sends the authorization information to the cloud media agent.

[0256] In some embodiments, the desktop sharer selects authorized members from the list of attendees and sends an authorization message to the cloud media agent, wherein the authorization information includes the authorized member's account and authorization instructions.

[0257] Step 1702: The cloud-based media agent sends the authorization information to the media resource center.

[0258] Step 1703: Share the desktop client to perform mouse and keyboard operations, providing the authorized member's language and authentication information.

[0259] Step 1704: The Media Resource Center authenticates the authorized members based on their member accounts and forwards the authorized members' language and identity verification information to the cloud desktop server.

[0260] In some embodiments, if the identified authorized member is an authorized member, the keyboard and mouse operation information is forwarded to the cloud desktop server; if the member is not authorized, no action is taken.

[0261] Step 1705: The cloud desktop server sends the desktop stream to the video conferencing module and media resource center corresponding to the desktop sharer.

[0262] Step 1706: The Media Resource Center forwards the desktop stream to the shared desktop of the members.

[0263] Fifthly, refer to Figure 18 This invention provides an electronic device comprising:

[0264] One or more processors 1801;

[0265] The memory 1802 stores one or more programs that, when executed by one or more processors, enable the one or more processors to implement the video conferencing method described above.

[0266] One or more I / O interfaces 1803 are connected between the processor and memory and configured to enable information exchange between the processor and memory.

[0267] Among them, processor 1801 is a device with data processing capabilities, including but not limited to central processing unit (CPU); memory 1802 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); I / O interface (read-write interface) 1803 is connected between processor 1801 and memory 1802, and can realize information interaction between processor 1801 and memory 1802, including but not limited to data bus (Bus).

[0268] In some embodiments, the processor 1801, memory 1802, and I / O interface 1803 are interconnected via a bus, and thus connected to other components of the computing device.

[0269] Sixthly, embodiments of the present invention provide a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described video conferencing implementation methods.

[0270] Those skilled in the art will understand that all or some of the steps, systems, or apparatuses in the methods, systems, and apparatuses described above can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0271] This document has described exemplary embodiments, and while specific terminology has been used, it is intended and should be interpreted only in a general illustrative sense and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in conjunction with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in conjunction with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of the invention as set forth in the appended claims.

Claims

1. A method for implementing video conferencing, characterized in that, Applied to cloud desktop clients, the method includes the following steps: Retrieve the main video conference window information from the video conference window information; the video conference window information includes the main video conference window information and the auxiliary video conference window information; wherein, the auxiliary video conference window information includes the video conference menu bar; the main video conference window information includes the specific content of the video conference; The main video conference window information and the auxiliary video conference window information are displayed synchronously, wherein the auxiliary video conference window information is displayed on the cloud desktop server and the main video conference window information is displayed on the cloud desktop client.

2. The method according to claim 1, characterized in that, Before obtaining the main video conference window information from the video conference window information, the process also includes: A Transmission Control Protocol (TCP) server is created based on the address of the video conferencing server (MCU) and the current user's account; wherein, the current user's account is the account of the user who initiated the video conference; A connection is established between the TCP server and the video conferencing server.

3. The method according to claim 2, characterized in that, After establishing a connection with the video conferencing server through the TCP server, the process further includes: The type of video conference and the participants are sent to the video conference server so that the video conference server can create the video conference and invite the participants. Receive video conference response messages returned by the cloud desktop clients corresponding to the meeting members.

4. The method according to claim 3, characterized in that, After receiving the video conferencing response message returned by the cloud desktop client corresponding to the meeting member, the method further includes: Initiate a desktop sharing message; wherein, the desktop sharing message includes the conference number of the video conference and the initiator's account; A connection is established with the media resource center based on the desktop sharing connection parameters and the account of the desktop sharing initiator; wherein, the desktop sharing connection parameters are parameters generated by the media resource center based on the video conference ID and the initiator's account; Receive the desktop stream from the desktop sharing initiator forwarded by the media resource center.

5. The method according to claim 4, characterized in that, After receiving the desktop stream from the desktop sharing initiator forwarded by the media resource center, the process further includes: Initiate an application sharing message to allow the cloud media agent to encapsulate application window information and occlusion window information; wherein, the application window information is the information that displays the application window, and the occlusion window information is the information that occludes the application window.

6. The method according to claim 5, characterized in that, After receiving the desktop stream from the desktop sharing initiator forwarded by the media resource center, the process further includes: Based on the occlusion window information, the application window information is cropped to obtain the application window display information; The application window information is displayed synchronously in the main video conference window.

7. The method according to claim 6, characterized in that, After initiating the desktop sharing message, the following is also included: Send a document modification authorization message and the authorized member account to authenticate the authorized member account.

8. The method according to claim 7, characterized in that, Before receiving the desktop stream from the desktop sharing initiator forwarded by the media resource center, the method further includes: In response to the document modification authorization information, authentication information is provided so that the media resource center can authenticate the authorized member account.

9. The method according to claim 1, characterized in that, Before synchronizing the display of the main video conference window information and the auxiliary video conference window information, the method further includes: The main video conference window information and the auxiliary video conference window information are simultaneously sent to the cloud desktop client and the cloud desktop server, respectively. The cloud desktop client displays the acquired main video conference window information in real time, while the cloud desktop server displays the acquired auxiliary video conference window information in real time.

10. A method for implementing video conferencing, characterized in that, The method is applied to cloud-based media proxies. Includes the following steps: In response to a video conference join message, the main video conference window information in the video conference window information is forwarded to the cloud desktop client so that the main video conference window information and the auxiliary video conference window information in the video conference window information are displayed synchronously. The join message is a confirmation message sent by a participant through the cloud desktop client to join the video conference. The main video conference window information is displayed on the cloud desktop client, and the auxiliary video conference window information is displayed on the cloud desktop server. The auxiliary video conference window information includes the video conference menu bar, and the main video conference window information includes the specific content of the video conference.

11. The method according to claim 10, characterized in that, Before forwarding the main video conference window information from the video conference window information to the cloud desktop client, the process also includes: A connection is established between the cloud desktop client and the video conferencing module based on the cloud desktop client's IP address and listening port; wherein, the video conferencing module is set up on the cloud desktop client.

12. The method according to claim 11, characterized in that, After the IP address and listening port of the cloud desktop client establish a connection with the video conferencing module, the following is also included: Receive a video conference invitation message, wherein the video conference invitation message includes the video conference ID and the account of the initiator of the video conference; The video conference invitation message is sent to the cloud desktop client corresponding to the participating member; wherein, the participating member is the member determined by the initiator of the video conference to participate in the video conference.

13. The method according to claim 12, characterized in that, The step of forwarding the main video conference window information from the video conference window information to the cloud desktop client also includes: In response to a desktop sharing request, a connection is established with the cloud desktop server; wherein, the desktop sharing request is initiated by a member of the video conference; Receive desktop stream from the cloud desktop server, wherein the desktop stream is sent by the initiator of desktop sharing, and the desktop stream includes the video conference main window information.

14. The method according to claim 13, characterized in that, After receiving the desktop stream from the cloud desktop server, the process further includes: Extract application window information and occlusion window information from the desktop stream; wherein, the application window information is the information that displays the application window, and the occlusion window information is the information that occludes the application window; The application window information and the occlusion window information are encapsulated to obtain application encapsulation information; The application encapsulation information is sent to the cloud desktop client.

15. The method according to claim 13, characterized in that, After establishing a connection with the cloud desktop server in response to the desktop sharing request, the process also includes: The document modification authorization message and the authorized member account are forwarded to the media resource center for the media resource center to verify the authorized member account.

16. A device for implementing video conferencing, characterized in that, Applied to cloud desktop clients, it includes: The client acquisition module is used to acquire the main video conference window information from the video conference window information; the video conference window information includes the main video conference window information and the auxiliary video conference window information; wherein, the auxiliary video conference window information includes the video conference menu bar, and the main video conference window information includes the specific content of the video conference; The display module is used to synchronize the display of the main video conferencing window information and the auxiliary video conferencing window information, wherein the auxiliary video conferencing window information is displayed on the cloud desktop server and the main video conferencing window information is displayed on the cloud desktop client.

17. A device for implementing video conferencing, characterized in that, Applications in cloud-based media proxies include: The sending module is used to join a video conference by forwarding the main video conference window information from the video conference window information to the cloud desktop client, so that the main video conference window information and the auxiliary video conference window information are displayed synchronously. The main video conference window information is displayed on the cloud desktop client, and the auxiliary video conference window information is displayed on the cloud desktop server. The auxiliary video conference window information includes the video conference menu bar, and the main video conference window information includes the specific content of the video conference.

18. An electronic device comprising: One or more processors; A storage device having stored one or more programs thereon, which, when executed by the one or more processors, cause the one or more processors to implement the method of implementing video conferencing according to any one of claims 1 to 9 or claims 10 to 15; One or more I / O interfaces are connected between the processor and the memory and configured to enable information interaction between the processor and the memory.

19. A storage medium for computer-readable storage, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the steps of the video conferencing implementation method according to any one of claims 1 to 9 or claims 10 to 15.

Images