Screen sharing method and related equipment

Abstract

The embodiment of the invention discloses a screen sharing method and related equipment, which are used for reducing computing resources occupied by screen sharing. The method is applied to a first server side, and a remote desktop system where the first server side is located further comprises a first client side, a second server side and a second client side. The method comprises the steps that a sharing request from a conference program is acquired, the sharing request indicates a second client accessing a conference to share desktop content of a first client, and the conference program runs on a desktop of the first client; and processing the desktop data of the first client to obtain a first code stream. And sending the first code stream to the first client. And in response to the sharing request, sending the first code stream to the conference program, so that the conference program forwards the first code stream to the second client through the second server.

Description

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on July 7, 2023, with application number 202310833347.3 and invention name “A conference sharing method and device”, the entire contents of which are incorporated by reference in this application. Technical Field

[0002] The embodiments of the present application relate to the field of computers, and in particular to a screen sharing method and related devices. Background Art

[0003] Screen sharing is a technology that allows you to share the screen content of a client with other devices. Through screen sharing, you can share files, images, videos and other content between different devices. Screen sharing is widely used in conference scenarios, allowing all parties to simultaneously view and operate the same content.

[0004] In the existing method, when screen sharing is performed, the desktop data of the shared client is processed to obtain a code stream for displaying the conference screen on the shared client. In addition, the same desktop data needs to be processed again to obtain another code stream and transmit it to the conference server, so that the conference server packages the code stream and sends it to the audience of the conference. Based on this, screen sharing can be realized.

[0005] In this method, it is necessary to collect, encode and send multiple streams of the same desktop data at the same time, which wastes computing resources. Summary of the invention

[0006] The present application provides a screen sharing method and related devices for reducing the computing resources occupied by screen sharing.

[0007] In a first aspect, the present application provides a screen sharing method, which is applied to a first server, including:

[0008] The first server is included in the remote desktop system, which also includes a first client, a second server and a second client. The first server is used to provide desktop services for the first client, so that the desktop of the first client can display the picture. The second server can also be called a conference server, which provides conference services for the first client and the second client, so that the first client and the second client can access the conference. The first server can obtain a sharing request from the conference program, which indicates that the second client accessing the conference shares the desktop content of the first client, that is, there is a screen sharing demand at this time, and the desktop content of the first client needs to be shared to the second client. The conference program runs on the desktop of the first client. The first server processes the desktop data of the first client to obtain a first code stream. The desktop data of the first client includes document data, image data, video data, etc. that can be displayed on the desktop of the first client, and the specific details are not limited here. The method of processing the desktop data of the first client includes capturing and encoding the desktop data, and the obtained first code stream includes at least one coded frame. The first server sends the first code stream to the first client, so that the first client can display the picture on the screen. The first server can package the first code stream, and the first client can perform unpacking, decoding and rendering operations to display the picture on the screen. The first server can respond to the sharing request and send the first code stream to the conference program, so that the conference program forwards the first code stream to the second client through the second server, so that the second client also displays the desktop content of the first client, that is, screen sharing is realized.

[0009] It can be seen from the above technical scheme that the present application has the following advantages: when performing screen sharing, the first server will process the desktop data of the first client to obtain the first code stream, and send the first code stream to the first client and the conference program respectively, so that the first client and the second client connected to the second server corresponding to the conference program can both display the desktop content of the first client, thereby realizing screen sharing. In this scheme, the first server reuses the first code stream to the conference program, and does not need to process the same desktop data corresponding to the first client multiple times, which reduces the amount of data processed and reduces the computing resources occupied.

[0010] In some optional implementations of the first aspect, the first server may obtain a sharing request from the conference program through a conference plug-in, that is, the sharing request is sent by the conference program to the conference plug-in, and then forwarded to the first server by the conference plug-in. The running location of the conference plug-in is the same as that of the conference program, both running on the desktop of the first client.

[0011] In the present application, a communication connection is established between the first server and the conference program through a conference plug-in, so that sharing requests can be transmitted between the conference program and the first server, providing technical support for the implementation of screen sharing and enhancing the feasibility of the technical solution of the present application.

[0012] In some optional implementations of the first aspect, the conference plug-in may forward the sharing request from the conference program through a first inter-process communication method, where the first inter-process communication method includes any one of shared memory, socket communication, or pipe communication.

[0013] In the present application, there are multiple possible underlying implementation methods for forwarding sharing requests through conference plug-ins, which can be selected according to the needs of actual applications, enriching the implementation methods of the technical solution of the present application and improving the flexibility of the technical solution.

[0014] In some optional implementations of the first aspect, the first server may send the first code stream to the conference program through the conference plug-in, that is, the first code stream is sent by the first server to the conference plug-in, and then forwarded to the conference program by the conference plug-in. The running location of the conference plug-in is the same as the running location of the conference program, both running on the desktop of the first client.

[0015] In the present application, a communication connection between the first server and the conference program is established through a conference plug-in, so that the first code stream can be transmitted between the conference program and the first server, providing technical support for the implementation of screen sharing and enhancing the feasibility of the technical solution of the present application.

[0016] In some optional implementations of the first aspect, the conference plug-in may send the first code stream to the conference program through a second inter-process communication method, where the second inter-process communication method includes any one of shared memory, socket communication, or pipe communication.

[0017] In the present application, there are multiple possible underlying implementation methods for forwarding the first code stream through the conference plug-in, which can be selected according to the needs of the actual application, enriching the implementation methods of the technical solution of the present application and improving the flexibility of the technical solution.

[0018] In some optional implementations of the first aspect, the screen sharing method provided by the present application can be applied in a cloud platform scenario, in which case the first client is a cloud desktop server, and the desktop of the first client is a cloud desktop. In other words, both the conference program and the conference plug-in run on the cloud desktop of the first client. The conference program, the conference plug-in, and the second server provide conference services for the first client, and the conference service mentioned here may be a cloud conference service or may not be a cloud conference service, which is not specifically limited here.

[0019] The screen sharing method provided in the present application can be applied in cloud platform scenarios, and accordingly, it also has the beneficial effects generally possessed by cloud platform scenarios, such as: easy maintenance, convenient use, and further power saving when multiple clients use cloud desktop services and enable screen sharing through conference programs.

[0020] In some optional implementations of the first aspect, the screen sharing method provided by the present application can also be applied to a remote assistance scenario, in which case the first server is a remote desktop server and the desktop of the first client is a remote desktop. In other words, the first client is a remotely controlled client. In the remote assistance scenario, both the conference program and the conference plug-in run on the remote desktop of the first client.

[0021] In some optional implementations of the first aspect, in a remote assistance scenario, the remote desktop system also includes a third client, and the first server can also obtain a sharing request from the third client, and the sharing request from the third client indicates sharing the desktop content of the first client. Among them, the third client is the client that initiates remote control. The first server can respond to the sharing request from the third client and send a first code stream to the third client so that the third client displays the desktop content of the first client to achieve screen sharing. Therefore, in this implementation, in a remote assistance conference scenario, not only the first client and the second client that access the conference display the desktop content of the first client, but also the third client that initiates remote control also displays the desktop content of the first client.

[0022] The screen sharing method provided in this application can be applied in a variety of scenarios, enriching the implementation method of the technical solution of this application, being able to adapt to different needs, and improving the flexibility of the technical solution.

[0023] In a second aspect, the present application provides a communication device, the communication device is a first server, the remote desktop system where the first server is located also includes a first client, a second server and a second client. The communication device includes: a processing unit and a transceiver unit.

[0024] The transceiver unit is used to send a first code stream to a first client, obtain a sharing request from a conference program, and the sharing request indicates that a second client accessing the conference shares the desktop content of the first client, and the conference program runs on the desktop of the first client.

[0025] The processing unit is used to process the desktop data of the first client to obtain a first code stream.

[0026] The transceiver unit is further configured to send the first code stream to the first client and, in response to the sharing request, send the first code stream to the conference program so that the conference program forwards the first code stream to the second client via the second server.

[0027] It should be noted that the second aspect or any implementation method of the second aspect is a device implementation corresponding to the first aspect or any implementation method of the first aspect, and the description in the first aspect or any implementation method of the first aspect is applicable to the second aspect or any implementation method of the second aspect, and will not be repeated here.

[0028] The third aspect of the present application provides a computing device, including: a memory and a processor, wherein the processor stores instructions, and when the instructions are executed on the processor, the screen sharing method in the first aspect or any implementation of the first aspect is implemented.

[0029] In a fourth aspect, the present application provides a computing device cluster, including at least one computing device, each computing device including a processor and a memory. The processor of at least one computing device is used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster implements the screen sharing method in the cloud platform scenario in the first aspect or any one of the implementations of the first aspect.

[0030] In a fifth aspect, the present application provides a computer program product comprising instructions. When the instructions are executed by a computing device cluster, the computing device cluster implements the screen sharing method in the cloud platform scenario in the first aspect or any one of the implementations of the first aspect. When the instructions are executed by a processor, the processor implements the screen sharing method in the non-cloud platform scenario in the first aspect or any one of the implementations of the first aspect.

[0031] In a sixth aspect, the present application provides a computer-readable storage medium, including computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster implements the screen sharing method in the cloud platform scenario in the first aspect or any one of the implementations of the first aspect. When the computer program instructions are executed on a processor, the screen sharing method in the non-cloud platform scenario in the first aspect or any one of the implementations of the first aspect is implemented.

[0032] The beneficial effects shown in any one of the second to sixth aspects of the present application are similar to those of the aforementioned first aspect or any possible implementation method of the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A schematic diagram of a system architecture provided for an embodiment of the present application;

[0034] Figure 2 A schematic diagram of the architecture of a remote desktop system provided in an embodiment of the present application;

[0035] Figure 3 A schematic diagram of a flow chart of a screen sharing method provided in an embodiment of the present application;

[0036] Figure 4 Another schematic diagram of the architecture of the remote desktop system provided in the embodiment of the present application;

[0037] Figure 5 A schematic diagram of the structure of a communication device provided in an embodiment of the present application;

[0038] Figure 6 A schematic diagram of the structure of a communication system provided in an embodiment of the present application;

[0039] Figure 7 A schematic diagram of a structure of a computing device provided in an embodiment of the present application;

[0040] Figure 8 A schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The embodiments of the present application provide a screen sharing method and related devices for reducing the computing resources occupied by screen sharing.

[0042] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0043] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged in appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attribute in the embodiments of the present application when describing. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment containing a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment. In addition, "at least one" refers to one or more, and "multiple" refers to two or more. "And / or", describes the association relationship of associated objects, indicating that three relationships can exist, for example, A and / or B, can represent: A exists alone, A and B exist simultaneously, and B exists alone, wherein A, B can be singular or plural. The character " / " generally represents that the associated objects before and after are a kind of "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0044] The screen sharing method provided in the embodiment of the present application has multiple application scenarios, which can be applied to cloud platform scenarios as well as non-cloud scenarios. The following describes the system architecture of different scenarios in conjunction with the schematic diagrams.

[0045] First, see Figure 1 , Figure 1 This is a schematic diagram of the system architecture provided in the embodiment of the present application. Specifically, Figure 1 This is an architecture diagram of the screen sharing method provided in the embodiment of the present application applied in a cloud platform scenario.

[0046] like Figure 1 As shown, the cloud platform 30 may provide interfaces related to cloud computing services, such as configuration pages (i.e., interfaces) or application programming interfaces (APIs) for users to access cloud services. As the top-level object for managing cloud services and / or cloud resources, users use the client 10 to log in to the cloud platform 30 via the Internet 20 using the account and password registered on the cloud platform 30 to configure and use cloud services, which are provided by the infrastructure managed by the cloud platform 30. The cloud platform 30 manages the infrastructure, which includes multiple cloud data centers located in different regions, with each region having at least one cloud data center. For example Figure 1 The region 1 shown includes cloud data center 1 and cloud data center 2, and the region 2 includes cloud data center 3 and cloud data center 4. Each cloud data center is provided with multiple servers, and service instances are running on the servers.

[0047] Among them, the business instance is an instance belonging to the user for running its business, including one or any combination of virtual machines, containers, and dedicated hosts. A virtual machine (VM) refers to a complete computer system with complete hardware system functions that runs in a completely isolated environment and is simulated by software. When creating a virtual machine in a server, part of the hard disk and memory capacity of the physical machine needs to be used as the hard disk and memory capacity of the virtual machine. Each virtual machine has an independent hard disk and operating system, and the user of the virtual machine can operate the virtual machine like using a server. Container is a virtualization technology in computer operating systems. This technology allows processes to run in a relatively independent and isolated environment (including independent file systems, namespaces, resource views, etc.), thereby simplifying the software deployment process, enhancing the portability and security of the software, and improving the utilization of system resources. Dedicated host (DeH) refers to dedicated physical host resources that users can enjoy exclusively. Users create cloud servers on dedicated hosts to meet their higher requirements for isolation, security, and performance. Users of dedicated hosts do not need to share the physical resources of the host with other users.

[0048] In the embodiment of the present application, cloud desktop service and conference service are deployed in the business instance. Users can purchase cloud desktop service on cloud platform 30 through client 10, and realize screen sharing through conference program running on cloud desktop. In summary, client 10 is the first client, and the desktop of the first client seen by the user is actually the cloud desktop provided by cloud platform 30. Users access the conference through the conference program running on the cloud desktop, and can use the screen sharing function of the conference program to share the desktop content of the first client with other clients accessing the conference.

[0049] Next, we will briefly describe the remote desktop system provided by this application in the cloud platform scenario. Figure 2 , Figure 2 A schematic diagram of the architecture of a remote desktop system provided in an embodiment of the present application.

[0050] like Figure 2 As shown in the figure, in the cloud platform scenario, the remote desktop system provides cloud desktop services and conference services. Figure 2 In the illustrated embodiment, the first client applies the cloud desktop service and the conference service, and the second client applies the conference service.

[0051] In the cloud platform scenario, the first server can also be called a cloud desktop server, which provides cloud desktop services to the first client; the desktop of the first client is a cloud desktop. In general, after adopting a cloud desktop, users no longer need to purchase a computer host, but can connect a monitor through a thin client. After installing the client, users can access the virtual machine on the backend server to implement interactive operations, achieving the same experience as a traditional computer.

[0052] The second server may also be referred to as a conference server, and the conference server, conference plug-in, and conference program are used to provide conference services. The conference service may be a cloud conference service or may not be a cloud conference service, which is not limited here. The conference program and conference plug-in are running on the cloud desktop of the first client, and the screen sharing function of the conference enables multiple clients accessing the conference to display the same screen.

[0053] exist Figure 2 In the illustrated embodiment, the first client may also be referred to as a thin client, or a local client. The desktop of the first client is a cloud desktop, and a conference program is running on the cloud desktop of the first client. In other words, the desktop seen by the user using the first client is actually a cloud desktop, and the user can use the conference service through the conference program running on the cloud desktop. In addition, a conference plug-in is also running on the cloud desktop, and the conference plug-in establishes a communication connection between the cloud desktop server and the conference program, providing technical support for implementing the screen sharing method provided in the embodiment of the present application, and the specific implementation process is described in detail later.

[0054] It should be noted that the present application does not limit the specific type of conference server. For example, the conference server can be a selective forwarding unit (SFU), or it can be a cloud conference server or other server that can provide conference services. No specific limitation is made here.

[0055] In general, for a user using the first client, he can use the conference service through the cloud desktop of the first client and share the screen with other clients in the conference. Figure 2 In the illustrated embodiment, the first client and the second client can implement screen sharing, and the desktop content of the first client is displayed on both the first client and the second client.

[0056] Next, the technical solution of applying the screen sharing method provided in the embodiment of the present application to the cloud platform scenario is described. Figure 3 , Figure 3 A flowchart of a screen sharing method provided in an embodiment of the present application includes:

[0057] 301. The first server obtains a sharing request from a conference program.

[0058] As mentioned above, in the cloud platform scenario, the first server is the cloud desktop server. When screen sharing is performed through a conference, the conference program sends a sharing request to the first server, which instructs the first client and the second client accessing the conference to share the desktop content of the first client. It is understandable that in the cloud platform scenario, the conference program runs on the cloud desktop of the first client. In other words, users using the first client can initiate screen sharing on the conference program of the cloud desktop to achieve that both the first client and the second client participating in the conference display the desktop content of the first client.

[0059] In some optional implementations, the first server can obtain a sharing request from the conference program through a conference plug-in. That is, when performing screen sharing, the conference program sends a sharing request to the conference plug-in, and the conference plug-in forwards the sharing request to the first server. It can be understood that the conference program establishes a communication connection with the first server through the conference plug-in in order to use the capabilities of the first server. In the screen sharing method of an embodiment of the present application, this capability includes multiplexing the code stream obtained by processing desktop data by the first server. Through the conference plug-in, the conference program can call the open interface (such as an API) of the first server to obtain the code stream through the interface.

[0060] The running position of the conference plug-in is the same as the running position of the conference program. In the cloud platform scenario, both the conference plug-in and the conference program run on the cloud desktop of the first client.

[0061] In some optional implementations, the conference plug-in and the first server can communicate through a first inter-process communication mode. That is, the conference plug-in forwards the sharing request through the first inter-process communication mode. The first inter-process communication mode includes any one of shared memory, socket communication, or pipe communication. In addition, the conference program and the conference plug-in can also interact through the inter-process communication mode, which is not specifically limited here.

[0062] In the embodiment of the present application, a communication connection between the first server and the conference program is established through the conference plug-in, so that the sharing request can be transmitted between the conference program and the first server, providing technical support for the implementation of screen sharing and enhancing the feasibility of the technical solution of the present application. In addition, there are multiple possible underlying implementation methods for forwarding sharing requests through the conference plug-in, which can be selected according to the needs of the actual application, enriching the implementation method of the technical solution of the present application and improving the flexibility of the technical solution.

[0063] 302. The first server processes the desktop data of the first client to obtain a first code stream.

[0064] The first server can collect, encode, and process the desktop data of the first client to obtain a first code stream. The first code stream includes at least one encoded frame. As the specific content of the screen sharing is different, the desktop data of the first client may also be in various forms, such as text data, image data, video data, etc., which are not specifically limited here.

[0065] 303. The first server sends a first code stream to the first client.

[0066] After the first server obtains the first bitstream, it can package the first bitstream and send the first bitstream to the first client in the form of a data packet. The first client decrypts the data packet to obtain the first bitstream, and performs operations such as decoding and rendering on the first bitstream, and displays it on the screen of the first client.

[0067] 304. The first server sends the first code stream to the conference program.

[0068] After the first server obtains the first code stream, it responds to the sharing request and sends the first code stream to the conference program, so that the conference program forwards the first code stream to the second client through the second server, so that the second client also displays the desktop content of the first client, that is, screen sharing is realized.

[0069] In some optional implementations, the first server may send the first code stream to the conference program via a conference plug-in. That is, the first server sends the first code stream to the conference plug-in, and the conference plug-in forwards the first code stream to the conference program. Among them, the conference plug-in and the first server may communicate via a second inter-process communication method. That is, the conference plug-in forwards the sharing request via a second inter-process communication method. The second inter-process communication method includes any one of shared memory, socket communication, or pipe communication. It can be understood that the second inter-process communication here is similar to the first inter-process communication method in step 302. The difference is that the second inter-process communication here emphasizes the communication method when the first server sends the first code stream to the conference plug-in, and the first inter-process communication in the previous text emphasizes the communication method when the conference plug-in forwards the sharing request to the first server.

[0070] In this application, a communication connection between the first server and the conference program is established through the conference plug-in, so that the first code stream can be transmitted between the conference program and the first server, providing technical support for the implementation of screen sharing and enhancing the feasibility of the technical solution of this application. In addition, there are multiple possible underlying implementation methods for forwarding the first code stream through the conference plug-in, which can be selected according to the needs of the actual application, enriching the implementation method of the technical solution of this application and improving the flexibility of the technical solution.

[0071] 305. The conference program sends the first code stream to the second server.

[0072] After receiving the first code stream, the conference program running on the cloud desktop of the first client can package the first code stream and send the first code stream to the second server providing conference services for the first client in the form of a data packet. The second server can also be called a conference server.

[0073] It is understandable that the conference server can also provide conference services for the second client. After the second server receives the data packet including the first code stream, it sends the data packet to the second client. The second client decrypts the data packet to obtain the first code stream, and performs operations such as decoding and rendering on the first code stream, and displays it on the screen of the second client, thereby realizing screen sharing between the second client and the first client. It should be noted that if the second client also applies the cloud service, then the conference program of the second client runs on the cloud desktop of the second client. If the second client does not apply the cloud service, that is, the second client is not a thin client, then the conference program of the second client runs on the desktop of the second client. The conference server sends the first code stream to the second client, which is actually a conference program sent to the second client. The conference program of the second client decodes, renders, and other operations on the first code stream, thereby displaying the desktop content of the first client corresponding to the first code stream on the second client.

[0074] In summary, when the screen sharing method provided by the implementation of the present application is applied in a cloud platform scenario, when screen sharing is performed, the first server will process the desktop data of the first client to obtain a first code stream, and send the first code stream to the first client and the conference program respectively, so that the first client and the second client connected to the second server corresponding to the conference program can both display the desktop content of the first client, thereby realizing screen sharing. In this solution, the first server reuses the first code stream to the conference program, and does not need to process the same desktop data corresponding to the first client multiple times, which reduces the amount of data processed and the computing resources occupied. In addition, when the screen sharing method provided by the present application is applied in a cloud platform scenario, it accordingly has the beneficial effects generally possessed by cloud platform scenarios, such as: easy maintenance, convenient use, and further power saving when multiple clients use cloud desktop services and turn on screen sharing through conference programs.

[0075] The above description is based on the cloud platform scenario. The screen sharing method provided in the embodiment of the present application can also be applied to the remote assistance scenario, which can also be called the remote desktop scenario. The following describes the remote desktop system provided by the present application in the remote assistance scenario. Figure 4 , Figure 4 A schematic diagram of the architecture of a remote desktop system is provided for an embodiment of the present application.

[0076] In the remote assistance scenario, the remote desktop system also includes a third client, which is a client directly operated by the user. The user operates the first client through the third client. For example, assume that the first client is the client of user A in the office, and the third client is the client of user A at home. User A can initiate remote control to the first client on the third client to display the desktop content of the first client on the desktop of the third client. The screen sharing between the first client and the third client is the screen sharing in the traditional remote assistance scenario: the first server, as the remote desktop server, can obtain a sharing request from the third client, and the sharing request from the third client instructs the third client to share the desktop content of the first client. Among them, the third client is the client that initiates remote control. The remote desktop server can respond to the sharing request from the third client and send a first code stream to the third client so that the third client displays the desktop content of the first client to achieve screen sharing.

[0077] The second client is connected to the same conference as the first client. When screen sharing is performed during the conference process, the second client can also display the desktop content of the first client. In an embodiment of the present application, the first server transmits a first code stream corresponding to the desktop content of the first client to the second server through a conference plug-in and a conference program. In other words, the screen sharing between the first client and the second client applies the screen sharing method provided in an embodiment of the present application. Specifically, the first server receives a sharing request from the conference program through a conference plug-in, and the sharing request instructs the second client connected to the conference to share the desktop content of the first client. After the first server processes the desktop data of the first client to obtain the first code stream, it can not only send the first code stream to the first client, but also send the first code stream to the conference program through the conference plug-in, so that the conference program forwards the first code stream to the second client through the second server. The specific implementation process is similar to Figure 3 The embodiments shown are similar, and you can refer to the relevant descriptions in the previous text, which will not be repeated here. It should be noted that, unlike the cloud platform scenario described above, in the remote assistance scenario, the first server that communicates with the conference program and conference plug-in is the remote desktop server, not the cloud desktop server; in addition, the conference program and conference plug-in both run on the remote desktop of the first client, not the cloud desktop of the first client.

[0078] In general, Figure 4In the illustrated embodiment, the first client, the second client, and the third client can all display the desktop content of the first client. The screen sharing between the first client and the second client is implemented based on the conference program, and the screen sharing between the first server and the second client is implemented based on remote assistance. For example, assume that the first client is the client of user A in the office, the third client is the client of user A at home, and the second client is the client of user B. User A can remotely control the first client on the third client, and the desktop of the third client displays the desktop content of the first client. At this time, user A can also conduct a meeting through the conference program running on the first client, and turn on screen sharing during the meeting, so that the second client accessing the meeting also displays the screen of the first client.

[0079] By comparing the descriptions of the cloud platform scenario and the non-cloud platform scenario (remote assistance scenario) in the previous article, it can be seen that for the user, no matter in which scenario, what they see is the desktop content of the first client. The difference is that in the cloud platform scenario, the user uses the first client and directly operates the first client. In the remote assistance scenario, the user uses the third client and operates the first client through the third client.

[0080] In addition, based on the foregoing description, it can be understood that in the screen sharing method provided in the embodiment of the present application, the first server reuses the first code stream, that is, the first client and the second client accessing the conference both display the screen corresponding to the first code stream. Without considering factors such as network quality, the fluency (frame rate) displayed by the first client and the second client is the same, and the viewing party and the sharing party of the conference remain consistent. In the existing scheme, the first client and the second client accessing the conference display the screen corresponding to different code streams. Usually, the smoothness of the screen displayed by the conference viewing party (second client) is lower, which can easily lead to problems such as unclear pictures. In general, the first server has a higher requirement for the frame rate. The first code stream reused by the technical solution of the present application is the code stream obtained by the first server. That is to say, the code stream frame rate reused by different clients in the technical solution of the present application is higher, and the smoothness of the picture is better.

[0081] The following is a description of the related equipment provided by this application. Figure 5 , Figure 5 A schematic diagram of the structure of the communication device provided in the present application, wherein the communication device is a first server, and the remote desktop system where the first server is located further includes a first client, a second server and a second client.

[0082] like Figure 5 As shown, the communication device 500 includes a transceiver unit 501 and a processing unit 502 .

[0083] In some optional implementations, the transceiver unit 501 is used to obtain a sharing request from a conference program, where the sharing request instructs a second client accessing the conference to share the desktop content of the first client, and the conference program runs on the desktop of the first client.

[0084] The processing unit 502 is used to process the desktop data of the first client to obtain a first code stream.

[0085] The transceiver unit 501 is further configured to: send the first code stream to the first client, and in response to a sharing request, send the first code stream to the conference program, so that the conference program forwards the first code stream to the second client via the second server.

[0086] In some optional implementations, the transceiver unit 501 is specifically configured to obtain a sharing request from a conference program through a conference plug-in, and the conference plug-in runs on the desktop of the first client.

[0087] In some optional implementations, the conference plug-in forwards the sharing request through a first inter-process communication method, where the first inter-process communication method includes any one of shared memory, socket communication, or pipe communication.

[0088] In some optional implementations, the transceiver unit 501 is specifically configured to send the first code stream to the conference program via a conference plug-in, and the conference plug-in runs on the desktop of the first client.

[0089] In some optional implementations, the transceiver unit 501 is specifically used to send the first code stream to the conference program through the second inter-process communication mode of the conference plug-in, and the second inter-process communication mode includes any one of shared memory, socket communication, or pipe communication.

[0090] In some optional implementations, if the first server is a cloud desktop server, the desktop of the first client is a cloud desktop.

[0091] In some optional implementations, if the first server is a remote desktop server, the desktop of the first client is a remote desktop.

[0092] The communication device 500 is used to execute the operations performed by the first server in the aforementioned embodiment to implement the screen sharing method provided in the embodiment of the present application.

[0093] See also Figure 6 , Figure 6 A schematic diagram of the structure of a communication system provided in an embodiment of the present application.

[0094] like Figure 6As shown, the communication system includes a communication device 601, a conference plug-in 602, a conference program 603, a second server 604, a first client 605 and a second client 606. The conference program 603 includes a media management module 631 and a second media transmission module 632. The communication device 601 includes a screen acquisition module 611, a screen encoding module 612 and a first media transmission module 613.

[0095] The media management module 631 sends a sharing request to the conference plug-in 602, which indicates sharing the desktop content of the first client 605 with the second client 606. The conference plug-in 602 sends the sharing request to the communication device 601. The screen acquisition module 611 included in the communication device 601 acquires the desktop data of the first client and sends the acquired desktop data to the screen encoding module 612.

[0096] The screen encoding module 612 encodes the collected desktop data to obtain a first code stream, and sends the first code stream to the first media transmission module 613 , and also responds to the sharing request and sends the first code stream to the conference plug-in 602 .

[0097] The conference plug-in 602 sends the first code stream to the second media transmission module 632, and the second media transmission module 632 sends the first code stream to the second server 604, so that the second server 604 forwards the first code stream to the second client 606. The second client 606 displays the desktop content of the first client corresponding to the first code stream.

[0098] At the same time, the first client 605 receives the first code stream sent by the first media transmission module 613, and displays the desktop content of the first client corresponding to the first code stream.

[0099] Based on the above process, screen sharing between the first client and the second client is achieved.

[0100] The communication device 601 is used to execute the operations performed by the first server in the aforementioned embodiment to implement the screen sharing method provided in the embodiment of the present application.

[0101] See also Figure 7 , Figure 7 A schematic diagram of the structure of a computing device provided in an embodiment of the present application.

[0102] like Figure 7 As shown, the computing device 700 includes: a bus 703, a memory 704, a processor 705 and a communication interface 706. The processor 705, the memory 704 and the communication interface 706 communicate with each other through the bus 703. The computing device 700 can be a server or a terminal device. It should be understood that the present invention does not limit the number of processors and memories in the computing device 700.

[0103] The bus 703 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 The bus 703 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 703 may include a path for transmitting information between various components of the computing device 700 (for example, the memory 704, the processor 705, and the communication interface 706).

[0104] The processor 705 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0105] The memory 704 may include a volatile memory, such as a random access memory (RAM). The processor 705 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0106] The memory 704 stores executable program codes, and the processor 705 executes the executable program codes to respectively implement the functions of the unit 701 (corresponding to the aforementioned transceiver unit 501) and the processing unit 702 (corresponding to the aforementioned processing unit 502), thereby implementing the screen sharing method. That is, the memory 704 stores instructions for the cloud service platform to execute the screen sharing method provided in the embodiment of the present application.

[0107] The communication interface 706 uses a transceiver unit such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 700 and other devices or a communication network.

[0108] The computing device 700 is used to execute the operations performed by the first server in the aforementioned embodiment to implement the screen sharing method provided in the embodiment of the present application.

[0109] An embodiment of the present invention further provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device may be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device may also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0110] See also Figure 8 , Figure 8 A schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application.

[0111] like Figure 8 As shown, the computing device cluster includes at least one computing device 700. Instructions may be stored in the memory 704 in one or more computing devices 700 in the computing device cluster, and are used to execute the screen sharing method provided in the embodiment of the present application.

[0112] It should be noted that the memory 704 in different computing devices 700 in the computing device cluster can store different instructions for executing some functions of the cloud platform. That is, the instructions stored in the memory 704 in different computing devices 700 can implement the functions of one or more modules in the transceiver unit 701 and the processing unit 702.

[0113] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be a software or program product including instructions that can be run on a computing device or stored in any available medium. When the instructions are run on at least one computer device included in the computing device cluster, the at least one computer device included in the computing device cluster executes the above-mentioned screen sharing method for application in the cloud platform scenario. When the instructions are run on a processor, the processor executes the above-mentioned screen sharing method for application in a non-cloud platform scenario.

[0114] The embodiment of the present application also provides a computer-readable storage medium, which can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid-state hard disk), etc. The computer-readable storage medium includes computer program instructions, which are used to execute the above-mentioned screen sharing method. Specifically, when the computer program instructions are executed by a computing device cluster, the computing device cluster implements the above-mentioned screen sharing method for application in a cloud platform scenario. When the computer program instructions are executed by a processor, the processor implements the above-mentioned screen sharing method for application in a non-cloud platform scenario.

[0115] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0116] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0117] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A screen sharing method, characterized in that: The method is applied to a first server, and the remote desktop system where the first server is located also includes a first client, a second server, and a second client. The method includes: Acquire a sharing request from a conference program, wherein the sharing request instructs the second client accessing the conference to share the desktop content of the first client, and the conference program runs on the desktop of the first client; Processing the desktop data of the first client to obtain a first code stream; Sending the first code stream to the first client; In response to the sharing request, the first code stream is sent to the conference program, so that the conference program forwards the first code stream to the second client through the second server.

2. The method according to claim 1, characterized in that The obtaining of a sharing request from a conference program includes: The sharing request from the conference program is obtained through a conference plug-in, and the conference plug-in runs on the desktop of the first client.

3. The method according to claim 2, characterized in that The conference plug-in forwards the sharing request through a first inter-process communication mode, where the first inter-process communication mode includes any one of shared memory, socket communication, or pipe communication.

4. The method according to any one of claims 1 to 3, characterized in that The sending the first code stream to the conference program includes: The first code stream is sent to the conference program via a conference plug-in, and the conference plug-in runs on the desktop of the first client.

5. The method according to claim 4, characterized in that The sending the first code stream to the conference program through the conference plug-in includes: The first code stream is sent to the conference program through a second inter-process communication mode of the conference plug-in, where the second inter-process communication mode includes any one of shared memory, socket communication, or pipe communication.

6. The method according to any one of claims 1 to 5, characterized in that If the first server is a cloud desktop server, the desktop of the first client is a cloud desktop.

7. The method according to any one of claims 1 to 5, characterized in that If the first server is a remote desktop server, the desktop of the first client is a remote desktop.

8. A communication device, characterized in that: The communication device is a first server, the remote desktop system where the first server is located also includes a first client, a second server and a second client, and the communication device includes: A transceiver unit, configured to obtain a sharing request from a conference program, wherein the sharing request instructs a second client accessing the conference to share the desktop content of the first client, and the conference program runs on the desktop of the first client; A processing unit, configured to process the desktop data of the first client to obtain a first code stream; The transceiver unit is further used for: Sending the first code stream to the first client; In response to the sharing request, the first code stream is sent to the conference program, so that the conference program forwards the first code stream to the second client through the second server.

9. The device according to claim 8, characterized in that The transceiver unit is specifically used for: The sharing request from the conference program is obtained through a conference plug-in, and the conference plug-in runs on the desktop of the first client.

10. The device according to claim 9, characterized in that The conference plug-in forwards the sharing request through a first inter-process communication mode, where the first inter-process communication mode includes any one of shared memory, socket communication, or pipe communication.

11. The device according to any one of claims 8 to 10, characterized in that The transceiver unit is specifically used for: The first code stream is sent to the conference program via a conference plug-in, and the conference plug-in runs on the desktop of the first client.

12. The device according to claim 11, characterized in that The transceiver unit is specifically used for: The first code stream is sent to the conference program through a second inter-process communication mode of the conference plug-in, where the second inter-process communication mode includes any one of shared memory, socket communication, or pipe communication.

13. The device according to any one of claims 8 to 12, characterized in that If the first server is a cloud desktop server, the desktop of the first client is a cloud desktop.

14. The device according to any one of claims 8 to 12, characterized in that If the first server is a remote desktop server, the desktop of the first client is a remote desktop.

15. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 6.

16. A computer program product comprising instructions, characterized in that When the instruction is executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 6; When the instructions are executed by a computer, the computer is caused to perform the method according to any one of claims 1 to 5 and 7.

17. A computer-readable storage medium, characterized in that: includes computer program instructions; When the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 6; When the computer program instructions are executed on a processor, the method according to any one of claims 1 to 5 and 7 is implemented.

Images