Screen sharing method, device and intelligent interactive tablet

Abstract

The present invention discloses a screen sharing method, device and intelligent interactive tablet. The method comprises: a first client enters a collaborative mode of a whiteboard application by joining a target virtual room and displays a first canvas, and the target virtual room comprises a second client; the first client receives a screen sharing instruction, and generates a first media player in the first canvas according to the screen sharing instruction; the first client determines a shared window; the first client plays a first video stream corresponding to the shared window in the first media player, broadcasts the first video stream to the second client, and after receiving the first video stream broadcast by the first client, the second client generates a second media player and plays the first video stream. The present invention solves the technical problem that when multiple speaking clients act as host clients to share screens, the host client needs to be frequently switched to realize the screen sharing function of each speaking user, which causes troublesome user operation and affects the efficiency of the meeting.

Description

Technical Field

[0001] The present invention relates to the field of display technology, and in particular to a screen sharing method, device and intelligent interactive tablet. Background Art

[0002] The conference collaboration whiteboard has a screen sharing function. During the conference, each client runs the conference collaboration whiteboard application. When using the conference collaboration whiteboard application for remote discussions, different users need to show their prepared documents and materials. Usually, this is done in the form of a host. One user uses his client as the host client and uses screen sharing to show the materials. At this time, other users can only watch. When others need to show document information, they first need to turn off the screen sharing of the previous user, then use their own client as the host client, turn on their own screen sharing, and show their own materials to other users. This results in the need to frequently switch screen sharing, which makes user operation troublesome and affects efficiency.

[0003] In a collaborative meeting, screen sharing is usually performed by the host client, but the users who speak during the discussion may change frequently, resulting in the need to frequently switch the host client to achieve the screen sharing function for each speaking user, which in turn causes trouble for users to operate and affects meeting efficiency.

[0004] With regard to the above-mentioned prior art, when multiple speaking user terminals act as host clients to share screens, it is necessary to frequently switch host clients to realize the screen sharing function of each speaking user, which leads to cumbersome user operations and affects meeting efficiency. No effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a screen sharing method, device and smart interactive tablet to at least solve the technical problem in the prior art that when multiple speaking clients act as host clients to share screens, the host client needs to be frequently switched to realize the screen sharing function of each speaking user, which causes troublesome user operation and affects the meeting efficiency.

[0006] According to one aspect of an embodiment of the present invention, a screen sharing method is provided, including: a first client enters a collaborative mode of a whiteboard application by joining a target virtual room, and displays a first canvas in the collaborative mode, wherein the first client is any client in the target virtual room, and the target virtual room also includes at least one second client; the first client receives a screen sharing instruction, and generates a first media player in the first canvas according to the screen sharing instruction; the first client determines a shared window; the first client plays a first video stream corresponding to the shared window in the first media player, and broadcasts the first video stream to a second client, wherein after receiving the first video stream broadcast by the first client, the second client generates a second media player and plays the first video stream in the second media player.

[0007] According to another aspect of an embodiment of the present invention, a screen sharing device is also provided, including: a display module, used for a first client to enter a collaborative mode of a whiteboard application by joining a target virtual room, and display a first canvas in the collaborative mode, wherein the first client is any client in the target virtual room, and the target virtual room also includes at least one second client; a receiving module, used for the first client to receive a screen sharing instruction, and generate a first media player in the first canvas according to the screen sharing instruction; a determination module, used for the first client to determine a shared window; a playing module, used for the first client to play a first video stream corresponding to the shared window in the first media player, and broadcast the first video stream to the second client, wherein the second client generates a second media player and plays the first video stream in the second media player after receiving the first video stream broadcast by the first client.

[0008] According to another aspect of an embodiment of the present invention, a computer storage medium is provided, wherein the computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing any one of the method steps described above.

[0009] According to another aspect of an embodiment of the present invention, there is also provided a smart interactive tablet, comprising: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing any one of the above method steps.

[0010] In an embodiment of the present invention, the first client enters the collaborative mode of the whiteboard application by joining the target virtual room, and displays the first canvas in the collaborative mode, wherein the first client is any client in the target virtual room, and the target virtual room also includes at least one second client; the first client receives the screen sharing instruction, and generates a first media player in the first canvas according to the screen sharing instruction; the first client determines the shared window; the first client plays the first video stream corresponding to the shared window in the first media player, and broadcasts the first video stream to the second client, wherein the second client generates a second media player and plays the first video stream in the second media player after receiving the first video stream broadcast by the first client. When the above scheme shares the screen in the multi-client collaborative mode, after the screen sharing of the first client is displayed on the conference collaborative whiteboard application, the screen of the second client can be directly shared on the above conference collaborative whiteboard application, without turning off the screen sharing of the first client, and then turning on the screen sharing of the second client to display the documents and materials on the screen of its own client to other clients. In the specific screen sharing process, all clients that enter the target virtual room through the unified target virtual room identifier can start their own screen sharing, and then share the content on the screen with other clients. During the meeting discussion, each user can synchronously see the information shared by other clients through his own client, which is very convenient and efficient. It solves the technical problem in the prior art that when multiple speaking user terminals share the screen as the host client, the host client needs to be frequently switched to realize the screen sharing function of each speaking user, which causes trouble for user operation and affects the meeting efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0012] Figure 1 is a schematic diagram of a canvas and a media player provided according to an embodiment of the present application;

[0013] Figure 2 is a flow chart of a screen sharing method provided according to an embodiment of the present application;

[0014] Figure 3 is a schematic diagram of a candidate window provided according to an embodiment of the present application;

[0015] Figure 4a is a schematic diagram of the layout of a first media player and a third media player provided according to an embodiment of the present application;

[0016] Figure 4bis a schematic diagram of the layout of another first media player and a third media player provided according to an embodiment of the present application;

[0017] Figure 4c is a schematic diagram of the layout of another first media player and a third media player provided according to an embodiment of the present application;

[0018] Figure 5 is a schematic diagram of a first media player moving on a first canvas according to an embodiment of the present application;

[0019] Figure 6 is a schematic diagram of a second media player moving on a second canvas provided according to an embodiment of the present application;

[0020] Figure 7 is a schematic diagram of a first media player scaling along four directions of upper left, lower left, upper right and lower right on a first canvas according to an embodiment of the present application;

[0021] Figure 8 is based on Figure 7 A schematic diagram of a first media player scaling the upper left corner of a first canvas is provided;

[0022] Fig. 9 is a schematic diagram of scaling a second media player on a second canvas provided according to an embodiment of the present application;

[0023] Fig.10 is a schematic diagram of a screen sharing device provided according to an embodiment of the present application;

[0024] Fig.11 It is a schematic diagram of a smart interactive tablet provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] Example 1

[0028] When using a conference collaborative whiteboard application, there are usually multiple users who speak. In the related art, one of the users uses his own client as the host client and uses screen sharing to display materials. At this time, other users can only watch. That is to say, when one user shares the screen of his own client, other clients cannot share the screen of their own clients. Only after each user uses his own client as the host client to display materials using the screen contribution, and then exits the screen sharing, can the clients of other users use their own clients as new host clients to share the screen, and then display their own documents and materials to other users, realizing the screen sharing function. Since screen sharing is usually performed by the client that serves as the host in a collaborative meeting, but the users who speak during the discussion may change frequently, it is necessary to frequently switch the host client to realize the screen sharing function of each speaking user, which in turn causes trouble for the user operation and affects the efficiency of the meeting. Based on such problems, this embodiment proposes a screen sharing method.

[0029] In the screen sharing method provided in the present embodiment, the first client and the second client are used for illustration, and both the first client and the second client run a conference collaboration whiteboard application to achieve synchronous display between the first client and the second client, wherein in this example, the first client and the second client are used as speaking clients for illustration, and the second client can be one or more. For example, in a conference collaboration demonstration scenario, when the first client turns on screen sharing and the second client does not turn on screen sharing, the conference collaboration whiteboard application of the second client will display the screen sharing interface of the first client with a delay; for another example, still in the conference collaboration demonstration scenario, when both the first client and the second client turn on screen sharing, the conference collaboration whiteboard application of the first client will display the screen sharing interface of the first client and the second client, and the conference collaboration whiteboard application of the second client will also display the screen sharing interface of the first client and the second client.

[0030] Before continuing to describe the screen sharing method in this embodiment, the layout of the conference collaboration whiteboard application of the first client in an example in the following embodiment is described. In this example, Figure 1 As shown, the first media player of the first client is hosted in the first canvas, and the size of the first canvas is consistent with the width and height of the screen visible area of ​​the first client.

[0031] like Figure 1 As shown, the first canvas is used to carry the first media player, and the first canvas completely overlaps the screen visible area.

[0032] The width of the first canvas is width, and the height is height, where width also represents the width of the screen visible area, and height also represents the height of the screen visible area.

[0033] The first canvas has a first media player, and the width of the first media player is initWidth, and the height is initHeight.

[0034] Define the upper left corner of the first canvas as the origin (x0, y0) and the vertex of the first media player (initX, initY).

[0035] When the first media player is moved or scaled, only the movement or scaling of the first media player can be seen in the visible area of ​​the screen, and the first canvas does not change.

[0036] According to an embodiment of the present invention, a method embodiment of screen sharing is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0037] Figure 2 is a flow chart of a screen sharing method according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0038] Step S202: The first client enters the collaborative mode of the whiteboard application by joining the target virtual room and displays the first canvas in the collaborative mode, wherein the first client is any client in the target virtual room, and the target virtual room also includes at least one second client.

[0039] Specifically, the first client and the second client may be smart devices with a human-computer interaction interface, such as a mobile phone, a tablet computer, a smart interactive tablet, etc. The first client and the second client enter the collaborative mode of the whiteboard application and join the same target virtual room through a network connection, wherein the first client is any client in the current collaborative meeting, and the second client is a client other than the first client.

[0040] The above-mentioned target virtual room is a group created by the user through the server to represent the conference collaboration whiteboard application. Multiple clients joining the same target virtual room are in the same group. When the collaborative function needs to be used, one of the clients can create a target virtual room, and then generate a target virtual room identifier for the target virtual room. Other clients can join the target virtual room through the target virtual room identifier. The target virtual room identifier can be a target virtual room number or a target virtual room QR code, etc. Other clients can join the target virtual room by entering the target virtual room number or scanning the target virtual room QR code. After joining the target virtual room, the client in the target virtual room can set the client name or number to distinguish different clients belonging to different user operations. For example, it can be set to Zhang San, Li Si, or User One, User Two, etc.

[0041] Taking the conference collaboration demonstration scenario as an example, a client starts the conference collaboration whiteboard application to generate a conference room (and the above-mentioned target virtual room). The conference collaboration whiteboard application generates a corresponding QR code for its conference room. The client shares the generated QR code with other objects that need to participate in the conference. Other objects use their respective clients to scan the QR code and join the above-mentioned conference room, so that many participating clients join the same common conference room.

[0042] Step S204: the first client receives a screen sharing instruction, and generates a first media player in the first canvas according to the screen sharing instruction.

[0043] The above screen sharing instruction is issued by the user through the screen sharing control provided by the first client, and the first client generates the first media player in the first canvas according to the screen sharing instruction. For example, after user A joins the above collaboration mode through the first client, the "share screen" control is triggered, and the first client receives the screen sharing instruction and generates the first media player in the first canvas. Figure 1 As shown, in an optional embodiment, the first media player in the first canvas can be displayed in the center of the first canvas by default, or can be located anywhere in the first canvas, as long as the first media player is located in the first canvas as a whole; the size of the first media player in the first canvas can also be a default size, for example, Figure 1 As shown in the size, the width and height of the first media player are smaller than the width and height of the first canvas. The size of the first media player may also be adjusted according to specific circumstances to meet the needs of the user.

[0044] Step S206: The first client determines a shared window.

[0045] During the specific meeting, the user determines the sharing window according to the content of the presentation. For example, the user can use his or her PowerPoint application as the sharing window, or the browser interface where the Chrome tab is located can be used as the sharing window.

[0046] In an optional embodiment, the screen of the device where the first client is located may have multiple candidate windows, such as an application interface, a browser interface, etc., and one of the above interfaces may be selected as a shared window.

[0047] In another optional embodiment, the screen of the device where the first client is located may have multiple candidate windows, such as the entire screen interface, an application interface, and a browser interface, wherein the entire screen interface may have multiple applications, and when the entire screen interface is used as a shared window, multiple applications may be displayed in the shared window.

[0048] Step S208: The first client plays the first video stream corresponding to the shared window in the first media player, and broadcasts the first video stream to the second client, wherein after receiving the first video stream broadcast by the first client, the second client generates a second media player and plays the first video stream in the second media player.

[0049] Specifically, the above-mentioned media player is used to play the video stream on the canvas, and the acquisition of the local desktop video stream can be achieved by using the WebRtc technology. The video stream information of the current screen, application window and browser tab can be obtained through the WebRtc technology.

[0050] It should be noted that the conference collaboration whiteboard applications run by the first client and the second client can provide a canvas through the canvas tag in the browser HTML, use the video tag in HTML to generate a video stream container in the canvas, and initialize a media player in the video stream container to play the video stream.

[0051] After receiving the first video stream broadcast by the first client, the second client generates a second media player and plays the first video stream in the second media player. Specifically, the following steps are included: first, the first client sends the first video stream to the server, and the server sends the first video stream to the second client. After the second client receives the first video stream sent by the server, a second media player is generated in the second canvas of the second client. Finally, the first video stream is played in the second media player to achieve an effect of screen sharing between the second client and the first client.

[0052] It should be noted that when the first video stream corresponding to the shared window is played in the first media player, the second media player will delay playing the first video stream according to the network bandwidth, the current network rate, and the hardware network configuration conditions of the first client and the second client.

[0053] For example, when the network bandwidth is 100 Mbps, the play delay between the second client and the first client is 200 milliseconds, and when the network bandwidth is 1000 Mbps, the play delay between the first client and the second client is 50 milliseconds.

[0054] For another example, when the network bandwidth is 1000 Mbps and the hardware network configuration of the second client is low, the play delay between the first client and the second client is 120 milliseconds. In an ideal case, when the network bandwidth is good enough and the hardware network configuration of the first client and the second client is good enough, the play delay between the first client and the second client can be approximately zero, that is, the first video stream in the first media player can be played synchronously in the second media player.

[0055] As can be seen from the above, in the above embodiment of the present application, the first client enters the collaborative mode of the whiteboard application by joining the target virtual room, and displays the first canvas in the collaborative mode, wherein the first client is any client in the target virtual room, and the target virtual room also includes at least one second client; the first client receives the screen sharing instruction, and generates a first media player in the first canvas according to the screen sharing instruction; the first client determines the shared window; the first client plays the first video stream corresponding to the shared window in the first media player, and broadcasts the first video stream to the second client, wherein the second client generates a second media player and plays the first video stream in the second media player after receiving the first video stream broadcast by the first client. When the above scheme shares the screen in the multi-client collaborative mode, after the screen sharing of the first client is displayed on the conference collaborative whiteboard application, the screen of the second client can be directly shared on the above conference collaborative whiteboard application, without turning off the screen sharing of the first client, and then turning on the screen sharing of the second client to display the documents and materials on the screen of its own client to other clients. In the specific screen sharing process, all clients that enter the target virtual room through the unified target virtual room identifier can start their own screen sharing, and then share the content on the screen with other clients. During the meeting discussion, each user can synchronously see the information shared by other clients through his own client, which is very convenient and efficient. It solves the technical problem in the prior art that when multiple speaking user terminals share the screen as the host client, the host client needs to be frequently switched to realize the screen sharing function of each speaking user, which causes trouble for user operation and affects the meeting efficiency.

[0056] The following describes a method for the first client to initialize the first canvas. The first client enters the collaborative mode of the whiteboard application by joining the target virtual room, including: the first client initializes the first canvas in the collaborative mode, determines the width information and the height information of the first canvas; the first client displays the first canvas with the width information and the height information of the first canvas.

[0057] like Figure 1 As shown, a canvas tag in HTML is used to initialize a canvas on the drawing board, which can be the first canvas, used to carry the video stream container, and the video stream container is used to carry the first media player.

[0058] In the specific initialization process, firstly, the screen width and height clientHeight and clientWidth of the current client, i.e., the first client, are obtained, and the width and height width and height of the first canvas are set, and the coordinates are x0=0 and y0=0. By setting the width and height of the first canvas and the coordinates, the specific size and position of the first canvas can be determined, so that the first canvas is displayed in the specified area, wherein the width and height of the first canvas are less than or equal to the width and height of the first client.

[0059] In an optional embodiment, the height of the first client is height=clientHeight, the width is width=clientWidth, and the coordinates are x0=0, y0=0, that is, the size of the first canvas is consistent with the width and height of the screen visible area of ​​the first client, so that the first canvas and the screen of the first client completely overlap.

[0060] As an optional embodiment, the first client receives a screen sharing instruction and generates a first media player in the first canvas according to the screen sharing instruction. The above method also includes: the first client initializes the first media player on the first canvas and determines the width information and height information of the first media player; the first client generates the first media player with the width information and height information of the first media player.

[0061] Combination Figure 1 As shown, a media player is initialized on the canvas using the video tag in HTML to support the playback of the video stream on the canvas.

[0062] In the specific initialization process, the initial width and height initWidth and initHeight of a media player are defined on the canvas, and the initial position is (initX, initY). The initial position of the media player is used as the origin, and the media player is generated with the initial width information and the initial height information. Through this initialization setting, the specific position of the media player on the canvas and the size of the media player can be determined, wherein the width and height of the media player are less than or equal to the width and height of the canvas.

[0063] For example, the width and height of the first canvas are width and height respectively, the first media player is located in the middle of the first canvas, and the width initWidth=1 / 3width, and the height initHeight=1 / 3height.

[0064] As an optional embodiment, the first client determines a shared window, including: the first client displays a shared window selection interface, wherein the shared window selection interface includes at least one candidate window allowed to be shared; the first client receives shared window selection information and selects a shared window from the candidate windows.

[0065] When a user needs to share the content on the client screen in conference collaboration mode, he can determine all the application interfaces to be shared according to his own shared content and display them in the shared window selection interface of the first client. The user uses all the above application interfaces as candidate windows. After the first client receives the user's selection information, it selects a shared window from multiple candidate windows to determine the shared window that needs to be shared.

[0066] For example, combined with Figure 3 As shown, when there are three candidate windows in the window selection interface of the client, namely: the entire screen of the client, a certain application window of the client and a certain browser tab of the client, the user determines the window to be shared according to the information that he currently needs to share. The user can use human-computer interaction, such as touch, to make the client receive the selection information of the shared window. After receiving the selection information of the shared window, the client selects the window to be shared from the candidate windows.

[0067] As an optional embodiment, after the first client plays the first video stream corresponding to the shared window in the first media player and broadcasts the first video stream to the second client, the method further includes: the first client receives the second video stream broadcast by the second client, wherein the second client broadcasts the second video stream to the first client after receiving the screen sharing instruction; the first client generates a third media player on the first canvas according to the second video stream, and displays the second video stream in the third media player, wherein the third media player and the first media player are displayed according to a preset layout.

[0068] In the scheme of this embodiment, at least two clients are included in the same target virtual room, and each client can perform screen sharing. Specifically, when the first client receives the screen sharing instruction, after the first client obtains the local desktop video stream of the first client as the first video stream through the WebRtc technology, it is played in the shared window of the first media player. During the playback process, the first video stream is broadcast to the second media player of the second client with a delay according to the network bandwidth and the hardware network configuration of the client; in this process, the second client can also perform screen sharing. After the second client receives the screen sharing instruction, the second client can also obtain the local desktop video stream of the second client as the second video stream through the above technology, and the second client can also generate a fourth media player different from the second media player locally, display the second video stream through the fourth media player, and then broadcast the second video stream to other clients in the target virtual room; after the first client receives the second video stream, it generates a third media player on the first canvas, and plays the second video stream in the third media player, thereby realizing the scheme of screen sharing of multiple clients at the same time.

[0069] It should be noted that, in some cases, the screen sizes of the first client and the second client are different. For example, the width and height of the screen of the first client are twice the width and height of the screen of the second client, and the sizes of the first canvas and the second canvas are the same as the screen of the first client and the second client, respectively. Based on the sizes of the screens of the first client and the second client, a first media player and a third media player are generated on the first canvas, and a second media player and a fourth media player are generated on the second canvas, wherein the first media player is twice the size of the fourth media player, and the third media player is twice the size of the second media player. In a specific broadcasting process, when the second video stream corresponding to the shared window played in the second media player is broadcast to the third media player of the first client, the shared window corresponding to the second video stream needs to be enlarged twice and displayed in the third media player. Similarly, when the first video stream corresponding to the shared window played in the first media player is broadcast to the fourth media player of the second client, the shared window corresponding to the first video stream needs to be reduced by two times and displayed in the fourth media player.

[0070] The first media player and the third media player can be displayed according to a preset layout, combined with Figure 4a-4c As shown. Figure 4a In the example, the first media player and the third media player can be horizontally preset on both sides of the center point of the first canvas; Figure 4b As shown, the first media player and the third media player can be vertically preset on both sides of the center point of the first canvas. Figure 4cAs shown, the first media player and the third media player can be set on the diagonal line of the first canvas and located on both sides of the center point. When the specific preset layout is used, it can be specifically set according to the specific situation. This embodiment does not limit the positions of the first media player, the third media player and other media players.

[0071] As an optional embodiment, after the first client plays the first video stream corresponding to the shared window in the first media player and broadcasts the first video stream to the second client, the method also includes: the first client receives a movement operation; the first client controls the first media player to translate on the first canvas according to the movement operation; the first client sends movement information corresponding to the movement operation to the second client, wherein the second client controls the second media player to translate on the second canvas according to the movement information.

[0072] Combination Figure 5 and Figure 6 As shown, after the first client receives the moving operation instruction, the first media player moves from point (x1, y1) to point (x2, y2) on the first canvas. After the first media player moves on the first canvas, the first client sends the moving operation information of the first media player to the second client, and the second client controls the second media player to move from point (x3, y3) to a new position accordingly according to the moving information.

[0073] In an optional embodiment, the first client sends movement information corresponding to the movement operation to the second client, including: the first client determines the movement information according to the position of the first media player before translation and the position after translation, wherein the movement information includes: a horizontal offset and a vertical offset; the first client sends the movement information to the second client, wherein the second client controls the second media player to move on the second canvas according to the position of the second media player before translation and the movement information.

[0074] Combination Figure 5 and Figure 6 As shown, the width and height of the first canvas of the first client are N times the width and height of the second canvas of the second client. The user of the first client moves the first media player from point (x1, y1) to point (x2, y2). The user of the second client needs to update the position of the second media player in real time to ensure that the positions of the first media player and the second media player are synchronized. The calculation process is as follows:

[0075] 1. Calculate the movement information of the first media player in the x and y directions of the first canvas: horizontal offset dx=x2-x1, vertical offset dy=y2-y1.

[0076] 2. The first client sends a broadcast message to the second client in the same target virtual room, and the broadcast message carries the movement information.

[0077] 3. After receiving the mobile information broadcast by the first client, the second client parses dx and dy from the mobile information.

[0078] 4. The second client updates the position of the second media player according to the size of the second canvas, and the new position is point (x3+dx / N, y3+dy / N).

[0079] In an optional implementation, the width and height of the first canvas are the same as those of the second canvas, and the new position of the second media player on the second canvas is point (x3+dx, y3+dy). Figure 6 As shown, the width and height of the first canvas mentioned above are twice the width and height of the second canvas, so the new position of the second media player on the second canvas is point (x3+dx / 2, y3+dy / 2).

[0080] As an optional embodiment, after the first client plays the first video stream corresponding to the shared window in the first media player and broadcasts the first video stream to the second client, the method further includes: the first client receives a zoom operation; the first client controls the first media player to zoom according to the zoom operation; the first client sends zoom information corresponding to the zoom operation to the second client, wherein the second client controls the second media player to zoom on the second canvas according to the zoom information.

[0081] Combination Figure 8 As shown, after the first client receives the zoom operation instruction, the first media player zooms from point (x1, y1) to point (x2, y2) on the first canvas. The width before zooming is w1 and the height is h1, and the width after zooming is w2 and the height is h2. After the first media player zooms on the first canvas, the first client sends the zoom operation information of the first media player to the second client accordingly. The second client controls the second media player to zoom to a new position from point (x3, y3) with a width and height of w3 and h4 according to the zoom information.

[0082] In an optional embodiment, the first client controls the first media player to zoom according to the zoom operation, including: the first client determines the zoom ratio according to the position of the first media player before zooming and the position after zooming; the first client determines the width and height information after zooming according to the width and height information of the first media player before zooming and the zoom ratio; the first client displays the first media player according to the position after zooming and the width and height information after zooming. The zoom information includes: the position and width and height information of the first media player after zooming, and the second client zooms the second media player according to the position after zooming and the width and height information after zooming.

[0083] Combination Figure 7 and Figure 8 The width and height of the first canvas of the first client are N times the width and height of the second canvas of the second client. The user of the first client scales the first media player from point (x1, y1) to point (x2, y2). The user of the second client needs to update the position of the second media player in real time to ensure that the positions of the first media player and the second media player are synchronized. The calculation process is as follows:

[0084] 1. Calculate the scaling ratio of the first media player on the first canvas: ratioX=x2 / x1, ratioY=y2 / y1.

[0085] 2. Calculate the width and height of the scaled first media player according to the scaling ratio: w2=ratioX*w1, h2=ratioY*h1.

[0086] 3. The first client updates the coordinates of the first media player to (x2, y2), and the width and height to w2 and h2.

[0087] 4. The first client broadcasts the scaling information to the second client, and the second client adaptively updates the position of the second media player according to the size of the second canvas. The new position is adaptively updated by the scaling direction, scaling information and the size of the second canvas.

[0088] In an optional embodiment, in combination Figure 8 and Fig. 9 As shown, when the width and height of the first canvas of the first client are twice the width and height of the second canvas of the second client, the user of the first client scales the first media player from point (x1, y1) to point (x2, y2), then the new position updated by the second media player on the second canvas is Width is Height is For another example, the width and height of the first canvas of the first client described above are consistent with the width and height of the second canvas of the second client. The user of the first client scales the first media player from point (x1, y1) to point (x2, y2), then the new position of the second media player updated on the second canvas is (x3*ratioX, y3*ratioY), the width is w3*ratioX, and the height is h3*ratioY.

[0089] Example 2

[0090] According to an embodiment of the present application, a screen sharing device is provided. Fig.10 is a schematic diagram of a screen sharing device according to an embodiment of the present application, such as Fig.10 As shown, the screen sharing device includes:

[0091] The display module 100 is used for a first client to enter a collaborative mode of a whiteboard application by joining a target virtual room and display a first canvas in the collaborative mode, wherein the first client is any client in the target virtual room, and the target virtual room also includes at least one second client;

[0092] The receiving module 102 is used for the first client to receive the screen sharing instruction and generate a first media player in the first canvas according to the screen sharing instruction;

[0093] A determination module 104, configured for the first client to determine a shared window;

[0094] The playing module 106 is used for the first client to play the first video stream corresponding to the shared window in the first media player, and broadcast the first video stream to the second client, wherein the second client generates a second media player and plays the first video stream in the second media player after receiving the first video stream broadcast by the first client.

[0095] As an optional embodiment, the first client enters the collaborative mode of the whiteboard application by joining the target virtual room, including:

[0096] A first determination module, configured for the first client to initialize the first canvas in the collaborative mode and determine the width information and the height information of the first canvas;

[0097] The first display module is used for the first client to display the first canvas using the width information and height information of the first canvas.

[0098] As an optional embodiment, the first client receives a screen sharing instruction, and generates a first media player in the first canvas according to the screen sharing instruction, and the receiving module includes:

[0099] A determination submodule, configured for the first client to initialize the first media player on the first canvas and determine the width information and the height information of the first media player;

[0100] The generating submodule is used for the first client to generate the first media player using the width information and height information of the first media player.

[0101] As an optional embodiment, the determining module includes:

[0102] A second display module, configured to display a shared window selection interface on the first client, wherein the shared window selection interface includes at least one candidate window that is allowed to be shared;

[0103] The selection module is used for the first client to receive the shared window selection information and select a shared window from the candidate windows.

[0104] As an optional embodiment, after the first client plays the first video stream corresponding to the shared window in the first media player and broadcasts the first video stream to the second client, the playing module further includes:

[0105] A first receiving module, configured for a first client to receive a second video stream broadcasted by a second client, wherein the second client broadcasts the second video stream to the first client after receiving a screen sharing instruction;

[0106] The third display unit is used for the first client to generate a third media player on the first canvas according to the second video stream, and display the second video stream in the third media player, wherein the third media player and the first media player are displayed according to a preset layout.

[0107] As an optional embodiment, after the first client plays the first video stream corresponding to the shared window in the first media player and broadcasts the first video stream to the second client, the playing module further includes:

[0108] A second receiving module, used for the first client to receive a moving operation;

[0109] A first control module, used for the first client to control the first media player to move horizontally on the first canvas according to the movement operation;

[0110] The first sending module is used for the first client to send movement information corresponding to the movement operation to the second client, wherein the second client controls the second media player to move horizontally on the second canvas according to the movement information.

[0111] As an optional embodiment, the sending module includes:

[0112] A first determining unit, configured for the first client to determine movement information according to the position of the first media player before translation and the position of the first media player after translation, wherein the movement information includes: a horizontal offset and a vertical offset;

[0113] The sending unit is used for the first client to send the movement information to the second client, wherein the second client controls the second media player to move on the second canvas according to the position of the second media player before translation and the movement information.

[0114] As an optional embodiment, after the first client plays the first video stream corresponding to the shared window in the first media player and broadcasts the first video stream to the second client, the playing module further includes:

[0115] A third receiving module, configured for the first client to receive a zoom operation;

[0116] A second control module, used for the first client to control the first media player to zoom in and out according to the zoom operation;

[0117] The second sending module is used for the first client to send zoom information corresponding to the zoom operation to the second client, wherein the second client controls the second media player to zoom on the second canvas according to the zoom information.

[0118] As an optional embodiment, the second control module includes:

[0119] A second determining unit, configured for the first client to determine a scaling ratio according to a position of the first media player before scaling and a position of the first media player after scaling;

[0120] A third determining unit, configured for the first client to determine the width and height information after scaling according to the width and height information and the scaling ratio before scaling by the first media player;

[0121] The display unit is used for the first client to display the first media player according to the scaled position and the scaled width and height information.

[0122] As an optional embodiment, the scaling information includes: the scaled position and width and height information of the first media player, and the second client scales the second media player according to the scaled position and the scaled width and height information.

[0123] Example 3

[0124] According to an embodiment of the present application, an embodiment of a computer storage medium is provided, wherein the computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing any one of the method steps in embodiment 1. In this embodiment, a first client enters the collaborative mode of a whiteboard application by joining a target virtual room, and displays a first canvas in the collaborative mode, wherein the first client is any client in the target virtual room, and the target virtual room also includes at least one second client; the first client receives a screen sharing instruction, and generates a first media player in the first canvas according to the screen sharing instruction; the first client determines a shared window; the first client plays a first video stream corresponding to the shared window in the first media player, and broadcasts the first video stream to the second client, wherein the second client generates a second media player and plays the first video stream in the second media player after receiving the first video stream broadcast by the first client. When the above scheme shares the screen in a multi-client collaborative mode, after the screen sharing of the first client is displayed on the conference collaborative whiteboard application, the screen of the second client can be directly shared on the above conference collaborative whiteboard application, without turning off the screen sharing of the first client, and then turning on the screen sharing of the second client to display the documents and materials on the screen of the client to other clients. In the specific screen sharing process, all clients that enter the target virtual room through the unified target virtual room identifier can start their own screen sharing, and then share the content on the screen with other clients. During the meeting discussion, each user can synchronously see the information shared by other clients through his own client, which is very convenient and efficient. It solves the technical problem in the prior art that when multiple speaking user terminals share the screen as the host client, the host client needs to be frequently switched to realize the screen sharing function of each speaking user, which causes trouble for user operation and affects the meeting efficiency.

[0125] Example 4

[0126] According to an embodiment of the present application, a smart interactive tablet is provided, comprising: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing any one of the method steps in Example 1.

[0127] Fig.11 is a schematic diagram of a smart interactive tablet provided in an embodiment of the present application, the smart interactive tablet comprises the above-mentioned interactive device body and touch frame, combined with Fig.11 As shown, the smart interactive tablet 1000 may include: at least one processor 1001 , at least one network interface 1004 , a user interface 1003 , a memory 1005 , and at least one communication bus 1002 .

[0128] The communication bus 1002 is used to realize the connection and communication between these components.

[0129] The user interface 1003 may include a display screen (Display) and a camera (Camera), and the optional user interface 1003 may also include a standard wired interface and a wireless interface.

[0130] The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0131] Among them, the processor 1001 may include one or more processing cores. The processor 1001 uses various interfaces and lines to connect the various parts of the entire intelligent interactive tablet 1000, and executes various functions and processes data of the intelligent interactive tablet 1000 by running or executing instructions, programs, code sets or instruction sets stored in the memory 1005, and calling data stored in the memory 1005. Optionally, the processor 1001 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 1001 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processor (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 1001, and it can be implemented separately through a chip.

[0132] Among them, the memory 1005 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 1005 includes a non-transitory computer-readable storage medium. The memory 1005 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 1005 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 1005 may also be optionally at least one storage device located away from the aforementioned processor 1001.

[0133] exist Fig.11 In the smart interactive tablet 1000 shown, the user interface 1003 is mainly used to provide an input interface for the user and obtain data input by the user; and the processor 1001 can be used to call the operating application of the smart interactive tablet stored in the memory 1005, and specifically perform any one of the operations in Example 1.

[0134] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0135] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0136] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0137] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0138] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0139] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0140] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A screen sharing method, characterized in that: include: The first client enters the collaborative mode of the whiteboard application by joining a target virtual room, and displays a first canvas in the collaborative mode, wherein the first client is any client in the target virtual room, and the target virtual room also includes at least one second client; The first client receives a screen sharing instruction, and generates a first media player in the first canvas according to the screen sharing instruction; The first client determines a shared window; The first client plays the first video stream corresponding to the shared window in the first media player, and broadcasts the first video stream to the second client, wherein the second client generates a second media player and plays the first video stream in the second media player after receiving the first video stream broadcast by the first client, wherein the second media player is generated in a second canvas of the second client; After the first client plays the first video stream corresponding to the shared window in the first media player and broadcasts the first video stream to the second client, the method further includes: the first client receives the second video stream broadcast by the second client, wherein the second client broadcasts the second video stream to the first client after receiving the screen sharing instruction; the first client generates a third media player on the first canvas according to the second video stream, and displays the second video stream in the third media player, wherein the third media player and the first media player are displayed according to a preset layout.

2. The method according to claim 1, characterized in that The first client enters the collaborative mode of the whiteboard application by joining the target virtual room, including: The first client initializes the first canvas in the collaboration mode and determines width information and height information of the first canvas; The first client displays the first canvas using the width information and the height information of the first canvas.

3. The method according to claim 1, characterized in that The first client receives a screen sharing instruction, and generates a first media player in the first canvas according to the screen sharing instruction, including: The first client initializes a first media player on the first canvas, and determines width information and height information of the first media player; The first client generates the first media player using the width information and height information of the first media player.

4. The method according to claim 1, characterized in that: The first client determines the shared window, including: The first client displays a sharing window selection interface, wherein the sharing window selection interface includes at least one candidate window allowed to be shared; The first client receives the shared window selection information and selects one of the shared windows from the candidate windows.

5. The method according to claim 1, characterized in that After the first client plays the first video stream corresponding to the shared window in the first media player and broadcasts the first video stream to the second client, the method further includes: The first client receives a move operation; The first client controls the first media player to move horizontally on the first canvas according to the movement operation; The first client sends movement information corresponding to the movement operation to the second client, wherein the second client controls the second media player to move horizontally on the second canvas according to the movement information.

6. The method according to claim 5, characterized in that The first client sends movement information corresponding to the movement operation to the second client, including: The first client determines the movement information according to the position of the first media player before translation and the position of the first media player after translation, wherein the movement information includes: a horizontal offset and a vertical offset; The first client sends the movement information to the second client, wherein the second client controls the second media player to move on the second canvas according to the position of the second media player before translation and the movement information.

7. The method according to claim 5, characterized in that After the first client plays the first video stream corresponding to the shared window in the first media player and broadcasts the first video stream to the second client, the method further includes: The first client receives a zoom operation; The first client controls the first media player to zoom according to the zoom operation; The first client sends scaling information corresponding to the scaling operation to the second client, wherein the second client controls the second media player to scale on the second canvas according to the scaling information.

8. The method according to claim 7, characterized in that The first client controls the first media player to zoom according to the zoom operation, including: The first client determines a scaling ratio according to a position of the first media player before scaling and a position of the first media player after scaling; The first client determines the width and height information after scaling according to the width and height information before scaling by the first media player and the scaling ratio; The first client displays the first media player according to the scaled position and the scaled width and height information.

9. The method according to claim 8, characterized in that The scaling information includes: the position and width and height information of the first media player after scaling, and the second client scales the second media player according to the scaled position and the scaled width and height information.

10. A screen sharing device, characterized in that: include: A display module, configured for a first client to enter a collaborative mode of a whiteboard application by joining a target virtual room and displaying a first canvas in the collaborative mode, wherein the first client is any client in the target virtual room, and the target virtual room further includes at least one second client; A receiving module, configured for the first client to receive a screen sharing instruction and generate a first media player in the first canvas according to the screen sharing instruction; A determination module, used by the first client to determine a shared window; a playing module, configured for the first client to play the first video stream corresponding to the shared window in the first media player, and broadcast the first video stream to the second client, wherein the second client generates a second media player and plays the first video stream in the second media player after receiving the first video stream broadcast by the first client, wherein the second media player is generated in a second canvas of the second client; After the first client plays the first video stream corresponding to the shared window in the first media player and broadcasts the first video stream to the second client, the playback module also includes: a first receiving module, used for the first client to receive the second video stream broadcast by the second client, wherein the second client broadcasts the second video stream to the first client after receiving the screen sharing instruction; a third display unit, used for the first client to generate a third media player on the first canvas according to the second video stream, and display the second video stream in the third media player, wherein the third media player and the first media player are displayed according to a preset layout.

11. A computer storage medium, characterized in that: The computer storage medium stores a plurality of instructions, which are suitable for being loaded by a processor and executing the method steps according to any one of claims 1 to 9.

12. A smart interactive tablet, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, the computer program being suitable for being loaded by the processor and executing the method steps as claimed in any one of claims 1 to 9.

Images