Multi-terminal cooperation display update method and device

Abstract

The present invention discloses a multi-terminal cooperative display update method and device. The method includes the steps of: a first terminal displays a first application interface of a first cooperative whiteboard application, the first cooperative whiteboard application and a second cooperative whiteboard application of a second terminal are in a cooperative mode, and the second terminal displays a second application interface of the second cooperative whiteboard application; receiving control parameters transmitted by the second terminal, the control parameters being generated when the second terminal accepts a control operation on the second application interface; and updating the content of the first application interface based on the control parameters by the first terminal, where the control operation is confirmed to be a movement operation, the first canvas moves based on the control parameters, and when the edges of the first canvas and the first application interface in the same direction overlap, the first terminal re-renders the content of the first canvas. The present invention solves the problem of the prior art that when the canvas of the terminal to be followed expands infinitely, it is difficult for the following terminal to follow.

Description

[Technical field]

[0001] The present invention relates to the field of display technology, and more particularly to a multi-terminal cooperative display update method and device. [Background technology]

[0002] Since the conference linked whiteboard has the functions of demonstration and lecture, during a conference, multiple terminals participating in the same conference linked whiteboard process can realize the function of demonstration following exhibition. For example, a conference linked whiteboard application is executed on each terminal, and a speaker can transfer the content of the terminal he / she uses to other terminals in the same conference linked whiteboard process, and the terminal used by the speaker is the follow-up target terminal in this process, and the other terminals are follow-up terminals. The application interface of the follow-up terminal can synchronously display the application interface of the follow-up target terminal, thereby realizing the demonstration follow-up function in the conference linked whiteboard application.

[0003] Typically, the terminal to be tracked needs to use a function such as a paintbrush to draw content in a web drawing application or web drawing function such as a conference collaboration whiteboard, and the terminal to be tracked needs to track this operation, but since it uses a canvas (single canvas) that is the same size as the application interface, it is necessary to recalculate the position of each point and redraw the content of the canvas to achieve the canvas extension function every time the canvas is moved to follow the operation of the terminal to be tracked. Since the canvas is cleared and redrawn every time the canvas is moved, if the amount of data in the canvas is large, the redrawing time will be long and even freezing will occur, and redrawing will be triggered every time it is moved, resulting in low efficiency and low scalability.

[0004] When the canvas of the device to be tracked expands infinitely, the device to be tracked has difficulty tracking, a problem that has been encountered with conventional technology, but no effective technology has been proposed to date. Summary of the Invention

[0005] The embodiments of the present invention provide a method and apparatus for multi-terminal cooperative display update to at least solve the problem in the prior art that visual angle tracking is not very effective due to pixel differences between a tracked terminal and a tracked terminal.

[0006] According to one aspect of an embodiment of the present invention, there is provided a multi-terminal cooperative display update method, the display update method being used for realizing synchronous display of an operation by a first terminal to a second terminal, the first terminal providing a first application interface of a first cooperative whiteboard application, a first canvas for carrying drawing content of the first cooperative whiteboard application, and a first container for carrying the first canvas, the second terminal providing a second application interface of a second cooperative whiteboard application, a second canvas for carrying drawing content of the second cooperative whiteboard application, and a second container for carrying the second canvas, a first length of the first canvas is greater than a length of the first application interface, and a width of the first canvas is greater than a width of the first application interface, and the multi-terminal cooperative display update method is The method includes the steps of: displaying a first application interface of the application, where the first coordinated whiteboard application and a second coordinated whiteboard application of the second terminal are in a coordinated mode and the second terminal displays a second application interface of the second coordinated whiteboard application; receiving control parameters transmitted by the second terminal, the control parameters being generated when the second terminal accepts a control operation on the second application interface; and updating content of the first application interface based on the control parameters by the first terminal, where the control operation is confirmed to be a move operation, the first canvas moves based on the control parameters, and if edges of the first canvas and the first application interface in the same direction overlap, the first terminal re-renders the content of the first canvas.

[0007] According to another aspect of the embodiment of the present invention, a multi-terminal cooperative display update device is provided, and a display update method is used for realizing synchronous display of an operation by a first terminal to a second terminal, the first terminal provides a first application interface of a first cooperative whiteboard application, a first canvas for carrying drawing content of the first cooperative whiteboard application, and a first container for carrying the first canvas, the second terminal provides a second application interface of a second cooperative whiteboard application, a second canvas for carrying drawing content of the second cooperative whiteboard application, and a second container for carrying the second canvas, a first length of the first canvas is greater than a length of the first application interface, and a width of the first canvas is greater than a width of the first application interface, and the multi-terminal cooperative display update device is the first terminal displays a first application interface of the first coordinated whiteboard application, where the first coordinated whiteboard application and a second coordinated whiteboard application of the second terminal are in a coordinated mode and the second terminal displays the second application interface of the second coordinated whiteboard application; a receiving module used to receive control parameters transmitted by the second terminal, where the control parameters are generated when the second terminal accepts a control operation on the second application interface; and an update module used by the first terminal to update content of the first application interface based on the control parameters, where the control operation is confirmed to be a move operation, the first canvas is moved based on the control parameters, and when edges of the first canvas and the first application interface in the same direction overlap, the first terminal re-renders content of the first canvas.

[0008] According to another aspect of an embodiment of the present invention there is provided a computer storage medium storing a plurality of instructions, the instructions being suitable for being loaded by a processor to perform the steps of the method according to any one of the above claims.

[0009] According to another aspect of an embodiment of the present invention there is provided an intelligent interactive tablet comprising a processor and a memory, the memory storing a computer program suitable for being loaded by the processor to perform the steps of the method according to any one of the above claims.

[0010] In an embodiment of the present invention, a first terminal displays a first application interface of a first coordinated whiteboard application, the first coordinated whiteboard application and a second coordinated whiteboard application of a second terminal are in a coordinated mode, and the second terminal displays a second application interface of the second coordinated whiteboard application, receives control parameters sent by the second terminal, the control parameters are generated when the second terminal accepts a control operation on the second application interface, the first terminal updates content of the first application interface based on the control parameters, it is confirmed that the control operation is a move operation, the first canvas moves based on the control parameters, and if edges in the same direction of the first canvas and the first application interface overlap, the first terminal re-renders the content of the first canvas. The above technology, when multiple terminals are linked, sets a multiple-fold canvas in the linked whiteboard application of the following terminal, eliminating the need for the following terminal to clear and redraw the canvas content each time the canvas is moved when following the movement of the terminal to be followed; and when an edge in the same direction of the canvas and the application interface overlap in the linked whiteboard application of the following terminal, the canvas content of the linked whiteboard application is re-rendered, allowing the following terminal to follow this effect when the terminal to be followed moves on the infinite canvas in the linked whiteboard application, thereby solving the problem of the conventional technology that it is difficult for the following terminal to follow when the canvas of the terminal to be followed expands infinitely. [Brief description of the drawings]

[0011] The accompanying drawings described herein are for further understanding of the present invention and constitute a part of this application, but the schematic embodiments of the present invention and their explanations are for interpreting the present invention and do not constitute undue limitations on the present invention. The drawings are as follows: [Figure 1] FIG. 2 is an exemplary schematic diagram of a double canvas according to an embodiment of the present application. [Diagram 2] 2 is a flowchart of a multi-terminal cooperative display update method according to an embodiment of the present application; [Figure 3a] 2 is an exemplary schematic diagram of a canvas edge and a viewport edge according to an embodiment of the present application. [Figure 3b] FIG. 2 is another exemplary schematic diagram of canvas edges and viewport edges according to an embodiment of the present application. [Figure 4] FIG. 2 is an exemplary schematic diagram of a canvas before it is moved according to an embodiment of the present application; [Diagram 5] FIG. 13 is an exemplary schematic diagram after the canvas has been moved according to an embodiment of the present application; [Figure 6] FIG. 2 is a schematic diagram of determining whether an edge of a canvas and an edge of an application interface overlap according to an embodiment of the present application; [Figure 7a] FIG. 2 is a schematic diagram of origin-based scaling according to an embodiment of the present application; [Figure 7b] FIG. 13 is another schematic diagram of origin-based scaling according to an embodiment of the present application. [Figure 8a] FIG. 1 is a schematic diagram of target point based scaling according to an embodiment of the present application; [Figure 8b] FIG. 13 is another schematic diagram of target point based scaling according to an embodiment of the present application. [Figure 9] FIG. 2 is a schematic diagram of a writing stroke made up of multiple points according to an embodiment of the present application; [Figure 10] 1 is a schematic diagram of a multi-terminal cooperation display update device according to an embodiment of the present invention; [Figure 11]1 is a schematic diagram of an intelligent interactive tablet according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] In order to allow those skilled in the art to better understand the technology of the present invention, the following will clearly and completely describe the technical technology in the embodiments of the present invention in combination with the drawings of the embodiments of the present invention, and obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, other embodiments that those skilled in the art can think of without creative efforts shall all belong to the protection scope of the present invention.

[0013] It should be noted that the terms "first," "second," and the like in the present specification and claims and the drawings are intended to distinguish between similar objects and are not necessarily used to describe a particular order or sequence. Such used data should be understood to be interchangeable where appropriate, such that the embodiments of the present invention described herein can be performed in other orders than those illustrated or described herein. Also, the terms "comprise" and "have" and any variations thereof are intended to cover non-exclusive inclusions, e.g., a process, method, system, product, or device that includes a series of steps or units need not be limited to explicitly recited steps or units, but may include other steps or units not explicitly recited or inherent to these processes, methods, products, or devices.

[0014] Example 1 In this embodiment, a first terminal and a second terminal are used for explanation, and a coordinated whiteboard application is executed on both the first terminal and the second terminal to realize the synchronous display between the first terminal and the second terminal, and in this example, the second terminal is the terminal to be followed and the first terminal is the following terminal, and the first terminal may be one or more. For example, in a conference cooperation demonstration scenario, the second terminal is the terminal used by the conference performer, the first terminal is the terminal used by the audience, and the first terminal follows the second terminal to display synchronously.

[0015] In a specific implementation scenario, the linked whiteboard applications executed on the first terminal and the second terminal can all implement web drawing functions, and the canvas is an area for carrying the drawing content in the web drawing application. The browser html canvas tag may be used for the canvas, and canvas is a common tag in browser html, which can be used to better render in the browser. The chrome kernel is used as a carrier to realize the cross-platform effect, and the div tag is used for the container, using the browser's absolute position and changing the offset value in real time to achieve the canvas movement effect. The canvas is used to carry the drawing content of the whiteboard application, and a transparent layer is covered on the canvas, and the transparent layer has a toolbar, and when the toolbar tool is clicked and operated, this is reflected in the change of the canvas content.

[0016] A container is used to carry the canvas, and moving the canvas is essentially moving the container. After the canvas is moved, its offset from the viewport changes, which may include changing the horizontal and vertical offsets.

[0017] The application interface of the collaborative whiteboard application is a viewport, which is an area on the terminal screen in which a web drawing application can be displayed, and which is a rectangular area relative to the device, and the coordinate unit is a "pixel" relative to the device.

[0018] In the process of using a linked whiteboard application, in order to expand the drawing range and realize the effect of an infinite canvas, the second terminal can use a canvas that is multiple times larger than the application interface in the linked whiteboard application, and the user can perform various operations such as translation, stretching, and writing on the second terminal, but the first terminal needs to follow these operations to realize synchronous display with the second terminal. Since a canvas (single canvas) with the same size as the application interface is used, each time the canvas is moved following the operation of the terminal to be followed, the position of each point needs to be recalculated and the content of the canvas needs to be redrawn to achieve the canvas expansion function. Since the canvas needs to be cleared and redrawn every time the canvas is moved, if the amount of data in the canvas is large, the redrawing time will be long and even freezes will occur, and redrawing will be triggered every time it is moved, resulting in low efficiency and low scalability. Based on such problems, this embodiment provides a multi-terminal linked display update method.

[0019] Before describing the multi-terminal cooperative display update method in this embodiment, we will first describe the layout of the terminal cooperative whiteboard application of the first terminal in the example shown in the following embodiment. In this example, the first canvas of the first terminal is a double canvas. FIG. 1 is an exemplary schematic diagram of a double canvas according to an embodiment of the present application. As shown in FIG. 1, the length of the first canvas is multiple times the length of the second application interface, and the width of the first canvas is also multiple times the width of the second application interface.

[0020] As shown in FIG. 1, container represents a first container for carrying a first canvas, and view represents a first application interface of the size of the entire screen that is visible to the user.

[0021] The width of the container is canvasWidth and the height is canvasHeight.

[0022] canvasWidth also represents the width of the canvas, which is twice the width of the view.

[0023] canvasHeight also represents the height of the canvas, which is twice the height of the view.

[0024] clientWidth equals the width of the view, clientHeight equals the height of the view, and the view is centered within the container.

[0025] Define the top left corner of the container as the origin, the left offset of the view relative to the vertex as left, and the top offset as top.

[0026] When performing a move operation, the top and left offsets of the container are used to achieve the effect of moving the canvas. That is, when the content of the canvas is moved, the canvas movement is displayed in the view, but the view does not actually change, but the underlying container changes.

[0027] FIG. 2 is a flowchart of a multi-terminal cooperative display update method according to an embodiment of the present application, in which the display update method is used to realize synchronous display of an operation by a first terminal to a second terminal, in which the first terminal provides a first application interface of a first cooperative whiteboard application, a first canvas for carrying drawing content of the first cooperative whiteboard application, and a first container for carrying the first canvas, and the second terminal provides a second application interface of a second cooperative whiteboard application, a second canvas for carrying drawing content of the second cooperative whiteboard application, and a second container for carrying the second canvas, in which a first length of the first canvas is greater than a length of the first application interface, and a width of the first canvas is greater than a width of the first application interface. As shown in FIG. 2, the multi-terminal cooperative display update method includes the following steps:

[0028] In step S201, the first terminal displays a first application interface of a first coordinated whiteboard application, the first coordinated whiteboard application and a second coordinated whiteboard application of the second terminal are in a coordinated mode, and the second terminal displays a second application interface of the second coordinated whiteboard application.

[0029] Specifically, the first terminal and the second terminal may be intelligent devices with a human-computer interaction interface, such as a mobile phone, a tablet computer, an intelligent interactive tablet, etc. The first terminal and the second terminal are in a cooperation mode to indicate that one of the first terminal and the second terminal is used as a follow-up target terminal, and the other is used as a follow-up terminal, and the contents displayed in the linked whiteboard application interface of both terminals are the same. That is, any display on the linked whiteboard application interface of the follow-up target terminal is also displayed on the linked whiteboard application interface of the follow-up terminal. For example, when a multimedia content is displayed on the application interface of the linked whiteboard application of the follow-up target terminal, the same multimedia content is displayed on the application interface of the linked whiteboard application of the follow-up terminal, and for example, by performing an operation on the application interface of the linked whiteboard application of the follow-up target terminal, the application interface changes, and a similar change occurs in the application interface of the linked whiteboard application of the follow-up terminal. In this embodiment, the second terminal is the follow-up target terminal, and the first terminal is the follow-up terminal.

[0030] In one optional embodiment, in a conference collaboration demonstration scenario, a user who is a performer can play multimedia content on his / her own conference whiteboard (i.e., the second terminal) and share the display content of the conference whiteboard with the conference whiteboards of other viewing users (i.e., the above-mentioned first terminal), thereby allowing the same multimedia content to be displayed synchronously on the display interfaces of the conference whiteboards of the other viewing users.

[0031] Furthermore, in different application scenarios, an intelligent device that is a tracking terminal may be used as a tracking target terminal, and an intelligent device that is a tracking target terminal may be used as a tracking terminal, and the definitions of the above first terminal and second terminal are determined according to the current performance state of the intelligent device.

[0032] In step S204, the control parameters sent by the second terminal are received, and the control parameters are generated when the second terminal accepts a control operation for the second application interface.

[0033] The second terminal, as the tracked terminal, is used to realize control operations on the second application interface to change and adjust the display screen, such as translation operations, scaling operations, and writing on the screen currently being displayed in the second application interface.

[0034] When the second terminal accepts the control operation, the second application interface changes in response to the control operation, and the change parameters of these changes are the control parameters. When the second terminal accepts the control operation on the second application interface, the second terminal transmits the generated control parameters to the first terminal, and the first terminal can receive the control parameters. The control parameters may be used to adjust the contents of the first application interface and the second application interface to maintain consistency. For example, the control parameters may be coordinate parameters, and the user translates a current screen in the second application interface of the second terminal, which is the terminal to be followed, and the second terminal generates coordinate parameters based on coordinate information of the translation of the screen and transmits them to the first terminal, which is the follower terminal.

[0035] In step S206, the first terminal updates the content of the first application interface based on the control parameters, and if it is determined that the control operation is a move operation, the first canvas moves based on the control parameters; and if the edges of the first canvas and the first application interface in the same direction overlap, the first terminal re-renders the content of the first canvas.

[0036] The first terminal updates the content of the first application interface according to the control parameters, and actually adjusts the first canvas according to the control parameters to realize the update effect of the content of the first application interface.

[0037] For example, when the operation on the second application interface is a translation operation, the first terminal translates the first canvas based on the control parameters; when the operation on the second application interface is a scaling operation, the first terminal scales the first canvas based on the control parameters; and when the operation on the second application interface is a writing operation, the first terminal generates a corresponding handwriting stroke on the first canvas based on the control parameters.

[0038] If the control operation is a moving operation, the second application interface moves according to the moving operation and sends a control parameter to the first terminal, and the first canvas of the first terminal moves based on the control parameter. The moving operation may be that the user drags the second application interface with a finger, a touch pen, or an external control device (such as a mouse).

[0039] When the first terminal controls the first canvas to move based on the control parameters, the following two states are distinguished.

[0040] In the first state, the edges of the first canvas and the first application interface do not overlap. As shown in FIG. 1, take the first application interface in the center of the first canvas as an example. Because the first canvas is a multiplexed canvas, when the horizontal movement distance is less than the current left and the vertical movement distance is less than the current top, the edges of the first application interface do not overlap with the edges of the first canvas, and the effect is equivalent to the first application interface moving on the first canvas, and the content rendered on the first canvas meets the movement needs, so there is no need to re-render.

[0041] In the second state, the first canvas and a certain edge of the first application interface overlap. Take the first canvas moving to the right as an example, as shown in FIG. 1, when the distance the first canvas moves to the right reaches the current left, the right edge of the first canvas overlaps with the right edge of the first application interface. At this time, if the first canvas continues to move to the right, the first canvas cannot continue to move to the right, but needs to re-render the content of the first canvas to follow the movement of the second canvas, thereby intuitively achieving the effect of the first canvas continuing to move.

[0042] As shown in Fig. 3a or 3b, the first canvas includes a first edge and a third edge, the first application interface includes a second edge and a fourth edge, the first edge is parallel to the third edge, the second edge is parallel to the fourth edge, the first edge and the second edge belong to edges in the same direction, and when the first canvas is moved to the right, the first canvas is triggered to be re-rendered if the third edge and the fourth edge belong to edges in the same direction, and when the first canvas is moved to the left, the third edge and the fourth edge overlap, and the first canvas is triggered to be re-rendered. Similarly, when the upper edge of the first canvas and the first application interface overlap, or the lower edge of the first canvas and the first application interface overlap, the first canvas is also triggered to be re-rendered.

[0043] As can be seen from the above, in an embodiment of the present application, a first terminal displays a first application interface of a first coordinated whiteboard application, the first coordinated whiteboard application and a second coordinated whiteboard application of a second terminal are in a coordinated mode, and the second terminal displays a second application interface of the second coordinated whiteboard application, receives control parameters sent by the second terminal, the control parameters are generated when the second terminal accepts a control operation on the second application interface, the first terminal updates content of the first application interface based on the control parameters, it is confirmed that the control operation is a move operation, the first canvas moves based on the control parameters, and if edges in the same direction of the first canvas and the first application interface overlap, the first terminal re-renders the content of the first canvas. In the above technology, when multiple terminals are linked, a multiple-fold canvas is set in the linked whiteboard application of the following terminal, so that when the following terminal follows the movement of the terminal to be followed, there is no need to clear the content of the canvas and redraw the canvas every time the canvas is moved; and when an edge in the same direction of the canvas and the application interface overlap in the linked whiteboard application of the following terminal, the content of the canvas of the linked whiteboard application is re-rendered, so that the following terminal can follow this effect when the terminal to be followed moves the infinite canvas in the linked whiteboard application, thereby solving the problem of the conventional technology that it is difficult for the following terminal to follow when the canvas of the terminal to be followed expands infinitely.

[0044] The manner in which the first terminal re-renders the first canvas is described below. As one selectable embodiment, the first canvas includes a first edge and a third edge that are parallel, the first application interface includes a second edge and a fourth edge that are parallel, the first edge and the second edge are on the same side, the third edge and the fourth edge are on the same side, the first canvas includes at least a first area close to the third edge and a second area close to the first edge, the first area is adjacent to the second area, a first content is drawn in the first area, and a second content is drawn in the second area, the movement direction of the move operation is from the side where the first edge is located to the side where the third edge is located, and when the edges of the first canvas and the first application interface that are in the same direction overlap, the step of the first terminal re-rendering the content of the first canvas includes: If the first edge overlaps the second edge, the method includes moving the first canvas along a reverse direction of the move operation to a position where the third edge and the fourth edge overlap, and triggering the first region to be rendered as the second content and the second region to be rendered as the third content.

[0045] Note that the first and third edges may be both upper and lower edges, and similarly the second and fourth edges may be both upper and lower edges. The canvas can move horizontally, vertically, or at a specific angle with the application interface.

[0046] The above technique will be described in conjunction with the one shown in FIG. 4. The first container includes a first region A near the third edge and a second region B near the first edge, and the first region A and the second region B each represent a 1x canvas, and together they form a 2x canvas, and the first region A and the second region B respectively have corresponding first and second contents drawn therein. FIG. 4 also includes a region C, and the content of the region C is the third content, but in this case, C is virtual and is not displayed, and is only used to match the display. In this case, the moving direction of the move operation is from the side where the first edge is to the side where the third edge is, that is, the moving direction is the direction in which the first container is moved to the right, and if the first edge of the first container does not overlap with the second edge of the first application interface view, only the canvas moves and no rendering is performed. During the move, the corresponding content is displayed in the first application interface view based on the offset amount of the move.

[0047] When controlling the first canvas to move from the direction of the first edge to the direction of the third edge, after the first edge and the second edge overlap, the canvas content is triggered to be re-rendered, and the canvas moves in the opposite direction to the movement direction to the position where the third edge and the fourth edge overlap. Of course, if it continues to move in the same direction, the above process is repeated, that is, if the first edge does not touch the second edge, the canvas content is not re-rendered, and after they touch, the canvas content is re-rendered and moves in the opposite direction to the position where the third edge and the fourth edge overlap.

[0048] For example, as shown in FIG. 5, when the first canvas continues to move based on FIG. 4, the first edge of the container and the second edge of the view touch, which triggers a canvas redraw operation. Specifically, the first canvas moves backward until the third and fourth edges overlap, where area B is the first area after update, and area C is the second area after update, i.e., the first area displays the second content of area B, and the second area displays the third content of area C, i.e., the previous content of canvas AB is re-rendered as the content of canvas BC, and the user does not feel anything this time because the canvas size is small and the redraw speed is fast, and the view at this time becomes the content of area B and area C, and as the container continues to move right, the view still displays the content of BC until the first and second edges touch again. Further, the entire process can be understood to include only area A and area B, and after being re-rendered, the content of area A will be redrawn as the content of area B, and the content of area B will be redrawn as the content of area C.

[0049] In the above technology, the example of the first canvas moving to the right is used to explain how to re-render when the left edge of the first canvas and the first application interface overlap; however, the method is similar when the right edge of the first canvas and the first application interface overlap when the first canvas moves to the left, or when the bottom edge of the first canvas and the first application interface overlap when the first canvas moves up, or when the top edge of the first canvas and the first application interface overlap when the first canvas moves down, so the explanation is omitted here.

[0050] As one optional embodiment, before the first terminal displays the same content in the first application interface of the first terminal as in the second application interface of the second terminal, the method further includes a step of at least one of the first terminal and the second terminal joining the same virtual room.

[0051] The virtual room is used to represent a group in a collaborative whiteboard application, and multiple terminals that join the same virtual room are in the same group. When it is necessary to use this collaborative function, one virtual room can be created, and then a virtual room identifier for this virtual room can be generated, and other terminals can join this virtual room by this virtual room identifier. The virtual room identifier can be a virtual room number or a virtual room two-dimensional code, etc., and other terminals can join the virtual room by inputting the virtual room number or scanning the virtual room two-dimensional code. After joining the virtual room, the terminal in the virtual room may be set with a terminal ID, for example, may be set as a follow terminal or a follow target terminal.

[0052] Taking a conference collaboration demonstration scenario as an example, a terminal launches a collaborative whiteboard application to generate a conference room (i.e., the above-mentioned virtual room), and the collaborative whiteboard application generates a two-dimensional code corresponding to the conference room. The terminal shares the generated two-dimensional code with other objects that need to participate in the conference, and the other objects use their respective terminals to scan the two-dimensional code to join the above-mentioned conference room, so that many terminals join one joint conference room.

[0053] After multiple terminals join the same virtual room and the IDs of the first terminal and the second terminal are set, the second terminal can perform synchronous display on the first terminal.

[0054] As one selectable example, the step of generating the control parameters when the second terminal accepts a control operation on the second application interface includes the step of the second terminal accepting a control operation to be executed on the second application interface, in which the second canvas changes in response to the control operation, the step of the second terminal determining, as the control parameter, a change parameter of the change in the second canvas generated in response to the control operation, and the step of the second terminal transmitting the control parameter to the first terminal.

[0055] Taking a conference collaboration demonstration scenario as an example, a speaker performs a control operation on a second application interface of a second terminal, and the second collaborative whiteboard application executes the control operation to change the second application interface, for example, if the control operation is a moving operation, the second collaborative whiteboard application controls the second canvas to move according to the control operation, if the control operation is a scaling operation, the second collaborative whiteboard application controls the second canvas to scale according to the control operation, and if the control operation is a writing operation, the second collaborative whiteboard application generates corresponding handwriting on the second canvas.

[0056] In this process, the second collaborative whiteboard application not only controls the second application interface to change in response to the control operation, but also sends change parameters of the change to the first terminal as control parameters, so that the first terminal can track the change of the second application interface based on the change parameters.

[0057] For example, if the control operation is a movement operation, the control parameter may be a position offset amount generated by the second canvas when executing the control operation; if the control operation is a scaling operation, the control parameter may be a scaling ratio and a position offset amount generated when the second collaborative whiteboard application controls the second canvas to perform a scaling operation in response to the control operation; and if the control operation is a writing operation, the control parameter may be handwriting data when the second collaborative whiteboard application generates corresponding handwriting on the second canvas.

[0058] As one selectable embodiment, when the first resolution parameter of the first application interface and the second resolution parameter of the second application interface are different, the step of the first terminal updating the content of the first application interface based on the control parameters includes: the first terminal updating the control parameters based on the first resolution parameters and the second resolution parameters; and the first terminal updating the content of the first application interface based on the updated control parameters.

[0059] Specifically, the first resolution parameter of the first application interface is used to indicate the number of pixel points in the vertical and horizontal directions of the first application interface, and the second resolution parameter of the second application interface is used to indicate the number of pixel points in the vertical and horizontal directions of the second application interface, and the units of both are all in px.

[0060] When the first application interface and the second application interface have the same resolution, the first collaborative whiteboard application can directly control the first application interface based on the control parameters, so that the first application interface tracks the second application interface.

[0061] However, when the resolutions of the first application interface and the second application interface are different, if the first linked whiteboard directly uses the obtained control parameters to control the first application interface, the control effect will be different from the display effect of the second application interface because the resolutions of the two application interfaces are different. Therefore, the first terminal updates the control parameters based on the first resolution of the first application interface and the resolution of the second application interface, and updates the first application interface based on the updated parameters.

[0062] The above-mentioned control parameter update may be to adjust the control parameter to a control parameter that matches the first resolution parameter of the first display interface, and the first terminal, as a follow-up terminal, updates the control parameter provided by the second terminal based on the first resolution parameter and the second resolution parameter, so that the first terminal can adjust the display content of the first application interface based on the updated control parameter, and further the first application interface can match the display content of the second application interface.

[0063] In the above technology, the first terminal adjusts the first canvas based on control parameters that match the first resolution of its first application interface, thereby achieving the technical effect of synchronizing the visual angles of the first application interface of the first terminal, which is a follower terminal, and the second application interface of the second terminal, which is a follow target terminal.

[0064] As one optional embodiment, the step of the first terminal updating the control parameter based on the first resolution parameter and the second resolution parameter includes the steps of: the first terminal reading the second resolution parameter transmitted by the second terminal; the first terminal obtaining a ratio between the first resolution parameter and the second resolution parameter; and the first terminal multiplying the ratio by the control parameter to obtain an updated control parameter.

[0065] Specifically, the ratio between the first resolution parameter and the second resolution parameter includes the height ratio and width ratio of the two, and if the height ratio and the width ratio of the two are the same, one of the ratios is selected and multiplied by the control parameter to obtain the updated control parameter, and if the two are different, the smaller ratio is selected and multiplied by the control parameter to obtain the updated ratio.

[0066] In one selectable embodiment, the control operation is a writing operation, the control parameters are a writing point generated by the writing operation, and the width of the second resolution parameter is A_Width, the width of the first resolution parameter is B_Width, and the coordinate information of a certain writing point in the second application interface is (X A ,Y A ), the coordinate information of the corrected image point in the first application interface (X B , Y B ) is as follows: X B =X A *(B_Width / A_Width), Y B =Y A *(B_Width / A_Width).

[0067] where X B is the horizontal coordinate of the updated icon in the first application interface, Y Bis the ordinate of the updated icon in the first application interface, and B_Width / A_Width is the ratio of the width of the first application interface to the width of the second application interface. Through the above calculation, the first terminal can obtain updated coordinate data corrected based on the resolution.

[0068] In this embodiment, the first terminal displays the same content as the second application interface of the second terminal in the first application interface, the second terminal generates control parameters according to the user's control operation on the second application interface and sends the control parameters to the first terminal, the first terminal updates the control parameters according to the first resolution parameters and the second resolution parameters, and then adjusts the content of the first application interface according to the updated control parameters. The first terminal, which is the tracking terminal, obtains control parameters matching the first application interface by updating the original control parameters sent by the first terminal according to the respective resolution parameters, thereby achieving the purpose of always maintaining the display content of the application interfaces of the first terminal and the second terminal consistent with each other, and further realizing the equal proportional visual angle tracking of the tracking terminal and the tracking target terminal, solving the problem of poor visual angle tracking effect due to different pixels of the tracking target terminal and the tracking terminal.

[0069] As one selectable embodiment, the control parameters include a starting coordinate parameter and a moving increment, the starting coordinate includes a coordinate of the upper left corner of the first application interface, a coordinate of an absolute origin of the first canvas, a first horizontal deviation and a second vertical deviation, and the moving increment includes a first horizontal increment and a second vertical increment when the second canvas moves in the second application interface, and the step of moving the first canvas according to the control parameters includes: The method includes the steps of: the first terminal updating a first deviation by a first increment; the first terminal updating a second deviation by a second increment; determining a horizontal offset amount of the first canvas based on the first deviation and a width of the first application interface; determining a vertical offset amount of the first canvas based on the second deviation and a height of the first application interface; and controlling the first canvas to move horizontally by the horizontal offset amount and to move vertically by the vertical offset amount.

[0070] Specifically, the absolute origin of the first canvas may be the upper left corner point of the first canvas or the upper left corner point of the first container. The starting coordinate is the deviation of the origin from the upper left corner of the first canvas before moving, including a first deviation in the horizontal direction and a second deviation in the vertical direction, and the moving increment represents the increment in two directions, the horizontal direction and the vertical direction, generated by the second application interface when moving the second canvas.

[0071] In the above technology, the starting coordinates (including the first deviation and the second deviation) of the first canvas are updated based on the increments in two directions, the horizontal and vertical directions, generated by the second application interface when moving the second canvas, the actual offset amount (including the horizontal offset amount and the vertical offset amount) of the first canvas is determined based on the updated coordinates, and the first canvas is controlled to move based on the actual offset amount, thereby enabling the first application interface to follow the canvas movement operation in the second application interface.

[0072] The above technique takes advantage of the browser's Canvas tag and uses the hammer open source gesture library to intercept events thrown by the Canvas and identify canvas movements.

[0073] In one alternative embodiment, SinglePanStart indicates that the mouse (or touch) has been triggered to start moving, SinglePanMove indicates that the mouse (or touch) has been triggered to move, and SinglePanEnd indicates that the mouse movement (or touch) has ended. The events must be to trigger SinglePanStart, then SinglePanMove, and finally end with SinglePanEnd.

[0074] When the mouse is moved while pressing the space key in the second application interface, the second canvas is triggered to move. When SinglePanStart is triggered by pressing the space key, the horizontal coordinate to start the movement is temporarily stored using moveOriginPointX, and the vertical coordinate to start the movement is temporarily stored using moveOriginPointY. When the mouse is triggered by pressing the space key, the horizontal offset amount dx (i.e., the above-mentioned first increment) is calculated using the position after the movement curOriginPointX-moveOriginPointX, and the vertical offset amount dy (i.e., the above-mentioned second increment) is calculated using the position after the movement curOriginPointY-moveOriginPoinX. After dx and dy are obtained, they are saved using the saved position information variable drawOrigin.

[0075] drawOriginX2=drawOriginX1+dx, drawOriginY2=drawOriginY1+dy, drawOrigin(drawOriginX1, drawOriginY1) represents the distance from the upper left corner of the first application interface to the origin (and the above starting coordinates), and after obtaining the new drawOrigin(drawOriginX2, drawOriginY2), the offset amount of the canvas is recalculated.

[0076] As one selectable embodiment, the step of determining a horizontal offset amount of the first canvas based on the first deviation and the width of the first application interface includes a step of determining a first remainder obtained by dividing the updated first deviation by the width of the first application interface as the horizontal offset amount when the updated first deviation is greater than 0, and a step of determining a second remainder obtained by dividing the updated first deviation by the width of the first application interface as the horizontal offset amount when the updated first deviation is less than 0, and the step of determining a sum of the width of the first application interface and the second remainder as the horizontal offset amount when the updated first deviation is greater than 0. The step of determining a vertical offset amount of the first canvas based on the second deviation and the height of the first application interface includes a step of determining a third remainder obtained by dividing the updated second deviation by the height of the first application interface as the vertical offset amount when the updated second deviation is greater than 0, and a step of determining a fourth remainder obtained by dividing the updated second deviation by the height of the first application interface as the vertical offset amount when the updated second deviation is less than 0.

[0077] After obtaining the updated first deviation and second deviation, calculate the offset amount of the first canvas. In one alternative embodiment, the width of the first application interface is represented by baseWidth, the height of the first application interface is represented by baseHeight, the horizontal offset of the first canvas is represented by left, and the vertical offset of the first canvas is represented by top.

[0078] if drawOriginX>0, left=‐(drawOriginPointX%baseWidth) If drawOriginX<0, then left=‐(baseWidth+drawOriginPointX%baseWidth) if drawOriginY>0 then top=‐(drawOriginPointY%baseHeight) If drawOriginX<0, then top=‐(baseHeight+drawOriginPointY%baseHeight) Finally, left and top are offset values ​​in the actual visual angle of the canvas tag, and the above process enables the control parameters of the first linked whiteboard application to follow the movement operation of the second linked whiteboard application.

[0079] In one optional embodiment, it is determined that edges of the first canvas and the first application interface in the same direction overlap if the abscissa of any corner of the first canvas is the same as the abscissa of any corner of the first application interface, or if the ordinate of any corner of the first canvas is the same as the ordinate of any corner of the first application interface.

[0080] In the above step, if the horizontal coordinate of any corner of the first canvas is the same as the horizontal coordinate of any corner of the first application interface, or the vertical coordinate of any corner of the first canvas is the same as the vertical coordinate of any corner of the first application interface, it indicates that one side of the first canvas overlaps with one side of the first application interface.

[0081] As shown in FIG. 6, the upper left corner of the first canvas 30 is the origin (0,0), and the coordinates of the upper left corner of the first application interface 31 are (drawOriginPointX, drawOriginPointY). When the left edge of the first canvas 30 and the left edge of the first application interface 31 overlap, drawOriginPointX is 0, that is, the horizontal coordinates of both are the same. When the right edge of the first canvas 30 and the right edge of the first application interface 31 overlap, the horizontal coordinates of both are also the same. However, when the right edge of the first canvas 30 and the upper edge or lower edge of the first application interface 31 overlap, the vertical coordinates of both are the same.

[0082] As one selectable embodiment, the width of the first canvas is a preset multiple of the width of the first application interface, and the height of the first canvas is a preset multiple of the height of the first application interface, the preset multiple being greater than 1 and less than 3.

[0083] The application interface is an area for displaying a collaborative whiteboard application on the screen, and when the first canvas is low, for example, a canvas five times the size of the application interface is used, and the application interface is in the middle of the canvas, and the canvas is moved to calculate the offset amount, and the content of a different position of the canvas is displayed in the application interface according to this offset amount, thereby realizing the canvas expansion function. However, in order to realize the expansion of the size of the canvas, the size of the canvas needs to be increased, and every time rendering or drawing is performed, all the content of the infinite canvas needs to be cleared and re-rendered first, and as a result, as the canvas becomes larger, the clearing and redrawing area increases, and when the canvas redraws the graphics data, the delay becomes large and the freezing phenomenon is significant. If a canvas with the same size as the application interface (single canvas) is used, every time the canvas is moved, the position of each point needs to be recalculated and the content of the canvas needs to be redrawn to achieve the canvas expansion function. Since the canvas is cleared and redrawn every time the canvas is moved, if the amount of data in the canvas is large, the redrawing time becomes long and even freezing occurs, and every time a redrawing is triggered, resulting in low efficiency and low scalability.

[0084] In the above technology, the canvas size is selected as a multiple of 1 to 3, so there is no need to re-render every time the canvas is moved, and the canvas area is not too large when redrawing, causing too much delay, so a canvas size of 1x to 3x provides a better user experience.

[0085] As one alternative embodiment, the width of the first canvas is twice the width of the first application interface, and the height of the first canvas is twice the height of the first application interface.

[0086] In the above technique, the first canvas is a double canvas, that is, the width of the first canvas is twice the width of the first application interface, and the height of the first canvas is also twice the height of the first application interface.

[0087] In one alternative embodiment, if the first canvas and the first application interface have non-overlapping edges in the same direction, the re-rendering of the canvas content is inhibited.

[0088] As shown in Fig. 3a, when the first canvas is controlled to move from the direction of the first edge to the direction of the third edge, if the first edge of the first canvas does not overlap with the second edge of the first application interface view, only the canvas moves and no rendering is performed. During the movement, the content corresponding to the first application interface is displayed based on the movement offset amount.

[0089] As one optional embodiment, when the control operation is a scaling operation based on one target point, the control parameters include a scaling ratio parameter and an offset parameter, where the offset parameter is a deviation between the coordinates of the target point and the coordinates of the upper left corner of the second canvas, and the step of the first terminal updating the content of the first application interface based on the control parameters includes: controlling the first canvas to scale according to the updated scaling ratio parameter based on the origin, thereby obtaining a first canvas scaled based on the origin; determining an adjusted offset parameter based on the scaling ratio parameter and the offset parameter; controlling the first canvas scaled based on the origin to move according to the adjusted offset parameter, thereby obtaining a first canvas scaled based on the target point; and re-rendering elements of the first canvas scaled based on the target point to update the content of the first application interface.

[0090] Specifically, the target point is any image point of the second display interface, for example, when a performer clicks a point of the second application interface with a mouse and performs a zoom operation based on this point, this point is the target point of this scaling operation. The offset parameter is the deviation between the coordinate of the target point and the coordinate of the upper left corner of the second canvas, and includes two values ​​in the horizontal and vertical directions, and is usually related to the size of the second canvas, for example, the width and height of the second canvas are x and y respectively, and the offset parameter is (-x,-y).

[0091] The scaling ratio parameter indicates the scaling ratio of the screen before and after the scaling operation, and when the initial value of the scaling ratio parameter is 1, it indicates that the magnification factor of the image of the display interface after the scaling operation is the same as that of the interface image when no operation is performed. The offset parameters include an offset value in the X direction (i.e., the deviation between the abscissa of the target electric energy and the origin) and an offset value in the Y direction (i.e., the deviation between the ordinate of the target point and the origin).

[0092] When the target point is the origin, the offset parameter is 0, and as shown in FIG. 7a, the second canvas 40 has a second application interface 41, and the dotted area is the area of ​​the image before the enlargement operation. After the scaling operation is triggered, it is enlarged from the origin of the interface to the lower right corner, as shown in FIG. 7b, and the position of the origin in the upper left corner of the interface remains unchanged, and other pixel points are enlarged based on the scaling ratio parameter, so that the area area of ​​the image becomes farther away from the origin of the interface with the enlargement. After the enlargement operation, the second application interface is filled with the enlarged second canvas 40, and the enlargement of the image is realized, as shown in FIG. 7b.

[0093] When the target point is not the origin, as shown in Fig. 8a, the performer performs a zoom operation in the second application interface 50 based on the target point A, the target point A is a pixel point in the second application interface, the scaling ratio parameter of the second application interface is the scaling ratio parameter curScale, and the offset parameters of the target point A include an offset value canvasTranslateOffsetX in the X direction and an offset value canvasTranslateOffsetY in the Y direction, where curScale is increased by 0.1 each time, i.e., 0.1, 0.2..., 1, 1.1, 1.2, 1.3..., and both the minimum value and the increment value are 0.1.

[0094] If the first resolution parameter and the second resolution parameter are different, an updated scaling parameter curScale1, an updated X-direction offset value canvasTranslateOffsetX1, and an updated Y-direction offset value canvasTranslateOffsetY1 for the first terminal are obtained based on the width A_Width of the second resolution parameter and the width B_Width of the first resolution parameter.

[0095] curScale1=curScale*(B_Width / A_Width), canvasTranslateOffsetX1=canvasTranslateOffsetX*(B_Width / A_Width), canvasTranslateOffsetY1=canvasTranslateOffsetY*(B_Width / A_Width), First, the first canvas is controlled to be scaled according to the updated scaling ratio parameter based on the origin, and a first canvas scaled based on the origin is obtained, that is, the position of the origin at the upper left corner of the first application interface remains unchanged, and other image points except the origin of the interface are enlarged based on the updated scaling ratio parameter curScale1, and the image within the dotted line is obtained as an enlarged image in the first application interface 53 of the first canvas 52, as shown in Fig. 8b. At this time, the result of enlarging based on the origin is obtained, but since the target point is not the origin, it is necessary to offset the result of enlarging based on the origin.

[0096] When offsetting the enlargement result based on the origin, first, the offset parameter of the target point is adjusted based on the scaling ratio to obtain the offset parameter corresponding to the enlargement result based on the origin, and the position of the scaled first canvas is obtained by moving the first canvas scaled based on the origin based on the adjusted offset parameter, and by re-rendering based on the position of the scaled first canvas, it is possible to realize the first coordinated whiteboard application following the scaling operation of the second coordinated whiteboard application.

[0097] As one selectable embodiment, the step of determining the adjusted offset parameters based on the scaling ratio parameter and the offset parameter includes the steps of obtaining relative coordinates of the target point in the second application interface; projecting the target point onto the single container based on the offset parameters to obtain absolute coordinates of the target point in the single container; obtaining a first intermediate value by subtracting the absolute coordinates from a sum of the relative coordinates and the offset coordinates of the second application interface, where the offset coordinates of the second application interface are used to indicate the difference between the coordinates of the upper left corner of the second application interface and the coordinates of the origin of the second canvas; obtaining a second intermediate value by multiplying the difference between the scaling ratio parameter and 1 by the absolute coordinates; and obtaining the adjusted offset parameters by subtracting the first intermediate value from the second intermediate value.

[0098] Specifically, the relative coordinates of the target point in the second application interface are coordinates directly obtained according to a scaling operation, for example, when a user clicks a point in the second application interface to scale, the coordinates of this point in the second application interface are the relative coordinates. The single container is used to represent a 1x container, which is a container of the original size when not scaled. When performing the current scaling operation, since a scaling operation may have been performed before, it is necessary to map and calculate the offset parameters to the single container.

[0099] In one optional embodiment, when the position when performing the scaling operation in the second application interface is a relative position (clickX, clickY), the coordinate difference between the upper left corner of the second application interface and the origin of the second canvas is (drawOriginPointX, drawOriginPointY), and the above offset parameter is canvasTranslateOffsetX, projecting the target point into a single container based on the offset parameter is as follows:

[0100] posX=(X1+drawOriginPointX+canvasTranslateOffset) / curScale, posY=X1+drawOriginPointX+canvasTranslateOffset / curScale, where (posX,posY) are the absolute coordinates, and curScale is the scaling factor of the first canvas before this scaling operation.

[0101] In addition, canvasTranslateOffset is updated based on the scaling ratio curScale of this scaling and the above absolute coordinates to obtain canvasTranslateOffset2.

[0102] canvasTranslateOffsetX:2=(curScale‐1)*posX‐(clickX+drawOriginPointX‐posX), canvasTranslateOffsetY2=(curScale‐1)*posY‐(clickY+drawOriginPointY‐posY, Here, clickX+drawOriginPointX-posX constitutes the first intermediate value, and (curScale-1)*posY constitutes the second intermediate value.

[0103] As one optional embodiment, when the control operation is an element addition operation, the control parameters include coordinate parameters of the handwriting, and the step of the first terminal updating the content of the first application interface based on the control parameters includes the step of the first terminal generating a handwriting based on the coordinate parameters of the handwriting, the coordinate parameters of the handwriting including coordinate parameters of handwriting points for constituting the handwriting, and each time the second terminal generates a new handwriting point, sending the coordinate parameters of all the generated handwriting points of the current handwriting to the first terminal.

[0104] Specifically, the element adding operation may be an operation of the performer writing on the second display interface. In one selectable embodiment, the data written by the performer with a brush on the second display interface is defined as being composed of a plurality of points (i.e., writing points), and is a line composed of a plurality of points written by the performer, as shown in Fig. 9. When the performer writes, the points constituting the writing line continue to increase, and the coordinate parameters of each writing point are transmitted to the first terminal in real time, and the first terminal can draw after receiving the coordinate parameters of each writing point, so that the same handwriting is obtained, and tracking of the element adding operation is realized.

[0105] In addition, when the first resolution parameter of the first application interface and the second resolution parameter of the second application interface are different, the first terminal updates the received control parameters (including the origin coordinate, stretch rate, and offset parameters of the interface) based on the width A_Width of the first resolution parameters and the width B_Width of the second resolution parameters, and realizes operations such as scaling and parallel translation of the brush strokes through calculation, thereby realizing the effect of proportional visual angle tracking of the brush strokes.

[0106] In one alternative embodiment, to realize the brush writing function using the web canvas technology, the data structure is defined as follows: Id: used to represent the unique identifier of the element, x: absolute horizontal coordinate of the element on the artboard, y: absolute vertical coordinate of the element on the artboard, width: width of the element, height: height of the element, type: type of element, such as brush, isDeleted: defines whether the definition is deleted, Points: defines the points on the artboard, one line is composed of many points. When the speaker user performs a writing operation on the second display interface and the brush data of the second display interface is updated, the data in Points continues to increase and is transferred to the first terminal in real time, thereby realizing the real-time update of data, and in order to ensure the consistency of data, the points are fully exchanged every time the data is transferred.

[0107] In this embodiment, the function of proportional visual angle tracking (lecturer mode) of multi-terminal cooperation is implemented based on web technology, and the data transmission between the tracking terminal and the tracking target terminal uses only four variables to update different resolutions, the amount of data transmitted is small, and the technology is lightweight. If the user does not feel it, the function of unlimited visual angle tracking (lecturer mode) of multi-terminal cooperation is implemented on a limited interactive whiteboard, which is extremely easy to maintain, and can realize visual angle tracking of writing, and the combination of art board layout and writing function is better separated, and the extensibility is extremely high.

[0108] In one optional embodiment, the step of the first terminal generating the handwriting based on the coordinate parameters of the handwriting includes a step of triggering the first canvas to move in a direction opposite to the contacting edge when the handwriting touches any edge of the first application interface, and a step of the first terminal redrawing the content of the first canvas when edges in the same direction of the first canvas and the first application interface overlap.

[0109] In the above technology, when a user writes on the second application interface, if the handwriting touches the boundary of the second application interface, the second application interface is triggered to move in the opposite direction of the touched edge, providing the user with a wider writing range. In this case, in order to follow the writing action of the second terminal, the first terminal touches the edge of the first application interface when generating the handwriting, and the first canvas also moves in the opposite direction of the touched edge. That is, the user does not directly move the second canvas, but the writing operation triggers the movement of the second application interface, and the first application interface also needs to move the first canvas following this action.

[0110] In the process of moving the first canvas, it is necessary to determine whether the edge of the first canvas overlaps with the edge of the first application interface, and therefore to determine whether the content of the first canvas needs to be re-rendered. If the edges of the first canvas and the first application interface in the same direction overlap, the first terminal redraws the content of the first canvas, and the redrawing step is the same as the re-rendering step in the above embodiment, and the description is omitted here.

[0111] Example 2 According to an embodiment of the present application, an embodiment of a multi-terminal cooperative display update device is provided, and a display update method is used to realize synchronous display of an operation by a first terminal to a second terminal, the first terminal provides a first application interface of a first cooperative whiteboard application, a first canvas for carrying drawing content of the first cooperative whiteboard application, and a first container for carrying the first canvas, the second terminal provides a second application interface of a second cooperative whiteboard application, a second canvas for carrying drawing content of the second cooperative whiteboard application, and a second container for carrying the second canvas, wherein a first length of the first canvas is greater than a length of the first application interface, and a width of the first canvas is greater than a width of the first application interface. Figure 10 is a schematic diagram of a multi-terminal cooperative display update device according to an embodiment of the present invention. As shown in FIG. 10, the multi-terminal cooperative display update device includes a display module 100 used by a first terminal to display a first application interface of a first cooperative whiteboard application, the first cooperative whiteboard application and a second cooperative whiteboard application of a second terminal being in a cooperative mode, and the second terminal displaying a second application interface of the second cooperative whiteboard application; a receiving module 102 used to receive control parameters sent by the second terminal, the control parameters being generated when the second terminal accepts a control operation on the second application interface; and an update module 104 for the first terminal to update content of the first application interface based on the control parameters, and the method further includes a movement module used to confirm that the control operation is a move operation, the first canvas moves based on the control parameters, and the first terminal re-renders content of the first canvas when edges in the same direction of the first canvas and the first application interface overlap.

[0112] As one selectable embodiment, the first canvas includes a first edge and a third edge that are parallel, the first application interface includes a second edge and a fourth edge that are parallel, the first edge and the second edge are on the same side, and the third edge and the fourth edge are on the same side, the first canvas includes at least a first area close to the third edge and a second area close to the first edge, the first area is adjacent to the second area, a first content is drawn in the first area, and a second content is drawn in the second area, a moving direction of the move operation is a moving direction from the side where the first edge is located to the side where the third edge is located, and the update module includes a rendering submodule for triggering, when the first edge overlaps with the second edge, to move the first canvas along a reverse direction of the move operation to a position where the third edge overlaps with the fourth edge, and to render the first area as the second content and to render the second area as the third content.

[0113] As one optional embodiment, the above apparatus further includes a joining module for at least one of the first terminal and the second terminal to join the same virtual room before the first terminal displays the same content in the first application interface of the first terminal as in the second application interface of the second terminal.

[0114] As one optional embodiment, the second terminal accepts a control operation performed on a second application interface, the second canvas changes in response to the control operation, the second terminal determines a change parameter of the change in the second canvas generated in response to the control operation as a control parameter, and the second terminal transmits the control parameter to the first terminal.

[0115] As one selectable embodiment, when a first resolution parameter of the first application interface and a second resolution parameter of the second application interface are different, the update module includes a first update submodule for the first terminal to update control parameters based on the first resolution parameters and the second resolution parameters, and a second update submodule for the first terminal to update content of the first application interface based on the updated control parameters.

[0116] As one optional embodiment, the first update submodule includes: a reading unit for the first terminal to read the second resolution parameter sent by the second terminal; an acquisition unit for the first terminal to acquire a ratio between the first resolution parameter and the second resolution parameter; and a multiplication module for the first terminal to multiply the ratio and the control parameter to obtain an updated control parameter.

[0117] As one selectable embodiment, the control parameters include a starting coordinate parameter and a movement increment, the starting coordinates include the coordinate of the upper left corner of the first application interface, the coordinate of the absolute origin of the first canvas, a first horizontal deviation and a second vertical deviation, and the movement increment includes a first horizontal increment and a second vertical increment when the second canvas moves in the second application interface, and the movement module includes a third updating submodule for the first terminal to update the first deviation by the first increment, a fourth updating submodule for the first terminal to update the second deviation by the second increment, a first determination submodule for determining a horizontal offset amount of the first canvas based on the first deviation and a width of the first application interface, a second determination submodule for determining a vertical offset amount of the first canvas based on the second deviation and a height of the first application interface, and a first control submodule for controlling the first canvas to move horizontally by the horizontal offset amount and move vertically by the vertical offset amount.

[0118] As one selectable embodiment, the first determination submodule includes: a first determination unit for determining a first remainder by dividing the updated first deviation by the width of the first application interface as the horizontal offset amount when the updated first deviation is greater than 0; and a first acquisition unit for obtaining a second remainder by dividing the updated first deviation by the width of the first application interface as the horizontal offset amount when the updated first deviation is less than 0, and determining a sum of the width of the first application interface and the second remainder as the horizontal offset amount; and the second determination submodule includes: a second determination unit for determining a third remainder by dividing the updated second deviation by the height of the first application interface as the vertical offset amount when the updated second deviation is greater than 0; and a second acquisition unit for obtaining a fourth remainder by dividing the updated second deviation by the height of the first application interface as the vertical offset amount when the updated second deviation is less than 0.

[0119] In one optional embodiment, it is determined that edges of the first canvas and the first application interface in the same direction overlap if the abscissa of any corner of the first canvas is the same as the abscissa of any corner of the first application interface, or if the ordinate of any corner of the first canvas is the same as the ordinate of any corner of the first application interface.

[0120] As one selectable embodiment, the width of the first canvas is a preset multiple of the width of the first application interface, and the height of the first canvas is a preset multiple of the height of the first application interface, the preset multiple being greater than 1 and less than 3.

[0121] As one alternative embodiment, the width of the first canvas is twice the width of the first application interface, and the height of the first canvas is twice the height of the first application interface.

[0122] In one alternative embodiment, if the first canvas and the first application interface have non-overlapping edges in the same direction, the re-rendering of the canvas content is inhibited.

[0123] As one selectable embodiment, when the control operation is a scaling operation based on one target point, the control parameters include a scaling ratio parameter and an offset parameter, where the offset parameter is the deviation between the coordinates of the target point and the coordinates of the upper left corner of the second canvas, and the update module includes: a second control submodule for controlling the first canvas to scale according to the updated scaling ratio parameter based on the origin, thereby obtaining a first canvas scaled based on the origin; a third determination submodule for determining an adjusted offset parameter based on the scaling ratio parameter and the offset parameter; a movement submodule for controlling the first canvas scaled based on the target point to move according to the adjusted offset parameter, thereby obtaining a first canvas scaled based on the target point; and a rendering submodule for re-rendering elements of the first canvas scaled based on the target point to update the content of the first application interface.

[0124] As one optional embodiment, the third determination submodule includes: a third acquisition unit for obtaining a relative coordinate of the target point in the second application interface; a projection unit for projecting the target point onto the single container based on the offset parameter to obtain an absolute coordinate of the target point in the single container; a first arithmetic unit used to obtain a first intermediate value by subtracting the absolute coordinate from the sum of the relative coordinate and the offset coordinate of the second application interface, where the offset coordinate of the second application interface is used to indicate the difference between the coordinate of the upper left corner of the second application interface and the coordinate of the origin of the second canvas; a second arithmetic unit for obtaining a second intermediate value by multiplying the difference between the scaling ratio parameter and 1 by the absolute coordinate; and a third arithmetic unit for obtaining an adjusted offset parameter by subtracting the first intermediate value from the second intermediate value.

[0125] As one optional embodiment, when the control operation is an element addition operation, the control parameters include coordinate parameters of the handwriting, and the update module includes a generating sub-module for the first terminal to generate a handwriting based on the coordinate parameters of the handwriting, the coordinate parameters of the handwriting include coordinate parameters of the handwriting points for constituting the handwriting, and each time the second terminal generates a new handwriting point, sending the coordinate parameters of all the generated handwriting points of the current handwriting to the first terminal.

[0126] As one optional embodiment, the generating submodule is used to trigger the first canvas to move in a direction opposite to the contacting edge when the writing stroke contacts any edge of the first application interface, and the first terminal includes a trigger unit to redraw the content of the first canvas when edges in the same direction of the first canvas and the first application interface overlap.

[0127] Example 3 According to an embodiment of the present application, an embodiment of a computer storage medium is provided, the computer storage medium storing a plurality of instructions, the instructions being suitable for being loaded by a processor to execute the steps of the method according to any one of the claims of embodiment 1. In this embodiment, the first terminal displays the same content as that of the second display interface of the second terminal on the first display interface, the second terminal transmits control parameters to the first terminal according to a user's control operation on the second display interface, the first terminal updates the control parameters according to the first resolution parameter and the second resolution parameter, and then adjusts the content of the first display interface according to the updated control parameters. The first terminal, which is the tracking terminal, obtains control parameters matching the first terminal by updating the original control parameters transmitted by the first terminal according to the respective resolution parameters, thereby achieving the purpose of constantly maintaining the display images of the display interfaces of the first terminal and the second terminal consistent with each other, and further realizing the equal proportion visual angle tracking of the tracking terminal and the tracking target terminal, and solving the problem of poor visual angle tracking effect due to different pixels of the tracking target terminal and the tracking terminal.

[0128] Example 4 According to an embodiment of the present application, there is provided an intelligent interactive tablet comprising a processor and a memory, the memory storing a computer program suitable for being loaded by the processor to perform the steps of the method according to any one of claims of embodiment 1.

[0129] 11 is a schematic diagram of an intelligent interactive tablet according to an embodiment of the present application. The intelligent interactive tablet includes the above-mentioned interactive device body and touch frame. As shown in FIG. 11, the intelligent interactive tablet 1000 can include at least one processor 1001, at least one network interface 1004, a user interface 1003, a memory 1005, and at least one communication bus 1002.

[0130] Here, a communication bus 1002 is used to realize connecting communication between these components.

[0131] Here, the user interface 1003 can include a display and a camera, and optionally, the user interface 1003 can further include a standard wired interface and a wireless interface.

[0132] Here, the network interface 1004 can selectively include a standard interface, a wireless interface (such as a WI-FI interface), etc.

[0133] Here, the processor 1001 may include one or more processing cores. The processor 1001 connects to each part in the whole intelligent interactive tablet 1000 through various interfaces or lines, and executes or operates instructions, programs, code sets or instruction sets stored in the memory 1005, and calls data stored in the memory 1005, thereby performing various functions of the intelligent interactive tablet 1000 and performing data processing. Optionally, the processor 1001 may be realized by adopting at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), programmable logic array (PLA). The processor 1001 may integrate one or several combinations of a central processing unit (CPU), a graphics processing unit (GPU), a modem, and the like. Here, the CPU mainly processes the operating system, the user interface, and application programs, etc., the GPU is used for rendering and drawing the content that needs to be displayed on the display, and the modem is used for processing wireless communication. It can be understood that the modem may not be integrated into the processor 1001, but may be realized independently in one chip.

[0134] Here, the memory 1005 may include a random access memory (RAM) and may also include a read-only memory. Optionally, the memory 1005 includes a non-transitory computer-readable storage medium. The memory 1005 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 1005 may include a program storage area and a data storage area, where the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a voice playback function, an image playback function, etc.), instructions for implementing the above-mentioned method embodiments, etc., and the data storage area may store data related to the above-mentioned method embodiments, etc. The memory 1005 may optionally be at least one storage device separate from the processor 1001. As shown in FIG. 8, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an operating application program of the intelligent interactive tablet.

[0135] In the intelligent interactive tablet 1000 shown in FIG. 8, the user interface 1003 is mainly used to provide an input interface for the user and obtain data input by the user, while the processor 1001 may also be used to call up the operating application program of the intelligent interactive tablet stored in the memory 1005 and specifically perform any operation of Example 1.

[0136] The above embodiment numbers of the present invention are for illustrative purposes only and do not represent the superiority or inferiority of the embodiments.

[0137] The above embodiments of the present invention have different emphasis in the description of each embodiment, so for the parts not described in detail in one embodiment, reference can be made to the relevant descriptions of other embodiments.

[0138] In some embodiments provided in the present application, it is understood that the disclosed technical contents may be realized in other ways. Here, the above device embodiments are only illustrative, for example, the division of the units may be a logical functional division, and there may be other division ways in actual implementation, for example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection via some interfaces, units or modules, and may be in an electrical or other form.

[0139] The units described as separate elements may or may not be physically separate, and the elements shown as units may or may not be physical units, i.e., they may be located in one place or distributed among multiple units. Some or all of the units may be selected according to actual needs to achieve the objectives of the technology of the present embodiment.

[0140] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be realized in the form of hardware or in the form of a software functional unit.

[0141] The integrated unit may be realized in the form of a software functional unit and stored in one computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technology of the present invention may be realized in the form of a software product, either essentially or in part contributing to the prior art, or all or part of the technology, and the computer software product is stored in a storage medium including some instructions for making a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present invention. The storage medium includes various media that can store program code, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0142] What has been described above is only a preferred embodiment of the present invention, and a person skilled in the art may make some improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered as within the scope of protection of the present invention.

Claims

1. A display update method for multi-terminal cooperation, comprising: The display update method is used to realize synchronous display of an operation by a first terminal to a second terminal, the first terminal providing a first application interface of a first coordinated whiteboard application, a first canvas for carrying drawing content of the first coordinated whiteboard application, and a first container for carrying the first canvas, the second terminal providing a second application interface of a second coordinated whiteboard application, a second canvas for carrying drawing content of the second coordinated whiteboard application, and a second container for carrying the second canvas, a first length of the first canvas is greater than a length of the first application interface, and a width of the first canvas is greater than a width of the first application interface, the display update method comprising: a first terminal displaying a first application interface of the first coordinated whiteboard application, the first coordinated whiteboard application and a second coordinated whiteboard application of the second terminal being in a coordinated mode, and the second terminal displaying a second application interface of the second coordinated whiteboard application; receiving a control parameter transmitted by the second terminal, the control parameter being generated when the second terminal receives a control operation on the second application interface; and a step of the first terminal updating content of the first application interface based on the control parameters, wherein if the control operation is determined to be a move operation, the first canvas moves based on the control parameters, and edges of the first canvas and the first application interface in the same direction overlap, the first terminal re-renders the content of the first canvas.

2. the first canvas includes a first edge and a third edge which are parallel to each other, the first application interface includes a second edge and a fourth edge which are parallel to each other, the first edge and the second edge are on the same side, the third edge and the fourth edge are on the same side, the first canvas includes at least a first area close to the third edge and a second area close to the first edge, the first area is adjacent to the second area, a first content is drawn in the first area, and a second content is drawn in the second area, a movement direction of the move operation is a movement direction from a side where the first edge is located to a side where the third edge is located, and when sides of the first canvas and the first application interface which are in the same direction overlap, the first terminal re-renders the content of the first canvas by 2. The method of claim 1, further comprising: if the first edge overlaps the second edge, moving the first canvas along a reverse direction of the move operation to a position where the third edge overlaps the fourth edge, triggering rendering of the first region as the second content and rendering of the second region as a third content.

3. Before the first terminal displays the same content as the second application interface of the second terminal in the first application interface of the first terminal, the method includes:

2. The method of claim 1, further comprising the step of: at least one of the first terminal and the second terminal joining the same virtual room.

4. The step of generating the control parameters when the second terminal accepts a control operation on the second application interface includes: a step of the second terminal accepting the control operation executed on the second application interface, the second canvas changing in response to the control operation; A step in which the second terminal determines, as a control parameter, a change parameter of the change in the second canvas generated in response to the control operation; 2. The method of claim 1, further comprising the step of: said second terminal transmitting said control parameters to said first terminal.

5. When a first resolution parameter of the first application interface and a second resolution parameter of the second application interface are different, the first terminal updates the content of the first application interface based on the control parameter, the first terminal updating the control parameters based on the first resolution parameters and the second resolution parameters; 2. The method of claim 1, further comprising: the first terminal updating content of the first application interface based on the updated control parameters.

6. The step of updating the control parameter by the first terminal based on the first resolution parameter and the second resolution parameter includes: the first terminal reading the second resolution parameters transmitted by the second terminal; The first terminal obtains a ratio between the first resolution parameter and the second resolution parameter; and a step of: said first terminal multiplying said ratio by said control parameter to obtain an updated control parameter.

7. The control parameters include a starting coordinate and a movement increment, the starting coordinate including a coordinate of an upper left corner of the first application interface and a coordinate of an absolute origin of the first canvas, a first deviation in a horizontal direction and a second deviation in a vertical direction, and the movement increment including a first increment in a horizontal direction and a second increment in a vertical direction when the second canvas moves in the second application interface, and the step of moving the first canvas based on the control parameters includes: the first terminal updating the first deviation by the first increment; the first terminal updating the second deviation by the second increment; determining a horizontal offset of the first canvas based on the first deviation and a width of the first application interface; determining a vertical offset of the first canvas based on the second deviation and a height of the first application interface; 2. The method of claim 1, further comprising: controlling the first canvas to move horizontally by the horizontal offset amount and to move vertically by the vertical offset amount.

8. determining a horizontal offset of the first canvas based on the first deviation and a width of the first application interface, If the updated first deviation is greater than 0, determining a first remainder obtained by dividing the updated first deviation by a width of the first application interface as a horizontal offset amount; if the updated first deviation is less than 0, obtaining a second remainder by dividing the updated first deviation by a width of the first application interface, and determining a sum of the width of the first application interface and the second remainder as a horizontal offset amount; determining a vertical offset of the first canvas based on the second deviation and the height of the first application interface, if the updated second deviation is greater than 0, determining a third remainder obtained by dividing the updated second deviation by a height of the first application interface as a vertical offset amount; 8. The method of claim 7, further comprising: if the updated second deviation is less than 0, obtaining a fourth remainder by dividing the updated second deviation by the height of the first application interface, and determining a sum of the height of the first application interface and the fourth remainder as a vertical offset amount.

9. 9. The method of claim 8, further comprising determining that edges of the first canvas and the first application interface in the same direction overlap if the horizontal coordinate of any corner of the first canvas is the same as the horizontal coordinate of any corner of the first application interface, or if the vertical coordinate of any corner of the first canvas is the same as the vertical coordinate of any corner of the first application interface.

10. 2. The method of claim 1, wherein the width of the first canvas is a preset multiple of the width of the first application interface and the height of the first canvas is a preset multiple of the height of the first application interface, the preset multiple being greater than 1 and less than 3.

11. 11. The method of claim 10, wherein the width of the first canvas is twice the width of the first application interface and the height of the first canvas is twice the height of the first application interface.

12. 2. The method of claim 1, further comprising: inhibiting re-rendering of the content of the first canvas if same-direction edges of the first canvas and the first application interface do not overlap.

13. When the control operation is a scaling operation based on one target point, the control parameters include a scaling ratio parameter and an offset parameter, and the offset parameter is a deviation between the coordinate of the target point and the coordinate of the upper left corner of the second canvas. The step of the first terminal updating the content of the first application interface based on the control parameters includes: controlling the first canvas to scale according to the updated scaling ratio parameter based on an origin of the first canvas to obtain a first canvas scaled based on the origin; determining an adjusted offset parameter based on the scaling ratio parameter and an offset parameter; controlling a first canvas scaled based on the origin to move according to the adjusted offset parameter to obtain a first canvas scaled based on the target point; and re-rendering elements of the first canvas scaled based on the target point to update content of the first application interface.

14. The step of determining an adjusted offset parameter based on the scaling ratio parameter and the offset parameter includes: obtaining relative coordinates of the target point in the second application interface; projecting the target point onto a single container based on the offset parameters to obtain absolute coordinates of the target point in the single container, the single container being a container of original size when not scaled; obtaining a first intermediate value by subtracting the absolute coordinate from a sum of the relative coordinate and an offset coordinate of the second application interface, the offset coordinate of the second application interface being used to indicate a difference between a top left corner of the second application interface and a coordinate of an origin of the second canvas; multiplying the absolute coordinate by the difference between the scaling ratio parameter and 1 to obtain a second intermediate value; and subtracting the first intermediate value from the second intermediate value to obtain an adjusted offset parameter.

15. When the control operation is an element adding operation, the control parameters include handwriting coordinate parameters, and the step of updating the content of the first application interface by the first terminal based on the control parameters includes:

2. The method according to claim 1, further comprising the steps of: the first terminal generating a handwriting stroke based on the coordinate parameters of the handwriting stroke, the coordinate parameters of the handwriting stroke including coordinate parameters of the handwriting points for constituting the handwriting stroke; and each time the second terminal generates a new handwriting point, transmitting the coordinate parameters of all the generated handwriting points of the current handwriting stroke to the first terminal.

16. The step of generating handwriting by the first terminal based on the coordinate parameters of the handwriting includes:

16. The method of claim 15, further comprising: triggering the first canvas to move in a direction opposite to the contacting edge when the writing stroke touches any edge of the first application interface; and causing the first terminal to redraw the content of the first canvas when edges of the first canvas and the first application interface in the same direction overlap.

17. A display update device for multi-terminal cooperation, the display update device is used to realize synchronous display of an operation by a first terminal to a second terminal, the first terminal providing a first application interface of a first coordinated whiteboard application, a first canvas for carrying drawing content of the first coordinated whiteboard application, and a first container for carrying the first canvas, the second terminal providing a second application interface of a second coordinated whiteboard application, a second canvas for carrying drawing content of the second coordinated whiteboard application, and a second container for carrying the second canvas, a first length of the first canvas is greater than a length of the first application interface, and a width of the first canvas is greater than a width of the first application interface, the display update device for multi-terminal coordination: a display module, the first terminal being used to display a first application interface of the first coordinated whiteboard application, the first coordinated whiteboard application and a second coordinated whiteboard application of the second terminal being in a coordinated mode, and the second terminal displaying a second application interface of the second coordinated whiteboard application; a receiving module used to receive a control parameter sent by the second terminal, the control parameter being generated when the second terminal accepts a control operation on the second application interface; an update module, wherein the first terminal is used to update content of the first application interface based on the control parameters, and when the control operation is determined to be a move operation, the first canvas moves based on the control parameters, and when edges of the first canvas and the first application interface in the same direction overlap, the first terminal re-renders the content of the first canvas.

18. 1. A computer storage medium, comprising: A computer storage medium having stored thereon a plurality of instructions, said instructions being suitable for being loaded by a processor to perform the steps of the method of any one of claims 1 to 16.

19. 1. An intelligent interactive tablet, comprising: An intelligent interactive tablet comprising a processor and a memory, the memory storing a computer program suitable for being loaded by the processor to perform the steps of the method according to any one of claims 1 to 16.

Images