Interface display method and device

Abstract

The present application provides an interface display method and device, which relates to the field of electronic devices. When a user uses a PC mouse to move the cursor to the content of the projection interface, the content and / or the cursor can provide corresponding visual feedback. The first terminal displays the projection interface on the display screen of the first terminal, and the content of the projection interface is a mirror image of the content of the first interface displayed on the display screen of the second terminal; the first terminal receives a first operation input by the user using an input device of the first terminal, which is used to move the first cursor on the display screen of the first terminal; when the first cursor moves to the content of the projection interface, the cursor style of the first cursor changes, and / or the display mode of the content changes.

Description

Technical Field

[0001] The present application relates to the field of electronic devices, and in particular to an interface display method and device. Background Art

[0002] Currently, a user can have more terminals such as mobile phones, tablet computers, personal computers (PCs), etc. In scenarios where there is a need for collaborative use of multiple terminals, such as collaborative office work, users can connect multiple terminals and use them together.

[0003] For example, if a user has a PC and a mobile phone, the user can connect the PC and the mobile phone wirelessly or wiredly to use them together to achieve collaborative work between the PC and the mobile phone. In the scenario of collaborative work between the PC and the mobile phone, multi-screen collaboration realizes the projection of the mobile phone display interface onto the PC display screen by using mirroring. For the sake of convenience, the interface projected by the mobile phone displayed on the PC can be called the projection interface, such as Figure 1 The projection screen interface 102 is the mobile phone desktop 101. Afterwards, the user can use the PC mouse to perform mouse operations such as mouse clicks and mouse movements in the projection screen interface to operate the actual interface displayed on the mobile phone.

[0004] However, when a user uses a PC mouse to perform mouse operations in the projection interface, there is usually no visual feedback when the cursor moves to a control on the projection interface, such as no visual feedback on the control or no corresponding change in the cursor style. Summary of the invention

[0005] An embodiment of the present application provides an interface display method and device. When a user uses a PC mouse to move a cursor to a control on the projection interface, the control and / or cursor can provide corresponding visual feedback.

[0006] In order to achieve the above purpose, this application adopts the following technical solutions:

[0007] In a first aspect, an embodiment of the present application provides an interface display method, which is applied to a first terminal, where the first terminal is connected to a second terminal. The method may include:

[0008] The first terminal displays a projection interface on the display screen of the first terminal, and the content of the projection interface is a mirror image of the content of the first interface displayed on the display screen of the second terminal; the first terminal receives a first operation input by a user using an input device of the first terminal, and the first operation is used to move a first cursor on the display screen of the first terminal; wherein, when the first cursor moves to the first content of the projection interface, the cursor style of the first cursor is the first style, and / or the display mode of the first content is changed from the first mode to the second mode; when the first cursor moves to the second content of the projection interface, the cursor style of the first cursor is the second style, and / or the display mode of the second content is changed from the third mode to the fourth mode.

[0009] Among them, the content of the projection interface in this embodiment may refer to the elements displayed on the projection interface. In this embodiment, the first content and the second content are different elements displayed in the projection interface, and the cursor style when the first cursor moves to the first content is different from the cursor style when it moves to the second content, that is, the first style is different from the second style. In addition, the above-mentioned first method and the third method may be the same or different. The second method and the fourth method may be the same or different.

[0010] With the above technical solution, when the user moves the cursor to the content of the projection screen interface by operating the input device of the projection target terminal such as the above first terminal, such as a mouse or a touchpad, the content and / or cursor in the projection screen interface will provide corresponding visual feedback, such as the content in the projection screen interface presenting a highlighted background and the cursor style changing accordingly. In this way, the user can visually determine whether the control in the projection screen interface corresponding to the control displayed on the projection source terminal can perform the next operation, thereby improving the user experience.

[0011] In a possible implementation, the above-mentioned projection interface is displayed on a partial area of ​​the display screen of the first terminal; the method may also include: in response to the first operation, the first terminal displays an animation of the movement of a first cursor on the display screen of the first terminal; in the process of the first cursor moving on the display screen of the first terminal, when the first terminal determines that the first cursor enters the projection interface, the first terminal sends the initial coordinate position of the first cursor entering the projection interface to the second terminal, and sends the data of the first operation to the second terminal; wherein the initial coordinate position is the coordinate position of the first cursor relative to the first corner of the projection interface when the first cursor enters the projection interface, and is used for the second terminal to display the second cursor on the display screen of the second terminal; the data of the first operation is used to move the second cursor on the display screen of the second terminal, so that when the first cursor moves to the first content, the second cursor moves to the content corresponding to the first content of the first interface, and when the second cursor moves to the content corresponding to the first content, the cursor style of the second cursor is the first style, and also when the first cursor moves to the second content, the second cursor moves to the content corresponding to the second content of the first interface, and when the second cursor moves to the content corresponding to the second content, the cursor style of the second cursor is the second style.

[0012] When the first cursor moves to the first content of the projection interface, the first terminal receives a cursor type of the first style from the second terminal, and displays the first cursor according to the cursor type of the first style, so that the first cursor is displayed in the first style; when the first cursor moves to the second content of the projection interface, the first terminal receives a cursor type of the second style from the second terminal, and displays the first cursor according to the cursor type of the second style, so that the first cursor is displayed in the second style.

[0013] After the first cursor enters the projection interface, the first terminal sends the corresponding operation data to the second terminal so that the second terminal can move the cursor of the second terminal according to the operation data, and feeds back the cursor style to the first terminal, so that the cursor style of the first cursor on the first terminal can be changed accordingly, giving the user a visual effect of visual feedback of the cursor after the cursor moves to the corresponding content of the projection interface.

[0014] In another possible implementation, when the second cursor moves to the content corresponding to the first content, the display mode of the content corresponding to the first content in the first interface is changed from the first mode to the second mode; the method may also include: after the first cursor moves to the first content of the projection interface, the first terminal updates the projection interface, and the display mode of the first content in the projection interface before the update is the first mode, and the display mode of the first content in the projection interface after the update is the second mode.

[0015] In another possible implementation, when the second cursor moves to the content corresponding to the second content, the display mode of the content corresponding to the second content in the first interface is changed from the third mode to the fourth mode; the method may also include: after the first cursor moves to the second content of the projection interface, the first terminal updates the projection interface, and the display mode of the second content in the projection interface before the update is the third mode, and the display mode of the second content in the projection interface after the update is the fourth mode.

[0016] After the first cursor enters the projection interface, the first terminal sends the corresponding operation data to the second terminal so that the second terminal can move the cursor of the second terminal to the corresponding content according to the operation data, so that the content can provide corresponding visual feedback. The first terminal updates the projection interface to change the display mode of the corresponding content on the first terminal accordingly, giving the user a visual effect of visual feedback of the content after the cursor moves to the corresponding content on the projection interface.

[0017] In another possible implementation manner, the transparency of the second cursor is greater than a threshold.

[0018] In another possible implementation, the above-mentioned sending of data for the first operation to the second terminal may include: after the first cursor enters the projection interface, when the user uses the input device of the first terminal to input the first operation, the first terminal obtains the first operation parameter in the received first input event, and the first input event is a moving event corresponding to the first operation; the first terminal sends the first operation parameter to the second terminal, and the first operation parameter is used by the second terminal to simulate the first input event, and then to move the second cursor.

[0019] In another possible implementation, the method may also include: when the first cursor moves to the first content of the projection interface, the first terminal receives the second operation input by the user using the input device of the first terminal; the first terminal sends the data of the second operation to the second terminal, and the data of the second operation is used for the second terminal to display the second interface; the first terminal updates the projection interface, and the content of the updated projection interface is a mirror image of the content of the second interface. After the cursor moves to the content of the projection interface, if the user operates the content, the first terminal sends the corresponding operation data to the second terminal so that the second terminal responds accordingly. The first terminal updates the projection interface so that the updated interface of the second terminal can be projected onto the first terminal accordingly.

[0020] In another possible implementation, the first terminal sends data of the second operation to the second terminal, which may include: after the user uses the input device of the first terminal to input the second operation, the first terminal intercepts the second input event corresponding to the second operation; the first terminal obtains and sends the second operation parameter in the second input event to the second terminal, and the second operation parameter is used by the second terminal to simulate the second input event, and then to display the second interface.

[0021] In another possible implementation, the first operation corresponds to a moving event; after the first cursor enters the projection interface, the method may further include: the first terminal enables input event interception to intercept input events other than moving events; the first terminal sends a first indication message to the second terminal, and the first indication message is used to indicate the start of sharing.

[0022] In another possible implementation, after the first cursor moves out of the projection interface, the method may further include: the first terminal cancels the interception of the input event; the first terminal sends a second indication message to the second terminal, and the second indication message is used to indicate that the sharing is stopped.

[0023] In another possible implementation, after the first cursor enters the projection interface, the first terminal may adjust the transparency of the first cursor, and the transparency of the adjusted first cursor is greater than a threshold value; the first terminal may also intercept the first input event, and send the first operation parameter of the first input event to the second terminal, so that the second terminal simulates the first input event according to the first operation parameter, and then moves the second cursor. In this way, when the second cursor moves to the content corresponding to the first content on the first interface, the first terminal updates the projection interface; wherein, in the updated interface, the cursor moves to the first content of the projection interface, and the cursor style of the cursor in the updated projection interface is the first style; and / or, the display mode of the first content before the update is the first mode, and the display mode of the first content after the update is the second display mode. After the second cursor moves to the content corresponding to the second content on the first interface, the first terminal updates the projection interface; wherein, in the updated interface, the cursor moves to the second content of the projection interface, and the cursor style of the cursor in the updated projection interface is the second style; and / or, the display mode of the second content before the update is the first mode, and the display mode of the second content after the update is the second display mode. It is understandable that the first terminal can provide the user with a visual effect of providing corresponding visual feedback to the content and / or cursor in the projection interface when the user's cursor moves to the content of the projection interface by updating the projection interface.

[0024] In a second aspect, an embodiment of the present application provides an interface display method, which is applied to a second terminal, where the second terminal is connected to the first terminal. The method may include:

[0025] The second terminal displays a first interface and projects the first interface to the first terminal so that the first terminal displays the projection interface; when the first cursor of the first terminal enters the projection interface, the second terminal displays a second cursor on the first interface; the second terminal receives a first operation input by a user using an input device of the first terminal, and the first operation is used to move the second cursor on the display screen of the second terminal; when the second cursor moves to the first content of the first interface, the second terminal displays the second cursor in a first style, and / or changes the display mode of the first content from the first mode to the second mode, so that when the first cursor moves to the content corresponding to the first content on the projection interface, the first cursor is displayed in the first style, and / or the display mode of the content corresponding to the first content on the projection interface is changed from the first mode to the second mode; when the second cursor moves to the second content of the first interface, the second terminal displays the second cursor in a second style, and / or changes the display mode of the second content from the third mode to the fourth mode, so that when the first cursor moves to the content corresponding to the second content on the projection interface, the first cursor is displayed in the second style, and / or the display mode of the content corresponding to the second content on the projection interface is changed from the first mode to the second mode.

[0026] In a possible implementation, after the second terminal displays the second cursor in the first style, the method may further include: the second terminal sends a cursor type of the first style to the first terminal, which is used for the first terminal to display the first cursor, so that the first cursor is displayed in the first style; after the second terminal displays the second cursor in the second style, the method may further include: the second terminal sends a cursor type of the second style to the first terminal, which is used for the first terminal to display the first cursor, so that the first cursor is displayed in the second style. The second terminal can feed back the cursor style of the second cursor to the first terminal, so that the cursor style of the first cursor on the first terminal can be changed accordingly, giving the user a visual effect of visual feedback of the cursor after the cursor moves to the corresponding content of the projection interface.

[0027] In another possible implementation manner, the transparency of the second cursor is greater than a threshold.

[0028] In another possible implementation, when the first cursor of the first terminal enters the projection interface, the second terminal displays the second cursor on the first interface, which may include: the second terminal receives the initial coordinate position of the first cursor entering the projection interface from the first terminal; the second terminal determines the starting position according to the initial coordinate position, the size of the projection interface and the resolution of the second terminal, and the starting position may be the coordinate position relative to the first corner of the second terminal display screen; the second terminal displays the second cursor at the starting position.

[0029] In another possible implementation, the second terminal receives the first operation input by the user using the input device of the first terminal, which may include: the second terminal receives the first operation parameter from the first terminal, the first operation parameter is the operation parameter in the first input event received by the first terminal during the process of the user using the input device of the first terminal to input the first operation after the first cursor enters the projection interface, and the first operation parameter includes the relative displacement of the first cursor compared to the initial coordinate position; the second terminal determines the relative displacement of the second cursor compared to the starting position according to the relative displacement of the first cursor equivalent to the initial coordinate position; the second terminal simulates the first input event according to the determined relative displacement of the second cursor compared to the starting position and other parameters in the first operation parameter. By converting the relative displacement in the received operation parameter, the second cursor can be moved to the corresponding content after the first cursor moves.

[0030] The method may further include: the second terminal displaying an animation of a second cursor moving on a display screen of the second terminal according to the first input event.

[0031] In another possible implementation, the method may further include: when the second cursor moves to the first content of the first interface, the second terminal receives a second operation input by the user using the input device of the first terminal; in response to the second operation, the second terminal displays the second interface, and projects the second interface to the first terminal, so that the content of the projection interface updated by the first terminal is a mirror image of the content of the second interface.

[0032] In another possible implementation, the second terminal receives a second operation input by a user using an input device of the first terminal, which may include: the second terminal receives a second operation parameter from the first terminal, the second operation parameter being an operation parameter included in a second input event intercepted by the first terminal after the user inputs the second operation using the input device of the first terminal when the first cursor moves to the content corresponding to the first content on the projection interface; the second terminal simulates a second input event according to the second operation parameter, and the second input event is used to display the second interface.

[0033] In another possible implementation, when the first cursor of the first terminal enters the screen projection interface, the method may further include: the second terminal receives first indication information from the first terminal, and the first indication information is used to indicate the start of sharing.

[0034] In another possible implementation, the method may further include: the second terminal receives second indication information from the first terminal, the second indication information is used to indicate that the sharing is stopped, and the second indication information is sent by the first terminal after determining that the first cursor moves out of the projection interface.

[0035] In a third aspect, an embodiment of the present application provides an interface display device, which is applied to a first terminal, and the first terminal is connected to a second terminal. The device may include:

[0036] A display unit is used to display a projection interface on the display screen of the first terminal, and the content of the projection interface is a mirror image of the content of the first interface displayed on the display screen of the second terminal; an input unit is used to receive a first operation input by a user using an input device of the first terminal, and the first operation is used to move a first cursor on the display screen of the first terminal; wherein, when the first cursor moves to the first content of the projection interface, the cursor style of the first cursor is the first style, and / or the display mode of the first content is changed from the first mode to the second mode; when the first cursor moves to the second content of the projection interface, the cursor style of the first cursor is the second style, and / or the display mode of the second content is changed from the third mode to the fourth mode.

[0037] In a possible implementation, the above-mentioned projection interface is displayed on a partial area of ​​the display screen of the first terminal; the display unit is also used to display an animation of the movement of the first cursor on the display screen of the first terminal in response to the first operation; the device may also include: a sending unit, which is used to send the initial coordinate position of the first cursor entering the projection interface to the second terminal when determining that the first cursor enters the projection interface during the process of the first cursor moving on the display screen of the first terminal, and send data of the first operation to the second terminal; wherein the initial coordinate position is the coordinate position of the first cursor relative to the first corner of the projection interface when the first cursor enters the projection interface, and is used for the second terminal to display the second cursor on the display screen of the second terminal; the data of the first operation is used to move the second cursor on the display screen of the second terminal, so that when the first cursor moves to the first content, the second cursor moves to the content corresponding to the first content on the first interface, and when the second cursor moves to the content corresponding to the first content, the cursor style of the second cursor is the first style, and also when the first cursor moves to the second content, the second cursor moves to the content corresponding to the first content on the first interface, and when the second cursor moves to the content corresponding to the second content, the cursor style of the second cursor is the second style.

[0038] The device may further include: a receiving unit.

[0039] The receiving unit is used to receive a cursor type of a first style from a second terminal when the first cursor moves to the first content of the projection interface; the display unit is also used to display the first cursor according to the cursor type of the first style, so that the first cursor is displayed in the first style; the receiving unit is also used to receive a cursor type of a second style from the second terminal when the first cursor moves to the second content of the projection interface; the display unit is also used to display the first cursor according to the cursor type of the second style, so that the first cursor is displayed in the second style.

[0040] In another possible implementation method, when the second cursor moves to the content corresponding to the first content, the display mode of the content corresponding to the first content in the first interface is changed from the first mode to the second mode; the display unit is also used to update the projection interface after the first cursor moves to the first content of the projection interface, and the display mode of the first content in the projection interface before the update is the first mode, and the display mode of the first content in the projection interface after the update is the second mode.

[0041] In another possible implementation, when the second cursor moves to the content corresponding to the second content, the display mode of the content corresponding to the second content in the first interface is changed from the third mode to the fourth mode; the display unit is also used to update the projection interface after the first cursor moves to the second content of the projection interface, and the display mode of the second content in the projection interface before the update is the third mode, and the display mode of the second content in the projection interface after the update is the fourth mode.

[0042] In another possible implementation manner, the transparency of the second cursor is greater than a threshold.

[0043] In another possible implementation, the device may also include: an acquisition unit, used to acquire a first operation parameter in a received first input event when a user uses an input device of a first terminal to input a first operation after the first cursor enters the projection interface, the first input event being a moving event corresponding to the first operation; a sending unit, specifically used to send the first operation parameter to a second terminal, the first operation parameter being used by the second terminal to simulate the first input event and then to move the second cursor.

[0044] In another possible implementation, the input unit is also used to receive a second operation input by the user using the input device of the first terminal when the first cursor moves to the first content of the projection interface; the sending unit is also used to send data of the second operation to the second terminal, and the data of the second operation is used for the second terminal to display the second interface; the display unit is also used to update the projection interface, and the content of the updated projection interface is a mirror image of the content of the second interface.

[0045] In another possible implementation, the acquisition unit is also used to intercept a second input event corresponding to the second operation after the user uses the input device of the first terminal to input the second operation, and to obtain a second operation parameter in the second input event. The sending unit is specifically used to send the second operation parameter to the second terminal, and the second operation parameter is used by the second terminal to simulate the second input event and then to display the second interface.

[0046] In another possible implementation, the first operation corresponds to a movement event; the acquisition unit is further configured to enable interception of input events, and to intercept other input events except the movement event. The sending unit is further configured to send first indication information to the second terminal, and the first indication information is used to indicate the start of sharing.

[0047] In another possible implementation, after the first cursor moves out of the projection interface, the acquisition unit is further used to cancel the interception of the input event. The sending unit is further used to send second indication information to the second terminal, where the second indication information is used to indicate that the sharing is stopped.

[0048] In a fourth aspect, an embodiment of the present application provides an interface display device, which is applied to a second terminal, and the second terminal is connected to the first terminal. The device may include:

[0049] The display unit is used to display a first interface; the projection unit is used to project and display the first interface to the first terminal so that the first terminal displays the projection interface; the display unit is also used to display a second cursor on the first interface when a first cursor of the first terminal enters the projection interface; the receiving unit is used to receive a first operation input by a user using an input device of the first terminal, and the first operation is used to move the second cursor on the display screen of the second terminal; the display unit is also used to display the second cursor in a first style when the second cursor moves to the first content of the first interface, and / or, change the display mode of the first content from the first mode to the second mode, so as to When the first cursor moves to the content corresponding to the first content on the projection interface, the first cursor is displayed in the first style, and / or the display mode of the content corresponding to the first content on the projection interface is changed from the first mode to the second mode; the display unit is also used to display the second cursor in the second style when the second cursor moves to the second content on the first interface, and / or change the display mode of the second content from the third mode to the fourth mode, so that when the first cursor moves to the content corresponding to the second content on the projection interface, the first cursor is displayed in the second style, and / or the display mode of the content corresponding to the second content on the projection interface is changed from the third mode to the fourth mode.

[0050] In a possible implementation, the device may also include: a sending unit, used to send a cursor type of the first style to the first terminal after displaying the second cursor as the first style, so that the first terminal displays the first cursor so that the first cursor is displayed as the first style; the sending unit is also used to send a cursor type of the second style to the first terminal after displaying the second cursor as the second style, so that the first terminal displays the first cursor so that the first cursor is displayed as the second style.

[0051] In another possible implementation manner, the transparency of the second cursor is greater than a threshold.

[0052] In another possible implementation, the receiving unit is further used to receive the initial coordinate position of the first cursor entering the projection interface from the first terminal. The device may also include: a determination unit, used to determine the starting position according to the initial coordinate position, the size of the projection interface and the resolution of the second terminal, and the starting position may be the coordinate position relative to the first corner of the second terminal display screen; a display unit, specifically used to display the second cursor at the starting position.

[0053] In another possible implementation, the receiving unit is specifically used to receive a first operation parameter from the first terminal, where the first operation parameter is an operation parameter in a first input event received by the first terminal when a user uses an input device of the first terminal to input a first operation after the first cursor enters the projection interface, and the first operation parameter includes a relative displacement of the first cursor compared to an initial coordinate position; the determination unit is further used to determine a relative displacement of the second cursor compared to a starting position based on a relative displacement of the first cursor equivalent to the initial coordinate position; the device may further include: a simulation unit, used to simulate the first input event based on the determined relative displacement of the second cursor compared to the starting position, and other parameters in the first operation parameter.

[0054] The display unit is further used to display the animation of the second cursor moving on the display screen of the second terminal according to the first input event.

[0055] In another possible implementation, the receiving unit is further used to receive a second operation input by the user using an input device of the first terminal when the second cursor moves to the first content of the first interface; the display unit is further used to display the second interface in response to the second operation; the projection unit is further used to project the second interface to the first terminal, so that the content of the updated projection interface of the first terminal is a mirror image of the content of the second interface.

[0056] In another possible implementation, the receiving unit is specifically used to receive a second operation parameter from the first terminal, where the second operation parameter is an operation parameter included in a second input event intercepted by the first terminal after the user uses an input device of the first terminal to input a second operation when the first cursor moves to the content corresponding to the first content on the projection interface; the simulation unit is used to simulate a second input event according to the second operation parameter, and the second input event is used to display the second interface.

[0057] In another possible implementation manner, the receiving unit is further configured to receive first indication information from the first terminal, where the first indication information is used to indicate the start of sharing.

[0058] In another possible implementation, the receiving unit is further used to receive second indication information from the first terminal, where the second indication information is used to indicate that sharing is stopped, and the second indication information is sent by the first terminal after determining that the first cursor moves out of the projection interface.

[0059] In a fifth aspect, an embodiment of the present application provides an interface display device, which may include: a processor; a memory for storing processor executable instructions; wherein the processor is configured to execute the instructions so that the interface display device implements a method as described in the first aspect or any one of the possible implementations of the first aspect, or implements a method as described in the second aspect or any one of the possible implementations of the second aspect.

[0060] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by an electronic device, the electronic device implements a method as described in the first aspect or any one of the possible implementations of the first aspect, or implements a method as described in the second aspect or any one of the possible implementations of the second aspect.

[0061] In the seventh aspect, an embodiment of the present application provides an electronic device, which includes a display screen, one or more processors and a memory; the display screen, the processor and the memory are coupled; the memory is used to store computer program code, the computer program code includes computer instructions, and when the computer instructions are executed by the electronic device, the electronic device executes a method as described in the first aspect or any possible implementation of the first aspect, or the terminal executes a method as described in the second aspect or any possible implementation of the second aspect.

[0062] In an eighth aspect, an embodiment of the present application provides a computer program product, comprising a computer-readable code, or a non-volatile computer-readable storage medium carrying a computer-readable code. When the computer-readable code is executed in an electronic device, a processor in the electronic device executes the method described in the first aspect or any possible implementation of the first aspect, or executes the method described in the second aspect or any possible implementation of the second aspect.

[0063] In a ninth aspect, an embodiment of the present application provides an interface display system, which may include a first terminal and a second terminal, and the first terminal is connected to the second terminal.

[0064] The second terminal is used to display the first interface and project the first interface to the first terminal so that the first terminal displays the projection interface.

[0065] The first terminal is used to display a projection interface on the display screen of the first terminal, and the content of the projection interface is a mirror image of the content of the first interface displayed on the display screen of the second terminal; receive a first operation input by a user using an input device of the first terminal, and the first operation is used to move a first cursor on the display screen of the first terminal.

[0066] Among them, when the first cursor moves to the first content of the projection interface, the cursor style of the first cursor is the first style, and / or the display mode of the first content is changed from the first mode to the second mode; when the first cursor moves to the second content of the projection interface, the cursor style of the first cursor is the second style, and / or the display mode of the second content is changed from the third mode to the fourth mode.

[0067] In one possible implementation, a projection interface is displayed on a partial area of ​​a display screen of a first terminal; the first terminal is further used to display an animation of a first cursor moving on the display screen of the first terminal in response to a first operation; the second terminal is further used to display a second cursor on the first interface when the first cursor enters the projection interface; a first operation input by a user using an input device of the first terminal is received, the first operation being used to move the second cursor on the display screen of the second terminal; when the second cursor moves to a content on the first interface corresponding to the first content, the second cursor is displayed in a first style, and the cursor type of the first style is sent to the first terminal; the first terminal is further used to display the first cursor according to the cursor type of the first style; the second terminal is further used to display the second cursor in a second style when the second cursor moves to a content on the first interface corresponding to the second content, and the cursor type of the second style is sent to the first terminal; the first terminal is further used to display the first cursor according to the cursor type of the second style.

[0068] In another possible implementation, the second terminal is also used to change the display mode of the content corresponding to the first interface and the first content from the first mode to the second mode when the second cursor moves to the content corresponding to the first interface and the first content; the first terminal is also used to update the projection interface; the second terminal is also used to change the display mode of the content corresponding to the first interface and the second content from the third mode to the fourth mode when the second cursor moves to the content corresponding to the first interface and the second content; the first terminal is also used to update the projection interface.

[0069] It should be noted that, in this embodiment, the first corner may be any one of the upper left corner, lower left corner, upper right corner and lower right corner of the display screen.

[0070] It can be understood that the beneficial effects that can be achieved by the interface display device described in the third aspect and any possible implementation thereof, the interface display device described in the fourth aspect and any possible implementation thereof, the interface display device described in the fifth aspect, the computer-readable storage medium described in the sixth aspect, the terminal described in the seventh aspect, the computer program product described in the eighth aspect and the interface display system described in the ninth aspect can be referred to as the beneficial effects in the first aspect or the second aspect and any possible implementation thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 A schematic diagram of a display interface provided in an embodiment of the present application;

[0072] Figure 2 A simplified schematic diagram of a system architecture provided in an embodiment of the present application;

[0073] Figure 3 A schematic diagram of the structure of a mobile phone provided in an embodiment of the present application;

[0074] Figure 4 A schematic diagram of the composition of a software architecture provided in an embodiment of the present application;

[0075] Figure 5 A flowchart of an interface display method provided in an embodiment of the present application;

[0076] Figure 6 A schematic diagram of another display interface provided in an embodiment of the present application;

[0077] Figure 7 A schematic diagram of another display interface provided in an embodiment of the present application;

[0078] Figure 8 A schematic diagram of a display coordinate system provided in an embodiment of the present application;

[0079] Fig. 9 A schematic diagram of another display coordinate system provided in an embodiment of the present application;

[0080] Fig.10 A schematic diagram of another display interface provided in an embodiment of the present application;

[0081] Fig.11 A cursor style schematic diagram provided in an embodiment of the present application;

[0082] Fig.12 A schematic diagram of another display interface provided in an embodiment of the present application;

[0083] Fig.13 A schematic diagram of another display interface provided in an embodiment of the present application;

[0084] Fig.14 A schematic diagram of the composition of an interface display device provided in an embodiment of the present application;

[0085] Fig.15 A schematic diagram of the composition of another interface display device provided in an embodiment of the present application;

[0086] Fig.16 A schematic diagram of the composition of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0087] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0088] At present, in order to improve office efficiency, users can connect multiple terminals and use them together. For example, after two terminals are connected, collaborative office work between the two terminals can be achieved by using multi-screen collaboration. Multi-screen collaboration can use mirrored screen projection to project the interface displayed by one terminal onto the display screen of another terminal. In this embodiment, the terminal that projects its display interface can be called the projection source, and the terminal that receives the projection of the projection source and displays the display interface of the projection source is called the projection destination. The interface projected by the projection source displayed on the projection destination is called the projection interface, and the window used by the projection destination to display the projection interface is called the projection window.

[0089] For example, combined with Figure 1 , take the screen projection source end as a mobile phone and the screen projection destination end as a PC as an example. The mobile phone can project the interface displayed on its display screen (such as desktop 101) onto the display screen of the PC. The PC can display the interface projected by the mobile phone on the display screen of the PC, such as the PC displays the projection interface 102 in the projection window. Afterwards, the user can use the input device of the projection destination end to operate on the projection interface to realize the operation of the actual interface of the projection source end. For example, continue to combine Figure 1, taking the input device as a mouse as an example. The user can use the PC's mouse to perform mouse operations such as mouse clicks and mouse movements in the projection interface 102. After the PC receives the corresponding mouse operation, it can convert the coordinates of the user performing the mouse operation in the projection interface 102 into coordinates in the original interface of the mobile phone according to the size ratio between the projection interface 102 and the original interface (such as desktop 101) projected by the mobile phone. The PC sends the converted coordinates and operation type (such as move, click) to the mobile phone so that the mobile phone can generate corresponding touch events to simulate the operation of the actual interface (such as desktop 101), and project the operated interface onto the PC.

[0090] In addition, under normal circumstances, for the content contained in the original interface displayed on the PC, such as controls, when the user moves the cursor to the control by operating the mouse, in order to allow the user to visually know whether the control can perform the next operation, the control and / or cursor will have corresponding visual feedback, such as the control presenting a highlighted background, or the cursor changing from a normal selection style to a text selection style. However, in the above-mentioned multi-screen collaboration scenario, when the user operates the input device of the projection destination (such as a PC), such as moving the cursor to the control of the projection interface by operating the mouse, the projection destination usually will not have visual feedback, such as the control in the projection interface will not present a highlighted background, and the cursor style will not change accordingly. For example, continuing to combine Figure 1 When the user moves the cursor to the icon 103 of the application (APP) 1 in the projection interface 102 by operating the mouse of the PC, the icon 103 will not have any visual feedback (such as no highlighted background), and the cursor 104 will always be in the normal selection style without any change. This is not very user-friendly, and the user cannot visually know whether the icon 105 displayed on the mobile phone corresponding to the icon 103 can perform the next operation.

[0091] An embodiment of the present application provides an interface display method, which can be applied to the collaborative use of multiple terminals, where the screen projection source end projects the interface displayed on its display screen to the scene displayed on the display screen of the screen projection destination end. In the method provided by this embodiment, when the user moves the cursor to the content of the projection interface by operating the input device of the screen projection destination end, such as a mouse or touchpad, the content and / or cursor in the projection interface will provide corresponding visual feedback, such as the content in the projection interface presenting a highlighted background and the cursor style changing accordingly. In this way, the user can visually determine whether the content in the projection interface corresponds to the content displayed on the projection source end and whether the next operation can be performed, thereby improving the user experience.

[0092] It should be noted that the cursor described in this embodiment may also be referred to as a mouse pointer. The cursor may be an image, which may be dynamic or static, and the style of the cursor may also be different in different situations. The content in this embodiment may be an operable element displayed in the interface such as a control, or an inoperable element displayed in the interface. An element may include one or more of the following: text, button, icon, etc.

[0093] The implementation of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0094] Please refer to Figure 2 , is a simplified schematic diagram of a system architecture to which the above method can be applied provided by an embodiment of the present application. Figure 2 As shown, the system architecture may include at least: a first terminal 201 and a second terminal 202.

[0095] The first terminal 201 is connected to the input device 201-1 (eg Figure 2 ), or including an input device 201-1 ( Figure 2 As an example, the input device 201-1 may be a mouse, a touch pad, etc. Figure 2 In the figure, it is taken as an example that the input device 201-1 is a mouse.

[0096] In this embodiment, the first terminal 201 and the second terminal 202 can establish a connection in a wired or wireless manner. Based on the established connection, the first terminal 201 and the second terminal 202 can be used together. In this embodiment, the wireless communication protocol used when the first terminal 201 and the second terminal 202 establish a connection in a wireless manner can be a wireless fidelity (Wi-Fi) protocol, a Bluetooth (Bluetooth) protocol, a ZigBee protocol, a Near Field Communication (NFC) protocol, etc., and can also be various cellular network protocols, which are not specifically limited here.

[0097] After the first terminal 201 is connected to the second terminal 202, the screen projection source in the first terminal 201 and the second terminal 202 can project the interface displayed on its display screen to the screen of the screen projection destination for display. For example, take the first terminal 201 as the screen projection destination and the second terminal 202 as the screen projection source. The second terminal 202 can project the interface displayed on its display screen to the display screen of the first terminal 201 for display. Afterwards, the user uses the input device 201-1 of the first terminal 201 to operate on the projection interface displayed on the display screen of the first terminal 201, and can realize the operation of the actual interface displayed in the second terminal 202.

[0098] In an embodiment of the present application, when a user operates the projection interface displayed on the display screen of the first terminal 201, the user moves the cursor on the display screen of the first terminal 201 to the content of the projection interface, such as a control, by operating the input device 201-1 of the first terminal 201, such as a mouse or a touchpad. With the help of keyboard and mouse sharing technology, the first terminal 201 can make the controls and / or cursor in the projection interface provide corresponding visual feedback, such as presenting a highlighted background for the controls in the projection interface and changing the cursor style accordingly, so that the user can visually know whether the control in the projection interface corresponds to the control displayed on the second terminal 202 and can perform the next operation.

[0099] It should be noted that the terminals in the embodiments of the present application, such as the first terminal 201 and the second terminal 202, can be mobile phones, tablet computers, handheld computers, PCs, cellular phones, personal digital assistants (PDAs), wearable devices (such as smart watches), car computers, game consoles, and augmented reality (AR) and virtual reality (VR) devices. This embodiment does not impose any special restrictions on the specific form of the terminal. Among them, Figure 2 In the example, the first terminal 201 is a PC and the second terminal 202 is a mobile phone. In addition, the technical solution provided in this embodiment can be applied to other electronic devices, such as smart home devices (such as TVs), in addition to the above terminals (or mobile terminals).

[0100] In this embodiment, the terminal is taken as a mobile phone. Figure 3 , is a schematic diagram of the structure of a mobile phone provided in an embodiment of the present application. The methods in the following embodiments can be implemented in a mobile phone having the above hardware structure.

[0101] like Figure 3 As shown, the mobile phone may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, and a display screen 194. Optionally, the mobile phone may also include a mobile communication module 150, a subscriber identification module (SIM) card interface 195, and the like.

[0102] Among them, the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0103] It is to be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the mobile phone. In other embodiments, the mobile phone may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0104] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0105] The controller can be the nerve center and command center of the mobile phone. The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.

[0106] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0107] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM interface, and / or a USB interface, etc.

[0108] The charging management module 140 is used to receive charging input from the charger. While the charging management module 140 is charging the battery 142, it can also power the mobile phone through the power management module 141. The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 can also receive input from the battery 142 to power the mobile phone.

[0109] The wireless communication function of the mobile phone can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor.

[0110] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the mobile phone can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0111] When the mobile phone includes a mobile communication module 150, the mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the mobile phone. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0112] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0113] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as Wi-Fi networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), NFC, infrared technology (IR), etc., which are applied to mobile phones. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and filters it, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0114] In some embodiments, the antenna 1 of the mobile phone is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the mobile phone can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), Beidou navigation satellite system (BDS), quasi-zenith satellite system (QZSS) and / or satellite based augmentation system (SBAS).

[0115] The mobile phone realizes the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 194 and the application processor. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0116] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode or an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the mobile phone may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0117] The mobile phone can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194 and the application processor, etc. In some embodiments, the mobile phone can include 1 or N cameras 193, where N is a positive integer greater than 1.

[0118] The external memory interface 120 can be used to connect an external memory card, such as a MicroSD card, to expand the storage capacity of the mobile phone. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and videos can be stored in the external memory card.

[0119] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the mobile phone by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area can store data created during the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0120] The mobile phone can realize audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor.

[0121] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be set on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. When a touch operation is applied to the display screen 194, the mobile phone detects the intensity of the touch operation according to the pressure sensor 180A. The mobile phone can also calculate the touch position according to the detection signal of the pressure sensor 180A.

[0122] The gyroscope sensor 180B can be used to determine the motion posture of the mobile phone. The air pressure sensor 180C is used to measure air pressure. The magnetic sensor 180D includes a Hall sensor. The mobile phone can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. The acceleration sensor 180E can detect the magnitude of the acceleration of the mobile phone in all directions (generally three axes). The distance sensor 180F is used to measure the distance. The mobile phone can use the proximity light sensor 180G to detect that the user holds the mobile phone close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used for automatic unlocking and locking of the leather case mode and pocket mode. The ambient light sensor 180L is used to sense the brightness of the ambient light. The fingerprint sensor 180H is used to collect fingerprints. The mobile phone can use the collected fingerprint characteristics to realize fingerprint unlocking, access application locks, fingerprint photography, fingerprint answering calls, etc. The temperature sensor 180J is used to detect temperature.

[0123] The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the mobile phone, which is different from the position of the display screen 194.

[0124] The bone conduction sensor 180M can obtain vibration signals. The button 190 includes a power button, a volume button, etc. The button 190 can be a mechanical button. It can also be a touch button. The motor 191 can generate a vibration prompt. The motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. The indicator 192 can be an indicator light, which can be used to indicate the charging status, power changes, messages, missed calls, notifications, etc.

[0125] When the mobile phone includes a SIM card interface 195, the SIM card interface 195 is used to connect the SIM card. The SIM card can be connected to and separated from the mobile phone by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195. The mobile phone can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The mobile phone interacts with the network through the SIM card to achieve functions such as calls and data communications. In some embodiments, the mobile phone uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the mobile phone and cannot be separated from the mobile phone.

[0126] Combination Figure 2 In this embodiment of the application, the software system of the first terminal 201 is a Windows system and the software system of the second terminal 202 is an Android system as an example to illustrate the software structure of the first terminal 201 and the second terminal 202. Please refer to Figure 4 , which is a schematic diagram of the composition of a software architecture provided in an embodiment of the present application.

[0127] Among them, Figure 4 As shown, the software architecture of the first terminal 201 may include: an application layer and a windows system (windows shell). In some embodiments, the application layer may include various applications installed in the first terminal 201. The applications of the application layer may directly interact with the windows system. Exemplarily, the application layer may also include a screen projection service module.

[0128] The software system of the second terminal 202 may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. For example, the software system of the second terminal 202 is a layered architecture. The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, Figure 4 As shown, the second terminal 202 may include an application layer and a framework layer (framework, FWK). The application layer may include a series of application packages. For example, the application package may include settings, calculator, camera, short message, music player and other applications. The applications included in the application layer may be system applications of the second terminal 202 or third-party applications, and the embodiments of the present application are not specifically limited here. The application layer may also include a screen projection service module. The framework layer is mainly responsible for providing an application programming interface (API) and a programming framework for applications in the application layer. Of course, the second terminal 202 may also include other layers, such as a kernel layer ( Figure 4 The kernel layer is a layer between hardware and software. The kernel layer may at least include display driver, camera driver, audio driver, sensor driver, etc.

[0129] In the embodiment of the present application, the first terminal 201 is used as the screen projection destination and the second terminal 202 is used as the screen projection source. After the second terminal 202 projects the interface displayed on its display screen onto the display screen of the first terminal 201 for display, if the user moves the cursor on the display screen of the first terminal 201 to the content of the screen projection interface, such as a control, by operating the input device 201-1 of the first terminal 201, such as a mouse or a touchpad, and the control corresponding to the control displayed on the second terminal 202 can be operated, based on the above software architecture and with the help of keyboard and mouse sharing technology, the first terminal 201 can make the control and / or cursor in the screen projection interface perform corresponding visual feedback, such as the control presenting a highlighted background and the cursor style changing accordingly. In this way, the user can visually know whether the control in the screen projection interface corresponds to the control displayed on the second terminal 202 and can perform the next operation. Among them, the keyboard and mouse sharing technology can refer to the technology of using the input device (such as a mouse, a touchpad) of one terminal to realize the control of other terminals.

[0130] The following combination Figure 2 and Figure 4 Taking the first terminal 201 as a PC, the second terminal 202 as a mobile phone, and the input device 202-1 as a mouse as an example, the interface display method provided in the embodiment of the present application is described in detail in conjunction with the accompanying drawings.

[0131] Figure 5 The following is a flow chart of an interface display method provided in an embodiment of the present application. Figure 5 As shown, the method may include the following S501-S511.

[0132] S501. After the mobile phone is connected to the PC, the mobile phone projects the interface displayed on the mobile phone display screen onto the PC display screen, and the PC displays the projection interface.

[0133] When a user wants to use multi-screen collaboration to achieve collaborative work between multiple terminals, after the connection between these multiple terminals is established, the terminal as the screen projection source can project the interface displayed on its display screen to the terminal display screen as the screen projection destination. For example, take a mobile phone as the screen projection source and a PC as the screen projection destination. The mobile phone establishes a connection with the PC. After that, the mobile phone can project the interface displayed on its display screen to the PC's display screen. The PC can display the projection interface on the PC's display screen.

[0134] There are many ways to establish a connection between the mobile phone and the PC. In some embodiments, the mobile phone and the PC can be connected in a wired manner. For example, the mobile phone and the PC can establish a wired connection via a data cable.

[0135] In some other embodiments, the mobile phone and the PC can establish a connection wirelessly. There are two requirements for establishing a connection between terminals wirelessly. One is that the terminals know each other's connection information, and the other is that each terminal has transmission capability. The connection information can be the device identifier of the terminal, such as the Internet Protocol (IP) address, port number, or the account number logged in by the terminal. The account number logged in by the terminal can be an account provided by the operator for the user, such as a Huawei account. The account number logged in by the terminal can also be an application account, such as account, Account, etc. The transmission capability of the terminal may be a near field communication capability or a long distance communication capability. That is to say, the wireless communication protocol used to establish a connection between terminals, such as a mobile phone and a PC, may be a near field communication protocol such as a Wi-Fi protocol, a Bluetooth protocol, or an NFC protocol, or a cellular network protocol. For example, a user may touch the NFC tag of a PC with a mobile phone, and the mobile phone reads the connection information stored in the NFC tag, such as the IP address of the PC included in the connection information. Afterwards, the mobile phone may establish a connection with the PC using the NFC protocol according to the IP address of the PC. For another example, both the mobile phone and the PC have turned on the Bluetooth function and the Wi-Fi function. The PC may broadcast a Bluetooth signal to discover surrounding terminals, such as the PC may display a list of discovered devices, and the list of discovered devices may include the identifier of the mobile phone discovered by the PC. In addition, during the process of device discovery by the PC, connection information such as an IP address may be exchanged with the discovered devices. Afterwards, after the PC receives an operation in which the user selects the identifier of the mobile phone in the displayed device list, the PC may establish a connection with the mobile phone using the Wi-Fi protocol according to the IP address of the mobile phone. For another example, both the mobile phone and the PC are connected to a cellular network, and the mobile phone and the PC are logged in to the same Huawei account. The mobile phone and PC can establish a connection based on the cellular network based on the Huawei account.

[0136] Exemplary, combined Figure 2In this embodiment, the example of establishing a connection between a mobile phone and a PC by wireless means is taken. In the scenario where the user uses multi-screen collaboration to realize collaborative work between a mobile phone and a PC, the user can manually turn on the screen projection service function of the PC. The screen projection service function of the PC can also be turned on automatically, such as when the PC is turned on. After the screen projection service function of the PC is turned on, the screen projection service module of the PC application layer can start network monitoring to monitor whether there is a terminal connected to the PC. When the user wants to project the display interface of the mobile phone to the PC and use the mobile phone to work collaboratively with the PC, the user can turn on the NFC switch of the mobile phone and touch the NFC tag of the PC with the mobile phone. The mobile phone can read the IP address of the PC saved in the NFC tag. After that, the mobile phone and the PC will display confirmation interfaces respectively to ask the user whether to confirm that the display interface of the mobile phone is projected on the PC for display. When the user selects confirmation on the confirmation interface, the PC, such as the screen projection service module of the PC, can send a message notifying the screen projection to the mobile phone (such as the screen projection service module of the mobile phone). After the mobile phone receives the message notifying the screen projection, it can establish a connection with the PC according to the IP address of the PC obtained above.

[0137] After the connection is successfully established, the mobile phone as the screen projection source can project the interface displayed on the mobile phone display to the display of the PC as the screen projection destination. The PC displays the projection interface. The content displayed in the projection interface is the same as the interface (such as the first interface) displayed on the mobile phone display, or the content in the projection interface is a mirror image of the content of the interface displayed on the mobile phone display. For example, Figure 6 As shown, the mobile phone display currently displays the setting interface 601. After the connection between the mobile phone and the PC is successfully established, the mobile phone can project the setting interface 601 onto the display screen of the PC. The PC displays the projection screen interface 602. It can be seen that the content in the projection screen interface 602 is the same as the content in the setting interface 601.

[0138] The window used by the PC to display the projection interface can be called the projection window. Figure 2 Before the mobile phone projects the interface displayed on the mobile phone display screen to the display screen of the PC, the screen projection service module of the PC application layer can display the screen projection window. For example, the screen projection service module of the PC can display the screen projection window after the screen projection service function of the PC is turned on, or after the screen projection service function of the PC is turned on and the connection with other terminals (such as the above-mentioned mobile phone) is successfully established. The PC can display the screen projection window on its entire display screen, that is, the screen projection window occupies the entire PC display screen. The PC can also display the screen projection window on a partial area of ​​its display screen, that is, the screen projection interface in the screen projection window is only a part of the interface on the PC display screen, and this embodiment does not make specific restrictions here.

[0139] In this embodiment, the specific implementation of the mobile phone projecting the interface displayed on the mobile phone display screen to the PC display screen can be: the mobile phone, such as the screen projection service module of the mobile phone, can obtain the data corresponding to the current display interface of the mobile phone and send it to the PC. After the PC receives the data, it can display the projection interface in the projection window on the PC display screen according to the data. For example, the screen projection service module of the mobile phone can obtain the data corresponding to the current display interface of the mobile phone, such as the screen recording data, through the display manager of the mobile phone (such as the display manager is a module of the mobile phone framework layer), and send it to the PC, so as to realize the projection display of the mobile phone display interface to the PC display screen.

[0140] In some embodiments, a distributed multimedia protocol (Distributed Multi-media Protocol, DMP) can be used to implement the projection display of the mobile phone display interface to the PC display screen. For example, after the mobile phone receives a message from the PC notifying the screen projection, the screen projection service module of the mobile phone can use the display manager (DisplayManager) of the mobile phone to create a virtual display (VirtualDisplay). For example, the screen projection service module of the mobile phone sends a request to create a VirtualDisplay to the display manager of the mobile phone. After the display manager of the mobile phone completes the creation of the VirtualDisplay, it can return the created VirtualDisplay to the screen projection service module of the mobile phone. After that, the screen projection service module of the mobile phone can move the drawing of the interface displayed on the mobile phone display to the VirtualDisplay. In this way, the screen projection service module of the mobile phone can obtain the screen recording data. After the screen projection service module of the mobile phone obtains the screen recording data, the screen recording data can be encoded and sent to the PC. The screen projection service module of the PC can receive the corresponding data, and the screen recording data can be obtained after decoding the data. After that, the screen projection service module of the PC cooperates with the framework layer of the PC to draw the corresponding interface according to the screen recording data and display it in the projection window. For example, the framework layer of the PC can provide a surfaceview to realize the projection display of the projection interface on the PC side.

[0141] In some other embodiments, wireless projection (Miracast) can also be used to realize the projection display of the mobile phone display interface on the PC display screen, that is, the mobile phone can obtain all layers of the mobile phone display interface, and then integrate all the obtained layers into a video stream (or called screen recording data) and encode it and send it to the PC through the real-time streaming protocol (RTSP) protocol. After receiving the video stream, the PC can decode and play it to realize the projection display of the mobile phone display interface on the PC display screen. Alternatively, the mobile phone can extract the instructions of the mobile phone display interface to obtain the instruction stream, and obtain the layer information of the mobile phone display interface, and then send the instruction stream and layer information to the PC, so that the PC can restore the interface displayed on the mobile phone display screen to realize the projection display of the mobile phone display interface on the PC.

[0142] S502: The mobile phone creates a virtual input device.

[0143] In this embodiment, the mobile phone as the screen projection source can also create a virtual input device, which is used to simulate the corresponding input event on the mobile phone when the user uses the input device (such as a mouse) of the screen projection destination, such as a PC, to operate the screen projection interface displayed on the PC. The mobile phone can realize the control of the mobile phone by the PC input device by responding to the simulated input event. In other words, the user can use the input device of the screen projection destination to not only control the screen projection destination, but also control the screen projection source, and realize the sharing of keyboard and mouse between the screen projection destination and the screen projection source.

[0144] As an exemplary implementation, when the keyboard and mouse sharing mode of the PC is turned on, keyboard and mouse sharing between the PC and the mobile phone can be achieved, that is, the user can use the input device of the PC to control both the PC and the mobile phone.

[0145] For example, in some embodiments, after the other terminal successfully establishes a connection with the PC, the PC may display a pop-up window. The pop-up window is used to ask the user whether to turn on the keyboard and mouse sharing mode. If the user selects to turn on the keyboard and mouse sharing mode, the PC may turn on the keyboard and mouse sharing mode.

[0146] After turning on the keyboard and mouse sharing mode, the PC can notify all terminals that have established a connection with itself, or terminals that have established a connection with itself and projected the interface to the PC, that the keyboard and mouse sharing mode has been turned on. If the PC has established a connection with the mobile phone, and the mobile phone projects the interface to the PC, the PC will notify the mobile phone that the keyboard and mouse sharing mode has been turned on. For example, the PC can send a notification message to the mobile phone, and the notification message can be used to indicate that the keyboard and mouse sharing mode of the PC has been turned on. After receiving the notification, the mobile phone can create a virtual input device, which has the same function as conventional input devices such as a mouse and a touchpad, and can be used to simulate corresponding input events on the mobile phone. For example, taking the input device as a mouse as an example, the virtual input device created by the mobile phone has the same function as a conventional mouse, and can be regarded as a mouse shared by the PC to the mobile phone, which can be used to simulate mouse events on the mobile phone to realize the control of the mobile phone by the PC mouse. Exemplary, take the mobile phone's operating system as an Android system as an example. The mobile phone can use the uinput capability of Linux to create a virtual input device. Among them, uinput is a kernel layer module that can simulate input devices. By writing to the / dev / uinput (or / dev / input / uinput) device, the process can create a virtual input device with a specific function. Once the virtual input device is created, it can simulate the corresponding event. Similarly, other terminals that have established a connection with the PC will also create virtual input devices based on the received notification. It should be noted that if the operating system of the terminal receiving the notification is the Android system, the creation of the virtual input device can be achieved by using the uinput capability of Linux, or the creation of the virtual input device can be achieved by using the human-computer interaction device (HID) protocol. If the operating system of the terminal receiving the notification is other operating systems such as the IOS system or the Windows system, the HID protocol can be used to create the virtual input device. In addition, the above embodiment is described by taking the creation of the virtual input device as an example after the terminal receives a notification for notifying that the keyboard and mouse sharing mode of the PC has been turned on. In some other embodiments, after the terminal receives the above notification, a pop-up window can also be displayed to ask the user whether he wants to use the input device of the PC to control the device. If the user selects to use the input device of the PC to control the device, the virtual input device is created, otherwise the virtual input device is not created.

[0147] For another example, in some other embodiments, it can also be set by default that after other terminals, such as mobile phones, are connected to PCs, the PC automatically turns on the keyboard and mouse sharing mode without the need for the user to turn it on manually. In other terminals, such as after the mobile phone mentioned above is connected to the PC, the virtual input device can also be created automatically without the PC sending a notification. Alternatively, after other terminals are connected to the PC, a pop-up window can be displayed to ask the user whether he wants to use the PC's input device to control the device. If the user selects to use the PC's input device to control the device, the virtual input device is automatically created, otherwise the virtual input device is not created. Alternatively, it can also be set by default that after the terminal as the source of the screen projection projects the interface to the PC as the destination of the screen projection, the PC automatically turns on the keyboard and mouse sharing mode without the need for the user to turn it on manually. The terminal as the source of the screen projection can also create a virtual input device after projecting the interface to the PC, or after receiving a message from the PC notifying the screen projection.

[0148] In addition, combined Figure 2 Since the mouse is an input device of the PC, when the PC opens the keyboard and mouse sharing mode, other terminals, such as mobile phones, create virtual input devices. Generally, the PC temporarily responds to the mouse operation, or the user can temporarily control the PC using the mouse. In this embodiment, after the keyboard and mouse sharing mode is turned on, the PC can trigger other terminals that have established a connection with the PC and created virtual input devices, such as mobile phones, to respond to the mouse operation after determining that the mouse shuttle condition is met, that is, triggering the keyboard and mouse sharing between the PC and the mobile phone.

[0149] Exemplarily, the mouse shuttle condition may be a cursor displayed on the PC display screen, such as cursor 1 sliding into the projection screen displayed on the PC display screen. Taking the input device as a mouse as an example, the user can move the mouse to make cursor 1 displayed on the PC display screen slide into the projection screen displayed on the PC display screen to trigger the keyboard and mouse sharing between the PC and the mobile phone. For example, the method also includes the following S503-S504.

[0150] S503: The PC receives a mouse movement event, and displays an animation of the movement of the cursor 1 on the display screen of the PC according to the mouse movement event.

[0151] Among them, cursor 1 can be the first cursor in the embodiment of the present application.

[0152] S504. When the PC determines that the cursor 1 enters the screen projection interface, the PC sends a first indication message to the mobile phone to indicate the start of keyboard and mouse sharing.

[0153] When the user wants to operate on the screen projection interface of the screen projection destination to operate the actual interface of the screen projection source, the user can operate the input device of the screen projection destination, such as inputting a first operation, so that the cursor displayed on the screen projection destination moves to the screen projection interface. In this embodiment, when the cursor displayed on the screen projection destination enters the screen projection interface, the screen projection destination and the screen projection source can start sharing the keyboard and mouse.

[0154] Let's continue with the example where the input device is a mouse, the screen projection destination is a PC, and the screen projection source is a mobile phone. The user can move the mouse of the PC so that the cursor 1 moves on the display screen of the PC. During the movement of the cursor 1 on the display screen of the PC, the PC can determine whether the cursor 1 enters the projection interface displayed on the display screen of the PC. For example, as described above, the projection interface is displayed in the projection window of the PC, and the projection window can be used to monitor whether the cursor 1 enters the projection interface. For example, when the cursor 1 enters the projection interface, the projection window can detect a corresponding event, which is used to indicate that the cursor 1 enters the projection window, and the PC can determine that the cursor 1 enters the projection interface based on the event. When it is determined that the cursor 1 enters the projection interface displayed on the PC display screen, the PC can determine that the mouse shuttle conditions are met, and then the keyboard and mouse sharing can be started with the mobile phone. In addition, after determining that the mouse shuttle has started, the PC can also send the above-mentioned first indication information to the mobile phone to indicate to the mobile phone that the keyboard and mouse sharing has started.

[0155] For example, combined with Figure 2 and Figure 6 When a user uses an input device of a PC to input a movement operation, such as when the user moves a mouse, the PC may receive a corresponding input event, such as a movement event, which may be referred to as a mouse movement event. Based on the received mouse movement event, the windows system of the PC may draw an animation of the movement of the cursor 1 and display it on the display screen of the PC. Figure 7 As shown, as the mouse 701 moves, the PC displays an animation of the movement of the cursor 703 on the display screen 702 of the PC, as shown in FIG. Figure 7The moving track of the cursor 703 shown in is shown as track 704. In the process of the cursor 703 moving on the display screen 702 of the PC, the screen projection service module of the PC application layer can determine whether the cursor 703 enters the screen projection interface 705 through the screen projection window. For example, when the cursor 703 enters the screen projection interface 705, the screen projection window can detect an event indicating that the cursor 703 enters the screen projection window. After detecting the event, the screen projection window can send a notification to the screen projection service module of the PC application layer to notify the screen projection service module of the PC that the cursor has entered the screen projection interface 705. After the screen projection service module of the PC determines that the cursor 703 enters the screen projection interface, it can be determined that the mouse shuttle condition is met. After that, the PC can start sharing the keyboard and mouse with the mobile phone. The screen projection service module of the PC can also send an indication message to the mobile phone (such as the screen projection service module of the mobile phone) to indicate the start of keyboard and mouse sharing. After receiving the indication message, the mobile phone can prepare to receive input events from the PC, such as mouse events. It should be noted that the above examples are illustrated by taking the communication between the PC and the mobile phone through the screen projection service module included in both as an example, that is, the screen projection service module has the function of communicating with other terminals. In some other embodiments, the screen projection service module may not have the function of communicating with other terminals. In this embodiment, the communication between the PC and the mobile phone can be achieved through other modules. For example, the PC and the mobile phone may also include a transmission management module, and the two can communicate through this module. This embodiment does not make specific restrictions here. For the convenience of description, the following embodiments are illustrated by taking the communication between the PC and the mobile phone through the screen projection service module as an example.

[0156] In addition, in order for the projection source to accurately locate the content on the projection source corresponding to the user's operation position on the projection interface, such as controls, when the user uses the input device of the projection destination to operate the projection interface, in this embodiment, a cursor, such as cursor 2, can be displayed on the projection source, and the cursor 2 can be moved with the user's operation on the input device of the projection destination. Among them, cursor 2 can be the second cursor in the embodiment of the present application. For example, the method also includes the following S505-S508.

[0157] S505. The PC sends the initial coordinate position of the cursor 1 when it enters the screen projection interface to the mobile phone.

[0158] The initial coordinate position is the entry point of the cursor 1 when it enters the projection interface relative to the origin of the projection interface (or projection window) (the origin may be a corner of the projection interface (such as the first corner), such as Figure 7 The coordinate position of the origin O1) shown in .

[0159] S506. The mobile phone displays a cursor 2 on the mobile phone display screen according to the initial coordinate position.

[0160] The cursor 2 is an invisible cursor, and its transparency is greater than a threshold value, for example, the transparency of the cursor 2 is very high, or completely transparent.

[0161] After determining that the mouse shuttle has started, the PC serving as the screen projection destination can obtain the coordinate position of the entry point of the cursor 1 relative to the origin of the screen projection interface when the cursor 1 enters the screen projection interface (i.e., obtain the above-mentioned initial coordinate position), and send the initial coordinate position to the mobile phone serving as the screen projection source.

[0162] As an example, after cursor 1 enters the projection interface, the PC can obtain the coordinate position of the entry point of cursor 1 in the display coordinate system of the PC when it enters the projection interface, such as coordinate position 1. The PC can determine the above initial coordinate position based on coordinate position 1 and the coordinate position of the upper left corner of the projection interface in the display coordinate system of the PC, such as coordinate position 2.

[0163] For example, see Figure 8 , the display coordinate system of the PC can be based on the upper left corner of the PC as the coordinate origin ( Figure 8 The X-axis points from the origin O2 to the right edge of the PC display, and the Y-axis points from the origin O2 to the bottom edge of the PC display. Figure 7 , use cursor 1 to enter the entry point of the screen projection interface. Figure 7 As shown in the entry point 706 in . After determining that the cursor 1 enters the projection interface, the PC can obtain the coordinate position 1 of the entry point 706 in the display coordinate system of the PC, such as the obtained coordinate position 1 is (a, b). When displaying the projection interface, the position of the projection window used to display the projection interface in the PC display coordinate system is known to the PC, and the position of the projection window is the position of the projection interface. Then the coordinate position of the upper left corner of the projection interface in the display coordinate system of the PC is called coordinate position 2, such as coordinate position 2 is (a, c). The PC can determine the coordinate position of the entry point 706 relative to the coordinate origin O1 based on the coordinate position 1 and coordinate position 2, that is, determine the initial coordinate position. For example, the initial coordinate position determined by the PC is (0, bc).

[0164] After receiving the initial coordinate position, the mobile phone can determine the starting position where the cursor 2 appears on the mobile phone according to the initial coordinate position. The mobile phone can display the cursor 2 at the starting position.

[0165] It is understandable that the window used to display the projection interface is the projection window, and the size of the projection interface is determined by the size of the projection window, such as the size of the projection interface is the same as the size of the projection window. The size of the projection window can be predefined, and it can be the same as or different from the resolution of the mobile phone. For example, if the size of the projection window is different from the resolution of the mobile phone, the projection interface in the projection window is the same as the interface projected by the mobile phone, but the projection interface is the interface after stretching and / or compressing the interface projected from the mobile phone.

[0166] When the size of the projection window is different from the resolution of the mobile phone, in order to make the starting position of cursor 2 displayed on the mobile phone consistent with the position where cursor 1 enters the projection interface, the mobile phone can convert the starting position of cursor 2 on the mobile phone based on the resolution of the mobile phone and the size of the projection interface, based on the initial coordinate position. In other words, after the mobile phone receives the initial coordinate position from the PC, it can determine the starting position of cursor 2 on the mobile phone based on the resolution of the mobile phone, the size of the projection interface (or the size of the projection window) and the initial coordinate position. The starting position is the coordinate position of cursor 2 relative to the origin of the mobile phone display (the origin can be a corner of the mobile phone display (such as the first corner)).

[0167] Among them, the size of the screen projection window can be sent by the PC to the mobile phone during the process of establishing a connection between the PC and the mobile phone, or after the connection is successfully established.

[0168] For example, Fig. 9 As shown, assuming that cursor 1 is displayed at the position shown by point 1 on the projection interface, the corresponding cursor 2 should be displayed at the position shown by point 2 on the mobile phone display. For example, the size of the projection window is A1*B1, the mobile phone resolution is A2*B2, the coordinates of point 1 in coordinate system 1 are (x1, y1), and the coordinates of point 2 in coordinate system 2 are (x2, y2). Among them, coordinate system 1 takes the upper left corner of the projection window as the coordinate origin (such as Fig. 9 The X-axis points from the origin O1 to the right edge of the projection window, and the Y-axis points from the origin O1 to the bottom edge of the projection window. Coordinate system 2 takes the upper left corner of the mobile phone display as the origin (such as Fig. 9O2 in the coordinate system, the X-axis points from the coordinate origin O2 to the right edge of the mobile phone display screen, and the Y-axis points from the coordinate origin O2 to the lower edge of the mobile phone display screen. The mobile phone can determine the conversion relationship between the coordinate position of cursor 1 in coordinate system 1 and the coordinate position of cursor 2 in coordinate system 2 according to the size of the projection window and the resolution of the mobile phone. For example, the coordinate position of cursor 1 on the X-axis in coordinate system 1 and the coordinate position of cursor 2 on the X-axis in coordinate system 2 should satisfy x1 / A1=x2 / A2. When the coordinate position of cursor 1 on the X-axis in coordinate system 1 is known, the coordinate position of cursor 2 on the X-axis in coordinate system 2 is x2=(A2 / A1)*x1, where the formula can be called conversion relationship 1, and A2 / A1 can be called conversion ratio value 1. Similarly, the coordinate position of cursor 1 on the Y axis in coordinate system 1 and the coordinate position of cursor 2 on the Y axis in coordinate system 2 should satisfy y1 / B1=y2 / B2. Then, when the coordinate position of cursor 1 on the Y axis in coordinate system 1 is known, the coordinate position of cursor 2 on the Y axis in coordinate system 2 is y2=(B2 / B1)*y1, where this formula can be called conversion relationship 2, and B2 / B1 can be called conversion ratio value 2. For example, if the mobile phone resolution is 2340*1080 and the size of the projection window is 1170*540, then conversion ratio value 1 is equal to 2, and conversion ratio value 2 is equal to 2.

[0169] In one possible implementation, after receiving the initial coordinate position from the PC, the mobile phone can determine the starting position where the cursor 2 appears on the mobile phone according to the above conversion relationship (such as conversion relationship 1 and / or conversion relationship 2). In another possible implementation, the mobile phone can predetermine the above conversion ratio value 1 and conversion ratio value 2. After receiving the initial coordinate position, the starting position where the cursor 2 appears on the mobile phone can be determined according to the predetermined conversion ratio value 1 and / or conversion ratio value 2. For example, in combination with Figure 8 In the example shown, the initial coordinate position is (0, bc), and the mobile phone determines that the starting position where the cursor 2 appears on the mobile phone is (0, (B2 / B1)*(bc)). The mobile phone can display the cursor 2 on the mobile phone display screen according to the determined starting position. Figure 7 As shown, the mobile phone displays a cursor 2 on the mobile phone display screen as shown in Figure 7 As shown in 707 in FIG. It can be seen that the starting position of the cursor 2 displayed on the mobile phone is consistent with the position of the entry point when the cursor 1 on the PC enters the projection interface.

[0170] The cursor 2 displayed by the mobile phone may be an invisible cursor, whose transparency is greater than a threshold value. For example, if the transparency of the cursor 2 is very high or completely transparent, it can be said that the cursor 2 is invisible to the user. The cursor 2 may also not be an invisible cursor and may be visible to the user, which is not limited in this embodiment. For ease of description, the drawings of the embodiments of the present application take the case where the cursor 2 is visible to the user as an example.

[0171] In addition, the above embodiment is described by taking the example that the projection destination obtains the initial coordinate position and sends it to the projection source, and the projection source determines the starting position of the cursor 2 on the projection source according to the initial coordinate position. In some other embodiments, after the projection destination obtains the initial coordinate position, the projection destination can also determine the starting position of the cursor 2 on the projection source according to the initial coordinate position, and send the starting position to the projection source for the projection source to display the cursor 2. The specific determination process is the same as the determination process of the projection source to determine the starting position of the cursor 2, and will not be repeated here. The device resolution of the projection source can be sent to the projection destination during the process of establishing a connection with the projection destination, or after the connection is successfully established. When the size of the projection window is the same as the resolution of the mobile phone, the mobile phone can directly display the cursor 2 on the mobile phone according to the initial coordinate position without the need for conversion.

[0172] S507. The PC sends the mouse operation parameter 1 included in the mouse movement event to the mobile phone.

[0173] S508: The mobile phone receives the mouse operation parameter 1, and simulates a mouse movement event according to the mouse operation parameter 1. The mobile phone displays an animation of the movement of the cursor 2 on the mobile phone display screen according to the mouse movement event.

[0174] The mouse movement event may be the first input event in the embodiment of the present application. The mouse operation parameter 1 may be the first operation parameter in the embodiment of the present application.

[0175] After cursor 1 enters the projection interface, the user may continue to operate the input device of the projection destination to move cursor 1 to the desired position in the projection interface. Since keyboard and mouse sharing has begun after cursor 1 enters the projection interface, after keyboard and mouse sharing begins, the projection destination may not respond to the input event received after the user operates the input device, but instead sends the operation parameters in the input event to the projection source of keyboard and mouse sharing so that the projection source responds to the input event.

[0176] Among them, when the input device is a mouse, the input event may include a mouse movement event, a mouse press event, a mouse lift event, etc. It should be noted that when the cursor 2 displayed on the mobile phone is an invisible cursor, the screen projection interface projected from the mobile phone to the PC does not contain a cursor, and the cursor 1 is displayed on the PC. Therefore, when the user moves the mouse, the cursor, such as cursor 1, can move with the movement of the mouse. After the keyboard and mouse sharing starts, the screen projection target end does not respond to the input events received after the user operates the input device. Specifically, it can be: the screen projection target end does not respond to other mouse events except mouse movement events, such as mouse press events and mouse lift events. Instead, it responds to mouse movement events so that after the user moves the mouse, the cursor 1 can move on the PC display accordingly.

[0177] As an exemplary implementation, the screen projection destination, such as a PC, can mount a hook after turning on the keyboard and mouse sharing mode. The mounted HOOK can be used to intercept (or shield) other input events except mouse movement events after the keyboard and mouse sharing starts. The mounted HOOK can also be used to obtain (or capture) the operation parameters contained in the corresponding input event (including mouse movement events and other input events) after the keyboard and mouse sharing starts. For example, if the input device is a mouse, the input event can be a mouse event. That is to say, after the cursor enters the screen projection interface, the keyboard and mouse sharing starts, and then the PC can use the mounted HOOK to intercept other input events except mouse movement events. The PC can also use the mounted HOOK to capture the operation parameters in the received mouse event, such as the so-called mouse operation parameters, and send the captured operation parameters to the screen projection source end that creates the virtual input device, so that the screen projection source end can use the created virtual input device to simulate the corresponding input event, such as a mouse event, and then respond to it. In this way, for mouse movement events, not only the screen projection destination can respond to the operation input by the input device, but the screen projection source end can also respond to the operation input by the input device. For input events other than mouse movement events, the mounted HOOK will intercept them, so the projection target end does not respond to them. Instead, the projection source end responds to the input device input operation according to the operation parameters sent by the projection target end.

[0178] The above mouse operation parameters may include: mouse button flag (used to indicate which operation the user has performed on the mouse, such as pressing, lifting, moving or scrolling the mouse), coordinate information (used to indicate the X-coordinate and Y-coordinate of the cursor movement when the user moves the mouse), scroll wheel information (used to indicate the X-axis distance and Y-axis distance of the scroll wheel when the user operates the mouse wheel), and key position information (used to indicate which of the left, middle or right buttons of the mouse the user has operated). As an example, after the keyboard and mouse sharing starts, the mounted HOOK can determine whether the input event is a mouse movement event based on the mouse button flag in the above mouse operation parameters. If it is a mouse movement event, it will not be intercepted; if it is not a mouse movement event, it will be intercepted.

[0179] Of course, the interception of input events and the capture of operating parameters therein can also be achieved in other ways (such as registering RAWINPUT in the PC). Alternatively, the interception of input events and the capture of operating parameters therein can also be achieved in different ways. For example, taking the input device as a mouse as an example, after the PC turns on the keyboard and mouse sharing mode, the HOOK can be mounted and RAWINPUT can be registered. After the keyboard and mouse sharing starts, the mounted HOOK can be used to intercept other mouse events except the mouse movement event, and the registered RAWINPUT can be used to capture the parameters in the mouse event. This embodiment does not limit the specific implementation of the interception of mouse events and the capture of parameters therein. For the convenience of description, the following embodiments are introduced by taking the interception of input events and the capture of operating parameters therein by mounting HOOK as an example.

[0180] For example, combined with Figure 2 and Figure 7 , take the input device as a mouse as an example. Take the user wanting to open the Bluetooth settings interface of a mobile phone as an example. Fig.10 As shown, after the user moves the PC mouse so that the cursor 1 enters the projection interface 1001, the user continues to move the mouse so that the cursor 1 moves to the position of the Bluetooth option 1002 on the projection interface 1001. Since the cursor 1 enters the projection interface 1001, the keyboard and mouse sharing has begun. After the keyboard and mouse sharing starts, the mounted HOOK can intercept other mouse events except the mouse movement event. Therefore, after the user continues to move the mouse, the PC can receive corresponding input events, such as mouse movement events, and the mouse movement event will not be intercepted by the mounted HOOK, and the event will be transmitted to the Windows system of the PC. Based on the received mouse movement event, the Windows system of the PC can continue to draw the animation of the movement of the cursor 1 and display it on the display screen of the PC. As shown Fig.10 As shown, as the mouse 1003 moves, the PC displays an animation of the cursor 1004 moving on the display screen of the PC, as shown in FIG. Fig.10 The movement track of the cursor 1004 is shown as track 1005, and it can be seen that the cursor 1004 moves to the position of the Bluetooth option 1002 on the projection interface 1001. After the cursor 1004 enters the projection interface, it moves before the Bluetooth option 1002. It can be considered that the cursor 1004 is displayed on an element of the projection interface, which can be the first content in this embodiment. It can be seen that on the first content, the cursor style of the cursor 1004 is style 1 (the style 1 can be the first style in this embodiment), that is, the normal selection style.

[0181] In addition, after the keyboard and mouse sharing starts, the mounted HOOK can capture the operation parameters in the input event. Therefore, while the user continues to move the mouse, the PC, such as the screen projection service module of the PC application layer, can use the mounted HOOK to capture the operation parameters in the received mouse movement event, such as the so-called mouse operation parameter 1, and send the mouse operation parameter 1 to the mobile phone at the screen projection source. As an example, the mouse operation parameter 1 can be: a mouse button flag indicating that the user has moved the mouse, coordinate information indicating the X and Y coordinates of the movement of the cursor (such as cursor 1), scroll wheel information (value is empty) and key position information (value is empty). Among them, the coordinate information is the relative displacement of cursor 1 compared to the position of cursor 1 when cursor 1 enters the screen projection interface during the movement of the mouse.

[0182] After receiving the mouse operation parameter 1, the mobile phone can simulate corresponding input events, such as mouse movement events, using the created virtual input device according to the mouse operation parameter 1, so that the cursor 2 on the mobile phone can also move to the position of the Bluetooth option on the actual interface displayed by the mobile phone.

[0183] Among them, as described in S506, the size of the projection window may be different from the resolution of the mobile phone. Therefore, after the user moves the PC mouse, in order to enable the cursor 2 to move to the position of the Bluetooth option in the actual interface, the mobile phone can calculate the relative displacement of the cursor 2 on the mobile phone compared to the starting position based on the coordinate information in the mouse operation parameter 1 according to the resolution of the mobile phone and the size of the projection interface.

[0184] For example, something like Fig. 9As described in , the size of the projection window is A1*B1, the resolution of the mobile phone is A2*B2, and after the user continues to move the mouse, the relative displacement of cursor 1 in coordinate system 1 compared to the entry point is (X3, Y3), and the relative displacement of cursor 2 in coordinate system 2 compared to the starting position is (X4, Y4) as an example. The mobile phone can determine the conversion relationship between the relative displacement of cursor 1 in coordinate system 1 compared to the entry point and the relative displacement of cursor 2 in coordinate system 2 compared to the starting position after the mouse moves according to the size of the projection window and the resolution of the mobile phone. For example, the relative displacement of cursor 1 on the X-axis in coordinate system 1 and the relative displacement of cursor 2 on the X-axis in coordinate system 2 should satisfy X3 / A1=X4 / A2. Then, when the relative displacement of cursor 1 on the X-axis in coordinate system 1 is known, the relative displacement of cursor 2 on the X-axis in coordinate system 2 is X4=(A2 / A1)*X3, where this formula can be called conversion relationship 3, and A2 / A1 can be called conversion ratio value 1. Similarly, the relative position of cursor 1 on the Y axis in coordinate system 1 and the relative displacement of cursor 2 on the Y axis in coordinate system 2 should satisfy Y3 / B1=Y4 / B2. Then, when the relative displacement of cursor 1 on the Y axis in coordinate system 1 is known, the relative displacement of cursor 2 on the Y axis in coordinate system 2 is Y4=(B2 / B1)*Y3, where this formula can be called conversion relationship 4, and B2 / B1 can be called conversion ratio value 2. For example: if the resolution of the mobile phone is 2340*1080 and the size of the projection window is 1170*540, then conversion ratio value 1 is equal to 2, and conversion ratio value 2 is equal to 2. That is to say, the distance moved by cursor 1 on the X axis and Y axis on the projection window is twice the original distance when converted to the mobile phone. Among them, the relative displacement (X3, Y3) of cursor 1 in coordinate system 1 compared to the entry point can be determined according to the coordinate information in mouse operation parameter 1.

[0185] In one possible implementation, after receiving the mouse operation parameter 1 from the PC, the mobile phone (such as the screen projection service module of the mobile phone application layer) can determine the relative displacement of the cursor 2 on the mobile phone compared to the starting position based on the coordinate information in the mouse operation parameter 1 and the above-mentioned conversion relationship (such as conversion relationship 3 and / or conversion relationship 4). Alternatively, the mobile phone can predetermine the above-mentioned conversion ratio value 1 and conversion ratio value 2, and after receiving the mouse operation parameter 1 from the PC, it can determine the relative displacement of the cursor 2 on the mobile phone compared to the starting position based on the coordinate information in the mouse operation parameter 1, and the predetermined conversion ratio value 1 and / or conversion ratio value 2. Based on the determined relative displacement and other parameters in the mouse operation parameter 1 (such as the mouse button flag, scroll wheel information and key position information), the mobile phone (such as the framework layer of the mobile phone) can use the created virtual input device to simulate corresponding input events, such as mouse movement events. Based on the mouse movement event, the framework layer of the mobile phone can draw an animation of the movement of the cursor 2 and display it on the display screen of the mobile phone. Continue to refer to Fig.10As shown, as the mouse 1003 moves, the mobile phone can display an animation of the cursor 1006 moving on the display screen of the mobile phone, such as Fig.10 The moving track of the cursor 1006 is shown as track 1007. It can be seen that since the coordinate information is the converted coordinates, the cursor 1006 moves to the position of the Bluetooth option 1009 on the actual interface 1008. In this way, by operating the mouse of the PC, the user can not only move the cursor 1 on the PC display to the desired position in the projection interface, but also move the cursor 2 on the mobile phone display to the corresponding position.

[0186] It is understandable that, as described in S506, when the cursor 2 displayed on the mobile phone is an invisible cursor, when the user moves the mouse of the PC to move the cursors on the mobile phone and the PC, the animation of the movement of the cursor 2 on the mobile phone is invisible to the user. Fig.10 The animation of the movement of cursor 2 is shown only for the convenience of understanding.

[0187] In addition, it should be noted that, due to the different operating systems of the PC and the mobile phone, the key values ​​of the mouse operation parameters in the mouse events of the two are different. Therefore, after the mobile phone receives the mouse operation parameter 1, it can convert the key code of the received mouse operation parameter 1 into a key code that the mobile phone can recognize according to the preset mapping relationship. Afterwards, the mobile phone can use the created virtual input device to simulate the input event that the mobile phone can recognize, such as the mouse movement event, according to the mouse operation parameter 1 after the key code is converted, so as to make a corresponding response.

[0188] S509: When the cursor 2 moves to a control on the current display interface of the mobile phone, the mobile phone changes the cursor 2 from style 1 to style 2.

[0189] S510. The mobile phone sends a cursor type of style 2 to the PC.

[0190] S511. The PC displays cursor 1 on the display screen of the PC according to the cursor type, and updates the screen projection interface.

[0191] Generally, for the content in the display interface, such as controls, in order to allow the user to visually know whether the next operation can be performed, when the user moves the cursor to it by operating an input device such as a mouse, for the controls that can perform the next operation, the control and / or cursor will have corresponding visual feedback. In this embodiment, when the cursor 1 moves to the position of a control in the projection interface, if the control corresponding to the control on the mobile phone is a control that can perform the next operation, the control and / or cursor 1 in the projection interface will provide corresponding visual feedback. For example, the control presents a highlighted background, and the style of the cursor 1 changes.

[0192] As described in S507-S508, when the user moves cursor 1 to a certain control on the projection interface by operating the mouse of the PC, cursor 2 on the mobile phone display screen also moves to the position of the corresponding control in the mobile phone display interface. For the mobile phone, when cursor 2 moves to the position of the control in the display interface, if the control can perform the next operation, the cursor style of cursor 2 will change, such as the cursor style of cursor 2 changes from style 1 to style 2. For example, if cursor 2 moves from one content in the interface (such as the content can be the first content in this embodiment) to another content (such as the content can be the second content in this embodiment), the cursor style of cursor 2 changes from style 1 (such as the first style in this embodiment) to style 2 (such as the second style in this embodiment).

[0193] It is understandable that there are multiple cursor styles, and different cursor styles have different cursor types. Fig.11 As shown, it is a schematic diagram of a cursor style and its corresponding cursor type provided in this embodiment. Among them, the cursor style may include: normal selection style, link selection style, text selection style, moving style and vertical adjustment style, etc. The corresponding cursor types include: normal selection, link selection, text selection, moving and vertical adjustment, etc. When the cursor is on different controls, the cursor style of the cursor can be different or the same. The correspondence between the specific control and the cursor style can be pre-defined by a third-party application developer or equipment manufacturer and stored in the mobile phone. As long as the cursor moves to the control, the cursor style changes. When the cursor 2 moves to the position of a control in the display interface, if the control can perform the next operation, the mobile phone can change the cursor style of the cursor 2 according to the pre-stored correspondence. For example, continue to combine Fig.10 In the example shown, cursor 2, i.e., cursor 1006, moves to the position of Bluetooth option 1009 on actual interface 1008, i.e., cursor 1006 moves from the position of the first content displayed on actual interface 1008 to the position of the second content, i.e., Bluetooth option 1009. The first content is an element displayed in the interface that cannot be operated. For example, when the predefined cursor 2 is in an inoperable position on the mobile phone, the cursor style is a normal selection style, and when the predefined cursor 2 is on Bluetooth option 1009, the cursor style is a link selection style. When cursor 2 moves from the position of the first content to the position of Bluetooth option 1009 on actual interface 1008, the cursor style of cursor 2 changes from a normal selection style to a link selection style.

[0194] After the cursor style of cursor 2 on the mobile phone changes, the mobile phone can send the cursor type of the changed cursor style to the PC. For example, after the mobile phone displays cursor 2, a cursor style listener can be registered in the framework layer of the mobile phone. In this way, after the cursor style of cursor 2 changes, the cursor style listener can listen to the event of the cursor style change. The framework layer of the mobile phone can obtain the changed cursor style, such as the cursor type of style 2, and send it to the PC through the screen projection service module of the mobile phone application layer.

[0195] After receiving the cursor type of style 2, the screen projection service module of the PC, such as the PC application layer, can display the cursor 1 on the display screen of the PC according to the cursor type. Fig.10 In the example shown, cursor 1, that is, cursor 1004, is changed from the normal selection style at the first content of the projection interface to the link selection style. In this way, the user is presented with a visual effect that the cursor style changes when cursor 1 moves to the position of the control in the projection interface.

[0196] In addition, when the cursor 2 moves to the position of the corresponding content in the mobile phone display interface, the content may have corresponding visual feedback. Exemplarily, the display mode of the first content can be changed from the first mode to the second mode, and the display mode of the second content can be changed from the third mode to the fourth mode. Among them, the display modes before the change of different contents can be the same or different. The display modes after the change of different contents can be the same or different. For example, if the content is a control, the first mode and the third mode are the same, that is, no highlighted background is presented, and the second mode and the fourth mode are the same, both of which present a highlighted background as an example. In this way, when the cursor 2 moves to the position of the corresponding control in the mobile phone display interface, the control changes from not presenting a highlighted background to presenting a highlighted background. It can be understood that in multi-screen collaboration, the mobile phone will project the interface displayed on the mobile phone display to the PC display in real time, so after the control presents a highlighted background, the change will also be projected onto the PC display. In this way, when the cursor 1 is presented to the user to move to the position of the control in the projection interface, the control gives the effect of responding to visual feedback.

[0197] For example, combined with Fig.10 For example, when cursor 2 moves to the position of a Bluetooth option in the mobile phone display interface, the Bluetooth option and cursor 2 both have visual feedback. Fig.12As shown, when the user moves the mouse of the PC to move cursor 1 to the position of Bluetooth option 1203 in the screen projection interface 1201, the Bluetooth option 1203 in the screen projection interface of the PC presents a highlighted background, and cursor 1 changes from the normal selection style to the link selection style. In addition, cursor 2 on the mobile phone also moves to the position of Bluetooth option 1204 in the actual interface 1202, and the Bluetooth option 1204 in the actual interface 1202 of the mobile phone presents a highlighted background, and cursor 2 changes from the normal selection style to the link selection style. It should be noted that cursor 2 may be invisible to the user.

[0198] After that, continue to combine Fig.12 If the user wants to set up the Bluetooth of the mobile phone, the user can use the input device of the PC, such as the mouse input of the PC to press the operation (the pressing operation can be the second operation in the embodiment of the present application, and the second operation can also be other operations). For example, the user can press the left button of the mouse. The PC can receive the corresponding input event, such as the mouse press event. Since the mouse press event is received after the keyboard and mouse sharing is turned on, the mounted HOOK will intercept the mouse press event so that the windows system of the PC does not respond to it. In addition, the PC (such as the screen projection service module of the PC application layer) can obtain the operation parameters in the mouse press event, such as the so-called mouse operation parameter 2 (such as the mouse operation parameter 2 captured by the mounted HOOK), and send the mouse operation parameter 2 to the mobile phone. Among them, the mouse press event can be the second input event in the embodiment of the present application, and the mouse operation parameter 2 can be the second operation parameter in the embodiment of the present application. The mouse operation parameter 2 may include: a mouse button flag for indicating that the user has pressed the mouse, coordinate information (the value is empty), wheel information (the value is empty) and key position information for indicating that the user has operated the left button of the mouse. It should be noted that only when the coordinate information value in the mouse operation parameters is not empty, the mobile phone needs to convert the coordinate information before simulating the corresponding input event (as described in the specific description of the corresponding content in S508 above). If the coordinate information value in the mouse operation parameters is empty, no conversion is required, and the corresponding input event can be simulated according to the received mouse operation parameters.

[0199] After the mobile phone (such as the screen projection service module of the mobile phone application layer) receives the mouse operation parameter 2, the framework layer of the mobile phone can convert the key code of the received mouse operation parameter 2 into a key code that the mobile phone can recognize according to the preset mapping relationship, and then use the created virtual input device to simulate the input event that the mobile phone can recognize, such as a mouse press event, according to the mouse operation parameter 2 after the key code is converted. According to the mouse press event, the mobile phone can respond accordingly, such as displaying the second interface. Fig.13As shown, the mobile phone displays the Bluetooth setting interface 1302. The mobile phone projects the Bluetooth setting interface 1302 displayed on the mobile phone display to the PC display, and the PC displays the projection interface 1301. In addition, it can be understood that after the display interface of the mobile phone changes, the control at the position of cursor 2 will also change. The mobile phone can use the cursor style monitor to monitor the style of cursor 2 in real time. If it is determined that the style of cursor 2 has changed, the cursor type of the changed cursor style is sent to the PC, so that the PC can make corresponding changes to the cursor style of cursor 1. For example, combined with Fig.13 , after the mobile phone displays the Bluetooth setting interface 1302, there is no control at the location of the cursor 1304, and the cursor 1304 is changed from the link selection style to the normal selection style. The mobile phone can send the cursor type of the normal selection style to the PC, so that the PC displays the cursor 1303 of the normal selection style on the projection window.

[0200] After the user completes the operation in the projection interface, the user may operate the PC's input device, such as moving the PC's mouse, to move the cursor 1 out of the projection interface. Similarly, the projection window for displaying the projection interface can also be used to monitor whether the cursor 1 moves out of the projection interface. For example, when the cursor 1 moves out of the projection interface, the projection window can detect a corresponding event, which is used to indicate that the cursor 1 moves out of the projection window, and the PC can determine that the cursor 1 moves out of the projection interface according to the event. After the cursor 1 moves out of the projection interface, the PC can determine that the keyboard and mouse sharing between the mobile phone and the mobile phone has stopped. The PC can send a second indication message to the mobile phone to indicate to the mobile phone that the keyboard and mouse sharing has stopped. The PC can also uninstall HOOK (or turn off HOOK), that is, cancel the interception of input events, such as mouse events, and the capture of the operating parameters therein. Afterwards, if the user operates the PC's input device, the PC will not intercept the received input event, but will send the received input event to the PC's windows system so that the PC's windows system responds to the input event, that is, the user can use the PC's mouse to control the PC. In addition, it can be understood that when the user moves the mouse of the PC so that the cursor 1 moves out of the projection interface, the invisible cursor 2 on the mobile phone will also move to the edge of the mobile phone display screen. In this embodiment, after receiving the above-mentioned second indication information, the mobile phone can restore the display of cursor 2, that is, set cursor 2 to be visible, so as to ensure that the cursor can be normally displayed on the mobile phone display screen after the mobile phone is directly connected to the mouse. The above is explained by taking the PC as an example to determine whether to stop keyboard and mouse sharing. In some other embodiments, the mobile phone can also determine whether to stop keyboard and mouse sharing. As described in the above embodiment, after the user moves the mouse of the PC, not only the cursor 1 on the PC display screen can move with it, but also the cursor 2 on the mobile phone display screen can move with it. Therefore, the mobile phone can monitor whether the cursor 2 moves out of the edge of the mobile phone display screen during the movement of the cursor 2. After determining that the cursor 2 moves out of the edge of the mobile phone display screen, the mobile phone can determine that the keyboard and mouse sharing between the mobile phone and the PC is stopped. The mobile phone can send the above-mentioned second indication information to the PC to indicate to the PC that the keyboard and mouse sharing is stopped. After receiving the second indication information, the PC can uninstall HOOK. The mobile phone can also restore the display of the cursor 2 after determining that the cursor 2 moves out of the edge of the mobile phone display screen.

[0201] It should be noted that the above embodiment is described by taking the example that when cursor 1 enters the projection interface (such as when cursor 1 is at the entry point), the cursor style of cursor 2 displayed at the starting position on the mobile phone does not change, that is, it is still the normal selection style. In some other embodiments, if the style of cursor 2 corresponding to the mobile phone changes when cursor 1 enters the projection interface, such as from the normal selection style to the link selection style, the mobile phone can send the changed cursor type to the PC so that the PC can correspondingly change the cursor style of cursor 1 at the entry point.

[0202] The above embodiment is described by taking the example that after the keyboard and mouse sharing starts, the mobile phone displays an invisible cursor 2, and after the style of the cursor 2 changes, the changed cursor type is sent to the PC, so that the PC changes the style of the cursor 1 accordingly, so as to realize the visual feedback of the cursor in the projection interface of the PC. In some other embodiments, after the keyboard and mouse sharing starts, the PC can hide the cursor on its display screen, such as cursor 1, and the mobile phone displays a visible cursor 2. In this way, when the cursor 2 on the mobile phone moves to the control that can perform the next operation, the style of the cursor 2 can change accordingly, and / or the control can provide visual feedback. Since the interface on the mobile phone will be projected onto the display screen of the PC in real time, when the style of the cursor 2 changes, and / or the control provides visual feedback, the corresponding content in the projection interface projected onto the PC display screen will also change accordingly, so that the user can also be given the effect of corresponding visual feedback of the control and / or cursor in the projection interface. The specific implementation is similar to the description in the above embodiment, except that after cursor 1 slides into the projection window, cursor 1 on the PC is hidden, and visible cursor 2 is displayed on the mobile phone; the mounted HOOK intercepts all input events after the keyboard and mouse sharing starts. Other descriptions are the same as those in the above embodiment, and will not be repeated here.

[0203] In addition, the above embodiment is described by taking the input device as a mouse as an example. In the present embodiment, the input device may also be a touch pad. When the input device is a touch pad, the user may use the button (left button or right button) of the touch pad to input a press operation, and slide a finger on the touch pad to input a move operation. The specific implementation of the method of the present embodiment by the user using the touch pad input operation is similar to the specific implementation of the method of the present embodiment using the mouse input operation, and will not be repeated here.

[0204] The technical solution provided by this embodiment is that when the user moves the cursor to a control on the projection screen interface by operating the input device of the projection target end, such as a mouse or touchpad, the control and / or cursor in the projection screen interface will provide corresponding visual feedback, such as a highlighted background for the control in the projection screen interface and a corresponding change in the cursor style. In this way, the user can visually determine whether the control in the projection screen interface corresponding to the control displayed on the projection source end can be operated on the next step, thereby improving the user experience.

[0205] Fig.14 This is a schematic diagram of the composition of an interface display device provided in an embodiment of the present application. Fig.14 As shown, the device can be applied to a first terminal (such as the above-mentioned PC), and the first terminal is connected to a second terminal. The device can include: a display unit 1401 and an input unit 1402.

[0206] The display unit 1401 is used to display a screen projection interface on the display screen of the first terminal, and the content of the screen projection interface is a mirror image of the first interface content displayed on the display screen of the second terminal.

[0207] The input unit 1402 is used to receive a first operation input by a user using an input device of the first terminal, where the first operation is used to move a first cursor on the display screen of the first terminal.

[0208] Among them, when the first cursor moves to the first content of the projection interface, the cursor style of the first cursor is the first style, and / or the display mode of the first content is changed from the first mode to the second mode; when the first cursor moves to the second content of the projection interface, the cursor style of the first cursor is the second style, and / or the display mode of the second content is changed from the third mode to the fourth mode.

[0209] Furthermore, the above-mentioned screen projection interface is displayed on a partial area of ​​the first terminal display screen.

[0210] The display unit 1401 is further configured to display an animation of a first cursor moving on a display screen of the first terminal in response to the first operation.

[0211] The device may also include: a sending unit 1403, which is used to send the initial coordinate position of the first cursor entering the projection interface to the second terminal when it is determined that the first cursor enters the projection interface during the process of the first cursor moving on the display screen of the first terminal, and send data of the first operation to the second terminal.

[0212] Among them, the initial coordinate position is the coordinate position of the first cursor relative to the first corner of the projection interface when the first cursor enters the projection interface, and is used by the second terminal to display the second cursor on the display screen of the second terminal; the data of the first operation is used to move the second cursor on the display screen of the second terminal, so that when the first cursor moves to the first content, the second cursor moves to the content corresponding to the first content on the first interface, and when the second cursor moves to the content corresponding to the first content, the cursor style of the second cursor is the first style; and also when the first cursor moves to the second content, the second cursor moves to the content corresponding to the second content on the first interface, and when the second cursor moves to the content corresponding to the second content, the cursor style of the second cursor is the second style.

[0213] The device may further include: a receiving unit 1404 .

[0214] The receiving unit 1404 is used to receive a cursor type of a first style from the second terminal when the first cursor moves to the first content of the projection interface.

[0215] The display unit 1401 is further configured to display the first cursor according to the cursor type of the first style, so that the first cursor is displayed in the first style.

[0216] The receiving unit 1404 is further configured to receive a cursor type of a second style from a second terminal when the first cursor moves to a second content on the projection interface.

[0217] The display unit 1401 is further configured to display the first cursor according to the cursor type of the second style, so that the first cursor is displayed in the second style.

[0218] Furthermore, when the second cursor moves to the content corresponding to the first content, the display mode of the content corresponding to the first content in the first interface is changed from the first mode to the second mode. The display unit 1401 is also used to update the projection interface after the first cursor moves to the first content in the projection interface, and the display mode of the first content in the projection interface before the update is the first mode, and the display mode of the first content in the projection interface after the update is the second mode.

[0219] In another possible implementation, when the second cursor moves to the content corresponding to the second content, the display mode of the content corresponding to the second content in the first interface is changed from the third mode to the fourth mode. The display unit 1401 is also used to update the projection interface after the first cursor moves to the second content in the projection interface, and the display mode of the second content in the projection interface before the update is the third mode, and the display mode of the second content in the projection interface after the update is the fourth mode.

[0220] In another possible implementation manner, the transparency of the second cursor is greater than a threshold.

[0221] In another possible implementation, the device may also include: an acquisition unit 1405, used to acquire a first operation parameter in a received first input event when the user uses an input device of the first terminal to input a first operation after the first cursor enters the projection interface, the first input event being a moving event corresponding to the first operation; a sending unit 1403, specifically used to send the first operation parameter to the second terminal, the first operation parameter being used by the second terminal to simulate the first input event and then to move the second cursor.

[0222] In another possible implementation, the input unit 1402 is also used to receive a second operation input by the user using the input device of the first terminal when the first cursor moves to the first content of the projection interface; the sending unit 1403 is also used to send data of the second operation to the second terminal, and the data of the second operation is used for the second terminal to display the second interface; the display unit 1401 is also used to update the projection interface, and the content of the updated projection interface is a mirror image of the content of the second interface.

[0223] In another possible implementation, the acquisition unit 1405 is also used to intercept a second input event corresponding to the second operation after the user uses the input device of the first terminal to input the second operation, and to obtain a second operation parameter in the second input event. The sending unit 1403 is specifically used to send the second operation parameter to the second terminal, and the second operation parameter is used by the second terminal to simulate the second input event and then to display the second interface.

[0224] In another possible implementation, the first operation corresponds to a moving event. The acquiring unit 1405 is further configured to enable interception of input events, and to intercept other input events except the moving event. The sending unit 1403 is further configured to send first indication information to the second terminal, and the first indication information is used to indicate the start of sharing.

[0225] In another possible implementation, after the first cursor moves out of the projection interface, the sending unit 1403 is further used to send a second indication message to the second terminal, where the second indication message is used to indicate that sharing is stopped; the acquiring unit 1405 is further used to cancel the interception of the input event.

[0226] Fig.15 This is a schematic diagram of another interface display device provided in an embodiment of the present application. The device can be applied to a second terminal (such as the mobile phone mentioned above), and the second terminal is connected to the first terminal. Fig.15 As shown, the device may include: a display unit 1501 , a projection unit 1502 and a receiving unit 1503 .

[0227] The display unit 1501 is used to display the first interface.

[0228] The projection unit 1502 is used to project and display the first interface to the first terminal so that the first terminal displays the projection interface.

[0229] The display unit 1501 is further configured to display a second cursor on the first interface when the first cursor of the first terminal enters the screen projection interface.

[0230] The receiving unit 1503 is configured to receive a first operation input by a user using an input device of the first terminal, where the first operation is used to move a second cursor on a display screen of the second terminal.

[0231] The display unit 1501 is also used to display the second cursor in the first style when the second cursor moves to the first content of the first interface, and / or change the display mode of the first content from the first mode to the second mode, so that when the first cursor moves to the content corresponding to the first content on the projection interface, the first cursor is displayed in the first style, and / or the display mode of the content corresponding to the first content on the projection interface is changed from the first mode to the second mode; the display unit 1501 is also used to display the second cursor in the second style when the second cursor moves to the second content of the first interface, and / or change the display mode of the second content from the third mode to the fourth mode, so that when the first cursor moves to the content corresponding to the second content on the projection interface, the first cursor is displayed in the second style, and / or the display mode of the content corresponding to the second content on the projection interface is changed from the third mode to the fourth mode.

[0232] In a possible implementation, the device may also include: a sending unit 1504, which is used to send a cursor type of the first style to the first terminal after displaying the second cursor as the first style, so that the first terminal displays the first cursor so that the first cursor is displayed as the first style; the sending unit 1504 is also used to send a cursor type of the second style to the first terminal after displaying the second cursor as the second style, so that the first terminal displays the first cursor so that the first cursor is displayed as the second style.

[0233] In another possible implementation manner, the transparency of the second cursor is greater than a threshold.

[0234] In another possible implementation, the receiving unit 1503 is further used to receive the initial coordinate position of the first cursor entering the projection interface from the first terminal. The device may also include: a determination unit 1505, used to determine the starting position according to the initial coordinate position, the size of the projection interface and the resolution of the second terminal, and the starting position may be the coordinate position relative to the first corner of the second terminal display screen; a display unit 1501, specifically used to display the second cursor at the starting position.

[0235] In another possible implementation, the receiving unit 1503 is specifically used to receive a first operation parameter from the first terminal, where the first operation parameter is an operation parameter in a first input event received by the first terminal when a user uses an input device of the first terminal to input a first operation after the first cursor enters the projection interface, and the first operation parameter includes a relative displacement of the first cursor compared to an initial coordinate position; the determination unit 1505 is further used to determine a relative displacement of the second cursor compared to a starting position based on a relative displacement of the first cursor to the initial coordinate position; the device may further include: a simulation unit 1506, used to simulate the first input event based on the determined relative displacement of the second cursor compared to the starting position, and other parameters in the first operation parameter.

[0236] The display unit 1501 is further configured to display an animation of a second cursor moving on a display screen of the second terminal according to the first input event.

[0237] In another possible implementation, the receiving unit 1503 is further used to receive a second operation input by the user using the input device of the first terminal when the second cursor moves to the first content of the first interface; the display unit 1501 is further used to display the second interface in response to the second operation; the projection unit 1502 is further used to project the second interface to the first terminal, so that the content of the updated projection interface of the first terminal is a mirror image of the content of the second interface.

[0238] In another possible implementation, the receiving unit 1503 is specifically used to receive a second operation parameter from the first terminal, where the second operation parameter is an operation parameter included in a second input event intercepted by the first terminal after the user uses an input device of the first terminal to input a second operation when the first cursor moves to the content corresponding to the first content on the projection interface; the simulation unit 1506 is used to simulate a second input event according to the second operation parameter, and the second input event is used to display the second interface.

[0239] In another possible implementation manner, the receiving unit 1503 is further configured to receive first indication information from the first terminal, where the first indication information is used to indicate the start of sharing.

[0240] In another possible implementation, the receiving unit 1503 is further used to receive second indication information from the first terminal, where the second indication information is used to indicate that sharing is stopped, and the second indication information is sent by the first terminal after determining that the first cursor moves out of the projection interface.

[0241] The embodiment of the present application also provides an interface display device, which can be applied to electronic devices, such as the first terminal or the second terminal in the above embodiment. The device may include: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions so that the interface display device implements the various functions or steps executed by the mobile phone or PC in the above method embodiment.

[0242] The embodiment of the present application also provides an electronic device (the electronic device can be a terminal, such as the first terminal or the second terminal in the above embodiment), which can include: a display screen, a memory and one or more processors. The display screen, the memory and the processor are coupled. The memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device can execute the various functions or steps performed by the mobile phone or PC in the above method embodiment. Of course, the electronic device includes but is not limited to the above display screen, the memory and one or more processors. For example, the structure of the electronic device can refer to Figure 3 The structure of the mobile phone shown.

[0243] The present application also provides a chip system, which can be applied to electronic devices, such as the terminal (such as the first terminal or the second terminal) in the above embodiments. Fig.16 As shown, the chip system includes at least one processor 1601 and at least one interface circuit 1602. The processor 1601 may be a processor in the above-mentioned electronic device. The processor 1601 and the interface circuit 1602 may be interconnected via a line. The processor 1601 may receive and execute computer instructions from the memory of the above-mentioned electronic device through the interface circuit 1602. When the computer instructions are executed by the processor 1601, the electronic device may execute the various steps executed by the mobile phone or PC in the above-mentioned embodiment. Of course, the chip system may also include other discrete devices, which are not specifically limited in the embodiments of the present application.

[0244] An embodiment of the present application also provides a computer-readable storage medium for storing computer instructions executed by an electronic device, such as the above-mentioned terminal (such as a mobile phone or a PC).

[0245] An embodiment of the present application also provides a computer program product, including an electronic device, such as computer instructions executed by the above-mentioned terminal (such as a mobile phone or a PC).

[0246] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0247] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0248] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0249] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0250] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0251] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An interface display method, characterized in that: Applied to a first terminal, the first terminal is connected to a second terminal, the method includes: The first terminal displays a screen projection interface on the display screen of the first terminal, and the content of the screen projection interface is a mirror image of the content of the first interface displayed on the display screen of the second terminal; The first terminal receives a first operation input by a user using an input device of the first terminal, where the first operation is used to move a first cursor on a display screen of the first terminal; Among them, when the first cursor moves to the first content of the projection interface, the first terminal receives a cursor type of a first style from the second terminal, and displays the first cursor according to the cursor type of the first style, so that the first cursor is displayed in the first style, and / or the display mode of the first content is changed from the first mode to the second mode; when the first cursor moves to the second content of the projection interface, the first terminal receives a cursor type of a second style from the second terminal, and displays the first cursor according to the cursor type of the second style, so that the first cursor is displayed in the second style, and / or the display mode of the second content is changed from the third mode to the fourth mode.

2. The method according to claim 1, characterized in that: The screen projection interface is displayed on a partial area of ​​the first terminal display screen; The method further comprises: In response to the first operation, the first terminal displays an animation of the first cursor moving on a display screen of the first terminal; During the process of the first cursor moving on the display screen of the first terminal, when the first terminal determines that the first cursor enters the projection interface, the first terminal sends the initial coordinate position of the first cursor entering the projection interface to the second terminal, and sends the data of the first operation to the second terminal; Among them, the initial coordinate position is the coordinate position of the first cursor relative to the first corner of the projection interface when the first cursor enters the projection interface, and is used by the second terminal to display the second cursor on the display screen of the second terminal; the data of the first operation is used to move the second cursor on the display screen of the second terminal, so that when the first cursor moves to the first content, the second cursor moves to the content corresponding to the first content on the first interface, and when the second cursor moves to the content corresponding to the first content, the cursor style of the second cursor is the first style; and also when the first cursor moves to the second content, the second cursor moves to the content corresponding to the first interface and the second content, and when the second cursor moves to the content corresponding to the second content, the cursor style of the second cursor is the second style.

3. The method according to claim 2, characterized in that When the second cursor moves to the content corresponding to the first content, the display mode of the content corresponding to the first content in the first interface is changed from the first mode to the second mode; The method further comprises: After the first cursor moves to the first content of the projection interface, the first terminal updates the projection interface, the display mode of the first content in the projection interface before the update is the first mode, and the display mode of the first content in the projection interface after the update is the second mode; When the second cursor moves to the content corresponding to the second content, the display mode of the content corresponding to the second content in the first interface is changed from the third mode to the fourth mode; The method further comprises: After the first cursor moves to the second content of the projection interface, the first terminal updates the projection interface, and the display mode of the second content in the projection interface before the update is the third mode, and the display mode of the second content in the projection interface after the update is the fourth mode.

4. The method according to claim 2 or 3, characterized in that: The transparency of the second cursor is greater than a threshold.

5. The method according to claim 2 or 3, characterized in that: The sending the data of the first operation to the second terminal includes: After the first cursor enters the screen projection interface, during the process in which the user uses the input device of the first terminal to input the first operation, the first terminal obtains a first operation parameter in the received first input event, where the first input event is a movement event corresponding to the first operation; The first terminal sends the first operation parameter to the second terminal, where the first operation parameter is used by the second terminal to simulate the first input event and further to move the second cursor.

6. The method according to any one of claims 1 to 3, characterized in that The method further comprises: When the first cursor moves to the first content of the projection interface, the first terminal receives a second operation input by the user using an input device of the first terminal; The first terminal sends data of the second operation to the second terminal, where the data of the second operation is used for the second terminal to display a second interface; The first terminal updates the screen projection interface, and the content of the updated screen projection interface is a mirror image of the content of the second interface.

7. The method according to claim 6, characterized in that The first terminal sending data of the second operation to the second terminal includes: After the user inputs the second operation using the input device of the first terminal, the first terminal intercepts a second input event corresponding to the second operation; The first terminal obtains and sends a second operation parameter in the second input event to the second terminal, where the second operation parameter is used by the second terminal to simulate the second input event and further to display the second interface.

8. The method according to claim 7, characterized in that The first operation corresponds to a moving event; After the first cursor enters the screen projection interface, the method further includes: The first terminal enables interception of input events to intercept other input events except the movement event; The first terminal sends first indication information to the second terminal, where the first indication information is used to indicate the start of sharing.

9. The method according to claim 8, characterized in that After the first cursor moves out of the screen projection interface, the method further includes: The first terminal cancels interception of the input event; The first terminal sends second indication information to the second terminal, where the second indication information is used to indicate that sharing is stopped.

10. An interface display method, characterized in that: Applied to a second terminal, the second terminal is connected to the first terminal, the method includes: The second terminal displays the first interface, and projects and displays the first interface to the first terminal, so that the first terminal displays the projection interface; When the first cursor of the first terminal enters the screen projection interface, the second terminal displays a second cursor on the first interface; The second terminal receives a first operation input by a user using an input device of the first terminal, where the first operation is used to move the second cursor on a display screen of the second terminal; When the second cursor moves to the first content of the first interface, the second terminal displays the second cursor in a first style and sends the cursor type of the first style to the first terminal, and / or changes the display mode of the first content from the first mode to the second mode, so that when the first cursor moves to the content corresponding to the first content on the projection interface, the first terminal receives the cursor type of the first style from the second terminal and displays the first cursor according to the cursor type of the first style, so that the first cursor is displayed in the first style, and / or the display mode of the content corresponding to the first content on the projection interface is changed from the first mode to the second mode; When the second cursor moves to the second content of the first interface, the second terminal displays the second cursor in the second style and sends the cursor type of the second style to the first terminal, and / or changes the display mode of the second content from the third mode to the fourth mode, so that when the first cursor moves to the content corresponding to the second content on the projection interface, the first terminal receives the cursor type of the second style from the second terminal and displays the first cursor according to the cursor type of the second style, so that the first cursor is displayed in the second style, and / or the display mode of the content corresponding to the second content on the projection interface is changed from the third mode to the fourth mode.

11. The method according to claim 10, characterized in that The transparency of the second cursor is greater than a threshold.

12. The method according to claim 10, characterized in that When the first cursor of the first terminal enters the screen projection interface, the second terminal displays a second cursor on the first interface, including: The second terminal receives, from the first terminal, an initial coordinate position of the first cursor entering the projection interface; The second terminal determines a starting position according to the initial coordinate position, the size of the projection interface and the resolution of the second terminal, where the starting position is a coordinate position relative to a first corner of the display screen of the second terminal; The second terminal displays the second cursor at the starting position.

13. The method according to claim 12, characterized in that The second terminal receives a first operation input by a user using an input device of the first terminal, including: The second terminal receives a first operation parameter from the first terminal, where the first operation parameter is an operation parameter in a first input event received by the first terminal during a process in which a user uses an input device of the first terminal to input the first operation after the first cursor enters the projection interface, and the first operation parameter includes a relative displacement of the first cursor compared to the initial coordinate position; The second terminal determines the relative displacement of the second cursor compared to the starting position according to the relative displacement of the first cursor corresponding to the initial coordinate position; The second terminal simulates the first input event according to the determined relative displacement of the second cursor compared to the starting position and other parameters in the first operation parameter; The method further includes: the second terminal displays the second cursor movement animation on a display screen of the second terminal according to the first input event.

14. The method according to any one of claims 10 to 13, characterized in that: The method further comprises: When the second cursor moves to the first content of the first interface, the second terminal receives a second operation input by the user using an input device of the first terminal; In response to the second operation, the second terminal displays a second interface and projects the second interface to the first terminal, so that the content of the updated projection interface of the first terminal is a mirror image of the content of the second interface.

15. The method according to claim 14, characterized in that The second terminal receives a second operation input by a user using an input device of the first terminal, including: The second terminal receives a second operation parameter from the first terminal, where the second operation parameter is an operation parameter included in a second input event intercepted by the first terminal after the user inputs the second operation using an input device of the first terminal when the first cursor moves to the content on the projection interface corresponding to the first content; The second terminal simulates the second input event according to the second operation parameter, and the second input event is used to display the second interface.

16. The method according to claim 15, characterized in that When the first cursor of the first terminal enters the screen projection interface, the method further includes: The second terminal receives first indication information from the first terminal, where the first indication information is used to indicate the start of sharing.

17. The method according to claim 16, characterized in that The method further comprises: The second terminal receives second indication information from the first terminal, where the second indication information is used to indicate that sharing is stopped, and the second indication information is sent by the first terminal after determining that the first cursor moves out of the projection interface.

18. An interface display device, characterized in that: include: processor; a memory for storing instructions executable by the processor; Wherein, the processor is configured to enable the interface display device to implement the method as described in any one of claims 1-9, or to implement the method as described in any one of claims 10-17 when executing the instruction.

19. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by an electronic device, the electronic device implements the method according to any one of claims 1 to 9, or implements the method according to any one of claims 10 to 17.

20. An interface display system, characterized in that: comprising a first terminal and a second terminal, wherein the first terminal is connected to the second terminal; The second terminal is used to display the first interface, and project the first interface to the first terminal so that the first terminal displays the projection interface; The first terminal is used to display the screen projection interface on the display screen of the first terminal, and the content of the screen projection interface is a mirror image of the content of the first interface displayed on the display screen of the second terminal; receiving a first operation input by a user using an input device of the first terminal, where the first operation is used to move a first cursor on a display screen of the first terminal; Among them, when the first cursor moves to the first content of the projection interface, the first terminal receives a cursor type of a first style from the second terminal, and displays the first cursor according to the cursor type of the first style, so that the first cursor is displayed in the first style, and / or the display mode of the first content is changed from the first mode to the second mode; when the first cursor moves to the second content of the projection interface, the first terminal receives a cursor type of a second style from the second terminal, and displays the first cursor according to the cursor type of the second style, so that the first cursor is displayed in the second style, and / or the display mode of the second content is changed from the third mode to the fourth mode.

21. The system according to claim 20, characterized in that The screen projection interface is displayed on a partial area of ​​the first terminal display screen; The first terminal is further configured to display an animation of the first cursor moving on a display screen of the first terminal in response to the first operation; The second terminal is further configured to display a second cursor on the first interface when the first cursor enters the screen projection interface; receiving the first operation input by a user using an input device of the first terminal, where the first operation is used to move the second cursor on a display screen of the second terminal; When the second cursor moves to the content corresponding to the first content on the first interface, displaying the second cursor in the first style, and sending the cursor type of the first style to the first terminal; The first terminal is further configured to display the first cursor according to the cursor type of the first style; The second terminal is further configured to display the second cursor in the second style and send the cursor type of the second style to the first terminal when the second cursor moves to the content corresponding to the second content on the first interface; The first terminal is further configured to display the first cursor according to the cursor type of the second style.

22. The system according to claim 21, characterized in that The second terminal is further used to change the display mode of the content corresponding to the first interface and the first content from the first mode to the second mode when the second cursor moves to the content corresponding to the first interface and the first content; the first terminal is also used to update the screen projection interface; The second terminal is also used to change the display mode of the content corresponding to the first interface and the second content from the third mode to the fourth mode when the second cursor moves to the content corresponding to the first interface and the second content; the first terminal is also used to update the projection interface.

Images