Interaction Method and Device Applicable to Mobile Terminals

Abstract

The present application provides an interaction method and device applicable to a mobile terminal. The method includes: when a mouse cursor pre-drawn on a user interface is within the corresponding range of a draggable control, during the process of a user pressing a single finger on the user interface for a third duration and then moving, moving the mouse cursor and displaying the result of performing a second simulated operation. Wherein, a progress bar starts to be displayed when the time for which the user presses a single finger on the user interface reaches a first duration, and a complete progress bar is completed when the time for which the user presses a single finger on the user interface reaches a second duration. The third duration is greater than or equal to the second duration, the second duration is greater than the first duration, and the second simulated operation is an operation of simulating moving a draggable control with a mouse, which enables a user to accurately select and drag small interface elements.

Description

Technical Field

[0001] The present invention relates to the field of interaction, and particularly to an interaction method and device applicable to a mobile terminal. Background Art

[0002] On mobile devices such as mobile phones and tablet computers that have a touch screen but do not support mouse operations, users can operate on interface elements presented on the touch screen through basic touch operations (including tapping, long pressing, and swiping) and multi-touch operations (including pinching and rotating). In this way, users directly touch the screen to perform gesture operations, so they can obtain an intuitive interaction experience. However, due to the lack of a cursor on the user interface, the sense of space is insufficient, and it is difficult for users to accurately select / drag small interface elements.

[0003] Therefore, how to provide an interaction solution applicable to a mobile terminal to facilitate users to accurately select / drag small interface elements has become a technical problem to be solved urgently. Summary of the Invention

[0004] In view of the technical problems existing in the prior art, embodiments of the present application provide an interaction method and device applicable to a mobile terminal.

[0005] In a first aspect, embodiments of the present application provide an interaction method applicable to a mobile terminal, including:

[0006] When a user presses and then lifts a single finger on the user interface, if it is determined that the time when the user holds the user interface with a single finger is greater than a first duration and less than a second duration, the result of performing a first simulated operation is displayed. Among them, a progress bar is displayed from the time corresponding to the first duration until the user lifts the single finger from the user interface. The first simulated operation is to simulate a right-click operation at the position corresponding to the mouse cursor pre-drawn on the user interface; and / or

[0007] When the mouse cursor pre-drawn on the user interface is within the corresponding range of the draggable control, during the process of the user moving a single finger on the user interface after pressing for a third duration, the mouse cursor is moved, and the result of performing a second simulated operation is displayed. Among them, the progress bar starts to be displayed when the time when the user presses a single finger on the user interface reaches the first duration, and a complete progress bar is completed when the time when the user presses a single finger on the user interface reaches the second duration. The third duration is greater than or equal to the second duration, the second duration is greater than the first duration, and the second simulated operation is to simulate the operation of moving the draggable control with the mouse.

[0008] In a second aspect, embodiments of the present application further provide an interaction device applicable to a mobile terminal, including:

[0009] The first display unit is configured to, when a user presses and then lifts a single finger on the user interface, if it is determined that the time for which the user holds the single finger on the user interface is greater than a first duration and less than a second duration, display the result of performing a first simulation operation. During the process from the time corresponding to the first duration until the user lifts the single finger from the user interface, a progress bar is displayed. The first simulation operation is to simulate a right-click operation at the position corresponding to the mouse cursor pre-drawn on the user interface; and / or

[0010] The second display unit is configured to, when the mouse cursor pre-drawn on the user interface is within the corresponding range of the draggable control, during the process of the user moving the single finger after pressing for a third duration on the user interface, move the mouse cursor and display the result of performing a second simulation operation. During this process, a progress bar starts to be displayed when the time for which the user presses the single finger on the user interface reaches the first duration, and a complete progress bar is completed when the time for which the user presses the single finger on the user interface reaches the second duration. The third duration is greater than or equal to the second duration, the second duration is greater than the first duration, and the second simulation operation is to simulate the operation of moving the draggable control with the mouse.

[0011] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of the interaction method applicable to a mobile terminal as described in the first aspect.

[0012] In a fourth aspect, an embodiment of the present application further provides an electronic device, including: a processor, a storage medium, and a bus. The storage medium stores machine-readable instructions executable by the processor. When the electronic device runs, communication between the processor and the storage medium is carried out through the bus. The processor executes the machine-readable instructions to execute the steps of the interaction method applicable to a mobile terminal as described in the first aspect.

[0013] In summary, the interaction method and device for a mobile terminal provided by the embodiments of the present application include at least one of Solution 1 and Solution 2. For Solution 1: when the user presses and then lifts a single finger on the user interface, if it is determined that the time when the user holds the user interface with a single finger is greater than the first duration and less than the second duration, the result of performing the first simulated operation is displayed. In this solution, a mouse cursor is pre-drawn on the user interface. If the user holds a single finger on the user interface for a certain period of time, the result of simulating the operation of clicking the right mouse button can be displayed. Thus, by simply moving the mouse cursor to the interface element, the function of clicking the right mouse button can be achieved by long-pressing on the user interface. Moreover, a progress bar is displayed from the time corresponding to the first duration during the process when the user holds a single finger on the user interface until the user lifts the single finger from the user interface, so as to guide the timing of lifting the single finger. For Solution 2: when the mouse cursor pre-drawn on the user interface is within the range corresponding to the draggable control, during the process when the user presses a single finger on the user interface for the third duration and then moves, the mouse cursor is moved and the result of performing the second simulated operation is displayed. In this solution, a mouse cursor is pre-drawn on the user interface. If the user holds a single finger on the user interface for a long time and then moves, the result of simulating the operation of moving the mouse to drag the draggable control can be displayed. Thus, by simply moving the mouse cursor to the interface element, the function of dragging the draggable control with the mouse can be achieved by long-pressing and then moving on the user interface. Moreover, the progress bar starts to be displayed when the time when the user presses a single finger on the user interface reaches the first duration, and a complete progress bar is completed when the time when the user presses a single finger on the user interface reaches the second duration, which can guide the timing of holding and then moving a single finger on the user interface. Therefore, through Solution 1 / Solution 2, the user can accurately select / drag small interface elements, and both Solution 1 and Solution 2 are implemented by long-pressing the user interface with a single finger. The involved user operations are relatively simple and have certain similarities, making it easy for the user to operate. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic flowchart of an interaction method for a mobile terminal provided by an embodiment of the present application;

[0015] Figure 2 It is a schematic structural diagram of an interaction device for a mobile terminal provided by an embodiment of the present application;

[0016] Figure 3 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of this application. It should be understood that the accompanying drawings in this application are only for the purposes of illustration and description, and are not used to limit the protection scope of this application. In addition, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate the operations implemented according to some embodiments of this application. It should be understood that the operations in the flowchart may not be implemented in sequence, and steps without a logical context relationship may be reversed or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart or remove one or more operations from the flowchart under the guidance of the content of this application.

[0018] In addition, the described embodiments are only some embodiments of this application, rather than all embodiments. The components of the embodiments of this application usually described and illustrated in the accompanying drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of this application claimed, but only represents the selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative efforts fall within the protection scope of this application.

[0019] It should be noted that the term "including" will be used in the embodiments of this application to indicate the existence of the features stated thereafter, but does not exclude adding other features.

[0020] Refer to Figure 1 As shown, the embodiments of this application provide a schematic flowchart of an interaction method applicable to a mobile terminal, including:

[0021] S10. When the user presses and then lifts a single finger on the user interface, if it is determined that the time for the user to hold the user interface with a single finger is greater than a first duration and less than a second duration, then display the result of performing a first simulated operation. Among them, a progress bar is displayed during the process from the time corresponding to the first duration to the time when the user lifts the single finger from the user interface. The first simulated operation is to simulate a right-click operation at the position corresponding to the mouse cursor pre-drawn on the user interface; and / or

[0022] S11. When the pre-drawn mouse cursor on the user interface is within the corresponding range of the draggable control, during the process of the user pressing with a single finger on the user interface for a third duration and then moving, move the mouse cursor and display the result of performing a second simulated operation. Among them, starting from the time when the user presses with a single finger on the user interface and reaches the first duration, a progress bar is displayed. By the time the user presses with a single finger on the user interface and reaches the second duration, a complete progress bar is completed. The third duration is greater than or equal to the second duration, the second duration is greater than the first duration, and the second simulated operation is an operation of simulating moving the draggable control with the mouse.

[0023] In this embodiment, it should be noted that in step S10, a pre-drawn mouse cursor is displayed on the user interface to prompt the user that the touch operation on the user interface simulates a mouse operation. Displaying the result of performing the first simulated operation includes two cases: First, for the user interface of the software on the mobile terminal, directly simulate performing a right-click operation at the mouse cursor, then on the mobile terminal, the result of simulating performing a right-click operation at the mouse cursor will be displayed, such as displaying a relevant context menu, etc.; Second, for the user interface corresponding to the cloud computer screen / cloud application screen displayed on the mobile terminal, it simulates a right-click operation at the corresponding position on the cloud computer screen / cloud application screen of the cloud server (the corresponding position is the position on the cloud computer screen / cloud application screen corresponding to the position of the mouse cursor on the user interface). Then, what is displayed on the mobile terminal is a combination of the mouse cursor and the cloud computer screen / cloud application screen rendered after the cloud server performs the simulated operation. It should be noted that when the cloud server performs the simulated operation, the mouse cursor may not be displayed, so what the user sees is a combination of the mouse cursor on the mobile terminal and the rendered screen, or the mouse cursor may be displayed when performing the simulated operation, so what the user sees is a combination of the mouse cursor on the mobile terminal and the rendered screen, and the mouse cursor on the mobile terminal covers the mouse cursor on the cloud computer screen / cloud application screen. The values of the first duration and the second duration can be set as needed. For example, the first duration and the second duration are set to 0.5 seconds and 2 seconds respectively. After the user presses with a single finger on the user interface, a progress bar will be displayed from the time corresponding to the first duration until the user lifts the single finger from the user interface. The duration required to complete a complete progress bar is the difference between the second duration and the first duration. The result of performing the first simulated operation will only be displayed if the user lifts the single finger before a complete progress bar is completed. That is to say, the way of displaying the progress bar can guide the timing of the user lifting the single finger. The form of the progress bar can be set as needed. For example, it can be set as a circular progress bar, and the circular progress bar can be displayed around the mouse cursor when it is displayed.

[0024] In step S11, a pre-drawn mouse cursor is displayed on the user interface to prompt the user that the touch operation on the user interface simulates a mouse operation. The result of performing the second simulation operation is displayed, including two cases: First, for the draggable control on the user interface of the software on the mobile terminal, the draggable control is moved according to the single-finger movement position, and at the same time the mouse cursor is moved, so what the user sees is the mouse cursor dragging the draggable control to move; Second, for the user interface corresponding to the cloud computer screen / cloud application screen displayed on the mobile terminal, it simulates the mouse moving the draggable control at the corresponding position on the cloud computer screen / cloud application screen of the cloud server (the corresponding position is the position on the cloud computer screen / cloud application screen corresponding to the position of the mouse cursor on the user interface), and at the same time the mouse cursor on the mobile terminal is moved, then what the mobile terminal displays is the combination of the movement of the mouse cursor and the cloud computer screen / cloud application screen rendered after the cloud server performs the simulation operation. It should be noted that when the cloud server performs the simulation operation, the mouse cursor may not be displayed, so what the user sees is the combination of the mouse cursor on the mobile terminal and the rendered screen, or when the simulation operation is performed, the mouse cursor may be displayed, so what the user sees is the combination of the mouse cursor on the mobile terminal and the rendered screen, and the mouse cursor on the mobile terminal covers the mouse cursor on the cloud computer screen / cloud application screen.

[0025] The progress bar starts to be displayed when the time when the user presses with a single finger on the user interface reaches the first duration, and a complete progress bar is completed when the time when the user presses with a single finger on the user interface reaches the second duration. After that, the corresponding draggable control is in a draggable state, and the corresponding draggable control can be moved by the movement of a single finger. That is to say, the way of displaying the progress bar can guide the timing of the single-finger movement.

[0026] The interaction method applicable to a mobile terminal provided by an embodiment of the present application includes at least one of Solution 1 and Solution 2. For Solution 1: When a user presses and then lifts a single finger on the user interface, if it is determined that the time when the user holds the user interface with a single finger is greater than a first duration and less than a second duration, the result of performing a first simulated operation is displayed. In this solution, a mouse cursor is pre-drawn on the user interface. If the user holds a single finger on the user interface for a certain period of time, the result of simulating the operation of clicking the right mouse button can be displayed. Thus, by simply moving the mouse cursor to an interface element, the function of clicking the right mouse button can be achieved by long-pressing on the user interface. Moreover, a progress bar is displayed during the process from the time corresponding to the first duration when the user holds a single finger on the user interface until the user lifts the single finger from the user interface, thereby guiding the timing of the user to lift the single finger. For Solution 2: When the mouse cursor pre-drawn on the user interface is within the corresponding range of a draggable control, during the process of moving after the user presses a single finger on the user interface for a third duration, the mouse cursor is moved and the result of performing a second simulated operation is displayed. In this solution, a mouse cursor is pre-drawn on the user interface. If the user moves after holding a single finger on the user interface for a long time, the result of simulating the operation of moving the mouse to drag the draggable control can be displayed. Thus, by simply moving the mouse cursor to an interface element, the function of dragging the draggable control with the mouse can be achieved by long-pressing and then moving on the user interface. Moreover, a progress bar is displayed starting from the time when the time when the user presses a single finger on the user interface reaches the first duration until a complete progress bar is completed when the time when the user presses a single finger on the user interface reaches the second duration, which can guide the timing of the user to move after holding a single finger on the user interface. Thus, through Solution 1 / Solution 2, the user can accurately select / drag small interface elements, and both Solution 1 and Solution 2 are implemented by long-pressing the user interface with a single finger. The involved user operations are relatively simple and have certain similarities, making it easy for the user to operate.

[0027] Based on the foregoing method embodiment, the displaying the result of performing the first simulated operation may include:

[0028] Simulating the operation of clicking the right mouse button at the mouse cursor on the user interface; or

[0029] Send the position of the mouse cursor on the user interface, the right mouse button press event data, and the right mouse button release event data to the cloud server, so that the cloud server generates right mouse button click event data based on the position of the mouse cursor on the user interface, the right mouse button press event data, and the right mouse button release event data, and sends the right mouse button click event data to the system of the cloud server to simulate the operation of clicking the right mouse button at the position corresponding to the mouse cursor on the rendered screen of the cloud server, or generate right mouse button click event data based on the position of the mouse cursor on the user interface, and send the right mouse button click event data to the system of the cloud server through the cloud server to simulate the operation of clicking the right mouse button at the position corresponding to the mouse cursor on the rendered screen of the cloud server.

[0030] In this embodiment, it should be noted that the mobile terminal can send the position of the mouse cursor on the user interface, the right mouse button press event data, and the right mouse button release event data to the cloud server. Among them, the right mouse button press event data may include a right mouse button identifier and a press identifier, and the right mouse button release event data may include a right mouse button identifier and a release identifier. After receiving the position of the mouse cursor on the user interface, the right mouse button press event data, and the right mouse button release event data, the cloud server can generate right mouse button click event data and send the right mouse button click event data to the system of the cloud server. After receiving the right mouse button click event data, the system of the cloud server can simulate the operation of clicking the right mouse button at the position corresponding to the mouse cursor on the rendered screen of the cloud server. After the rendered screen of the cloud server is transmitted to the mobile terminal, what the mobile terminal displays is the combination of the mouse cursor and the result screen obtained by performing the simulation operation on the cloud server. In this solution, the process of generating the right mouse button click event data can be executed by the cloud server. In addition to this, it can also be executed by the mobile terminal.

[0031] Based on the foregoing method embodiment, the displaying the result of performing the second simulation operation may include:

[0032] Simulate the operation of moving and dragging a control by operating the mouse on the user interface.

[0033] Based on the foregoing method embodiment, the method may further include:

[0034] When the user presses with a single finger on the user interface for a second duration, the position of the mouse cursor on the user interface and the mouse left button press event data are sent to the cloud server, so that the cloud server generates target mouse left button press event data based on the position of the mouse cursor on the user interface and the mouse left button press event data, and sends the target mouse left button press event data to the system of the cloud server to simulate the operation of pressing the mouse left button at the position corresponding to the mouse cursor on the rendering screen of the cloud server, or generates target mouse left button press event data based on the position of the mouse cursor on the user interface, and sends the target mouse left button press event data to the system of the cloud server through the cloud server to simulate the operation of pressing the mouse left button at the position corresponding to the mouse cursor on the rendering screen of the cloud server;

[0035] Among them, the display of the result of executing the second simulation operation may include:

[0036] Sending the mouse movement event data to the cloud server, so that the cloud server generates first target mouse movement event data based on the mouse movement event data, and sends the first target mouse movement event data to the system of the cloud server to simulate the operation of operating the mouse to move and drag a control on the rendering screen of the cloud server, or generating first target mouse movement event data based on the mouse movement event data, and sending the first target mouse movement event data to the system of the cloud server through the cloud server to simulate the operation of operating the mouse to move and drag a control on the rendering screen of the cloud server.

[0037] In this embodiment, it should be noted that the mobile terminal can send the position of the mouse cursor on the user interface and the mouse left button press event data to the cloud server. Among them, the mouse left button press event data may include a mouse left button identifier and a press identifier. After receiving the position of the mouse cursor on the user interface and the mouse left button press event data, the cloud server can generate target mouse left button press event data and send the target mouse left button press event data to the system of the cloud server. After receiving the target mouse left button press event data, the system of the cloud server can simulate the operation of clicking the mouse left button at the position corresponding to the mouse cursor on the rendering screen of the cloud server. After the rendered screen of the cloud server is transmitted to the mobile terminal, what the mobile terminal displays is the combination of the mouse cursor and the result screen obtained by performing the simulation operation on the cloud server. In this solution, the process of generating the target mouse left button press event data can be executed by the cloud server. In addition to this, it can also be executed by the mobile terminal.

[0038] The mouse movement event data may include data related to the movement position of the user's single finger, such as the movement position of the user's single finger, the movement displacement of the user's single finger, or the movement position / movement displacement of the draggable control calculated based on the movement position / movement displacement of the user's single finger. According to the mouse movement event data, first target mouse movement event data can be generated for triggering the operation of simulating the movement of the mouse to operate the draggable control on the rendering screen of the cloud server.

[0039] Based on the foregoing method embodiments, the method may further include:

[0040] When the user's single finger moves on the user interface, move the mouse cursor and display the result of performing a third simulation operation, where the third simulation operation is simulating the operation of moving the mouse; and / or

[0041] When the user's single finger presses and then lifts on the user interface, if it is determined that the time for which the user's single finger holds the user interface is less than or equal to the first duration, display the result of performing a fourth simulation operation, where the fourth simulation operation is simulating the operation of clicking the left mouse button at the position corresponding to the mouse cursor pre-drawn on the user interface; and / or

[0042] When the user's single finger presses and then lifts twice continuously on the user interface, if it is determined that the time for which the previous single finger holds the user interface is less than or equal to the first duration, the time interval from the previous single finger lift to the next single finger press is less than the fourth duration, and the time for which the next single finger holds the user interface is less than or equal to the first duration, display the result of performing a fifth simulation operation, where the fifth simulation operation is simulating the operation of double-clicking the left mouse button at the position corresponding to the mouse cursor pre-drawn on the user interface; and / or

[0043] When the user's two fingers press and then move on the user interface, display the result of performing a sixth simulation operation during the movement, where the sixth simulation operation is simulating the operation of sliding the mouse wheel.

[0044] In this embodiment, it should be noted that in addition to the functions of simulating clicking the right mouse button and simulating the function of dragging the draggable control with the mouse shown in the foregoing embodiments, the functions of simulating moving the mouse, clicking the left mouse button, double-clicking the left mouse button, and sliding the mouse wheel can also be simulated.

[0045] Specifically, when the user moves a single finger on the user interface, there are two cases: First, for the user interface of the software on the mobile terminal, directly calculate the movement position / movement displacement of the pre-drawn mouse cursor on the user interface according to the movement position / movement displacement of the user's single finger (for example, determine that the movement displacement of the mouse cursor is equal to the movement displacement of the user's single finger), move the mouse cursor according to the movement position / movement displacement of the mouse cursor, and simulate the execution of the mouse movement operation. Then, what is displayed on the user interface of the mobile terminal is the mouse cursor that moves with the movement of the single finger and the simulation execution result of the mouse movement operation. Second, for the user interface corresponding to the cloud computer screen / cloud application screen displayed on the mobile terminal, simulate the operation of moving the mouse cursor on the cloud computer screen / cloud application screen of the cloud server, and the mouse cursor on the user interface needs to be moved according to the movement position / movement displacement of the single finger. Then, what is displayed on the mobile terminal is the combination of the movement of the mouse cursor and the cloud computer screen / cloud application screen rendered after the cloud server executes the simulation operation. The mouse cursor may not be displayed on the cloud computer screen / cloud application screen rendered after the cloud server executes the simulation operation, or the mouse cursor on the user interface may cover the mouse cursor on the cloud computer screen / cloud application screen rendered after the cloud server executes the simulation operation. For example, assume that the mouse cursor on the user interface is at position A, and the user's single finger moves from position B to position C. Then, move the mouse cursor to position D, and simulate moving the mouse cursor from position M to position N on the cloud computer screen / cloud application screen in the cloud server, where position M is the position on the cloud computer screen / cloud application screen corresponding to position A, and positions D and N are determined according to positions B and C.

[0046] Specifically, when the user clicks with a single finger on the user interface, if the time interval between the pressing and lifting actions involved in the clicking process is short, then display the result of executing the fourth simulation operation, including two cases: First, for the user interface of the software on the mobile terminal, directly simulate the execution of the left mouse button click operation at the pre-drawn mouse cursor on the user interface, such as selecting a control, etc. Second, for the user interface corresponding to the cloud computer screen / cloud application screen displayed on the mobile terminal, simulate the left mouse button click operation at the corresponding position on the cloud computer screen / cloud application screen of the cloud server (the corresponding position is the position on the cloud computer screen / cloud application screen corresponding to the mouse cursor position on the user interface). Then, what is displayed on the mobile terminal is the combination of the mouse cursor and the cloud computer screen / cloud application screen rendered after the cloud server executes the simulation operation. The mouse cursor may not be displayed on the cloud computer screen / cloud application screen rendered after the cloud server executes the simulation operation, or the mouse cursor on the user interface may cover the mouse cursor on the cloud computer screen / cloud application screen rendered after the cloud server executes the simulation operation.

[0047] Specifically, when the user double-clicks on the user interface with a single finger, if the time intervals between the press and lift actions involved in each click during the double-click process are short, and the time interval between the lift action of the previous click and the press action of the next click is short, then the result of performing the fifth simulation operation is displayed, including two cases: First, for the user interface of the software on the mobile terminal, a double-click left mouse button operation is simulated directly at the mouse cursor pre-drawn on the user interface, such as opening a folder, etc.; Second, for the user interface corresponding to the cloud computer screen / cloud application screen displayed on the mobile terminal, what is simulated is a double-click left mouse button operation at the corresponding position on the cloud computer screen / cloud application screen of the cloud server (the corresponding position is the position on the cloud computer screen / cloud application screen corresponding to the mouse cursor position on the user interface), then what is displayed on the mobile terminal is a combination of the mouse cursor and the cloud computer screen / cloud application screen rendered after the cloud server performs the simulation operation. The mouse cursor may not be displayed on the cloud computer screen / cloud application screen rendered after the cloud server performs the simulation operation, or the mouse cursor on the user interface may cover the mouse cursor on the cloud computer screen / cloud application screen rendered after the cloud server performs the simulation operation.

[0048] Specifically, when the user moves two fingers on the user interface, then during the movement, the result of performing the sixth simulation operation is displayed, including two cases: First, for the user interface of the software on the mobile terminal, the sliding displacement is calculated based on the moving position / moving displacement of the two fingers, and the user interface is controlled to slide based on the sliding displacement; Second, for the user interface corresponding to the cloud computer screen / cloud application screen displayed on the mobile terminal, what is simulated is sliding the cloud computer screen / cloud application screen of the cloud server, then what is displayed on the mobile terminal is a combination of the mouse cursor and the cloud computer screen / cloud application screen rendered after the cloud server performs the simulation operation. The mouse cursor may not be displayed on the cloud computer screen / cloud application screen rendered after the cloud server performs the simulation operation, or the mouse cursor on the user interface may cover the mouse cursor on the cloud computer screen / cloud application screen rendered after the cloud server performs the simulation operation. Among them, the sliding displacement is calculated based on the moving position / moving displacement of the two fingers. For example, the sliding displacement can be the average value of the longitudinal displacements of the two fingers.

[0049] Based on the foregoing method embodiments, the display of the result of performing the third simulation operation may include:

[0050] Simulating a mouse movement operation at the mouse cursor on the user interface; or

[0051] Send the position of the mouse cursor on the user interface and the mouse movement event data to the cloud server, so that the cloud server generates second target mouse movement event data based on the position of the mouse cursor on the user interface and the mouse movement event data, and sends the second target mouse movement event data to the system of the cloud server to simulate the execution of a mouse movement operation on the rendered screen of the cloud server, or generate second target mouse movement event data based on the position of the mouse cursor on the user interface, and send the second target mouse movement event data to the system of the cloud server through the cloud server to simulate the execution of a mouse movement operation on the rendered screen of the cloud server.

[0052] In this embodiment, it should be noted that the mouse movement event data may include data related to the movement position of the user's single finger, such as the movement position of the user's single finger, the movement displacement of the user's single finger, or the movement position / movement displacement of the mouse cursor calculated according to the movement position / movement displacement of the user's single finger, etc. The mobile terminal can send the position of the mouse cursor on the user interface and the mouse movement event data to the cloud server to generate second target mouse movement event data through the cloud server, or directly generate second target mouse movement event data and send the second target mouse movement event data to the system of the cloud server through the cloud server. After receiving the second target mouse movement event data, the system of the cloud server can simulate the execution of a mouse movement operation on the rendered screen of the cloud server.

[0053] Based on the foregoing method embodiment, the display of the result of executing the fourth simulation operation may include:

[0054] Simulate the execution of a left mouse button click operation at the mouse cursor on the user interface; or

[0055] Send the position of the mouse cursor on the user interface, the left mouse button press event data, and the left mouse button release event data to the cloud server, so that the cloud server generates left mouse button click event data based on the position of the mouse cursor on the user interface, the left mouse button press event data, and the left mouse button release event data, and sends the left mouse button click event data to the system of the cloud server to simulate the execution of a left mouse button click operation corresponding to the position of the mouse cursor on the rendered screen of the cloud server, or generate left mouse button click event data based on the position of the mouse cursor on the user interface, and send the left mouse button click event data to the system of the cloud server through the cloud server to simulate the execution of a left mouse button click operation corresponding to the position of the mouse cursor on the rendered screen of the cloud server.

[0056] In this embodiment, it should be noted that the left mouse button press event data may include a left mouse button identifier and a press identifier, and the left mouse button release event data may include a left mouse button identifier and a release identifier. The mobile terminal may send the position of the mouse cursor on the user interface, the left mouse button press event data, and the left mouse button release event data to the cloud server to generate left mouse button click event data through the cloud server, or the mobile terminal may directly generate left mouse button click event data and send the left mouse button click event data to the system of the cloud server through the cloud server to simulate a left mouse button click operation at the position corresponding to the mouse cursor on the rendering screen of the cloud server.

[0057] Based on the foregoing method embodiment, the displaying the result of the fifth simulation operation may include:

[0058] Simulating a double left mouse button click operation at the mouse cursor on the user interface; or

[0059] When the user presses and then releases a single finger on the user interface each time, the mobile terminal sends the position of the mouse cursor on the user interface, the left mouse button press event data, and the left mouse button release event data to the cloud server, so that the cloud server generates left mouse button click event data based on the position of the mouse cursor on the user interface, the left mouse button press event data, and the left mouse button release event data, and sends the left mouse button click event data to the system of the cloud server to simulate a double left mouse button click operation at the position corresponding to the mouse cursor on the rendering screen of the cloud server, or generates left mouse button click event data based on the position of the mouse cursor on the user interface and sends the left mouse button click event data to the system of the cloud server through the cloud server to simulate a double left mouse button click operation at the position corresponding to the mouse cursor on the rendering screen of the cloud server.

[0060] Based on the foregoing method embodiment, the displaying the result of the sixth simulation operation during the movement may include:

[0061] Generating mouse wheel scroll event data during the movement and sending the mouse wheel scroll event data to the cloud server, so that the cloud server generates target mouse wheel scroll event data based on the mouse wheel scroll event data and sends the target mouse wheel scroll event data to the system of the cloud server to simulate a mouse wheel scroll operation on the rendering screen of the cloud server, or generating target mouse wheel scroll event data during the movement and sending the target mouse wheel scroll event data to the system of the cloud server through the cloud server to simulate a mouse wheel scroll operation on the rendering screen of the cloud server.

[0062] In this embodiment, it should be noted that the mouse wheel sliding event data may include data related to the moving positions of the user's two fingers, such as the moving positions of the user's two fingers, the moving displacements of the user's two fingers, or the sliding displacement of the mouse wheel calculated based on the moving positions / moving displacements of the user's two fingers. The mobile terminal may send the mouse wheel sliding event data to the cloud server to generate target mouse wheel sliding event data through the cloud server, or directly generate the target mouse wheel sliding event data and send the target mouse wheel sliding event data to the system of the cloud server through the cloud server. After receiving the target mouse wheel sliding event data, the system of the cloud server may simulate the operation of sliding the mouse wheel on the rendering screen of the cloud server.

[0063] Referring to Figure 2 As shown, it is a schematic structural diagram of an interaction device applicable to a mobile terminal provided by an embodiment of the present application. The device includes:

[0064] A first display unit 20, configured to, when the user's single finger presses and then lifts on the user interface, if it is determined that the time for which the user's single finger holds the user interface is greater than a first duration and less than a second duration, display the result of performing a first simulation operation. During this process, a progress bar is displayed from the time corresponding to the first duration until the user's single finger lifts from the user interface. The first simulation operation is to simulate a right-click operation at the position corresponding to the mouse cursor pre-drawn on the user interface; and / or

[0065] A second display unit 21, configured to, when the mouse cursor pre-drawn on the user interface is within the corresponding range of the draggable control, during the process of the user's single finger moving after pressing for a third duration on the user interface, move the mouse cursor and display the result of performing a second simulation operation. During this process, a progress bar is displayed starting from the time when the time for which the user's single finger presses on the user interface reaches the first duration until the time for which the user's single finger presses on the user interface reaches the second duration and a complete progress bar is completed. The third duration is greater than or equal to the second duration, the second duration is greater than the first duration, and the second simulation operation is to simulate the operation of moving the draggable control by operating the mouse.

[0066] The interaction device applicable to a mobile terminal provided by an embodiment of the present application includes at least one of Solution 1 and Solution 2. For Solution 1: When a user presses and then lifts a single finger on the user interface, if it is determined that the time when the user holds the user interface with a single finger is greater than a first duration and less than a second duration, the result of performing a first simulated operation is displayed. In this solution, a mouse cursor is pre-drawn on the user interface. If the user holds a single finger on the user interface for a certain period of time, the result of simulating the operation of clicking the right mouse button can be displayed. Thus, by simply moving the mouse cursor to an interface element, the function of clicking the right mouse button can be achieved by long-pressing on the user interface. Moreover, a progress bar is displayed during the process from the time corresponding to the first duration when the user holds a single finger on the user interface until the user lifts the single finger from the user interface, thereby guiding the timing of the user to lift the single finger. For Solution 2: When the mouse cursor pre-drawn on the user interface is within the corresponding range of a draggable control, during the process of the user pressing a single finger on the user interface for a third duration and then moving, the mouse cursor is moved and the result of performing a second simulated operation is displayed. In this solution, a mouse cursor is pre-drawn on the user interface. If the user holds a single finger on the user interface for a long time and then moves, the result of simulating the operation of moving the mouse to drag the draggable control can be displayed. Thus, by simply moving the mouse cursor to an interface element, the function of dragging the draggable control with the mouse can be achieved by long-pressing and then moving on the user interface. Moreover, the progress bar starts to be displayed when the time when the user presses a single finger on the user interface reaches the first duration, and a complete progress bar is completed when the time when the user presses a single finger on the user interface reaches the second duration, which can guide the timing of the user to hold and then move a single finger on the user interface. Therefore, through Solution 1 / Solution 2, the user can accurately select / drag small interface elements, and both Solution 1 and Solution 2 are implemented by long-pressing the user interface with a single finger. The involved user operations are relatively simple and have certain similarities, making it easy for the user to operate.

[0067] The implementation process of the interaction device applicable to a mobile terminal provided by an embodiment of the present application is consistent with the interaction method applicable to a mobile terminal provided by an embodiment of the present application, and the achieved effects are also the same as those of the interaction method applicable to a mobile terminal provided by an embodiment of the present application, which will not be elaborated here.

[0068] As Figure 3 shown, an electronic device provided by an embodiment of the present application includes: a processor 30, a memory 31, and a bus 32. The memory 31 stores machine-readable instructions executable by the processor 30. When the electronic device runs, the processor 30 communicates with the memory 31 through the bus 32. The processor 30 executes the machine-readable instructions to perform the steps of the interaction method applicable to a mobile terminal as described above.

[0069] Specifically, the above-mentioned memory 31 and processor 30 can be general-purpose memory and processor, which are not specifically limited here. When the processor 30 runs the computer program stored in the memory 31, it can execute the above-mentioned interaction method applicable to the mobile terminal.

[0070] Corresponding to the above-mentioned interaction method applicable to the mobile terminal, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, it executes the steps of the above-mentioned interaction method applicable to the mobile terminal.

[0071] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of changes or substitutions, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An interactive method applicable to a mobile terminal, characterized in that: include: When the user presses and then lifts a finger on the user interface, if it is determined that the time the user holds the user interface with the single finger is greater than a first duration and less than a second duration, a result of executing a first simulation operation is displayed, wherein a progress bar is displayed during the process from the time corresponding to the first duration to the time the user lifts the single finger from the user interface, and the first simulation operation is to simulate a right-click operation of a mouse at a position corresponding to a mouse cursor pre-drawn on the user interface; and / or When the mouse cursor pre-drawn on the user interface is within the corresponding range of the draggable control, the mouse cursor is moved during the process of moving after the user presses the user interface with one finger for a third time period, and the result of executing the second simulation operation is displayed, wherein a progress bar is displayed starting from when the user presses the user interface with one finger for a first time period, and a complete progress bar is displayed when the user presses the user interface with one finger for a second time period, the third time period is greater than or equal to the second time period, and the second time period is greater than the first time period, and the second simulation operation is a simulation operation of moving the mouse to move the draggable control.

2. The method according to claim 1, characterized in that The displaying of the result of executing the first simulation operation comprises: Simulate a right-click at the mouse cursor position on the user interface; or The position of the mouse cursor on the user interface, the right button of the mouse is pressed event data, and the right button of the mouse is lifted event data are sent to the cloud server, so that the cloud server generates the right button of the mouse click event data based on the position of the mouse cursor on the user interface, the right button of the mouse is pressed event data, and the right button of the mouse is lifted event data, and sends the right button of the mouse click event data to the cloud server's system to simulate the right button of the mouse click operation corresponding to the position of the mouse cursor on the rendering screen of the cloud server, or generates the right button of the mouse click event data based on the position of the mouse cursor on the user interface, and sends the right button of the mouse click event data to the cloud server's system through the cloud server to simulate the right button of the mouse click operation corresponding to the position of the mouse cursor on the rendering screen of the cloud server.

3. The method according to claim 1, characterized in that The displaying of the result of executing the second simulation operation comprises: Simulates the action of moving a mouse over a draggable control in the user interface.

4. The method according to claim 1, characterized in that Also includes: When the user presses a single finger on the user interface for a second duration, the position of the mouse cursor on the user interface and the left mouse button pressed event data are sent to the cloud server, so that the cloud server generates target left mouse button pressed event data based on the position of the mouse cursor on the user interface and the left mouse button pressed event data, and sends the target left mouse button pressed event data to the system of the cloud server, so as to simulate the left mouse button pressed operation corresponding to the position of the mouse cursor on the rendering screen of the cloud server, or generates target left mouse button pressed event data based on the position of the mouse cursor on the user interface, and sends the target left mouse button pressed event data to the system of the cloud server through the cloud server, so as to simulate the left mouse button pressed operation corresponding to the position of the mouse cursor on the rendering screen of the cloud server; The displaying of the result of executing the second simulation operation includes: The mouse movement event data is sent to the cloud server so that the cloud server generates first target mouse movement event data based on the mouse movement event data, and sends the first target mouse movement event data to the system of the cloud server to simulate the operation of moving the mouse to drag the control on the rendering screen of the cloud server, or the first target mouse movement event data is generated based on the mouse movement event data, and the first target mouse movement event data is sent to the system of the cloud server through the cloud server to simulate the operation of moving the mouse to drag the control on the rendering screen of the cloud server.

5. The method according to any one of claims 1 to 4, characterized in that: Also includes: When the user moves a single finger on the user interface, the mouse cursor is moved, and the result of executing the third simulation operation is displayed, wherein the third simulation operation is a simulation of moving the mouse; and / or When the user presses and then lifts a finger on the user interface, if it is determined that the time for which the user presses and holds the user interface with the single finger is less than or equal to the first duration, a result of executing a fourth simulation operation is displayed, wherein the fourth simulation operation is a simulation of a left-click operation of the mouse at a position corresponding to a mouse cursor pre-drawn on the user interface; and / or The user presses and lifts a finger on the user interface twice in succession, and if it is determined that the time the user pressed the user interface with a single finger the previous time is less than or equal to the first duration, and the interval between the previous time the user lifted the finger and the next time the user pressed the user finger is less than a fourth duration, and the next time the user pressed the user interface with a single finger is less than or equal to the first duration, then the result of executing the fifth simulation operation is displayed, wherein the fifth simulation operation is to simulate a double-click of the left mouse button at a position corresponding to a mouse cursor pre-drawn on the user interface; and / or When the user presses and moves two fingers on the user interface, the result of executing the sixth simulation operation is displayed during the movement, wherein the sixth simulation operation is a simulation of sliding the mouse wheel.

6. The method according to claim 5, characterized in that The displaying of the result of executing the third simulation operation comprises: Simulate moving the mouse at the mouse cursor position on the user interface; or The position of the mouse cursor on the user interface and the mouse movement event data are sent to the cloud server, so that the cloud server generates second target mouse movement event data based on the position of the mouse cursor on the user interface and the mouse movement event data, and sends the second target mouse movement event data to the cloud server's system to simulate the mouse movement operation on the cloud server's rendering screen, or generates second target mouse movement event data based on the position of the mouse cursor on the user interface, and sends the second target mouse movement event data to the cloud server's system through the cloud server to simulate the mouse movement operation on the cloud server's rendering screen.

7. The method according to claim 5, characterized in that The displaying of the result of executing the fourth simulation operation comprises: Simulate a left mouse button click at the mouse cursor position on the user interface; or The position of the mouse cursor on the user interface, the left mouse button pressed event data, and the left mouse button lifted event data are sent to the cloud server, so that the cloud server generates the left mouse button click event data based on the position of the mouse cursor on the user interface, the left mouse button pressed event data, and the left mouse button lifted event data, and sends the left mouse button click event data to the cloud server's system to simulate the left mouse button click operation corresponding to the position of the mouse cursor on the cloud server's rendering screen, or generates the left mouse button click event data based on the position of the mouse cursor on the user interface, and sends the left mouse button click event data to the cloud server's system through the cloud server to simulate the left mouse button click operation corresponding to the position of the mouse cursor on the cloud server's rendering screen.

8. The method according to claim 5, characterized in that The displaying of the result of executing the fifth simulation operation comprises: Simulate a double-click of the left mouse button at the mouse cursor position on the user interface; or Each time the user presses and lifts a single finger on the user interface, the position of the mouse cursor on the user interface, the left mouse button pressed event data and the left mouse button lifted event data are sent to the cloud server, so that the cloud server generates left mouse button click event data based on the position of the mouse cursor on the user interface, the left mouse button pressed event data and the left mouse button lifted event data, and sends the left mouse button click event data to the cloud server's system to simulate a double-click of the left mouse button at the position of the mouse cursor on the cloud server's rendering screen, or generates left mouse button click event data based on the position of the mouse cursor on the user interface, and sends the left mouse button click event data to the cloud server's system through the cloud server to simulate a double-click of the left mouse button at the position of the mouse cursor on the cloud server's rendering screen.

9. The method according to claim 5, characterized in that The step of displaying the result of executing the sixth simulation operation during the moving process includes: During the movement process, mouse wheel sliding event data is generated, and the mouse wheel sliding event data is sent to the cloud server, so that the cloud server generates target mouse wheel sliding event data based on the mouse wheel sliding event data, and sends the target mouse wheel sliding event data to the cloud server's system to simulate the sliding mouse wheel operation on the cloud server's rendering screen, or target mouse wheel sliding event data is generated during the movement process, and the target mouse wheel sliding event data is sent to the cloud server's system through the cloud server to simulate the sliding mouse wheel operation on the cloud server's rendering screen.

10. An interactive device suitable for a mobile terminal, characterized in that: include: A first display unit is used for displaying the result of executing a first simulation operation when a user presses and then lifts a finger on the user interface, if it is determined that the time for which the user presses and holds the user interface with a single finger is greater than a first duration and less than a second duration, wherein a progress bar is displayed during the process from the time corresponding to the first duration to the time when the user lifts the single finger from the user interface, and the first simulation operation is to simulate a right-click operation of a mouse at a position corresponding to a mouse cursor pre-drawn on the user interface; and / or The second display unit is used to move the mouse cursor and display the result of executing the second simulation operation during the process of moving the mouse cursor after the user presses the user interface with one finger for a third time period, when the mouse cursor pre-drawn on the user interface is within the corresponding range of the draggable control, wherein the progress bar starts to be displayed when the user presses the user interface with one finger for a first time period, and a complete progress bar is completed when the user presses the user interface with one finger for a second time period, the third time period is greater than or equal to the second time period, and the second time period is greater than the first time period, and the second simulation operation is a simulation operation of moving the mouse to move the draggable control.

Images