Interaction method and device suitable for mobile terminal

Abstract

The invention provides an interaction method and device suitable for a mobile terminal, and the method comprises the steps: under the condition that a mouse cursor pre-drawn on a user interface is located in a corresponding range of a draggable control, moving the mouse cursor in a process that the user presses the mouse cursor on the user interface for a third time length, the progress bar starts to be displayed when the time of pressing the single finger of the user on the user interface reaches the first time length, a complete progress bar is completed when the time of pressing the single finger of the user on the user interface reaches the second time length, the third time length is larger than or equal to the second time length, and the second time length is larger than the first time length. The second simulation operation is the operation for simulating the mouse to move the draggable control, so that the user can accurately select and drag the small interface elements.

Description

Technical Field

[0001] The present invention relates to the field of interaction, and in particular to an interaction method and device suitable for a mobile terminal. Background Art

[0002] On mobile devices with touch screens but no mouse support, such as mobile phones and tablets, users can operate the interface elements presented by the touch screen through basic touch operations (including tap, long press, and slide) and multi-touch operations (including pinch-to-zoom and rotate). In this way, users can directly touch the screen to perform gesture operations, thus obtaining an intuitive interactive experience. However, due to the lack of a cursor on the user interface, the sense of space is insufficient, and it is difficult for users to accurately select / drag small interface elements.

[0003] Therefore, how to provide an interactive solution suitable for mobile terminals so that users can accurately select / drag small interface elements has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] In view of the technical problems existing in the prior art, the embodiments of the present application provide an interaction method and device suitable for a mobile terminal.

[0005] In a first aspect, an embodiment of the present application provides an interaction method applicable to a mobile terminal, including: When the user presses and then lifts a finger on the user interface, if it is determined that the time the user holds the user interface with the single finger is greater than a first duration and less than a second duration, a result of executing a first simulation operation is displayed, wherein a progress bar is displayed during the process from the time corresponding to the first duration to the time the user lifts the single finger from the user interface, and the first simulation operation is to simulate a right-click operation of a mouse at a position corresponding to a mouse cursor pre-drawn on the user interface; and / or When the mouse cursor pre-drawn on the user interface is within the corresponding range of the draggable control, the mouse cursor is moved during the process of moving after the user presses the user interface with one finger for a third time period, and the result of executing the second simulation operation is displayed, wherein a progress bar is displayed starting from when the user presses the user interface with one finger for a first time period, and a complete progress bar is displayed when the user presses the user interface with one finger for a second time period, the third time period is greater than or equal to the second time period, and the second time period is greater than the first time period, and the second simulation operation is a simulation operation of moving the mouse to move the draggable control.

[0006] In a second aspect, an embodiment of the present application further provides an interactive device applicable to a mobile terminal, including: A first display unit is used for displaying the result of executing a first simulation operation when a user presses and then lifts a finger on the user interface, if it is determined that the time for which the user presses and holds the user interface with a single finger is greater than a first duration and less than a second duration, wherein a progress bar is displayed during the process from the time corresponding to the first duration to the time when the user lifts the single finger from the user interface, and the first simulation operation is to simulate a right-click operation of a mouse at a position corresponding to a mouse cursor pre-drawn on the user interface; and / or The second display unit is used to move the mouse cursor and display the result of executing the second simulation operation during the process of moving the mouse cursor after the user presses the user interface with one finger for a third time period, when the mouse cursor pre-drawn on the user interface is within the corresponding range of the draggable control, wherein the progress bar starts to be displayed when the user presses the user interface with one finger for a first time period, and a complete progress bar is completed when the user presses the user interface with one finger for a second time period, the third time period is greater than or equal to the second time period, and the second time period is greater than the first time period, and the second simulation operation is a simulation operation of moving the mouse to move the draggable control.

[0007] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the interaction method applicable to a mobile terminal as described in the first aspect are executed.

[0008] In a fourth aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to perform the steps of the interaction method applicable to the mobile terminal as described in the first aspect.

[0009] In summary, the interactive method and device for mobile terminals provided in the embodiments of the present application include at least one of Scheme 1 and Scheme 2. For Scheme 1: when a user presses and lifts a finger on a user interface, if it is determined that the time the user holds the user interface with a single finger is greater than a first duration and less than a second duration, the result of executing the first simulation operation is displayed. In this scheme, a mouse cursor is pre-drawn on the user interface. If the user holds down a single finger on the user interface for a certain period of time, the result of simulating the right-click operation of the mouse can be displayed. Therefore, the right-click function of the mouse can be implemented by long pressing on the user interface by simply moving the mouse cursor to the interface element. Moreover, a progress bar will be displayed during the process of the user holding down the user interface with a single finger from the time corresponding to the first duration to the user lifting the single finger from the user interface, thereby guiding the user to choose the timing of lifting the single finger. For Scheme 2: when the mouse cursor pre-drawn on the user interface is within the corresponding range of the draggable control, the user holds down the single finger. During the process of moving after pressing the third time length on the user interface, the mouse cursor is moved and the result of executing the second simulation operation is displayed. In this scheme, the mouse cursor is pre-drawn on the user interface. If the user presses the single finger on the user interface for a long time and then moves, the result of simulating the execution of the mouse movement and draggable control operation can be displayed. Therefore, the function of dragging the draggable control by long pressing on the user interface can be realized by moving the mouse cursor to the interface element. Moreover, the progress bar starts to be displayed when the user presses the single finger on the user interface for a first time length, and a complete progress bar is completed when the user presses the single finger on the user interface for a second time length. The timing of moving the single finger after pressing on the user interface can be guided. Therefore, through Scheme 1 / Scheme 2, the user can accurately select / drag small interface elements. Moreover, both Scheme 1 and Scheme 2 are realized by long pressing the user interface with one finger. The user operations involved are relatively simple and have certain similarities, which are easy for users to operate. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A flowchart of an interactive method applicable to a mobile terminal provided in an embodiment of the present application; Figure 2 A schematic diagram of the structure of an interactive device applicable to a mobile terminal provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0011] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0012] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0013] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0014] Reference Figure 1 As shown, the embodiment of the present application provides a flow chart of an interaction method applicable to a mobile terminal, including: S10, when the user presses and then lifts a finger on the user interface, if it is determined that the time the user presses and holds the user interface with a single finger is greater than a first duration and less than a second duration, displaying the result of executing a first simulation operation, wherein a progress bar is displayed during the process from the time corresponding to the first duration to the time the user lifts the single finger from the user interface, and the first simulation operation is to simulate a right-click operation of a mouse at a position corresponding to a mouse cursor pre-drawn on the user interface; and / or S11. When a mouse cursor pre-drawn on the user interface is within a range corresponding to the draggable control, the mouse cursor is moved during the process of moving the mouse after the user presses the user interface with one finger for a third time period, and a result of executing a second simulation operation is displayed, wherein a progress bar is displayed starting from when the user presses the user interface with one finger for a first time period, and a complete progress bar is displayed when the user presses the user interface with one finger for a second time period, the third time period is greater than or equal to the second time period, and the second time period is greater than the first time period, and the second simulation operation is a simulation operation of moving the mouse to move the draggable control.

[0015] In this embodiment, it should be noted that in step S10 , a pre-drawn mouse cursor is displayed on the user interface to prompt the user that the touch operation on the user interface simulates a mouse operation. The result of executing the first simulation operation is displayed, including two situations: first, for the user interface of the software on the mobile terminal, the right-click operation of the mouse is simulated directly at the mouse cursor, and the result of simulating the right-click operation of the mouse at the mouse cursor will be displayed on the mobile terminal, such as displaying a related context menu; second, for the user interface corresponding to the cloud computer screen / cloud application screen displayed on the mobile terminal, the right-click operation of the mouse is simulated at the corresponding position on the cloud computer screen / cloud application screen of the cloud server (the corresponding position is the position on the cloud computer screen / cloud application screen corresponding to the position of the mouse cursor on the user interface), and the mobile terminal displays a combination of the mouse cursor and the cloud computer screen / cloud application screen rendered by the cloud server after executing the simulation operation. It should be noted that the mouse cursor may not be displayed when the cloud server executes the simulation operation, and the user sees a combination of the mouse cursor on the mobile terminal and the rendered screen, or the mouse cursor may be displayed when the simulation operation is performed, and the user sees a combination of the mouse cursor on the mobile terminal and the rendered screen, and the mouse cursor on the mobile terminal covers the mouse cursor on the cloud computer screen / cloud application screen. The values ​​of the first duration and the second duration can be set as needed, for example, the first duration and the second duration can be set to 0.5 seconds and 2 seconds respectively. After the user presses a single finger on the user interface, a progress bar will be displayed from the time corresponding to the first duration to the time when the user lifts the single finger from the user interface. The time consumed to complete a complete progress bar is the difference between the second duration and the first duration. The result of executing the first simulation operation will be displayed only if the user lifts the single finger before a complete progress bar is completed. In other words, the way of displaying the progress bar can guide the user to lift the single finger. The form of the progress bar can be set as needed, for example, it can be set to a circular progress bar, and the circular progress bar can be displayed around the mouse cursor when displayed.

[0016] In step S11 , a pre-drawn mouse cursor is displayed on the user interface to prompt the user that the touch operation on the user interface simulates a mouse operation. The result of executing the second simulation operation is displayed, including two situations: first, for the draggable control on the user interface of the software on the mobile terminal, the movement of the draggable control is controlled according to the single-finger movement position, and the mouse cursor is moved at the same time, so that the user sees that the mouse cursor drags the draggable control to move; second, for the user interface corresponding to the cloud computer screen / cloud application screen displayed on the mobile terminal, it is simulated that the mouse moves the draggable control at the corresponding position on the cloud computer screen / cloud application screen of the cloud server (the corresponding position is the position on the cloud computer screen / cloud application screen corresponding to the position of the mouse cursor on the user interface), and the mouse cursor on the mobile terminal is moved at the same time, then the mobile terminal displays the combination of the movement of the mouse cursor and the cloud computer screen / cloud application screen rendered by the cloud server after executing the simulation operation. It should be noted that the mouse cursor may not be displayed when the cloud server executes the simulation operation, so that the user sees the combination of the mouse cursor on the mobile terminal and the rendered screen, or the mouse cursor may be displayed when executing the simulation operation, so that the user sees the combination of the mouse cursor on the mobile terminal and the rendered screen, and the mouse cursor on the mobile terminal covers the mouse cursor on the cloud computer screen / cloud application screen.

[0017] The progress bar starts to be displayed when the user presses the user interface with one finger for a first time period, and completes a full progress bar when the user presses the user interface with one finger for a second time period. After that, the corresponding draggable control is in a draggable state, and the corresponding draggable control can be moved by moving one finger. In other words, the way of displaying the progress bar can guide the user to make the timing of single-finger movement.

[0018] The interactive method applicable to a mobile terminal provided in an embodiment of the present application includes at least one of schemes 1 and 2. For scheme 1: when a user presses and lifts a finger on a user interface, if it is determined that the time the user holds the user interface with a single finger is greater than a first duration and less than a second duration, the result of executing the first simulation operation is displayed. In this scheme, a mouse cursor is pre-drawn on the user interface. If the user holds down the user interface with a single finger for a certain period of time, the result of simulating the execution of a right-click operation can be displayed. Therefore, the right-click function can be implemented by long pressing on the user interface by simply moving the mouse cursor to the interface element. Moreover, during the process of the user holding down the user interface with a single finger, a progress bar will be displayed from the time corresponding to the first duration to the process of the user lifting the single finger from the user interface, thereby guiding the user to select the timing to lift the single finger. For scheme 2: when the mouse cursor pre-drawn on the user interface is within the corresponding range of the draggable control, the user moves the single finger within the user interface. During the process of moving after pressing the third time length on the interface, the mouse cursor is moved and the result of executing the second simulation operation is displayed. In this scheme, the mouse cursor is pre-drawn on the user interface. If the user presses a single finger on the user interface for a long time and then moves, the result of simulating the execution of the mouse movement and draggable control operation can be displayed. Therefore, the function of dragging the mouse and dragging the draggable control can be realized by long pressing and moving on the user interface only by moving the mouse cursor to the interface element. Moreover, the progress bar starts to be displayed when the user presses a single finger on the user interface for a first time length, and a complete progress bar is completed when the user presses a single finger on the user interface for a second time length. The timing of moving the user after pressing a single finger on the user interface can be guided. Therefore, through Scheme 1 / Scheme 2, the user can accurately select / drag small interface elements. Moreover, both Scheme 1 and Scheme 2 are realized by long pressing the user interface with a single finger. The user operations involved are relatively simple and have certain similarities, which are easy for users to operate.

[0019] Based on the foregoing method embodiment, displaying the result of executing the first simulation operation may include: Simulate a right-click at the mouse cursor position on the user interface; or The position of the mouse cursor on the user interface, the right button of the mouse is pressed event data, and the right button of the mouse is lifted event data are sent to the cloud server, so that the cloud server generates the right button of the mouse click event data based on the position of the mouse cursor on the user interface, the right button of the mouse is pressed event data, and the right button of the mouse is lifted event data, and sends the right button of the mouse click event data to the cloud server's system to simulate the right button of the mouse click operation corresponding to the position of the mouse cursor on the rendering screen of the cloud server, or generates the right button of the mouse click event data based on the position of the mouse cursor on the user interface, and sends the right button of the mouse click event data to the cloud server's system through the cloud server to simulate the right button of the mouse click operation corresponding to the position of the mouse cursor on the rendering screen of the cloud server.

[0020] In this embodiment, it should be noted that the mobile terminal can send the position of the mouse cursor on the user interface, the right-button mouse press event data, and the right-button mouse lift event data to the cloud server, wherein the right-button mouse press event data may include the right-button mouse identifier and the press identifier, and the right-button mouse lift event data may include the right-button mouse identifier and the lift identifier. After receiving the position of the mouse cursor on the user interface, the right-button mouse press event data, and the right-button mouse lift event data, the cloud server can generate the right-button mouse click event data, and send the right-button mouse click event data to the cloud server system. After receiving the right-button mouse click event data, the cloud server system can simulate the right-button mouse click operation corresponding to the position of the mouse cursor on the rendering screen of the cloud server. After that, after the screen rendered by the cloud server is transmitted to the mobile terminal, the mobile terminal displays a combination of the mouse cursor and the result screen obtained by performing the simulated operation on the cloud server. The process of generating the right-button mouse click event data in this scheme can be performed by the cloud server, and in addition, it can also be performed by the mobile terminal.

[0021] Based on the foregoing method embodiment, displaying the result of executing the second simulation operation may include: Simulates the action of moving a mouse over a draggable control in the user interface.

[0022] Based on the foregoing method embodiment, the method may further include: When the user presses a single finger on the user interface for a second duration, the position of the mouse cursor on the user interface and the left mouse button pressed event data are sent to the cloud server, so that the cloud server generates target left mouse button pressed event data based on the position of the mouse cursor on the user interface and the left mouse button pressed event data, and sends the target left mouse button pressed event data to the system of the cloud server, so as to simulate the left mouse button pressed operation at the position of the mouse cursor on the rendered screen of the cloud server, or generates target left mouse button pressed event data based on the position of the mouse cursor on the user interface, and sends the target left mouse button pressed event data to the system of the cloud server through the cloud server, so as to simulate the left mouse button pressed operation at the position of the mouse cursor on the rendered screen of the cloud server; The displaying of the result of executing the second simulation operation may include: The mouse movement event data is sent to the cloud server so that the cloud server generates first target mouse movement event data based on the mouse movement event data, and sends the first target mouse movement event data to the system of the cloud server to simulate the operation of moving the mouse to drag the control on the rendering screen of the cloud server, or the first target mouse movement event data is generated based on the mouse movement event data, and the first target mouse movement event data is sent to the system of the cloud server through the cloud server to simulate the operation of moving the mouse to drag the control on the rendering screen of the cloud server.

[0023] In this embodiment, it should be noted that the mobile terminal can send the position of the mouse cursor on the user interface and the left mouse button pressed event data to the cloud server, wherein the left mouse button pressed event data may include a left mouse button identifier and a pressed identifier. After receiving the position of the mouse cursor on the user interface and the left mouse button pressed event data, the cloud server can generate target left mouse button pressed event data, and send the target left mouse button pressed event data to the cloud server system. After receiving the target left mouse button pressed event data, the cloud server system can simulate the left mouse button click operation at the position of the mouse cursor on the rendered screen of the cloud server. After that, after the screen rendered by the cloud server is transmitted to the mobile terminal, the mobile terminal displays a combination of the mouse cursor and the result screen obtained by performing the simulated operation on the cloud server. The process of generating the target left mouse button pressed event data in this scheme can be executed by the cloud server, and in addition, it can also be executed by the mobile terminal.

[0024] The mouse movement event data may include data related to the movement position of a single finger of the user, such as the movement position of the single finger of the user, the movement displacement of the single finger of the user, or the movement position / movement displacement of the draggable control calculated according to the movement position / movement displacement of the single finger of the user, etc. The first target mouse movement event data for triggering a simulation execution operation of moving the mouse to the draggable control on the rendering screen of the cloud server may be generated according to the mouse movement event data.

[0025] Based on the foregoing method embodiment, the method may further include: When the user moves a single finger on the user interface, the mouse cursor is moved, and the result of executing the third simulation operation is displayed, wherein the third simulation operation is a simulation of moving the mouse; and / or When the user presses and then lifts a finger on the user interface, if it is determined that the time for which the user presses and holds the user interface with a single finger is less than or equal to the first duration, a result of executing a fourth simulation operation is displayed, wherein the fourth simulation operation is a simulation of a left-click operation of the mouse at a position corresponding to a mouse cursor pre-drawn on the user interface; and / or The user presses and lifts a finger on the user interface twice in succession, and if it is determined that the time the user pressed the user interface with a single finger the previous time is less than or equal to the first duration, and the interval between the previous time the user lifted the finger and the next time the user pressed the user finger is less than a fourth duration, and the next time the user pressed the user interface with a single finger is less than or equal to the first duration, then the result of executing the fifth simulation operation is displayed, wherein the fifth simulation operation is to simulate a double-click of the left mouse button at a position corresponding to a mouse cursor pre-drawn on the user interface; and / or When the user presses and moves two fingers on the user interface, the result of executing the sixth simulation operation is displayed during the movement, wherein the sixth simulation operation is a simulation of sliding the mouse wheel.

[0026] In this embodiment, it should be noted that in addition to the simulated right-click function and the simulated mouse dragging function shown in the aforementioned embodiments, the mouse moving function, the left-click function, the left-double-click function and the mouse wheel sliding function can also be simulated.

[0027] Specifically, when the user moves a single finger on the user interface, there are two situations: first, for the user interface of the software on the mobile terminal, the moving position / movement displacement of the mouse cursor pre-drawn on the user interface is directly calculated according to the moving position / movement displacement of the user's single finger (for example, determining that the moving displacement of the mouse cursor is equal to the moving displacement of the user's single finger), the mouse cursor is moved according to the moving position / movement displacement of the mouse cursor, and the mouse moving operation is simulated. Then, what is displayed on the user interface of the mobile terminal is the mouse cursor that moves with the movement of the single finger and the simulated execution result of the mouse moving operation; second, for the user interface corresponding to the cloud computer screen / cloud application screen displayed on the mobile terminal, the operation of moving the mouse cursor on the cloud computer screen / cloud application screen of the cloud server is simulated, and it is necessary to calculate the moving position / movement displacement of the mouse cursor according to the moving position / movement displacement of the single finger. If the mouse cursor on the mobile user interface is moved, what is displayed on the mobile terminal is a combination of the movement of the mouse cursor and the cloud computer screen / cloud application screen rendered by the cloud server after performing the simulated operation. The mouse cursor may not be displayed on the cloud computer screen / cloud application screen rendered by the cloud server after performing the simulated operation, or the mouse cursor on the user interface may cover the mouse cursor on the cloud computer screen / cloud application screen rendered by the cloud server after performing the simulated operation. For example, assuming that the mouse cursor on the user interface is at position A, and the user moves from position B to position C with a single finger, the mouse cursor is moved to position D, and the cloud server is simulated to move the mouse cursor from position M on the cloud computer screen / cloud application screen to position N, where position M is the position on the cloud computer screen / cloud application screen corresponding to position A, and position D and position N are determined based on position B and position C.

[0028] Specifically, when the user clicks on the user interface with a single finger, if the time interval between the pressing and lifting actions involved in the clicking process is short, the result of executing the fourth simulation operation is displayed, including two situations: first, for the user interface of the software on the mobile terminal, the left-click operation of the mouse is simulated directly at the mouse cursor pre-drawn on the user interface, such as selecting a control; second, for the user interface corresponding to the cloud computer screen / cloud application screen displayed on the mobile terminal, the left-click operation of the mouse is simulated at the corresponding position on the cloud computer screen / cloud application screen of the cloud server (the corresponding position is the position on the cloud computer screen / cloud application screen corresponding to the position of the mouse cursor on the user interface). Then, the mobile terminal displays a combination of the mouse cursor and the cloud computer screen / cloud application screen rendered by the cloud server after the simulation operation is performed. The mouse cursor may not be displayed on the cloud computer screen / cloud application screen rendered by the cloud server after the simulation operation is performed, or the mouse cursor on the user interface may cover the mouse cursor on the cloud computer screen / cloud application screen rendered by the cloud server after the simulation operation is performed.

[0029] Specifically, when the user double-clicks with a single finger on the user interface, if the time interval between the pressing and lifting actions involved in each single-click during the double-click process is short, and the time interval between the lifting action involved in the previous single-click and the pressing action involved in the next single-click is short, the result of executing the fifth simulation operation is displayed, including two situations: first, for the user interface of the software on the mobile terminal, the double-clicking of the left mouse button operation is simulated directly at the mouse cursor pre-drawn on the user interface, such as opening a folder, etc.; second, for the user interface corresponding to the cloud computer screen / cloud application screen displayed on the mobile terminal, the simulation is a double-clicking of the left mouse button operation at a corresponding position on the cloud computer screen / cloud application screen of the cloud server (the corresponding position is the position on the cloud computer screen / cloud application screen corresponding to the position of the mouse cursor on the user interface). Then, the mobile terminal displays a combination of the mouse cursor and the cloud computer screen / cloud application screen rendered by the cloud server after the simulation operation is performed. The mouse cursor may not be displayed on the cloud computer screen / cloud application screen rendered by the cloud server after the simulation operation is performed, or the mouse cursor on the user interface may cover the mouse cursor on the cloud computer screen / cloud application screen rendered by the cloud server after the simulation operation is performed.

[0030] Specifically, when the user moves two fingers on the user interface, the result of executing the sixth simulation operation is displayed during the movement, including two situations: first, for the user interface of the software on the mobile terminal, the sliding displacement is calculated according to the moving position / moving displacement of the two fingers, and the user interface is controlled to slide based on the sliding displacement; second, for the user interface corresponding to the cloud computer screen / cloud application screen displayed on the mobile terminal, the simulation is to slide the cloud computer screen / cloud application screen of the cloud server, then the mobile terminal displays a combination of the mouse cursor and the cloud computer screen / cloud application screen rendered by the cloud server after performing the simulation operation, the mouse cursor may not be displayed on the cloud computer screen / cloud application screen rendered by the cloud server after performing the simulation operation, or the mouse cursor on the user interface may cover the mouse cursor on the cloud computer screen / cloud application screen rendered by the cloud server after performing the simulation operation, wherein the sliding displacement is calculated according to the moving position / moving displacement of the two fingers, for example, the sliding displacement may be the average value of the longitudinal displacements of the two fingers.

[0031] Based on the foregoing method embodiment, displaying the result of executing the third simulation operation may include: Simulate moving the mouse at the mouse cursor position on the user interface; or The position of the mouse cursor on the user interface and the mouse movement event data are sent to the cloud server, so that the cloud server generates second target mouse movement event data based on the position of the mouse cursor on the user interface and the mouse movement event data, and sends the second target mouse movement event data to the cloud server's system to simulate the mouse movement operation on the cloud server's rendering screen, or generates second target mouse movement event data based on the position of the mouse cursor on the user interface, and sends the second target mouse movement event data to the cloud server's system through the cloud server to simulate the mouse movement operation on the cloud server's rendering screen.

[0032] In this embodiment, it should be noted that the mouse movement event data may include data related to the moving position of a single finger of the user, such as the moving position of a single finger of the user, the moving displacement of a single finger of the user, or the moving position / movement displacement of the mouse cursor calculated based on the moving position / movement displacement of a single finger of the user. The mobile terminal may send the position of the mouse cursor on the user interface and the mouse movement event data to the cloud server to generate the second target mouse movement event data through the cloud server, or may directly generate the second target mouse movement event data and send the second target mouse movement event data to the system of the cloud server through the cloud server. After receiving the second target mouse movement event data, the system of the cloud server may simulate the execution of the mouse movement operation on the rendering screen of the cloud server.

[0033] Based on the foregoing method embodiment, displaying the result of executing the fourth simulation operation may include: Simulate a left mouse button click at the mouse cursor position on the user interface; or The position of the mouse cursor on the user interface, the left mouse button pressed event data, and the left mouse button lifted event data are sent to the cloud server, so that the cloud server generates the left mouse button click event data based on the position of the mouse cursor on the user interface, the left mouse button pressed event data, and the left mouse button lifted event data, and sends the left mouse button click event data to the cloud server's system to simulate the left mouse button click operation corresponding to the position of the mouse cursor on the cloud server's rendering screen, or generates the left mouse button click event data based on the position of the mouse cursor on the user interface, and sends the left mouse button click event data to the cloud server's system through the cloud server to simulate the left mouse button click operation corresponding to the position of the mouse cursor on the cloud server's rendering screen.

[0034] In this embodiment, it should be noted that the left mouse button press event data may include a left mouse button identifier and a press identifier, and the left mouse button lift event data may include a left mouse button identifier and a lift identifier. The mobile terminal may send the position of the mouse cursor on the user interface, the left mouse button press event data, and the left mouse button lift event data to the cloud server to generate the left mouse button click event data through the cloud server, or the mobile terminal may directly generate the left mouse button click event data and send the left mouse button click event data to the cloud server system through the cloud server to simulate the left mouse button click operation corresponding to the position of the mouse cursor on the rendering screen of the cloud server.

[0035] Based on the foregoing method embodiment, displaying the result of executing the fifth simulation operation may include: Simulate a double-click of the left mouse button at the mouse cursor position on the user interface; or Each time the user presses and lifts a single finger on the user interface, the position of the mouse cursor on the user interface, the left mouse button pressed event data and the left mouse button lifted event data are sent to the cloud server, so that the cloud server generates left mouse button click event data based on the position of the mouse cursor on the user interface, the left mouse button pressed event data and the left mouse button lifted event data, and sends the left mouse button click event data to the cloud server's system to simulate a double-click of the left mouse button at the position of the mouse cursor on the cloud server's rendering screen, or generates left mouse button click event data based on the position of the mouse cursor on the user interface, and sends the left mouse button click event data to the cloud server's system through the cloud server to simulate a double-click of the left mouse button at the position of the mouse cursor on the cloud server's rendering screen.

[0036] Based on the foregoing method embodiment, displaying the result of executing the sixth simulation operation during the moving process may include: During the movement process, mouse wheel sliding event data is generated, and the mouse wheel sliding event data is sent to the cloud server, so that the cloud server generates target mouse wheel sliding event data based on the mouse wheel sliding event data, and sends the target mouse wheel sliding event data to the cloud server's system to simulate the sliding mouse wheel operation on the cloud server's rendering screen, or target mouse wheel sliding event data is generated during the movement process, and the target mouse wheel sliding event data is sent to the cloud server's system through the cloud server to simulate the sliding mouse wheel operation on the cloud server's rendering screen.

[0037] In this embodiment, it should be noted that the mouse wheel sliding event data may include data related to the moving position of the user's two fingers, such as the moving position of the user's two fingers, the moving displacement of the user's two fingers, or the sliding displacement of the mouse wheel calculated based on the moving position / movement displacement of the user's two fingers. The mobile terminal may send the mouse wheel sliding event data to the cloud server to generate the target mouse wheel sliding event data through the cloud server, or may directly generate the target mouse wheel sliding event data and send the target mouse wheel sliding event data to the cloud server system through the cloud server. After receiving the target mouse wheel sliding event data, the cloud server system may simulate the mouse wheel sliding operation on the rendering screen of the cloud server.

[0038] Reference Figure 2 FIG. 1 is a schematic diagram of a structure of an interactive device applicable to a mobile terminal provided in an embodiment of the present application, the device comprising: A first display unit 20 is used for displaying the result of executing a first simulation operation when a user presses and then lifts a finger on the user interface, if it is determined that the time for which the user presses and holds the user interface with a single finger is greater than a first duration and less than a second duration, wherein a progress bar is displayed during the process from the time corresponding to the first duration to the time when the user lifts the single finger from the user interface, and the first simulation operation is to simulate a right-click operation of a mouse at a position corresponding to a mouse cursor pre-drawn on the user interface; and / or The second display unit 21 is used to move the mouse cursor and display the result of executing the second simulation operation during the process of moving the mouse cursor after the user presses the user interface with one finger for a third time period, when the mouse cursor pre-drawn on the user interface is within the corresponding range of the draggable control, wherein the progress bar starts to be displayed when the user presses the user interface with one finger for a first time period, and a complete progress bar is completed when the user presses the user interface with one finger for a second time period, the third time period is greater than or equal to the second time period, and the second time period is greater than the first time period, and the second simulation operation is a simulation operation of moving the mouse to move the draggable control.

[0039] An interactive device suitable for a mobile terminal provided in an embodiment of the present application includes at least one of schemes 1 and 2. For scheme 1: when a user presses and lifts a finger on a user interface, if it is determined that the time the user holds the user interface with a single finger is greater than a first duration and less than a second duration, the result of executing the first simulation operation is displayed. In this scheme, a mouse cursor is pre-drawn on the user interface. If the user holds down the user interface with a single finger for a certain period of time, the result of simulating the execution of a right-click operation can be displayed, so that the right-click function can be implemented by long pressing on the user interface by simply moving the mouse cursor to the interface element, and a progress bar will be displayed during the process of the user holding down the user interface with a single finger from the time corresponding to the first duration to the user lifting the single finger from the user interface, thereby guiding the user to lift the single finger; for scheme 2: when the mouse cursor pre-drawn on the user interface is within the corresponding range of the draggable control, the user moves the single finger within the user interface. During the process of moving after pressing the third time length on the interface, the mouse cursor is moved and the result of executing the second simulation operation is displayed. In this scheme, the mouse cursor is pre-drawn on the user interface. If the user presses a single finger on the user interface for a long time and then moves, the result of simulating the execution of the mouse movement and draggable control operation can be displayed. Therefore, the function of dragging the mouse and dragging the draggable control can be realized by long pressing and moving on the user interface only by moving the mouse cursor to the interface element. Moreover, the progress bar starts to be displayed when the user presses a single finger on the user interface for a first time length, and a complete progress bar is completed when the user presses a single finger on the user interface for a second time length. The timing of moving the user after pressing a single finger on the user interface can be guided. Therefore, through Scheme 1 / Scheme 2, the user can accurately select / drag small interface elements. Moreover, both Scheme 1 and Scheme 2 are realized by long pressing the user interface with a single finger. The user operations involved are relatively simple and have certain similarities, which are easy for users to operate.

[0040] The implementation process of the interactive device applicable to the mobile terminal provided in the embodiment of the present application is consistent with the interactive method applicable to the mobile terminal provided in the embodiment of the present application, and the effect that can be achieved is also the same as the interactive method applicable to the mobile terminal provided in the embodiment of the present application, which will not be repeated here.

[0041] like Figure 3 As shown, an electronic device provided by an embodiment of the present application includes: a processor 30, a memory 31 and a bus 32, wherein the memory 31 stores machine-readable instructions executable by the processor 30, and when the electronic device is running, the processor 30 communicates with the memory 31 through the bus 32, and the processor 30 executes the machine-readable instructions to perform the steps of the above-mentioned interaction method applicable to the mobile terminal.

[0042] Specifically, the above-mentioned memory 31 and processor 30 can be general-purpose memories and processors, which are not specifically limited here. When the processor 30 runs the computer program stored in the memory 31, it can execute the above-mentioned interaction method applicable to the mobile terminal.

[0043] Corresponding to the above-mentioned interaction method applicable to a mobile terminal, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned interaction method applicable to a mobile terminal are executed.

[0044] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An interactive method applicable to a mobile terminal, characterized in that: include: When the user presses and then lifts a finger on the user interface, if it is determined that the time the user holds the user interface with the single finger is greater than a first duration and less than a second duration, a result of executing a first simulation operation is displayed, wherein a progress bar is displayed during the process from the time corresponding to the first duration to the time the user lifts the single finger from the user interface, and the first simulation operation is to simulate a right-click operation of a mouse at a position corresponding to a mouse cursor pre-drawn on the user interface; and / or When the mouse cursor pre-drawn on the user interface is within the corresponding range of the draggable control, the mouse cursor is moved during the process of moving after the user presses the user interface with one finger for a third time period, and the result of executing the second simulation operation is displayed, wherein a progress bar is displayed starting from when the user presses the user interface with one finger for a first time period, and a complete progress bar is displayed when the user presses the user interface with one finger for a second time period, the third time period is greater than or equal to the second time period, and the second time period is greater than the first time period, and the second simulation operation is a simulation operation of moving the mouse to move the draggable control.

2. The method according to claim 1, characterized in that The displaying of the result of executing the first simulation operation comprises: Simulate a right-click at the mouse cursor position on the user interface; or The position of the mouse cursor on the user interface, the right button of the mouse is pressed event data, and the right button of the mouse is lifted event data are sent to the cloud server, so that the cloud server generates the right button of the mouse click event data based on the position of the mouse cursor on the user interface, the right button of the mouse is pressed event data, and the right button of the mouse is lifted event data, and sends the right button of the mouse click event data to the cloud server's system to simulate the right button of the mouse click operation corresponding to the position of the mouse cursor on the rendering screen of the cloud server, or generates the right button of the mouse click event data based on the position of the mouse cursor on the user interface, and sends the right button of the mouse click event data to the cloud server's system through the cloud server to simulate the right button of the mouse click operation corresponding to the position of the mouse cursor on the rendering screen of the cloud server.

3. The method according to claim 1, characterized in that The displaying of the result of executing the second simulation operation comprises: Simulates the action of moving a mouse over a draggable control in the user interface.

4. The method according to claim 1, characterized in that Also includes: When the user presses a single finger on the user interface for a second duration, the position of the mouse cursor on the user interface and the left mouse button pressed event data are sent to the cloud server, so that the cloud server generates target left mouse button pressed event data based on the position of the mouse cursor on the user interface and the left mouse button pressed event data, and sends the target left mouse button pressed event data to the system of the cloud server, so as to simulate the left mouse button pressed operation corresponding to the position of the mouse cursor on the rendering screen of the cloud server, or generates target left mouse button pressed event data based on the position of the mouse cursor on the user interface, and sends the target left mouse button pressed event data to the system of the cloud server through the cloud server, so as to simulate the left mouse button pressed operation corresponding to the position of the mouse cursor on the rendering screen of the cloud server; The displaying of the result of executing the second simulation operation includes: The mouse movement event data is sent to the cloud server so that the cloud server generates first target mouse movement event data based on the mouse movement event data, and sends the first target mouse movement event data to the system of the cloud server to simulate the operation of moving the mouse to drag the control on the rendering screen of the cloud server, or the first target mouse movement event data is generated based on the mouse movement event data, and the first target mouse movement event data is sent to the system of the cloud server through the cloud server to simulate the operation of moving the mouse to drag the control on the rendering screen of the cloud server.

5. The method according to any one of claims 1 to 4, characterized in that: Also includes: When the user moves a single finger on the user interface, the mouse cursor is moved, and the result of executing the third simulation operation is displayed, wherein the third simulation operation is a simulation of moving the mouse; and / or When the user presses and then lifts a finger on the user interface, if it is determined that the time for which the user presses and holds the user interface with a single finger is less than or equal to the first duration, a result of executing a fourth simulation operation is displayed, wherein the fourth simulation operation is a simulation of a left-click operation of the mouse at a position corresponding to a mouse cursor pre-drawn on the user interface; and / or The user presses and lifts a finger on the user interface twice in succession, and if it is determined that the time the user pressed the user interface with a single finger the previous time is less than or equal to the first duration, and the interval between the previous time the user lifted the finger and the next time the user pressed the user finger is less than a fourth duration, and the next time the user pressed the user interface with a single finger is less than or equal to the first duration, then the result of executing the fifth simulation operation is displayed, wherein the fifth simulation operation is to simulate a double-click of the left mouse button at a position corresponding to a mouse cursor pre-drawn on the user interface; and / or When the user presses and moves two fingers on the user interface, the result of executing the sixth simulation operation is displayed during the movement, wherein the sixth simulation operation is a simulation of sliding the mouse wheel.

6. The method according to claim 5, characterized in that The displaying of the result of executing the third simulation operation comprises: Simulate moving the mouse at the mouse cursor position on the user interface; or The position of the mouse cursor on the user interface and the mouse movement event data are sent to the cloud server, so that the cloud server generates second target mouse movement event data based on the position of the mouse cursor on the user interface and the mouse movement event data, and sends the second target mouse movement event data to the cloud server's system to simulate the mouse movement operation on the cloud server's rendering screen, or generates second target mouse movement event data based on the position of the mouse cursor on the user interface, and sends the second target mouse movement event data to the cloud server's system through the cloud server to simulate the mouse movement operation on the cloud server's rendering screen.

7. The method according to claim 5, characterized in that The displaying of the result of executing the fourth simulation operation comprises: Simulate a left mouse button click at the mouse cursor position on the user interface; or The position of the mouse cursor on the user interface, the left mouse button pressed event data, and the left mouse button lifted event data are sent to the cloud server, so that the cloud server generates the left mouse button click event data based on the position of the mouse cursor on the user interface, the left mouse button pressed event data, and the left mouse button lifted event data, and sends the left mouse button click event data to the cloud server's system to simulate the left mouse button click operation corresponding to the position of the mouse cursor on the cloud server's rendering screen, or generates the left mouse button click event data based on the position of the mouse cursor on the user interface, and sends the left mouse button click event data to the cloud server's system through the cloud server to simulate the left mouse button click operation corresponding to the position of the mouse cursor on the cloud server's rendering screen.

8. The method according to claim 5, characterized in that The displaying of the result of executing the fifth simulation operation comprises: Simulate a double-click of the left mouse button at the mouse cursor position on the user interface; or Each time the user presses and lifts a single finger on the user interface, the position of the mouse cursor on the user interface, the left mouse button pressed event data and the left mouse button lifted event data are sent to the cloud server, so that the cloud server generates left mouse button click event data based on the position of the mouse cursor on the user interface, the left mouse button pressed event data and the left mouse button lifted event data, and sends the left mouse button click event data to the cloud server's system to simulate a double-click of the left mouse button at the position of the mouse cursor on the cloud server's rendering screen, or generates left mouse button click event data based on the position of the mouse cursor on the user interface, and sends the left mouse button click event data to the cloud server's system through the cloud server to simulate a double-click of the left mouse button at the position of the mouse cursor on the cloud server's rendering screen.

9. The method according to claim 5, characterized in that The step of displaying the result of executing the sixth simulation operation during the moving process includes: During the movement process, mouse wheel sliding event data is generated, and the mouse wheel sliding event data is sent to the cloud server, so that the cloud server generates target mouse wheel sliding event data based on the mouse wheel sliding event data, and sends the target mouse wheel sliding event data to the cloud server's system to simulate the sliding mouse wheel operation on the cloud server's rendering screen, or target mouse wheel sliding event data is generated during the movement process, and the target mouse wheel sliding event data is sent to the cloud server's system through the cloud server to simulate the sliding mouse wheel operation on the cloud server's rendering screen.

10. An interactive device suitable for a mobile terminal, characterized in that: include: A first display unit is used for displaying the result of executing a first simulation operation when a user presses and then lifts a finger on the user interface, if it is determined that the time for which the user presses and holds the user interface with a single finger is greater than a first duration and less than a second duration, wherein a progress bar is displayed during the process from the time corresponding to the first duration to the time when the user lifts the single finger from the user interface, and the first simulation operation is to simulate a right-click operation of a mouse at a position corresponding to a mouse cursor pre-drawn on the user interface; and / or The second display unit is used to move the mouse cursor and display the result of executing the second simulation operation during the process of moving the mouse cursor after the user presses the user interface with one finger for a third time period, when the mouse cursor pre-drawn on the user interface is within the corresponding range of the draggable control, wherein the progress bar starts to be displayed when the user presses the user interface with one finger for a first time period, and a complete progress bar is completed when the user presses the user interface with one finger for a second time period, the third time period is greater than or equal to the second time period, and the second time period is greater than the first time period, and the second simulation operation is a simulation operation of moving the mouse to move the draggable control.

Images