Mouse control method and device, terminal equipment and storage medium

Abstract

The invention discloses a mouse control method and device, terminal equipment and a storage medium, and the method comprises the steps: determining a mouse operation mode of a current scene according to a mouse operation event in response to the recognized mouse operation event; based on the mouse operation mode, mouse drawing is carried out according to at least one item of a client mouse position, a server mouse position and a server mouse shape, the server mouse position and the server mouse shape are generated by a server and sent to the client, and the server mouse position and the server mouse shape are used for mouse drawing according to at least one item of the client mouse position, the server mouse position and the server mouse shape. The mouse operation mode of the current scene is determined according to the mouse operation event, and then the mouse drawing is executed based on the corresponding mouse operation mode, so that the adaptability of the mouse operation modes in different operation scenes in the cloud computer is improved, the advantages of drawing the mouse by a client or a server are fully played, and the user experience is improved. The smoothness and accuracy of mouse drawing in the cloud computer are considered, and the use experience of a user can be comprehensively improved.

Description

Technical Field

[0001] The present invention relates to the field of cloud computing technology, and in particular to a mouse control method, device, terminal equipment and storage medium. Background Art

[0002] Cloud PC is a virtual desktop solution based on cloud computing technology, which allows users to access and operate a virtual desktop environment located on a remote server through the Internet without relying on local hardware performance.

[0003] At present, the mouse mode of cloud computers mainly adopts client mode or server mode. The mouse response of client mode is faster, but it cannot distinguish and process the mouse movement inside the cloud computer and requires the terminal to operate the mouse to move to the specified position, and there is also the problem of inaccurate mouse response position. In server mouse mode, the local mouse of the terminal is hidden, and the mouse is drawn by the server. The mouse movement tracking is poor, and the mouse can only be operated within the cloud computer window. The toolbar cannot be operated. Additional manual operation is required to operate the client control interface, and the user experience is poor. Different mouse modes have their own advantages and disadvantages, but it is difficult to comprehensively play the advantages of the two to improve the user experience.

[0004] Therefore, it is necessary to propose a solution that comprehensively improves the user experience by improving the adaptability of the mouse operation mode in different operation scenarios in the cloud computer.

[0005] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0006] The main purpose of the present invention is to provide a mouse control method, device, terminal device and storage medium, aiming to comprehensively improve the user experience by improving the adaptability of the mouse operation mode in different operation scenarios in a cloud computer.

[0007] To achieve the above object, the present invention provides a mouse control method, which is applied to a client and includes:

[0008] In response to identifying a mouse operation event, determining a mouse operation mode of a current scene according to the mouse operation event;

[0009] Based on the mouse operation mode, mouse drawing is performed according to at least one of the client mouse position, the server mouse position and the server mouse shape, wherein the server mouse position and the server mouse shape are generated by the server and sent to the client.

[0010] In addition, to achieve the above object, the present invention also provides a mouse control method, which is applied to a server and includes the following steps:

[0011] Sending the server-side mouse position and / or the server-side mouse shape to the client, so that the client can perform mouse drawing according to at least one of the client-side mouse position, the server-side mouse position and the server-side mouse shape based on the determined mouse operation mode;

[0012] The mouse operation mode is determined by the client in response to identifying a mouse operation event and according to the mouse operation event.

[0013] In addition, to achieve the above-mentioned purpose, the present invention further provides a mouse control device, which is applied to a client and includes:

[0014] A response module, configured to respond to identifying a mouse operation event and determine a mouse operation mode of a current scene according to the mouse operation event;

[0015] A drawing module is used to perform mouse drawing based on the mouse operation mode and according to at least one of the client mouse position, the server mouse position and the server mouse shape, wherein the server mouse position and the server mouse shape are generated by the server and sent to the client.

[0016] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal device, which includes a memory, a processor, and a mouse control program stored in the memory and executable on the processor, and the mouse control program implements the steps of the mouse control method described above when executed by the processor.

[0017] In addition, to achieve the above-mentioned purpose, the present invention further provides a computer-readable storage medium, on which a mouse control program is stored, and when the mouse control program is executed by a processor, the steps of the mouse control method described above are implemented.

[0018] A mouse control method, apparatus, terminal device and storage medium proposed in an embodiment of the present invention, in response to identifying a mouse operation event, determine the mouse operation mode of the current scene according to the mouse operation event; based on the mouse operation mode, perform mouse drawing according to at least one of the client mouse position, the server mouse position and the server mouse shape, wherein the server mouse position and the server mouse shape are generated by the server and sent to the client, by determining the mouse operation mode of the current scene according to the mouse operation event, and then performing mouse drawing based on the corresponding mouse operation mode, thereby improving the adaptability of the mouse operation mode in different operation scenarios in the cloud computer, giving full play to the advantages of drawing the mouse by the client or the server, taking into account the fluency and accuracy of the mouse drawing in the cloud computer, and comprehensively improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a schematic diagram of the functional modules of the terminal device to which the mouse control device of the present invention belongs;

[0020] Figure 2 A flowchart of an exemplary embodiment of a mouse control method of the present invention;

[0021] Figure 3 for Figure 2 A schematic diagram of a specific process of step S20 in the embodiment;

[0022] Figure 4 Schematic diagram of the cloud computer hybrid mouse mode interaction process in an embodiment of the present invention;

[0023] Figure 5 is a flow chart of another exemplary embodiment of the mouse control method of the present invention;

[0024] Figure 6 Schematic diagram of the cloud computer mouse interaction system architecture in an embodiment of the present invention.

[0025] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0026] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0027] The main solution of the embodiment of the present invention is: by responding to the recognition of a mouse operation event, the mouse operation mode of the current scene is determined according to the mouse operation event; based on the mouse operation mode, the mouse is drawn according to at least one of the client mouse position, the server mouse position and the server mouse shape, wherein the server mouse position and the server mouse shape are generated by the server and sent to the client. By determining the mouse operation mode of the current scene according to the mouse operation event, and then performing mouse drawing based on the corresponding mouse operation mode, the adaptability of the mouse operation mode in different operation scenarios in the cloud computer is improved, the advantage of drawing the mouse by the client or the server is fully utilized, and the fluency and accuracy of the mouse drawing in the cloud computer are taken into account, which can comprehensively improve the user experience.

[0028] Since the cloud computer mouse mode in the prior art mainly adopts the client mode or the server mode, the mouse in the client mode is the mouse of the terminal, and the mouse movement and click events are sent to the server. The server moves the mouse in the cloud computer to the specified position, and then notifies the mouse position inside the cloud computer to the client. When the mouse shape in the cloud computer changes, the server sends the mouse shape data to the client, and the client draws the mouse shape at the current client mouse position. This client mode has a fast mouse time response, but it cannot distinguish and process the scene of the mouse movement inside the cloud computer and requires the client to operate the mouse to move to the specified position, and there is also the problem of inaccurate mouse response position. The server mode hides the local mouse control of the client and uses the window control to simulate the cursor. The mouse position is the offset of the mouse sent by the client to the center position of the window to the server, and the position of the mouse cursor is set according to the mouse position returned by the server. After each offset is obtained, the mouse cursor is reset to the center position of the window. In the server mouse mode, the local mouse of the client is hidden, and the mouse is drawn by the server. The mouse movement follow-up is poor, and the mouse can only be operated in the cloud computer window, and the toolbar cannot be operated. Additional manual operation is required to operate the client control interface, resulting in a poor user experience.

[0029] The present invention provides a solution, and adopts different mouse event processing methods for different usage scenarios of mouse operation in cloud computers. For events such as traditional mouse movement and clicking, high operation fluency and fast response speed are required. At this time, the client mouse normal mode is adopted, and the mouse is a local mouse of the client, the position is the local position of the terminal, and the shape is determined by the data sent by the server; when the client determines that the mouse is in a pressed state, the position needs to be accurate, and the mouse position is based on the mouse position returned by the server. In addition, the return and display channels of the server mouse position in the cloud computer are combined together, and the mouse position coordinates of the cloud computer are stored in the desktop screen display data and returned to the client by the server, thereby improving the accuracy of the mouse position in the client mouse drag mode scenario and the operation followability, and improving the user experience. In addition, based on the client mouse position sent by the client, the server can detect the freeze of the cloud computer while responding to the mouse position in the cloud computer. The client sends the mouse position to the server regularly. If no response is received within a certain period of time, it can be considered that freeze occurs, and the operation and maintenance system is reported in time to help troubleshoot problems and / or optimize products.

[0030] Specifically, refer to Figure 1 , Figure 1 The figure is a functional module diagram of the terminal device to which the mouse control device of the present invention belongs. The mouse control device can be a device that is independent of the terminal device and can control the mouse, and can be carried on the terminal device in the form of hardware or software. The terminal device can be a smart mobile terminal with data processing function such as a mobile phone or a tablet computer, or a fixed terminal device or server with data processing function.

[0031] In this embodiment, the terminal device to which the mouse control apparatus belongs at least includes an output module 110 , a processor 120 , a memory 130 and a communication module 140 .

[0032] The memory 130 stores an operating system and a mouse control program. The mouse control device can store information such as mouse operation events, mouse operation modes of the current scene determined according to the mouse operation events, client mouse positions, server mouse positions, and server mouse shapes in the memory 130. The output module 110 can be a display screen, etc. The communication module 140 can include a WIFI module, a mobile communication module, and a Bluetooth module, etc., and communicates with an external device or server through the communication module 140.

[0033] When the mouse control program in the memory 130 is executed by the processor, the following steps are implemented:

[0034] In response to identifying a mouse operation event, determining a mouse operation mode of a current scene according to the mouse operation event;

[0035] Based on the mouse operation mode, mouse drawing is performed according to at least one of the client mouse position, the server mouse position and the server mouse shape, wherein the server mouse position and the server mouse shape are generated by the server and sent to the client.

[0036] Furthermore, when the mouse control program in the memory 130 is executed by the processor, the following steps are also implemented:

[0037] If the mouse operation event is a mouse movement and / or a mouse click, determining that the mouse operation mode of the current scene is the first mouse mode;

[0038] If the mouse operation event is mouse dragging, it is determined that the mouse operation mode of the current scene is the second mouse mode.

[0039] Furthermore, when the mouse control program in the memory 130 is executed by the processor, the following steps are also implemented:

[0040] If the mouse operation mode of the current scene is the first mouse mode, mouse drawing is performed according to the client mouse position and the server mouse shape;

[0041] If the mouse operation mode of the current scene is the second mouse mode, the client mouse is hidden, and the mouse drawing is performed according to the server mouse position and server mouse shape.

[0042] Furthermore, when the mouse control program in the memory 130 is executed by the processor, the following steps are also implemented:

[0043] capturing the client mouse position; and / or,

[0044] Receive the server mouse shape sent by the server.

[0045] Furthermore, when the mouse control program in the memory 130 is executed by the processor, the following steps are also implemented:

[0046] If the server-side initial mouse shape is received, the client-side mouse shape is initialized according to the server-side initial mouse shape to obtain an initialized client-side mouse;

[0047] If the server-side updated mouse shape is received, the server-side updated mouse shape is used for mouse drawing;

[0048] If the server does not receive the updated mouse shape, the initialized client mouse is used to perform mouse drawing;

[0049] The server updates the mouse shape when the server detects that the server mouse shape changes.

[0050] Furthermore, when the mouse control program in the memory 130 is executed by the processor, the following steps are also implemented:

[0051] Sending the client mouse position to the server, so that the server determines the server mouse position according to the client mouse position, and sending the server mouse position to the client via desktop stream data;

[0052] Receive the server-side mouse position.

[0053] Furthermore, when the mouse control program in the memory 130 is executed by the processor, the following steps are also implemented:

[0054] Recapturing the client's mouse position at a preset interval;

[0055] The client mouse position is sent to the server so that the server can determine whether a freeze occurs.

[0056] Furthermore, when the mouse control program in the memory 130 is executed by the processor, the following steps are also implemented:

[0057] Sending the server-side mouse position and / or the server-side mouse shape to the client, so that the client can perform mouse drawing according to at least one of the client-side mouse position, the server-side mouse position and the server-side mouse shape based on the determined mouse operation mode;

[0058] The mouse operation mode is determined by the client in response to identifying a mouse operation event and according to the mouse operation event.

[0059] Furthermore, when the mouse control program in the memory 130 is executed by the processor, the following steps are also implemented:

[0060] Receiving a client mouse position sent by the client, determining the server mouse position according to the client mouse position, and sending the server mouse position to the client via desktop stream data;

[0061] Sending the server-side initial mouse shape to the client, so that the client initializes the shape of the client mouse according to the server-side initial mouse shape, thereby obtaining an initialized client mouse;

[0062] In response to detecting a change in the shape of a server-side mouse, determining that the server-side mouse shape is updated, and sending the updated mouse shape of the server-side to the client.

[0063] Furthermore, when the mouse control program in the memory 130 is executed by the processor, the following steps are also implemented:

[0064] In response to not receiving the client mouse position for more than a preset time, freeze information is generated and reported to the operation and maintenance system so that the operation and maintenance system can process the freeze information.

[0065] This embodiment adopts the above scheme, specifically by responding to the recognition of the mouse operation event, determining the mouse operation mode of the current scene according to the mouse operation event; based on the mouse operation mode, performing mouse drawing according to at least one of the client mouse position, the server mouse position and the server mouse shape, wherein the server mouse position and the server mouse shape are generated by the server and sent to the client, by determining the mouse operation mode of the current scene according to the mouse operation event, and then performing mouse drawing based on the corresponding mouse operation mode, the adaptability of the mouse operation mode in different operation scenarios in the cloud computer is improved, the advantage of drawing the mouse by the client or the server is fully utilized, and the fluency and accuracy of the mouse drawing in the cloud computer are taken into account, which can comprehensively improve the user experience.

[0066] Based on the above terminal device architecture but not limited to the above architecture, an embodiment of the method of the present invention is proposed.

[0067] The execution subject of the method of this embodiment may be a mouse control device or a terminal device, etc. This embodiment takes a mouse control device as an example.

[0068] Reference Figure 2 , Figure 2 The figure is a flowchart of an exemplary embodiment of a mouse control method of the present invention. The mouse control method comprises:

[0069] Step S10, in response to identifying a mouse operation event, determining a mouse operation mode of a current scene according to the mouse operation event;

[0070] Specifically, cloud computers are based on distributed cloud computing storage technology. By virtualizing various physical devices such as CPU and memory, a large number of virtual machines are deployed on the server side. Based on the SPICE desktop transmission protocol, the virtual desktop is transmitted to the client in the form of an image and displayed on the user terminal, thereby providing a large number of users with an actual experience close to that of a PC (Personal Computer) on a small number of servers. In scenarios where the accuracy of the mouse position and the mouse tracking are required to be high, especially in the case of poor network quality, this problem will be magnified. Using the mouse position returned through the display channel in the cloud computer can comprehensively improve the mouse operation experience, and using different mouse modes in different mouse operation scenarios can comprehensively improve the user experience.

[0071] In an embodiment of the present invention, the server and the client jointly complete the mouse interaction in the cloud computer scenario, wherein the client is responsible for obtaining the client mouse position, sending the mouse position to the server, determining the mouse operation usage scenario, and drawing the mouse shape, and the server is responsible for receiving the client mouse position and moving the mouse to the specified position.

[0072] Optionally, the step of determining the mouse operation mode of the current scene according to the mouse operation event includes:

[0073] If the mouse operation event is a mouse movement and / or a mouse click, determining that the mouse operation mode of the current scene is the first mouse mode;

[0074] If the mouse operation event is mouse dragging, it is determined that the mouse operation mode of the current scene is the second mouse mode.

[0075] Optionally, the mouse operation events in the embodiment of the present invention include mouse dragging, mouse movement and mouse clicking. When the mouse is in a long press state, it can be considered that the user is performing a drag operation, and the mouse operation event at this time is mouse dragging; when the user performs conventional mouse movement and click operations, the mouse operation events correspond to mouse movement and mouse click, respectively.

[0076] Optionally, different mouse operation events usually have different performance requirements for mouse drawing. For example, during mouse movement or mouse clicking, the smoothness of the operation is required to be higher. At this time, the mouse position of the client should be used as the basis, corresponding to the first mouse mode in the embodiment of the present invention; during mouse dragging, the accuracy of the position is required to be higher. At this time, the mouse position returned by the server should be used as the basis, corresponding to the second mouse mode in the embodiment of the present invention.

[0077] Step S20, based on the mouse operation mode, performing mouse drawing according to at least one of the client mouse position, the server mouse position and the server mouse shape, wherein the server mouse position and the server mouse shape are generated by the server and sent to the client.

[0078] Furthermore, after the mouse operation mode of the current scene is determined according to the mouse operation event, mouse drawing can be performed based on the determined mouse operation mode.

[0079] Optionally, in the first mouse mode, higher requirements are placed on the smoothness of the operation, in which case the client's mouse position shall prevail, but the mouse shape used is still the server's mouse shape, and mouse drawing is performed based on the client's mouse position and the server's mouse shape.

[0080] Optionally, in the second mouse mode, higher requirements are placed on position accuracy, in which case the mouse position returned by the server shall prevail, the mouse shape used shall also be the server mouse shape, the client mouse shall be hidden, and the mouse shall be drawn according to the server mouse position and shape.

[0081] In this embodiment, in response to identifying a mouse operation event, the mouse operation mode of the current scene is determined according to the mouse operation event; based on the mouse operation mode, mouse drawing is performed according to at least one of the client mouse position, the server mouse position and the server mouse shape, wherein the server mouse position and the server mouse shape are generated by the server and sent to the client. By determining the mouse operation mode of the current scene according to the mouse operation event, and then performing mouse drawing based on the corresponding mouse operation mode, the adaptability of the mouse operation mode in different operation scenarios in the cloud computer is improved, the advantage of drawing the mouse by the client or the server is fully utilized, and the fluency and accuracy of the mouse drawing in the cloud computer are taken into account, which can comprehensively improve the user experience.

[0082] Reference Figure 3 , Figure 3 for Figure 2 The specific flow chart of step S20 in the embodiment. Figure 2 In the embodiment shown, in this embodiment, the above step S20 includes:

[0083] Step S201, if the mouse operation mode of the current scene is the first mouse mode, mouse drawing is performed according to the client mouse position and the server mouse shape;

[0084] Step S202: If the mouse operation mode of the current scene is the second mouse mode, the client mouse is hidden, and mouse drawing is performed according to the server mouse position and server mouse shape.

[0085] Optionally, based on the mouse operation mode, before the step of performing mouse drawing according to at least one of the client mouse position, the server mouse position and the server mouse shape, the step further includes:

[0086] capturing the client mouse position; and / or,

[0087] Receive the server mouse shape sent by the server.

[0088] Optionally, the server mouse shape in an embodiment of the present invention includes a server initial mouse shape and / or a server updated mouse shape. The server initial mouse shape is used to initialize the client mouse shape. When the server detects that the mouse shape has changed, it can send the server updated mouse shape to the client so that the client can synchronously update the mouse shape.

[0089] Optionally, after the step of receiving the server mouse shape sent by the server, the step further includes:

[0090] If the server-side initial mouse shape is received, the client-side mouse shape is initialized according to the server-side initial mouse shape to obtain an initialized client-side mouse;

[0091] If the server-side updated mouse shape is received, the server-side updated mouse shape is used for mouse drawing;

[0092] If the server does not receive the updated mouse shape, the initialized client mouse is used to perform mouse drawing;

[0093] The server updates the mouse shape when the server detects that the server mouse shape changes.

[0094] Optionally, after the step of capturing the client mouse position, the step further includes:

[0095] Sending the client mouse position to the server, so that the server determines the server mouse position according to the client mouse position, and sending the server mouse position to the client via desktop stream data;

[0096] Receive the server-side mouse position.

[0097] Specifically, refer to Figure 4 , Figure 4 FIG. 1 is a schematic diagram of the cloud computer hybrid mouse mode interaction process in an embodiment of the present invention. Figure 4 As shown, when the server is connected to the client desktop, the client initializes the client mouse shape to the initial shape sent by the server, captures the client mouse position and sends it to the server.

[0098] Optionally, after receiving the client's mouse position, the server moves the mouse to a specified position, and then returns the current server's mouse position in the desktop stream data, and the client updates and saves the current server's mouse position. In the embodiment of the present invention, the return of the server's mouse position is combined with the display channel, and the cloud computer's mouse position coordinates are stored in the desktop screen display data and returned from the server to the client, thereby improving the accuracy of the mouse position and operation followability in the client's mouse drag mode scenario, and improving the user experience.

[0099] Optionally, the client determines the current user mouse operation scenario. When the mouse is long pressed, it is considered to be the client mouse drag mode, that is, the second mouse mode; other normal click and move events are considered to be the client mouse regular mode, that is, the first mouse mode.

[0100] Optionally, when it is the first mouse mode, a mouse shape is drawn at the current client mouse position. During this process, the server may not return the server mouse position to the client, which can reduce bandwidth; when it is the second mouse mode, the client mouse is hidden first, and then a virtual mouse shape is drawn at the saved server mouse position.

[0101] Optionally, after the step of capturing the client mouse position, the step further includes:

[0102] Recapturing the client's mouse position at a preset interval;

[0103] The client mouse position is sent to the server so that the server can determine whether a freeze occurs.

[0104] Optionally, based on the client mouse position sent by the client, the server can detect the lag of the cloud computer while responding to the mouse position in the cloud computer. The client sends the mouse position to the server at regular intervals. If no response is received within a certain period of time, it can be considered that a lag has occurred and the operation and maintenance system is reported in a timely manner to help troubleshoot the problem and / or optimize the product.

[0105] It should be noted that the preset time in the embodiment of the present invention can be adjusted according to actual conditions, and is not specifically limited in the embodiment of the present invention.

[0106] This embodiment adopts the above scheme, specifically by drawing the mouse according to the client mouse position and the server mouse shape when the mouse operation mode of the current scene is the first mouse mode; hiding the client mouse when the mouse operation mode of the current scene is the second mouse mode, and drawing the mouse according to the server mouse position and the server mouse shape, thereby improving the adaptability of the mouse operation mode in different operation scenarios in the cloud computer, and combining the return and display channels of the server mouse position, the mouse position coordinates of the cloud computer are stored in the desktop screen display data and returned from the server to the client, thereby improving the accuracy of the mouse position and operation followability in the client mouse drag mode scenario; based on the client mouse position sent by the client, the server can detect the freeze of the cloud computer while responding to the mouse position in the cloud computer, so as to handle the freeze in time, further improving the user experience.

[0107] Reference Figure 5 , Figure 5 FIG. 1 is a flow chart of another exemplary embodiment of a mouse control method of the present invention. The mouse control method comprises:

[0108] Step A10, sending the server-side mouse position and / or the server-side mouse shape to the client, so that the client can perform mouse drawing according to at least one of the client-side mouse position, the server-side mouse position and the server-side mouse shape based on the determined mouse operation mode;

[0109] The mouse operation mode is determined by the client in response to identifying a mouse operation event and according to the mouse operation event.

[0110] Specifically, refer to Figure 6 , Figure 6 Schematic diagram of the cloud computer mouse interaction system architecture in an embodiment of the present invention. Figure 6 As shown, in the present invention, the server and the client jointly complete the mouse interaction in the cloud computer scenario, wherein the client is responsible for obtaining the client mouse position, sending the mouse position to the server, determining the mouse operation usage scenario, and drawing the mouse shape; the server is responsible for receiving the client mouse position; moving the mouse to the specified position, and returning the current mouse position of the server. In the first mouse mode, the server may not return the mouse position to the client to reduce the bandwidth. When the server mouse shape changes, the mouse shape data is returned to the client.

[0111] Optionally, the server mouse shape in an embodiment of the present invention includes a server initial mouse shape and / or a server updated mouse shape. The server initial mouse shape is used to initialize the client mouse shape. When the server detects that the mouse shape has changed, it can send the server updated mouse shape to the client so that the client can synchronously update the mouse shape.

[0112] Optionally, the step of sending the server-side mouse position and / or the server-side mouse shape to the client includes at least one of the following:

[0113] Receiving a client mouse position sent by the client, determining the server mouse position according to the client mouse position, and sending the server mouse position to the client via desktop stream data;

[0114] Sending the server-side initial mouse shape to the client, so that the client initializes the shape of the client mouse according to the server-side initial mouse shape, thereby obtaining an initialized client mouse;

[0115] In response to detecting a change in the shape of a server-side mouse, determining that the server-side mouse shape is updated, and sending the updated mouse shape of the server-side to the client.

[0116] Optionally, after the step of receiving the client mouse position sent by the client, the step further includes:

[0117] In response to not receiving the client mouse position for more than a preset time, freeze information is generated and reported to the operation and maintenance system so that the operation and maintenance system can process the freeze information.

[0118] Optionally, the cloud computer mouse interaction system in an embodiment of the present invention includes a cloud computer client and a cloud computer server, wherein the cloud computer client includes a client mouse position capture module, a client mouse operation scene judgment module and a client mouse control drawing module; the cloud computer server includes a server mouse position receiving module, a server mouse position return module and a server mouse shape return module.

[0119] Optionally, the client mouse position capture module is mainly used to obtain the current mouse position of the client and send it to the server. The client mouse operation scene judgment module is mainly used to judge the scene of the current user's mouse operation. When the mouse is in a long press state, it is considered that the user is performing a drag operation, which is recorded as the second mouse mode. When the user is performing conventional mouse movement and clicking operations, it is considered to be the first mouse mode. The client mouse control drawing module is mainly used to draw the mouse. When in the first mouse mode, the mouse shape data sent by the server is drawn based on the local mouse position of the client. When in the second mouse mode, the virtual mouse shape is drawn based on the cloud computer mouse position sent by the server through the desktop stream data.

[0120] Optionally, the server-side mouse position receiving module is mainly used to receive the mouse position sent by the client and move the mouse to the specified position. The server-side mouse position returning module is mainly used to send the current server-side mouse position to the client through the desktop data. The return of the server-side mouse position is combined with the display channel, and the mouse position coordinates of the cloud computer are stored in the desktop screen display data and returned from the server to the client, thereby improving the accuracy of the mouse position and operation followability in the client mouse drag mode scenario, and improving the user experience. The server-side mouse shape returning module is mainly used to send the server-side initial mouse shape and the server-side updated mouse shape to the client. When the desktop is connected, the server-side initial mouse shape is sent to the client. When the server detects that the current mouse shape has changed, the server-side updated mouse shape is sent to the client.

[0121] Optionally, based on the client mouse position sent by the client, the server can detect the lag of the cloud computer while responding to the mouse position in the cloud computer. The client sends the mouse position to the server at regular intervals. If no response is received within a certain period of time, it can be considered that a lag has occurred and the operation and maintenance system is reported in a timely manner to help troubleshoot the problem and / or optimize the product.

[0122] This embodiment adopts the above scheme, specifically by sending the server-side mouse position and / or the server-side mouse shape to the client, so that the client can perform mouse drawing based on the determined mouse operation mode according to at least one of the client mouse position, the server mouse position and the server mouse shape; wherein the mouse operation mode is determined by the client in response to identifying a mouse operation event and according to the mouse operation event. By sending the server-side mouse position and / or the server-side mouse shape to the client, so that the client can perform mouse drawing according to the determined mouse operation mode, the adaptability of the mouse operation mode in different operation scenarios in the cloud computer is improved, the advantage of drawing the mouse by the client or the server is fully utilized, and the fluency and accuracy of mouse drawing in the cloud computer are taken into account, which can comprehensively improve the user experience.

[0123] In addition, an embodiment of the present invention further provides a mouse control device, which is applied to a client and includes:

[0124] A response module, configured to respond to identifying a mouse operation event and determine a mouse operation mode of a current scene according to the mouse operation event;

[0125] A drawing module is used to perform mouse drawing based on the mouse operation mode and according to at least one of the client mouse position, the server mouse position and the server mouse shape, wherein the server mouse position and the server mouse shape are generated by the server and sent to the client.

[0126] Compared with the existing cloud computer mouse interaction method, in the embodiment of the present invention, the cloud computer client determines the usage scenario of the mouse operation. When the user's mouse operation is a regular mouse click or mouse movement event, the first mouse mode is adopted, and the client sends the client's local mouse position to the server, and the server sends the mouse shape data to the client, and the client draws the shape at the current mouse position. When the client determines that the mouse operation scenario is a drag scenario, it enters the second mouse mode, and the client sends the client's mouse position to the server. The server sends the mouse position in the cloud computer and the desktop stream data to the client together. The client updates and saves the mouse position in the current cloud computer. When the mouse shape data sent by the server is received, the mouse is drawn at the saved mouse position. The present invention adopts this hybrid mouse mode, which can simultaneously meet the user's multi-operation fluency and mouse position accuracy requirements.

[0127] For the principle and implementation process of realizing mouse control in this embodiment, please refer to the above embodiments and will not be described in detail here.

[0128] In addition, an embodiment of the present invention further proposes a terminal device, which includes a memory, a processor, and a mouse control program stored in the memory and executable on the processor, wherein the mouse control program implements the steps of the mouse control method described above when executed by the processor.

[0129] Since the mouse control program adopts all the technical solutions of all the aforementioned embodiments when executed by the processor, it has at least all the beneficial effects brought by all the technical solutions of all the aforementioned embodiments, which will not be described one by one here.

[0130] In addition, an embodiment of the present invention further provides a computer-readable storage medium, on which a mouse control program is stored. When the mouse control program is executed by a processor, the steps of the mouse control method described above are implemented.

[0131] Since the mouse control program adopts all the technical solutions of all the aforementioned embodiments when executed by the processor, it has at least all the beneficial effects brought by all the technical solutions of all the aforementioned embodiments, which will not be described one by one here.

[0132] Compared with the prior art, the mouse control method, apparatus, terminal device and storage medium proposed in the embodiments of the present invention, in response to identifying a mouse operation event, determine the mouse operation mode of the current scene according to the mouse operation event; based on the mouse operation mode, perform mouse drawing according to at least one of the client mouse position, the server mouse position and the server mouse shape, wherein the server mouse position and the server mouse shape are generated by the server and sent to the client, and by determining the mouse operation mode of the current scene according to the mouse operation event, and then performing mouse drawing based on the corresponding mouse operation mode, the adaptability of the mouse operation mode in different operation scenarios in the cloud computer is improved, the advantage of drawing the mouse by the client or the server is fully utilized, and the fluency and accuracy of the mouse drawing in the cloud computer are taken into account, which can comprehensively improve the user experience.

[0133] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0134] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0135] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, a controlled terminal, or a network device, etc.) to execute the method of each embodiment of the present application.

[0136] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A mouse control method, characterized in that: The mouse control method is applied to a client, and the mouse control method comprises the following steps: In response to identifying a mouse operation event, determining a mouse operation mode of a current scene according to the mouse operation event; Based on the mouse operation mode, mouse drawing is performed according to at least one of the client mouse position, the server mouse position and the server mouse shape, wherein the server mouse position and the server mouse shape are generated by the server and sent to the client.

2. The mouse control method according to claim 1, characterized in that: The mouse operation mode includes a first mouse mode and a second mouse mode, the mouse operation event includes mouse dragging, mouse moving and mouse clicking, and the step of determining the mouse operation mode of the current scene according to the mouse operation event includes: If the mouse operation event is a mouse movement and / or a mouse click, determining that the mouse operation mode of the current scene is the first mouse mode; If the mouse operation event is mouse dragging, it is determined that the mouse operation mode of the current scene is the second mouse mode.

3. The mouse control method according to claim 2, characterized in that: The step of performing mouse drawing based on the mouse operation mode according to at least one of the client mouse position, the server mouse position and the server mouse shape comprises: If the mouse operation mode of the current scene is the first mouse mode, mouse drawing is performed according to the client mouse position and the server mouse shape; If the mouse operation mode of the current scene is the second mouse mode, the client mouse is hidden, and the mouse drawing is performed according to the server mouse position and server mouse shape.

4. The mouse control method according to claim 3, characterized in that: The step of performing mouse drawing based on the mouse operation mode according to at least one of the client mouse position, the server mouse position and the server mouse shape also includes: capturing the client mouse position; and / or, Receive the server mouse shape sent by the server.

5. The mouse control method according to claim 4, characterized in that: The server-side mouse shape includes an initial server-side mouse shape and / or an updated server-side mouse shape, and after the step of receiving the server-side mouse shape sent by the server, the step further includes: If the server-side initial mouse shape is received, the client-side mouse shape is initialized according to the server-side initial mouse shape to obtain an initialized client-side mouse; If the server-side updated mouse shape is received, the server-side updated mouse shape is used for mouse drawing; If the server does not receive the updated mouse shape, the initialized client mouse is used to perform mouse drawing; The server updates the mouse shape when the server detects that the server mouse shape changes.

6. The mouse control method according to claim 4, characterized in that: The step of capturing the client mouse position also includes: Sending the client mouse position to the server, so that the server determines the server mouse position according to the client mouse position, and sending the server mouse position to the client via desktop stream data; Receive the server-side mouse position.

7. The mouse control method according to claim 4, characterized in that: The step of capturing the client mouse position also includes: Recapturing the client's mouse position at a preset interval; The client mouse position is sent to the server so that the server can determine whether a freeze occurs.

8. A mouse control method, characterized in that: The mouse control method is applied to the server, and the mouse control method comprises the following steps: Sending the server-side mouse position and / or the server-side mouse shape to the client, so that the client can perform mouse drawing according to at least one of the client-side mouse position, the server-side mouse position and the server-side mouse shape based on the determined mouse operation mode; The mouse operation mode is determined by the client in response to identifying a mouse operation event and according to the mouse operation event.

9. The mouse control method according to claim 8, characterized in that: The server-side mouse shape includes the server-side initial mouse shape and / or the server-side updated mouse shape, and the step of sending the server-side mouse position and / or the server-side mouse shape to the client includes at least one of the following: Receiving a client mouse position sent by the client, determining the server mouse position according to the client mouse position, and sending the server mouse position to the client via desktop stream data; Sending the server-side initial mouse shape to the client, so that the client initializes the shape of the client mouse according to the server-side initial mouse shape, thereby obtaining an initialized client mouse; In response to detecting a change in the shape of a server-side mouse, determining that the server-side mouse shape is updated, and sending the updated mouse shape of the server-side to the client.

10. The mouse control method according to claim 9, characterized in that: After the step of receiving the client mouse position sent by the client, the step further includes: In response to not receiving the client mouse position for more than a preset time, freeze information is generated and reported to the operation and maintenance system so that the operation and maintenance system can process the freeze information.

11. A mouse control device, characterized in that: The mouse control device is applied to a client, and the mouse control device comprises: A response module, configured to respond to identifying a mouse operation event and determine a mouse operation mode of a current scene according to the mouse operation event; A drawing module is used to perform mouse drawing based on the mouse operation mode and according to at least one of the client mouse position, the server mouse position and the server mouse shape, wherein the server mouse position and the server mouse shape are generated by the server and sent to the client.

12. A terminal device, characterized in that: The terminal device includes a memory, a processor, and a mouse control program stored in the memory and executable on the processor. When the mouse control program is executed by the processor, the steps of the mouse control method according to any one of claims 1 to 10 are implemented.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a mouse control program, and when the mouse control program is executed by a processor, the steps of the mouse control method according to any one of claims 1 to 10 are implemented.

Images