METHODS AND DEVICES FOR CONTROLLING MOUSE, TERMINAL DEVICES, AND STORAGE MEDIA

Abstract

The invention relates to a method and apparatus for controlling a mouse, terminal, and storage medium. The method includes: responding to the recognition of a mouse operation event, following the mouse operation event, determining the mouse operation mode of the current scenario; and based on the mouse operation mode, drawing a mouse based on at least one of the following: client mouse position, server mouse position, and server mouse shape, wherein the server mouse position and server mouse shape are generated by the server and sent to the client.

Description

Mouse control method, device, terminal equipment and storage medium

[0001] Cross-references

[0002] This application claims priority to the Chinese patent application filed with the China Patent Office on November 14, 2023, with application number: 202311524016.8 and titled “Mouse control method, device, terminal device and storage medium”. The entire contents of the application are incorporated by reference into this application. Technical Field

[0003] The present application relates to the field of cloud computing technology, and in particular to a mouse control method, apparatus, terminal device, and storage medium. Background Art

[0004] Cloud PC is a virtual desktop solution based on cloud computing technology. It allows users to access and operate a virtual desktop environment located on a remote server through the Internet without relying on local hardware performance.

[0005] In some cases, the cloud computer mouse mode mainly adopts client mode or server mode. The mouse response in client mode is faster, but it cannot distinguish and process the mouse movement within the cloud computer and requires the terminal to operate the mouse to move to a specified position, and there is also the problem of inaccurate mouse response position. In server mouse mode, the local terminal mouse is hidden and the mouse is drawn by the server. The mouse movement tracking is poor, and the mouse can only be operated within the cloud computer window. The toolbar cannot be operated, and additional manual operation is required to operate the client control interface, which provides a poor user experience. Different mouse modes have their own advantages and disadvantages, but it is difficult to combine the advantages of both to improve the user experience.

[0006] The above content is only used to assist in understanding the technical solution of this application and does not constitute an admission that the above content is prior art.

[0007] Summary of the Invention

[0008] The main purpose of this application is to provide a mouse control method, device, terminal device and storage medium.

[0009] The present application provides a mouse control method, which is applied to a client. The mouse control method includes: in response to identifying a mouse operation event, determining the mouse operation mode of the current scene according to the mouse operation event; based on the mouse operation mode, performing mouse drawing according to at least one of the client mouse position, the server mouse position and the server mouse shape, wherein the server mouse position and the server mouse shape are generated by the server and sent to the client.

[0010] The present application also provides a mouse control method, which is applied to a server, and the mouse control method includes the following steps: sending the server mouse position and / or the server mouse shape to the client, so that the client can perform mouse drawing based on a determined mouse operation mode and according to at least one of the client mouse position, the server mouse position and the server mouse shape; wherein the mouse operation mode is determined by the client in response to identifying a mouse operation event and according to the mouse operation event.

[0011] The present application also provides a mouse control device, which is applied to a client. The mouse control device includes: a response module, which is used to respond to identifying a mouse operation event and determine the mouse operation mode of the current scene according to the mouse operation event; a drawing module, which is used to perform mouse drawing based on the mouse operation mode and at least one of the client mouse position, the server mouse position and the server mouse shape, wherein the server mouse position and the server mouse shape are generated by the server and sent to the client.

[0012] The present application also provides a terminal device, which includes a memory, a processor, and a mouse control program stored in the memory and executable on the processor. When the mouse control program is executed by the processor, the steps of the mouse control method described above are implemented.

[0013] The present application also provides a computer-readable storage medium, on which a mouse control program is stored. When the mouse control program is executed by a processor, the steps of the mouse control method described above are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] FIG1 is a schematic diagram of the functional modules of a terminal device to which the mouse control device of the present application belongs;

[0015] FIG2 is a flow chart of an exemplary embodiment of a mouse control method of the present application;

[0016] FIG3 is a schematic diagram of a specific process of step S20 in the embodiment of FIG2 ;

[0017] FIG4 is a schematic diagram of the cloud computer hybrid mouse mode interaction process in an embodiment of the present application;

[0018] FIG5 is a flow chart of another exemplary embodiment of the mouse control method of the present application;

[0019] FIG6 is a schematic diagram of the cloud computer mouse interaction system architecture in an embodiment of the present application.

[0020] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0021] It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application.

[0022] The main solution of the embodiment of the present application is: by responding to the recognition of a mouse operation event, determining the mouse operation mode of the current scene according to the mouse operation event; based on the mouse operation mode, performing mouse drawing according to at least one of the client mouse position, the server mouse position and the server mouse shape, wherein the server mouse position and the server mouse shape are generated by the server and sent to the client, by determining the mouse operation mode of the current scene according to the mouse operation event, and then performing mouse drawing based on the corresponding mouse operation mode, the adaptability of the mouse operation mode in different operation scenarios in the cloud computer is improved, the advantages of drawing the mouse by the client or the server are fully utilized, and the fluency and accuracy of the mouse drawing in the cloud computer are taken into account, which can comprehensively improve the user experience.

[0023] In some cases, cloud PC mouse modes primarily employ client or server modes. In client mode, the mouse is the terminal's mouse, sending mouse movement and click events to the server. The server then moves the mouse within the cloud PC to a specified location and notifies the client of the mouse's internal location. When the mouse shape within the cloud PC changes, the server sends the mouse shape data to the client, which then draws the mouse shape at the current client mouse location. This client-side mode offers fast mouse response time, but it cannot distinguish between scenarios where mouse movement within the cloud PC requires the client to manipulate the mouse to a specified location, and can also suffer from inaccurate mouse position responses. In server mode, the client's local mouse control is hidden, and window controls are used to simulate the cursor. The mouse position is determined by the client sending an offset from the window center to the server. The mouse cursor is set based on the mouse position returned by the server, and the mouse cursor is reset to the window center after each offset is retrieved. In server mode, the client's local mouse is hidden, and the server draws the mouse. Mouse movement tracking is poor, and the mouse can only be operated within the cloud PC window; toolbars cannot be manipulated. This requires additional manual operation of the client control interface, resulting in a poor user experience.

[0024] This application provides a solution that uses different mouse event processing methods for different usage scenarios of mouse operations in cloud computers. For traditional mouse movement and click events, high operation fluency and fast response speed are required. In this case, the client mouse normal mode is used. The mouse is the local mouse of the client, the position is the local position of the terminal, and the shape is determined by the data sent by the server. When the client determines that the mouse is in the pressed state, the position needs to be accurate. At this time, the mouse position is based on the mouse position returned by the server. In addition, the return and display channels of the server mouse position in the cloud computer are combined. The mouse position coordinates of the cloud computer are stored in the desktop screen display data and returned to the client by the server, thereby improving the accuracy of the mouse position and operation tracking in the client mouse drag mode scenario, and improving the user experience. In addition, based on the client mouse position sent by the client, the server can detect the cloud computer lag while responding to the mouse position in the cloud computer. The client sends the mouse position to the server regularly. If no response is received within a certain period of time, it can be considered that a lag has occurred and reported to the operation and maintenance system in a timely manner to help troubleshoot the problem and / or optimize the product.

[0025] Referring to Figure 1 , Figure 1 is a schematic diagram of the functional modules of a terminal device to which the mouse control device of the present application belongs. The mouse control device can be a device independent of the terminal device that is capable of controlling the mouse, and can be hosted on the terminal device in the form of hardware or software. The terminal device can be a smart mobile terminal with data processing capabilities, such as a mobile phone or tablet computer, or a fixed terminal device or server with data processing capabilities.

[0026] In this embodiment, the terminal device to which the mouse control apparatus belongs includes at least an output module 110 , a processor 120 , a memory 130 and a communication module 140 .

[0027] The memory 130 stores an operating system and a mouse control program. The mouse control device can store information such as mouse operation events, the current mouse operation mode determined based on the mouse operation events, the client mouse position, the server mouse position, and the server mouse shape in the memory 130. The output module 110 can be a display screen, etc. The communication module 140 can include a Wi-Fi module, a mobile communication module, and a Bluetooth module, etc., and communicates with external devices or servers through the communication module 140.

[0028] Among them, when the mouse control program in the memory 130 is executed by the processor, the following steps are implemented: in response to identifying a mouse operation event, determining the mouse operation mode of the current scene according to the mouse operation event; based on the mouse operation mode, performing mouse drawing according to at least one of the client mouse position, the server mouse position and the server mouse shape, wherein the server mouse position and the server mouse shape are generated by the server and sent to the client.

[0029] In one possible implementation, when the mouse control program in the memory 130 is executed by the processor, the following steps are also implemented: if the mouse operation event is mouse movement and / or mouse click, the mouse operation mode of the current scene is determined to be the first mouse mode; if the mouse operation event is mouse dragging, the mouse operation mode of the current scene is determined to be the second mouse mode.

[0030] In one possible implementation, when the mouse control program in the memory 130 is executed by the processor, the following steps are also implemented: if the mouse operation mode of the current scene is the first mouse mode, the mouse is drawn according to the client mouse position and the server mouse shape; if the mouse operation mode of the current scene is the second mouse mode, the client mouse is hidden, and the mouse is drawn according to the server mouse position and the server mouse shape.

[0031] In a possible implementation, when the processor executes the mouse control program in the memory 130 , the following steps are further implemented: capturing the client mouse position; and / or receiving the server mouse shape sent by the server.

[0032] In one possible implementation, the mouse control program in the memory 130 further implements the following steps when executed by the processor: if the server-side initial mouse shape is received, the shape of the client mouse is initialized according to the server-side initial mouse shape to obtain an initialized client mouse; if the server-side updated mouse shape is received, the server-side updated mouse shape is used for mouse drawing; if the server-side updated mouse shape is not received, the initialized client mouse is used for mouse drawing; wherein, the server-side updated mouse shape is sent when the server detects that the server-side mouse shape has changed.

[0033] In one possible implementation, when the mouse control program in the memory 130 is executed by the processor, the following steps are also implemented: sending the client mouse position to the server so that the server can determine the server mouse position based on the client mouse position, and sending the server mouse position to the client through desktop stream data; receiving the server mouse position.

[0034] In one possible implementation, when the mouse control program in the memory 130 is executed by the processor, the following steps are also implemented: recapturing the client mouse position at preset intervals; and sending the client mouse position to the server for the server to determine whether a freeze occurs.

[0035] In one possible implementation, when the mouse control program in the memory 130 is executed by the processor, the following steps are also implemented: sending the server mouse position and / or the server mouse shape to the client, so that the client can perform mouse drawing based on at least one of the client mouse position, the server mouse position and the server mouse shape based on the determined mouse operation mode; wherein, the mouse operation mode is determined by the client in response to identifying a mouse operation event and based on the mouse operation event.

[0036] In one possible implementation, the mouse control program in the memory 130 also implements the following steps when executed by the processor: receiving the client mouse position sent by the client, determining the server mouse position based on the client mouse position, and sending the server mouse position to the client through desktop stream data; sending the server initial mouse shape to the client, so that the client initializes the shape of the client mouse according to the server initial mouse shape to obtain an initialized client mouse; in response to detecting a change in the server mouse shape, determining that the server updates the mouse shape, and sending the server updated mouse shape to the client.

[0037] In one possible implementation, when the mouse control program in the memory 130 is executed by the processor, the following steps are also implemented: in response to not receiving the client mouse position for more than a preset time, generating freeze information and reporting it to the operation and maintenance system for the operation and maintenance system to process the freeze information.

[0038] Through the above scheme, this embodiment responds to the recognition of a mouse operation event and determines the mouse operation mode of the current scene according to the mouse operation event; based on the mouse operation mode, the mouse is drawn according to at least one of the client mouse position, the server mouse position and the server mouse shape, wherein the server mouse position and the server mouse shape are generated by the server and sent to the client. By determining the mouse operation mode of the current scene according to the mouse operation event and then executing mouse drawing based on the corresponding mouse operation mode, the adaptability of the mouse operation mode in different operation scenarios in the cloud computer is improved, the advantages of drawing the mouse by the client or the server are fully utilized, and the fluency and accuracy of the mouse drawing in the cloud computer are taken into account, which can comprehensively improve the user experience.

[0039] Based on the above terminal device architecture but not limited to the above architecture, an embodiment of the method of the present application is proposed.

[0040] The execution subject of the method of this embodiment may be a mouse control device or a terminal device, etc. This embodiment takes a mouse control device as an example.

[0041] 2 is a flowchart of an exemplary embodiment of a mouse control method of the present application, which includes steps S10 to S20.

[0042] Step S10 : In response to identifying a mouse operation event, determining a mouse operation mode of a current scene according to the mouse operation event.

[0043] In one possible implementation, cloud computing is based on distributed cloud computing storage technology. By virtualizing various physical devices such as the CPU and memory, a large number of virtual machines are deployed on the server side. Using the SPICE desktop transmission protocol, these virtual desktops are transmitted to the client as images and displayed on the user terminals. This allows a small number of servers to provide a user experience close to that of a PC (Personal Computer). In scenarios where mouse position accuracy and tracking are critical, especially over poor network conditions, this issue is magnified. Using the mouse position returned via the display channel within the cloud computing system can comprehensively improve the mouse operation experience. Using different mouse modes for different mouse operation scenarios can also improve the user experience.

[0044] In an embodiment of the present application, the server and the client jointly complete the mouse interaction in the cloud computer scenario, wherein the client is responsible for obtaining the client mouse position, sending the mouse position to the server, determining the mouse operation usage scenario, and drawing the mouse shape. The server is responsible for receiving the client mouse position and moving the mouse to the specified position.

[0045] Optionally, the step of determining the mouse operation mode of the current scene based on the mouse operation event includes: if the mouse operation event is mouse movement and / or mouse click, determining that the mouse operation mode of the current scene is the first mouse mode; if the mouse operation event is mouse dragging, determining that the mouse operation mode of the current scene is the second mouse mode.

[0046] Optionally, the mouse operation events in the embodiment of the present application include mouse dragging, mouse movement, and mouse clicking. When the mouse is in a long press state, it can be considered that the user is performing a drag operation, and the mouse operation event at this time is mouse dragging; when the user performs conventional mouse movement and click operations, the mouse operation events correspond to mouse movement and mouse click, respectively.

[0047] Optionally, different mouse operation events usually have different performance requirements for mouse drawing. For example, during mouse movement or mouse clicking, the smoothness of the operation is required to be higher. At this time, the mouse position of the client should be used as the basis, corresponding to the first mouse mode in the embodiment of the present application; during mouse dragging, the accuracy of the position is required to be higher. At this time, the mouse position returned by the server should be used as the basis, corresponding to the second mouse mode in the embodiment of the present application.

[0048] Step S20: Based on the mouse operation mode, perform mouse drawing according to at least one of the client mouse position, the server mouse position and the server mouse shape, wherein the server mouse position and the server mouse shape are generated by the server and sent to the client.

[0049] In a possible implementation, after the mouse operation mode of the current scene is determined according to the mouse operation event, mouse drawing can be performed based on the determined mouse operation mode.

[0050] Optionally, in the first mouse mode, higher requirements are placed on the smoothness of the operation. At this time, the client's mouse position should be used as the basis, but the mouse shape used is still the server's mouse shape, and the mouse is drawn according to the client's mouse position and the server's mouse shape.

[0051] Optionally, in the second mouse mode, higher requirements are placed on position accuracy. In this case, the mouse position returned by the server should be used as the basis, and the mouse shape used should also be the server mouse shape. The client mouse is hidden, and the mouse is drawn according to the server mouse position and server mouse shape.

[0052] In this embodiment, in response to identifying a mouse operation event, the mouse operation mode of the current scene is determined according to the mouse operation event; based on the mouse operation mode, mouse drawing is performed according to at least one of the client mouse position, the server mouse position and the server mouse shape, wherein the server mouse position and the server mouse shape are generated by the server and sent to the client. By determining the mouse operation mode of the current scene according to the mouse operation event, and then executing mouse drawing based on the corresponding mouse operation mode, the adaptability of the mouse operation mode in different operation scenarios in the cloud computer is improved, the advantage of drawing the mouse by the client or the server is fully utilized, and the smoothness and accuracy of the mouse drawing in the cloud computer are taken into account, which can comprehensively improve the user experience.

[0053] Referring to Figure 3, Figure 3 is a schematic diagram of a specific flow chart of step S20 in the embodiment of Figure 2. This embodiment is based on the embodiment shown in Figure 2 above. In this embodiment, step S20 includes step S201 and step S202.

[0054] Step S201: If the mouse operation mode of the current scene is the first mouse mode, mouse drawing is performed according to the client mouse position and the server mouse shape.

[0055] Step S202: If the mouse operation mode of the current scene is the second mouse mode, the client mouse is hidden, and mouse drawing is performed according to the server mouse position and server mouse shape.

[0056] Optionally, based on the mouse operation mode, before the step of performing mouse drawing according to at least one of the client mouse position, the server mouse position and the server mouse shape, the step also includes: capturing the client mouse position; and / or receiving the server mouse shape sent by the server.

[0057] Optionally, the server mouse shape in the embodiment of the present application includes the server initial mouse shape and / or the server updated mouse shape. The server initial mouse shape is used to initialize the client mouse shape. When the server detects that the mouse shape has changed, it can send the server updated mouse shape to the client so that the client can synchronously update the mouse shape.

[0058] Optionally, after the step of receiving the server mouse shape sent by the server, the step also includes: if the server initial mouse shape is received, initializing the shape of the client mouse according to the server initial mouse shape to obtain an initialized client mouse; if the server updated mouse shape is received, using the server updated mouse shape for mouse drawing; if the server updated mouse shape is not received, continuing to use the initialized client mouse for mouse drawing; wherein, the server updated mouse shape is sent when the server detects that the server mouse shape has changed.

[0059] Optionally, after the step of capturing the client mouse position, the step also includes: sending the client mouse position to the server, so that the server determines the server mouse position based on the client mouse position, and sending the server mouse position to the client via desktop stream data; receiving the server mouse position.

[0060] Refer to Figure 4, which is a schematic diagram of the cloud computer hybrid mouse mode interaction process in an embodiment of the present application. As shown in Figure 4, after the server is connected to the client desktop, the client initializes the client mouse shape to the initial shape sent by the server, captures the client mouse position and sends it to the server.

[0061] Optionally, after receiving the client's mouse position, the server moves the mouse to a specified position, then returns the current server-side mouse position in the desktop stream data, and the client updates and saves the current server-side mouse position. In this embodiment of the present application, the server-side mouse position return and display channel are combined, and the cloud computer's mouse position coordinates are stored in the desktop screen display data and returned by the server to the client, thereby improving the accuracy of the client's mouse position in the mouse drag mode scenario and the operation tracking, thereby enhancing the user experience.

[0062] Optionally, the client determines the current user mouse operation scenario. When the mouse is long pressed, it is considered to be the client mouse drag mode, that is, the second mouse mode; other normal click and move events are considered to be the client mouse regular mode, that is, the first mouse mode.

[0063] Optionally, when in the first mouse mode, a mouse shape is drawn at the current client mouse position. During this process, the server may not return the server mouse position to the client, which can reduce bandwidth. When in the second mouse mode, the client mouse is first hidden, and then a virtual mouse shape is drawn at the saved server mouse position.

[0064] Optionally, after the step of capturing the client mouse position, the step further includes: re-capturing the client mouse position at a preset time interval; and sending the client mouse position to the server for the server to determine whether a freeze occurs.

[0065] Optionally, based on the client mouse position sent by the client, the server can detect the cloud computer's lag while responding to the mouse position in the cloud computer. The client sends the mouse position to the server at regular intervals. If no response is received within a certain period of time, it can be considered that a lag has occurred and reported to the operation and maintenance system in a timely manner to help troubleshoot the problem and / or optimize the product.

[0066] It should be noted that the preset time in the embodiment of the present application can be adjusted according to actual conditions, and is not specifically limited in the embodiment of the present application.

[0067] This embodiment uses the above scheme, specifically by performing mouse drawing according to the client mouse position and the server mouse shape when the mouse operation mode of the current scene is the first mouse mode; hiding the client mouse when the mouse operation mode of the current scene is the second mouse mode, and performing mouse drawing according to the server mouse position and the server mouse shape, thereby improving the adaptability of the mouse operation mode in different operation scenarios in the cloud computer. By combining the return and display channels of the server mouse position, the mouse position coordinates of the cloud computer are stored in the desktop screen display data and returned to the client by the server, thereby improving the accuracy of the mouse position and operation followability in the client mouse drag mode scenario; based on the client mouse position sent by the client, the server can detect the cloud computer's freeze while responding to the mouse position in the cloud computer, so as to handle the freeze in time, further improving the user experience.

[0068] Referring to Figure 5, which is a flowchart of another exemplary embodiment of the mouse control method of the present application, the mouse control method includes: Step A10, sending the server-side mouse position and / or server-side mouse shape to the client, so that the client can perform mouse drawing based on at least one of the client-side mouse position, the server-side mouse position, and the server-side mouse shape based on a determined mouse operation mode; wherein the mouse operation mode is determined by the client in response to identifying a mouse operation event and based on the mouse operation event.

[0069] Refer to Figure 6, which is a schematic diagram of the cloud computer mouse interaction system architecture in an embodiment of the present application. As shown in Figure 6, in the present application, the server and the client jointly complete the mouse interaction in the cloud computer scenario, wherein the client is responsible for obtaining the client mouse position, sending the mouse position to the server, determining the mouse operation usage scenario, and drawing the mouse shape; the server is responsible for receiving the client mouse position; moving the mouse to the specified position, and returning the current mouse position to the server. In the first mouse mode, the server may not return the mouse position to the client to reduce bandwidth. When the server mouse shape changes, the mouse shape data is returned to the client.

[0070] Optionally, the server mouse shape in the embodiment of the present application includes the server initial mouse shape and / or the server updated mouse shape. The server initial mouse shape is used to initialize the client mouse shape. When the server detects that the mouse shape has changed, it can send the server updated mouse shape to the client so that the client can synchronously update the mouse shape.

[0071] Optionally, the step of sending the server mouse position and / or server mouse shape to the client includes at least one of the following: receiving the client mouse position sent by the client, determining the server mouse position based on the client mouse position, and sending the server mouse position to the client via desktop stream data; sending the server initial mouse shape to the client, so that the client initializes the shape of the client mouse according to the server initial mouse shape to obtain an initialized client mouse; in response to detecting a change in the server mouse shape, determining that the server updates the mouse shape, and sending the server updated mouse shape to the client.

[0072] Optionally, after the step of receiving the client mouse position sent by the client, the step also includes: in response to not receiving the client mouse position for more than a preset time, generating freeze information and reporting it to the operation and maintenance system, so that the operation and maintenance system can process the freeze information.

[0073] Optionally, the cloud computer mouse interaction system in the embodiment of the present application includes a cloud computer client and a cloud computer server, wherein the cloud computer client includes a client mouse position capture module, a client mouse operation scene judgment module and a client mouse control drawing module; the cloud computer server includes a server mouse position receiving module, a server mouse position return module and a server mouse shape return module.

[0074] Optionally, the client mouse position capture module is mainly used to obtain the current mouse position of the client and send it to the server. The client mouse operation scene judgment module is mainly used to judge the scene of the current user's mouse operation. When the mouse is in a long press state, it is considered that the user is performing a drag operation, which is recorded as the second mouse mode. When the user is performing conventional mouse movement and clicking operations, it is considered to be the first mouse mode. The client mouse control drawing module is mainly used to draw the mouse. When in the first mouse mode, the mouse shape data sent by the server is drawn based on the local mouse position of the client. When in the second mouse mode, the virtual mouse shape is drawn based on the cloud computer mouse position sent by the server through the desktop stream data.

[0075] Optionally, the server-side mouse position receiving module is mainly used to receive the mouse position sent by the client and move the mouse to the specified position. The server-side mouse position returning module is mainly used to send the current server-side mouse position to the client through the desktop data. The return of the server-side mouse position is combined with the display channel, and the mouse position coordinates of the cloud computer are stored in the desktop screen display data and returned from the server to the client, thereby improving the accuracy of the mouse position and operation followability in the client mouse drag mode scenario, and improving the user experience. The server-side mouse shape returning module is mainly used to send the server-side initial mouse shape and the server-side updated mouse shape to the client. When the desktop is connected, the server-side initial mouse shape is sent to the client. When the server detects that the current mouse shape has changed, the server-side updated mouse shape is sent to the client.

[0076] Optionally, based on the client mouse position sent by the client, the server can detect the cloud computer's lag while responding to the mouse position in the cloud computer. The client sends the mouse position to the server at regular intervals. If no response is received within a certain period of time, it can be considered that a lag has occurred and reported to the operation and maintenance system in a timely manner to help troubleshoot the problem and / or optimize the product.

[0077] This embodiment uses the above scheme to send the server-side mouse position and / or server-side mouse shape to the client, so that the client can perform mouse drawing based on the determined mouse operation mode and according to at least one of the client mouse position, the server mouse position and the server mouse shape; wherein, the mouse operation mode is determined by the client in response to identifying a mouse operation event and according to the mouse operation event. By sending the server-side mouse position and / or server-side mouse shape to the client, so that the client can perform mouse drawing according to the determined mouse operation mode, the adaptability of the mouse operation mode in different operation scenarios in the cloud computer is improved, the advantage of drawing the mouse by the client or the server is fully utilized, and the fluency and accuracy of mouse drawing in the cloud computer are taken into account, which can comprehensively improve the user experience.

[0078] In addition, an embodiment of the present application also proposes a mouse control device, which is applied to a client, and the mouse control device includes: a response module, which is used to respond to the recognition of a mouse operation event and determine the mouse operation mode of the current scene according to the mouse operation event; a drawing module, which is used to perform mouse drawing based on the mouse operation mode and at least one of the client mouse position, the server mouse position and the server mouse shape, wherein the server mouse position and the server mouse shape are generated by the server and sent to the client.

[0079] Compared with the existing cloud computer mouse interaction method, in the embodiment of the present application, the cloud computer client determines the usage scenario of the mouse operation. When the user's mouse operation is a regular mouse click or mouse movement event, the first mouse mode is adopted, and the client sends the client's local mouse position to the server. The server sends the mouse shape data to the client, and the client draws the shape at the current mouse position. When the client determines that the mouse operation scenario is a drag scenario, it enters the second mouse mode, and the client sends the client's mouse position to the server. The server sends the mouse position in the cloud computer and the desktop stream data to the client together. The client updates and saves the current mouse position in the cloud computer. When the mouse shape data sent by the server is received, the mouse is drawn at the saved mouse position. This application adopts this hybrid mouse mode, which can simultaneously meet the user's requirements for the smoothness of multiple operations and the accuracy of the mouse position.

[0080] For the principle and implementation process of realizing mouse control in this embodiment, please refer to the above embodiments and will not be described in detail here.

[0081] In addition, an embodiment of the present application also proposes a terminal device, which includes a memory, a processor, and a mouse control program stored in the memory and runnable on the processor. When the mouse control program is executed by the processor, the steps of the mouse control method described above are implemented.

[0082] Since the mouse control program adopts all the technical solutions of all the aforementioned embodiments when executed by the processor, it has at least all the beneficial effects brought by all the technical solutions of all the aforementioned embodiments, which will not be described one by one here.

[0083] In addition, an embodiment of the present application further provides a computer-readable storage medium, on which a mouse control program is stored. When the mouse control program is executed by a processor, the steps of the mouse control method described above are implemented.

[0084] Since the mouse control program adopts all the technical solutions of all the aforementioned embodiments when executed by the processor, it has at least all the beneficial effects brought by all the technical solutions of all the aforementioned embodiments, which will not be described one by one here.

[0085] Compared with related technologies, the mouse control method, device, terminal device and storage medium proposed in the embodiments of the present application determine the mouse operation mode of the current scene according to the mouse operation event in response to the recognition of the mouse operation event; based on the mouse operation mode, the mouse is drawn according to at least one of the client mouse position, the server mouse position and the server mouse shape, wherein the server mouse position and the server mouse shape are generated by the server and sent to the client. By determining the mouse operation mode of the current scene according to the mouse operation event, and then performing mouse drawing based on the corresponding mouse operation mode, the adaptability of the mouse operation mode in different operation scenarios in the cloud computer is improved, and the advantages of drawing the mouse by the client or server are fully utilized, while taking into account the smoothness and accuracy of the mouse drawing in the cloud computer, which can comprehensively improve the user experience.

[0086] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0087] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0088] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as mentioned above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, controlled terminal, or network device, etc.) to execute the method of each embodiment of the present application.

[0089] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the description and drawings of this application, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A mouse control method, applied to a client, comprising the following steps: In response to identifying a mouse operation event, determining a mouse operation mode of a current scene according to the mouse operation event; Based on the mouse operation mode, mouse drawing is performed according to at least one of the client mouse position, the server mouse position and the server mouse shape, wherein the server mouse position and the server mouse shape are generated by the server and sent to the client.

2. The mouse control method according to claim 1, wherein: The mouse operation mode includes a first mouse mode and a second mouse mode, the mouse operation event includes mouse dragging, mouse moving and mouse clicking, and the step of determining the mouse operation mode of the current scene according to the mouse operation event includes: If the mouse operation event is a mouse movement and / or a mouse click, determining that the mouse operation mode of the current scene is the first mouse mode; If the mouse operation event is mouse dragging, it is determined that the mouse operation mode of the current scene is the second mouse mode.

3. The mouse control method according to claim 2, wherein: The step of performing mouse drawing based on the mouse operation mode according to at least one of the client mouse position, the server mouse position and the server mouse shape comprises: If the mouse operation mode of the current scene is the first mouse mode, mouse drawing is performed according to the client mouse position and the server mouse shape; If the mouse operation mode of the current scene is the second mouse mode, the client mouse is hidden, and the mouse drawing is performed according to the server mouse position and server mouse shape.

4. The mouse control method according to claim 3, wherein: Before the step of performing mouse drawing based on the mouse operation mode according to at least one of the client mouse position, the server mouse position and the server mouse shape, the step further includes: capturing the client mouse position; and / or, Receive the server mouse shape sent by the server.

5. The mouse control method according to claim 4, wherein: The server-side mouse shape includes the server-side initial mouse shape and / or the server-side updated mouse shape. After the step of receiving the server-side mouse shape sent by the server, the step further includes: If the server-side initial mouse shape is received, the client-side mouse shape is initialized according to the server-side initial mouse shape to obtain an initialized client-side mouse; If the server-side updated mouse shape is received, the server-side updated mouse shape is used for mouse drawing; If the server does not receive the updated mouse shape, the initialized client mouse is used to perform mouse drawing; The server updates the mouse shape when the server detects that the server mouse shape changes.

6. The mouse control method according to claim 4, wherein: After the step of capturing the client mouse position, the method further includes: Sending the client mouse position to the server, so that the server determines the server mouse position according to the client mouse position, and sending the server mouse position to the client via desktop stream data; Receive the server-side mouse position.

7. The mouse control method according to claim 4, characterized in that: After the step of capturing the client mouse position, the method further includes: Recapturing the client's mouse position at a preset interval; The client mouse position is sent to the server so that the server can determine whether a freeze occurs.

8. A mouse control method, applied to a server, comprising the following steps: Sending the server-side mouse position and / or the server-side mouse shape to the client, so that the client can perform mouse drawing according to at least one of the client-side mouse position, the server-side mouse position and the server-side mouse shape based on the determined mouse operation mode; The mouse operation mode is determined by the client in response to identifying a mouse operation event and according to the mouse operation event.

9. The mouse control method according to claim 8, wherein: The server-side mouse shape includes the server-side initial mouse shape and / or the server-side updated mouse shape, and the step of sending the server-side mouse position and / or the server-side mouse shape to the client includes at least one of the following: Receiving a client mouse position sent by the client, determining the server mouse position according to the client mouse position, and sending the server mouse position to the client via desktop stream data; Sending the server-side initial mouse shape to the client, so that the client initializes the shape of the client mouse according to the server-side initial mouse shape, thereby obtaining an initialized client mouse; In response to detecting a change in the shape of a server-side mouse, determining that the server-side mouse shape is updated, and sending the updated mouse shape of the server-side to the client.

10. The mouse control method according to claim 9, wherein: After the step of receiving the client mouse position sent by the client, the step further includes: In response to not receiving the client mouse position for more than a preset time, freeze information is generated and reported to the operation and maintenance system so that the operation and maintenance system can process the freeze information.

11. A mouse control device, applied to a client, the mouse control device comprising: A response module, configured to respond to identifying a mouse operation event and determine a mouse operation mode of a current scene according to the mouse operation event; A drawing module is used to perform mouse drawing based on the mouse operation mode and according to at least one of the client mouse position, the server mouse position and the server mouse shape, wherein the server mouse position and the server mouse shape are generated by the server and sent to the client.

12. A terminal device, comprising a memory, a processor, and a mouse control program stored in the memory and executable on the processor, wherein the mouse control program implements the steps of the mouse control method according to any one of claims 1 to 10 when executed by the processor.

13. A computer-readable storage medium having a mouse control program stored thereon, wherein the mouse control program, when executed by a processor, implements the steps of the mouse control method according to any one of claims 1 to 10.

Images