Remote desktop mouse data processing method and device, storage medium and electronic equipment

Abstract

The application discloses a remote desktop mouse data processing method and device, a storage medium and electronic equipment, and relates to the technical field of information. The method comprises the following steps: collecting coordinates of each track point of a remote desktop mouse within a preset period; determining state information of the remote desktop mouse within the preset period based on the coordinates of each track point; dividing each track point into a plurality of motion primitives according to the state information; determining operation intention information corresponding to each motion primitive; performing data purification and reconstruction on each motion primitive according to the operation intention information to obtain reconstruction data corresponding to each motion primitive; and sending the reconstruction data to a remote server to enable the remote server to reproduce the mouse track. The application can avoid problems such as remote cursor drift and operation stickiness when the network condition is not good, and ensures user experience.

Description

Technical Field

[0001] This application relates to the field of information technology, and in particular to a remote desktop mouse data processing method, apparatus, storage medium, and electronic device. Background Technology

[0002] Remote desktop technology is a technology that allows users to remotely access and control another computer from their local computer via a network connection. This technology is widely used in remote work, IT operations and maintenance, and remote technical support, greatly improving work efficiency and flexibility.

[0003] Currently, when using this technology, the user's mouse coordinates are typically collected at a high frequency and continuously, and then the mouse data is sent to a remote server. However, this method of existing technology only works when the network conditions are good. When the network conditions are poor, phenomena such as remote cursor drift and sluggish operation will occur, resulting in a poor user experience. Summary of the Invention

[0004] In view of this, this application provides a remote desktop mouse data processing method, device, storage medium and electronic device, which mainly avoids problems such as remote cursor drift and operation stickiness even when the network conditions are poor, thus ensuring user experience.

[0005] According to a first aspect of this application, a remote desktop mouse data processing method is provided, the method comprising:

[0006] Collect the coordinates of various trajectory points of the remote desktop mouse within a preset period;

[0007] Based on the coordinates of each trajectory point, determine the state information of the remote desktop mouse within the preset period;

[0008] Based on the state information, each trajectory point is divided into multiple motion primitives;

[0009] Determine the operational intent information corresponding to each of the plurality of motion primitives;

[0010] Based on the operation intent information, data purification and reconstruction are performed on the plurality of motion primitives respectively to obtain the reconstructed data corresponding to the plurality of motion primitives respectively;

[0011] The reconstructed data is sent to a remote server so that the remote server can reproduce the mouse trajectory.

[0012] According to a second aspect of this application, a remote desktop mouse data processing device is provided, the device comprising:

[0013] The acquisition unit is used to acquire the coordinates of various trajectory points of the remote desktop mouse within a preset period;

[0014] The first determining unit is used to determine the state information of the remote desktop mouse within the preset period based on the coordinates of each trajectory point;

[0015] A segmentation unit is used to segment each trajectory point into multiple motion primitives based on the state information;

[0016] The second determining unit is used to determine the operation intention information corresponding to the plurality of motion primitives respectively;

[0017] The reconstruction unit is used to perform data purification and reconstruction on the plurality of motion primitives according to the operation intention information, so as to obtain the reconstruction data corresponding to the plurality of motion primitives respectively;

[0018] The sending unit is used to send the reconstructed data to a remote server so that the remote server can reproduce the mouse trajectory.

[0019] According to a third aspect of this application, a storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the above-described remote desktop mouse data processing method.

[0020] According to a fourth aspect of this application, an electronic device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the program to implement the above-described remote desktop mouse data processing method.

[0021] By employing the above technical solution, this application provides a remote desktop mouse data processing method, apparatus, storage medium, and electronic device. Compared with the prior art, firstly, based on the coordinates of each trajectory point of the remote desktop mouse, the state information of the remote desktop mouse within the preset period is determined. Then, based on the state information, each trajectory point is divided into multiple motion primitives, and the operation intention information corresponding to each of the multiple motion primitives is determined. Next, based on the operation intention information, data purification and reconstruction are performed on each of the multiple motion primitives to obtain the reconstructed data corresponding to each of the multiple motion primitives. Finally, the reconstructed data is sent to the remote server so that the remote server can reproduce the mouse trajectory. Therefore, this application, by determining the user's operation intention information and performing data purification and reconstruction on the trajectory point sequence within each motion primitive, can extract the truly useful core information and send it to the remote server. This makes the response quality of remote interaction independent of the network channel quality. Even during network fluctuations, there will be no phenomena such as remote cursor drift or operation stickiness, thus ensuring a good user experience.

[0022] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0023] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0024] Figure 1 A flowchart illustrating a remote desktop mouse data processing method provided in an embodiment of this application is shown.

[0025] Figure 2 A flowchart illustrating another remote desktop mouse data processing method provided in an embodiment of this application is shown;

[0026] Figure 3 A schematic diagram of the structure of a remote desktop mouse data processing device provided in an embodiment of this application is shown. Detailed Implementation

[0027] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.

[0028] Existing technologies can cause issues such as remote cursor drift and sluggish operation when network conditions are poor, resulting in a poor user experience.

[0029] To address the aforementioned problems, embodiments of the present invention provide a remote desktop mouse data processing method, applied to a client, such as... Figure 1 As shown, the method includes:

[0030] Step 10: Collect the coordinates of each trajectory point of the remote desktop mouse within a preset period.

[0031] The preset period can be set according to actual business needs, and the embodiments of the present invention do not impose specific limitations on it.

[0032] In this embodiment of the invention, a fixed preset period can be set to obtain the coordinates of the trajectory points of the remote desktop mouse within each preset period. For example, the coordinates of each trajectory point of the remote desktop mouse can be obtained every 16 milliseconds.

[0033] Step 20: Based on the coordinates of each trajectory point, determine the status information of the remote desktop mouse within the preset period.

[0034] The status information includes silent state and non-silent state.

[0035] This invention employs an asymmetric comparison rule to determine whether a remote desktop mouse is currently in a silent or active state. Specifically, step 20 includes: determining the pixel displacement between any two adjacent trajectory points based on their coordinates; if the pixel displacement is lower than a first preset threshold, determining that the remote desktop mouse has switched from an active to a silent state; if the pixel displacement is higher than a second preset threshold, determining that the remote desktop mouse has switched from a silent to an active state.

[0036] Wherein, the first preset threshold is less than the second preset threshold, and the first preset threshold and the second preset threshold can be set according to actual business needs.

[0037] For example, when the pixel displacement between any two adjacent trajectory points is less than 2 pixels, the remote desktop mouse is determined to switch from a non-silent state to a silent state; when the pixel displacement between any two adjacent trajectory points is greater than 5 pixels, it indicates that the mouse has made significant movement, and the remote desktop mouse is determined to switch from a silent state to a non-silent state. This avoids the system from switching back and forth between silent and non-silent states due to slight hand tremors when the user is moving slowly or preparing to stop, thus ensuring the stability of the state determination in this embodiment of the invention.

[0038] Step 30: Based on the state information, divide each trajectory point into multiple motion primitives.

[0039] In an embodiment of the present invention, the sequence of trajectory points generated by the remote desktop mouse between two adjacent silent states can be determined as an independent motion primitive.

[0040] Specifically, based on this status information, it can be determined whether the remote desktop mouse is in a silent or non-silent state at various points in time within a preset period. Treating the start and end of each silent state as a logical boundary, the silent and non-silent periods of the remote desktop mouse within the preset period can be defined. The sequence of trajectory points corresponding to the non-silent periods between two adjacent silent periods is defined as an independent motion primitive. For example, when a user quickly moves the remote desktop mouse from point A to point B and stops, all the trajectory points in this process are packaged into a single motion primitive, representing a complete and continuous operation by the user.

[0041] In some embodiments of the present invention, a boundary forced calibration strategy is also adopted to calibrate the boundaries of motion primitives. For this process, the method includes: for any one of the plurality of motion primitives, determining whether the remote desktop mouse has made a momentary movement based on the trajectory point sequence within the arbitrary motion primitive; if the remote desktop mouse has made a momentary movement, splitting the arbitrary motion primitive into two new motion primitives based on the target trajectory point to which the remote desktop mouse has made a momentary movement.

[0042] Specifically, some professional software, such as CAD, has an automatic mouse-attracting function. When the user's mouse approaches a point in the software, the software will forcibly jump the mouse cursor to that point instantly. This unconventional displacement is not user-operated and can interfere with the judgment. Therefore, this embodiment of the invention adds a monitoring unit to detect whether a spatiotemporal discontinuity event occurs in the trajectory point sequence within the motion primitive, that is, whether a physically impossible ultra-long displacement occurs in an extremely short time, such as moving more than 45 pixels within 16 milliseconds. Once this embodiment of the invention detects such an event, it will forcibly set that point as the end boundary of the current motion primitive and the starting boundary of the next motion primitive, thereby obtaining a new motion primitive.

[0043] Therefore, it can be seen that the embodiments of the present invention can accurately calibrate the boundaries of motion primitives by cleverly utilizing the rules of the remote software itself. The boundary forced calibration mechanism of the embodiments of the present invention can transform the interaction rules of the remote application itself (such as automatic adsorption) from an interference factor into an effective signal that can be used to accurately calibrate the intention boundary, which is particularly useful in professional design scenarios that require high-precision collaboration.

[0044] Step 40: Determine the operation intention information corresponding to each of the multiple motion primitives.

[0045] The operation intent information includes directional intent and descriptive intent. Directional intent means that the user's purpose is to go from the starting point to the destination, while descriptive intent means that the user is currently drawing or signing something.

[0046] In this embodiment of the invention, after obtaining the clearly defined motion primitives, it is necessary to analyze their corresponding operational intentions, that is, to determine the operational intention information corresponding to the motion primitives that have not been split and the new motion primitives, so as to simplify the data.

[0047] When specifically determining the operational intent information corresponding to a motion primitive, for any one of the plurality of motion primitives, the actual path length and straight-line distance corresponding to the motion primitive are determined based on the trajectory point sequence within the motion primitive; the ratio of the actual path length to the straight-line distance is calculated; if the ratio is less than a preset ratio, it is determined that the motion primitive has a directional intent; if the ratio is greater than or equal to the preset ratio, it is determined that the motion primitive has a descriptive intent.

[0048] The preset ratio can be set according to actual business needs. The actual path length is the total length of all points connected in the trajectory point sequence, and the straight-line distance is the straight-line distance between the start and end points in the trajectory point sequence within the motion element.

[0049] Specifically, based on the sequence of trajectory points within the motion primitive, the actual path length L and the straight-line distance D from the starting point to the ending point are determined. If the ratio of L / D is very small (e.g., less than 1.2), it indicates that the trajectory is very straight, and the motion primitive is a directional motion primitive, with the user's goal being to travel from the starting point to the ending point. If the ratio of L / D is very large (e.g., greater than or equal to 1.2), it indicates that the trajectory is very curved, and the motion primitive is a descriptive motion primitive, with the user possibly performing operations such as drawing or signing.

[0050] Step 50: Based on the operation intent information, perform data purification and reconstruction on the multiple motion primitives respectively to obtain the reconstructed data corresponding to the multiple motion primitives respectively.

[0051] In this embodiment of the invention, after determining the operation intent information corresponding to each motion primitive, it is necessary to simplify the data of the motion primitive according to the operation intent information. For this process, step 50 specifically includes: if any motion primitive has a directional intent, then delete all trajectory points in the trajectory point sequence within the motion primitive except for the starting trajectory point and the ending trajectory point, and determine the starting trajectory point and the ending trajectory point as the reconstructed data corresponding to the motion primitive; if any motion primitive has a descriptive intent, then use the Douglas-Puk algorithm to thin the trajectory point sequence within the motion primitive to obtain key turning points, and determine the key turning points as the reconstructed data corresponding to the motion primitive.

[0052] Specifically, for directional motion primitives, all intermediate trajectory points are discarded directly, and the reconstructed data only includes the starting point coordinates and the ending point coordinates; for descriptive motion primitives, in order to both preserve the trajectory shape and reduce the number of data points, this embodiment of the invention uses the Douglas-Puk algorithm to thin out the trajectory points, retaining only the key turning points.

[0053] When using the Douglas-Puk algorithm to thin out trajectory points, the starting and ending points of the trajectory point sequence within the motion primitive are first determined to define the initial boundaries. Then, all points in the trajectory point sequence except the starting and ending points are traversed, and the perpendicular distance from each point to the line connecting the starting and ending points is calculated. The specific calculation formula is as follows.

[0054]

[0055] in, It is the vertical distance. and These are the starting point coordinates and the ending point coordinates, respectively. These are points other than the starting point and the ending point.

[0056] Next, in the calculated vertical distance, find the maximum distance and its corresponding trajectory point. This point is the point furthest from the starting point and the ending point of the current distance. If this maximum distance is greater than the preset distance... If the point is retained, the trajectory point sequence corresponding to the motion primitive is divided into two segments: one from the starting point to the maximum distance point, and the other from the maximum distance point to the end point. The above processing is repeated for these two segments, and the simplification is performed recursively. Finally, all the points are merged in order, which is the final key turning point, i.e., the reconstructed data.

[0057] Among them, the preset distance , The preset fidelity coefficient, This is the straight-line distance from the starting point to the ending point.

[0058] The embodiments of the present invention use the Douglas-Puk algorithm to thin out the trajectory points of motion primitives, which can reduce the number of points while maintaining the basic shape of polylines or polygons.

[0059] Step 60: Send the reconstructed data to a remote server so that the remote server can reproduce the mouse trajectory.

[0060] Through the above processing, the information finally sent to the remote server in this embodiment of the invention is no longer a dense array of physical coordinate points, but rather purified and structured information that can directly express the user's operational intent.

[0061] In some embodiments, after determining the operational intent information corresponding to each motion primitive, an intent service level instruction can be assigned to it. For example, high-speed, long-distance directional motion primitives can be marked as efficient navigation instructions, low-speed, short-distance descriptive motion primitives can be marked as high-precision instructions, and silent states can be marked as low-power idle instructions. These intent service level instructions are sent to a remote server along with the data, and the server-side agent dynamically adjusts the rendering resource allocation strategy based on the instruction. For example, when receiving an efficient navigation instruction, the rendering frame rate can be appropriately reduced, while when receiving a high-precision instruction, the rendering quality and response speed can be improved, thereby achieving a reallocation of computing resources according to the user's operational intent.

[0062] Furthermore, embodiments of the present invention have also made specific optimizations for the mouse in a silent state. Regarding this optimization process, such as... Figure 2 As shown, it includes:

[0063] Step 70: Determine the sequence of trajectory points during the period when the remote desktop mouse is in the silent state.

[0064] In this embodiment of the invention, the start and end of each silent state of the remote desktop mouse are taken as a logical boundary, and the silent state period of the remote desktop mouse within a preset period can be determined.

[0065] Step 80: Calculate the center point coordinates of the remote desktop mouse based on the trajectory point sequence during the silent state.

[0066] In this embodiment of the invention, the mean horizontal coordinates and mean vertical coordinates of the trajectory point sequence during the silent state are calculated respectively, and the center point coordinates of the remote desktop mouse are obtained based on the mean horizontal coordinates and mean vertical coordinates.

[0067] Step 90: Send the center point coordinates to the remote server so that the remote server can reproduce the mouse trajectory.

[0068] In this embodiment of the invention, when a user tries to click a very small button, their hand will inevitably tremble slightly without conscious effort, generating a series of noisy coordinate points. This embodiment of the invention, upon determining that the remote desktop mouse has entered a silent state, does not immediately send the last trajectory point. Instead, it records all coordinate points generated by the tremors during this silent period, and then calculates the geometric center (average position) of these points at the end of the silent period. Finally, it sends this stable and accurate center point coordinate to the remote server.

[0069] Through the above optimization, the embodiments of the present invention can effectively filter out physiological hand tremors, allowing the remote cursor to stay steadily at the user's true intention position, thereby greatly improving the accuracy of fine operations.

[0070] In some embodiments, the present invention can also analyze the micro-movement patterns of a user when the mouse stops, thereby determining whether the user is hesitating to observe or preparing to click. If the analysis shows that the user is just observing (at this time, the hand tremors are more random), the client will delay submitting the data slightly to give the user room for fine-tuning. If the analysis shows that the user has already aimed at the target and is preparing to click (at this time, the hand tremors are more rhythmic), the client will submit the data immediately to ensure the response speed of the operation.

[0071] Based on this, the method further includes: determining a time interval sequence of trajectory point changes during the period when the remote desktop mouse is in the silent state; if the user's intention is determined to be exploratory based on the time interval sequence, then delaying the transmission of the center point coordinates; if the user's intention is determined to be decisive based on the time interval sequence, then immediately transmitting the center point coordinates.

[0072] Specifically, if the intervals in the time interval sequence are inconsistent in length and lack regularity, and are relatively random, it can be determined that the user is currently observing around, indicating an exploratory intent; if the intervals in the time interval sequence are relatively consistent in length and follow a pattern, it can be determined that the user has already targeted the target and is preparing to click, indicating a decision-making intent.

[0073] In some embodiments, the type of application software currently running on the remote server can also be identified, and core parameters such as the first preset threshold, the second preset threshold, and the preset ratio can be dynamically adjusted based on the application software type. The application software type includes CAD, Word, etc. Specifically, when the application software type is computer-aided design software or a first-person shooter game, a smaller first preset threshold and a smaller second preset threshold (e.g., 1 pixel displacement and 3 pixel displacement) are used; when the identified application software type is office automation software, a larger first preset threshold and a larger second preset threshold (e.g., 3 pixel displacement and 7 pixel displacement) are used to filter out unintentional jitter and improve operational stability.

[0074] In some embodiments, to prevent the system from freezing in extreme situations (such as when a user draws a continuous circle with the mouse), this embodiment of the invention also designs a safety mechanism. If the client cannot find a clear silent boundary to segment the motion primitive within a certain period of time (e.g., 1.6 seconds), it temporarily switches back to normal mode, i.e., directly forwards all coordinate points. It automatically switches back to the optimized mode of this embodiment of the invention until a clear silent state is detected again. This ensures that the system can always work normally without problems, regardless of user operation, further guaranteeing the user experience.

[0075] In some embodiments, the operation intent information also includes high-speed movement intent and fine-grained operation intent. When sending data, a service level instruction (SLE) is generated based on the operation intent information, and this SLE is sent to the remote server along with the data. Upon receiving the SLE, the remote server can dynamically adjust its rendering strategy according to the SLE. For example, when a high-speed movement intent is received, the rendering frame rate is reduced to ensure smoothness; when a fine-grained operation intent is received, the rendering quality is improved to ensure clarity. This enables the construction of an intelligent system closed loop from user input to server rendering.

[0076] The present invention provides a remote desktop mouse data processing method that, by determining the user's operation intention information, performs data purification and reconstruction on the trajectory point sequence within each motion primitive. This method can extract the truly useful core information and send it to a remote server, thereby making the response quality of remote interaction independent of the quality of the network channel. Even when the network fluctuates, there will be no phenomena such as remote cursor drift or operation stickiness, thus ensuring a good user experience.

[0077] Furthermore, as Figure 1 and Figure 2 The specific implementation of the method shown in this embodiment provides a remote desktop mouse data processing device, such as... Figure 3 As shown, the device includes: a data acquisition unit 101, a first determination unit 102, a segmentation unit 103, a second determination unit 104, a reconstruction unit 105, and a transmission unit 106.

[0078] The acquisition unit 101 can be used to acquire the coordinates of each trajectory point of the remote desktop mouse within a preset period.

[0079] The first determining unit 102 can be used to determine the status information of the remote desktop mouse within the preset period based on the coordinates of each trajectory point.

[0080] The segmentation unit 103 can be used to segment each trajectory point into multiple motion primitives according to the state information.

[0081] The second determining unit 104 can be used to determine the operation intention information corresponding to the plurality of motion primitives respectively.

[0082] The reconstruction unit 105 can be used to perform data purification and reconstruction on the plurality of motion primitives according to the operation intention information, so as to obtain the reconstruction data corresponding to the plurality of motion primitives respectively.

[0083] The sending unit 106 can be used to send the reconstructed data to a remote server so that the remote server can reproduce the mouse trajectory.

[0084] In some embodiments, the first determining unit 102 may be specifically used to determine the pixel displacement between any two adjacent trajectory points based on the coordinates of each trajectory point; if the pixel displacement is lower than a first preset threshold, then the remote desktop mouse is determined to switch from a non-silent state to a silent state; if the pixel displacement is higher than a second preset threshold, then the remote desktop mouse is determined to switch from a silent state to a non-silent state, wherein the first preset threshold is less than the second preset threshold.

[0085] In some embodiments, the segmentation unit 103 may be specifically used to determine the sequence of trajectory points generated by the remote desktop mouse between two adjacent silent states as an independent motion primitive.

[0086] In some embodiments, the apparatus further includes a determination unit and a splitting unit.

[0087] The determination unit can be used to determine whether the remote desktop mouse has moved instantaneously based on the sequence of trajectory points within any one of the plurality of motion primitives.

[0088] The splitting unit can be used to split any one motion primitive into two new motion primitives based on the target trajectory point to which the remote desktop mouse moves instantaneously if the remote desktop mouse moves instantaneously.

[0089] In some embodiments, the second determining unit 104 may be specifically used to determine the operation intention information corresponding to the unsplit motion primitive and the new motion primitive, respectively.

[0090] In some embodiments, the operation intent information includes directional intent and descriptive intent. The second determining unit 104 can also be specifically used to determine, for any one of the plurality of motion primitives, the actual path length and straight-line distance corresponding to the arbitrary motion primitive based on the trajectory point sequence within the arbitrary motion primitive; calculate the ratio of the actual path length to the straight-line distance; if the ratio is less than a preset ratio, then determine that the arbitrary motion primitive has a directional intent; if the ratio is greater than or equal to the preset ratio, then determine that the arbitrary motion primitive has a descriptive intent.

[0091] In some embodiments, the reconstruction unit 105 may be specifically used to: if any one of the motion primitives has a directional intention, delete all trajectory points in the trajectory point sequence within the motion primitive except for the starting trajectory point and the ending trajectory point, and determine the starting trajectory point and the ending trajectory point as the reconstruction data corresponding to the motion primitive; if any one of the motion primitives has a descriptive intention, use the Douglas-Puk algorithm to thin the trajectory point sequence within the motion primitive to obtain key turning points, and determine the key turning points as the reconstruction data corresponding to the motion primitive.

[0092] In some embodiments, the apparatus further includes a computing unit.

[0093] The first determining unit 102 can also be used to determine the sequence of trajectory points during the period when the remote desktop mouse is in the silent state.

[0094] The calculation unit can be used to calculate the center point coordinates of the remote desktop mouse based on the sequence of trajectory points during the silent state.

[0095] The sending unit 106 can also be used to send the center point coordinates to the remote server so that the remote server can reproduce the mouse trajectory.

[0096] In some embodiments, the apparatus further includes an adjustment unit.

[0097] The adjustment unit can be used to identify the type of application software currently running on the remote server; and dynamically adjust the first preset threshold and the second preset threshold based on the type of application software.

[0098] In some embodiments, the sending unit 106 may be specifically used to determine a time interval sequence of trajectory point changes during the period when the remote desktop mouse is in the silent state; if the user intention is determined to be exploratory based on the time interval sequence, the center point coordinates are sent with a delay; if the user intention is determined to be decisive based on the time interval sequence, the center point coordinates are sent immediately.

[0099] It should be noted that other corresponding descriptions of the functional units involved in the remote desktop mouse data processing device provided in this embodiment can be found in [reference needed]. Figure 1 and Figure 2 The corresponding descriptions in [the document] will not be repeated here.

[0100] Based on the above, Figure 1 and Figure 2 Accordingly, this embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the above-described method.Figure 1 and Figure 2 The method for processing remote desktop mouse data is shown.

[0101] Based on this understanding, the technical solution of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as CD-ROM, USB flash drive, mobile hard drive, etc.) and includes several instructions to cause an electronic device (such as personal computer, server, or network device, etc.) to execute the methods of various implementation scenarios of this application.

[0102] Based on the above, Figure 1 and Figure 2 The method shown, and Figure 3 To achieve the above objectives, the present application also provides an electronic device, specifically a personal computer, tablet computer, server, or other network device, as shown in the virtual device embodiment. This device includes a storage medium and a processor; the storage medium stores a computer program; the processor executes the computer program to achieve the above-described objectives. Figure 1 and Figure 2 The method for processing remote desktop mouse data is shown.

[0103] Optionally, the aforementioned physical devices may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Wi-Fi interfaces), etc.

[0104] Those skilled in the art will understand that the physical device structure provided in this embodiment does not constitute a limitation on the physical device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0105] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the aforementioned physical device, supporting the operation of information processing programs and other software and / or programs. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software in the information processing physical device.

[0106] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platform, or it can be implemented by hardware.

[0107] This invention, by determining the user's operational intent information, performs data purification and reconstruction on the trajectory point sequence within each motion primitive. This allows the truly useful core information to be extracted and sent to a remote server, thereby making the response quality of remote interaction independent of the quality of the network channel. Even when the network fluctuates, phenomena such as remote cursor drift and operation stickiness will not occur, thus ensuring a good user experience.

[0108] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing this application. Those skilled in the art will understand that the modules in the apparatus of the embodiment can be distributed within the apparatus of the embodiment as described, or can be modified to be located in one or more apparatuses different from this embodiment. The modules of the above-described embodiment can be combined into one module, or further divided into multiple sub-modules.

[0109] The serial numbers in this application are for descriptive purposes only and do not represent the superiority or inferiority of any particular implementation scenario. The above disclosures are merely a few specific implementation scenarios of this application; however, this application is not limited thereto, and any variations conceived by those skilled in the art should fall within the protection scope of this application.

Claims

1. A method for processing remote desktop mouse data, characterized in that, include: Collect the coordinates of various trajectory points of the remote desktop mouse within a preset period; Based on the coordinates of each trajectory point, determine the state information of the remote desktop mouse within the preset period; Based on the state information, each trajectory point is divided into multiple motion primitives; Determine the operational intent information corresponding to each of the plurality of motion primitives; Based on the operation intent information, data purification and reconstruction are performed on the plurality of motion primitives respectively to obtain the reconstructed data corresponding to the plurality of motion primitives respectively; The reconstructed data is sent to a remote server so that the remote server can reproduce the mouse trajectory.

2. The method according to claim 1, characterized in that, Determining the state information of the remote desktop mouse within the preset period based on the coordinates of each trajectory point includes: Based on the coordinates of each trajectory point, determine the pixel displacement between any two adjacent trajectory points; If the pixel displacement is lower than the first preset threshold, then the remote desktop mouse is determined to switch from a non-silent state to a silent state. If the pixel displacement is higher than the second preset threshold, then the remote desktop mouse is determined to switch from a silent state to a non-silent state, wherein the first preset threshold is less than the second preset threshold.

3. The method according to claim 2, characterized in that, The step of dividing each trajectory point into multiple motion primitives based on the state information includes: The sequence of trajectory points generated by the remote desktop mouse between two consecutive silent states is determined as an independent motion primitive.

4. The method according to claim 1, characterized in that, After dividing the trajectory points into multiple motion primitives based on the state information, the method further includes: For any one of the plurality of motion primitives, determine whether the remote desktop mouse has moved instantaneously based on the sequence of trajectory points within the arbitrary motion primitive; If the remote desktop mouse moves instantaneously, then based on the target trajectory point to which the remote desktop mouse moves instantaneously, any one motion primitive is split into two new motion primitives; The step of determining the operation intent information corresponding to the plurality of motion primitives includes: Determine the operational intent information corresponding to the unsplit motion primitive and the new motion primitive, respectively; and / or The operational intent information includes directional intent and descriptive intent. Determining the operational intent information corresponding to each of the plurality of motion primitives includes: For any one of the plurality of motion primitives, the actual path length and straight-line distance corresponding to the arbitrary motion primitive are determined based on the sequence of trajectory points within the arbitrary motion primitive. Calculate the ratio of the actual path length to the straight-line distance; If the ratio is less than a preset ratio, then it is determined that any one of the motion elements has a directional intention; If the ratio is greater than or equal to the preset ratio, then it is determined that any one of the motion primitives has a descriptive intent.

5. The method according to claim 4, characterized in that, The step of refining and reconstructing data for the plurality of motion primitives according to the operation intent information to obtain reconstructed data corresponding to the plurality of motion primitives includes: If any of the motion primitives has a directional intention, then delete all trajectory points in the trajectory point sequence within any of the motion primitives except for the starting trajectory point and the ending trajectory point, and determine the starting trajectory point and the ending trajectory point as the reconstructed data corresponding to any of the motion primitives; If any of the motion primitives has a descriptive intent, the Douglas-Puk algorithm is used to thin the sequence of trajectory points within the motion primitive to obtain key turning points, and the key turning points are determined as the reconstructed data corresponding to the motion primitive.

6. The method according to claim 2, characterized in that, The method further includes: Determine the sequence of trajectory points of the remote desktop mouse during the period when the mouse is in the silent state; Calculate the center point coordinates of the remote desktop mouse based on the sequence of trajectory points during the silent state; Send the center point coordinates to the remote server so that the remote server can reproduce the mouse trajectory; and / or The method further includes: Identify the type of application software currently running on the remote server; Based on the application software type, the first preset threshold and the second preset threshold are dynamically adjusted.

7. The method according to claim 6, characterized in that, Sending the center point coordinates to the remote server so that the remote server can reproduce the mouse trajectory includes: Determine the time interval sequence of trajectory point changes during the period when the remote desktop mouse is in the silent state; If the user's intent is determined to be exploratory based on the time interval sequence, then the transmission of the center point coordinates is delayed. If the user's intent is determined to be decisive based on the time interval sequence, the center point coordinates are sent immediately.

8. A remote desktop mouse data processing device, characterized in that, include: The acquisition unit is used to acquire the coordinates of various trajectory points of the remote desktop mouse within a preset period; The first determining unit is used to determine the state information of the remote desktop mouse within the preset period based on the coordinates of each trajectory point; A segmentation unit is used to segment each trajectory point into multiple motion primitives based on the state information; The second determining unit is used to determine the operation intention information corresponding to the plurality of motion primitives respectively; The reconstruction unit is used to perform data purification and reconstruction on the plurality of motion primitives according to the operation intention information, so as to obtain the reconstruction data corresponding to the plurality of motion primitives respectively; The sending unit is used to send the reconstructed data to a remote server so that the remote server can reproduce the mouse trajectory.

9. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

10. An electronic device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

Images