Cursor moving method, device and equipment based on VDI client and medium

Abstract

The invention relates to a cursor moving method and device based on a VDI client, equipment and a medium. In the scheme, if the VDI client runs in the target display server protocol architecture, the target virtual input device of the VDI client is created, when the cursor movement message sent by the VDI server is detected, the absolute coordinate position is determined through the cursor movement message, and the cursor synchronization event is generated by using the absolute coordinate position. And writing the cursor synchronization event into the kernel through the target virtual input device, so that the kernel moves the position of the cursor of the client by responding to the cursor synchronization event. Visibly, through the target virtual input device, the position of the client cursor can be adjusted in time through the cursor synchronization event and the target virtual input device after the cursor movement message is sent by the VDI server; moreover, by determining the absolute coordinate position, inaccurate displacement caused by cursor acceleration can be avoided, and the accuracy of cursor displacement is improved.

Description

Technical Field

[0001] This application relates to the field of cursor movement technology, and in particular to a cursor movement method, apparatus, device and medium based on a VDI client. Background Technology

[0002] In traditional solutions, with relative cursor functionality enabled on the Linux-X11 architecture, in scenarios where the server actively moves the cursor, the client can change its cursor position via software based on cursor movement messages sent by the server. For example, in scenarios supporting cursor snapping, the server needs to automatically move the cursor to the default button in a dialog box; after the server's cursor position changes, the client's cursor needs to move accordingly. In scenarios supporting cursor feedback, to ensure compatibility with certain peripherals, the cursor position needs to be sent back to the client. Therefore, in all these scenarios, the client software must actively move the client's cursor position. Wayland is a UNIX-like display server protocol architecture. Currently, on the Linux-Wayland architecture, due to security considerations, client software is not allowed to actively move the client's cursor position. Therefore, the aforementioned cursor movement messages are blocked. In other words, on the Wayland architecture, the traditional Linux-X11 method cannot be used to actively move the client's cursor position.

[0003] Therefore, how to actively move the client cursor based on cursor movement messages sent by the server under display server protocol architectures such as Wayland is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] This application provides a cursor movement method, apparatus, device, and medium based on a VDI client, to actively move the client cursor based on cursor movement messages sent by the server under a display server protocol architecture.

[0005] Firstly, a cursor movement method based on a VDI client is provided, including:

[0006] Determine if the VDI client is running on the target display server protocol architecture;

[0007] If running on the target display server protocol architecture, then create the target virtual input device for the VDI client;

[0008] Detect whether a cursor movement message is sent by the VDI server;

[0009] If a cursor movement message exists, the absolute coordinate position is determined based on the cursor movement message;

[0010] A cursor synchronization event is generated using the absolute coordinate position, and the cursor synchronization event is written to the kernel through the target virtual input device, so that the kernel can move the position of the client cursor by responding to the cursor synchronization event.

[0011] Optionally, determining the absolute coordinate position based on the cursor movement message includes:

[0012] Determine the coordinates of the top left corner of the session window; wherein, the session window is the session window between the VDI client and the VDI server;

[0013] The absolute coordinate position is determined based on the coordinate position of the upper left corner and the movement coordinate position in the cursor movement message.

[0014] Optionally, generating a cursor synchronization event using the absolute coordinate position includes:

[0015] Construct cursor synchronization events using event types, processing operations, and absolute displacement values;

[0016] The event type is an absolute movement event, and the processing operation includes X-axis displacement operation and Y-axis displacement operation. The absolute displacement value includes the X-axis coordinate value and Y-axis coordinate value of the absolute coordinate position.

[0017] Optionally, writing the cursor synchronization event into the kernel via the target virtual input device includes:

[0018] The cursor synchronization event is written to the kernel through the device descriptor of the target virtual input device, so that the kernel can move the client cursor to the position corresponding to the absolute displacement value by responding to the cursor synchronization event.

[0019] Optionally, before generating the cursor synchronization event using the absolute coordinate position, the method further includes:

[0020] Determine whether the absolute coordinate position is within the absolute cursor movement range of the target virtual input device;

[0021] If so, then execute the step of generating a cursor synchronization event using the absolute coordinate position;

[0022] If a change in system resolution is detected, the absolute cursor movement range is updated according to the changed system resolution, and the target virtual input device of the VDI client is recreated according to the updated absolute cursor movement range.

[0023] Optionally, the cursor movement method further includes:

[0024] Based on the cursor configuration information and absolute coordinate position of the cursor to be displayed, the cursor window is cropped to include a cursor shape area; wherein, the background color of the cursor window is the same as the color of the cursor to be displayed, the cursor shape area cropped by the cursor window has the same display style as the cursor to be displayed, and the cropped area of ​​the cursor window other than the cursor shape area is hidden.

[0025] The display screen is rendered based on the image window and the cursor window with the cursor shape area cropped. The image window is the parent window, which is used to listen for all events related to the display screen. The cursor window is the child window of the image window, and the cursor window with the cursor shape area cropped is displayed on top of the image window.

[0026] Optionally, the cursor movement method includes:

[0027] Receive the updated cursor configuration information sent by the VDI server;

[0028] Save the updated cursor configuration information so that the cursor window can be cropped to the cursor shape area based on the updated cursor configuration information and absolute coordinate position.

[0029] Secondly, this application provides a cursor movement device based on a VDI client, comprising:

[0030] The first judgment module is used to determine whether the VDI client is running on the target display server protocol architecture; if it is running on the target display server protocol architecture, the creation module is triggered.

[0031] A module is created to create the target virtual input device for the VDI client.

[0032] The detection module is used to detect whether there is a cursor movement message sent by the VDI server; if there is a cursor movement message, the determination module is triggered.

[0033] The determination module is used to determine the absolute coordinate position based on the cursor movement message;

[0034] The generation module is used to generate a cursor synchronization event using the absolute coordinate position;

[0035] The writing module is used to write the cursor synchronization event to the kernel through the target virtual input device, so that the kernel can move the position of the client cursor in response to the cursor synchronization event.

[0036] Thirdly, this application provides an electronic device, comprising:

[0037] Memory, used to store computer programs;

[0038] A processor for implementing the cursor movement method described above when executing the computer program.

[0039] Fourthly, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the cursor movement method described above.

[0040] Compared with the prior art, the technical solution provided in this application has the following advantages: This application provides a cursor movement method, apparatus, device, and medium based on a VDI client. In this solution, if the VDI client runs on a target display server protocol architecture, a target virtual input device for the VDI client needs to be created. After detecting a cursor movement message sent by the VDI server, the absolute coordinate position is determined through the cursor movement message, and a cursor synchronization event is generated using the absolute coordinate position. The cursor synchronization event is written to the kernel through the target virtual input device, so that the kernel can move the position of the client cursor in response to the cursor synchronization event. Therefore, by creating a target virtual input device, this application can adjust the position of the client cursor in a timely manner after a cursor movement message is sent by the VDI server, using the cursor synchronization event and the target virtual input device. Furthermore, by determining the absolute coordinate position, this application can avoid inaccurate displacement due to cursor acceleration, thus improving the accuracy of cursor displacement. Attached Figure Description

[0041] To more clearly illustrate the technical solutions of the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 A flowchart of a cursor movement method based on a VDI client is provided for embodiments of this application;

[0043] Figure 2 A cursor movement flowchart based on the Wayland architecture is provided as an embodiment of this application;

[0044] Figure 3 This is a schematic diagram illustrating the cursor implementation provided in an embodiment of this application;

[0045] Figure 4 The CAD cursor optimization implementation process provided in the embodiments of this application;

[0046] Figure 5 A schematic diagram of a cursor movement device based on a VDI client is provided for an embodiment of this application;

[0047] Figure 6 This is a schematic diagram of an electronic device structure provided in an embodiment of this application. Detailed Implementation

[0048] This specific embodiment is merely an explanation of this application and is not intended to limit it. After reading this specification, those skilled in the art can make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they are within the scope of this application.

[0049] It should be noted that, in the optional embodiments of this application, the data related to object information, when applied to specific products or technologies, requires the permission or consent of the object. Furthermore, the collection, use, and processing of this data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. In other words, if the embodiments of this application involve data related to an object, it must be obtained with the object's authorization and consent, the authorization and consent of relevant departments, and in accordance with the relevant laws, regulations, and standards of the country and region. If the embodiments involve personal information, the acquisition of all personal information requires the individual's consent. If sensitive information is involved, the separate consent of the information subject is required. The embodiments also need to be implemented with the object's authorization and consent.

[0050] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0051] Furthermore, the term "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document, unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship. To better understand and illustrate the solutions of the embodiments of this application, some technical terms involved in the embodiments of this application are briefly explained below.

[0052] Relative cursor: In Windows systems, relative cursor displacement indicates the distance the cursor moves in the current frame relative to the distance the cursor moved in the previous frame. This mode is applicable to almost all Windows applications (cursor movement within a Windows physical machine system is relative movement); while in VDI (Virtual Desktop Infrastructure) scenarios, almost all cursors are absolute, that is, the cursor position is determined based on the absolute xy coordinates in the window / screen.

[0053] VDI (Virtual Desktop Infrastructure): A server-based computing model that virtualizes and hosts a user's desktop environment (including operating system, applications, and data) on a central server in a data center. Users can remotely access their own desktop from various terminal devices via a network.

[0054] CAD (Computer-Aided Design): CAD software is a general term for CAD design software such as AutoCAD / ZWCAD;

[0055] CAD Cursor Optimization: CAD uses a crosshair cursor, which is drawn by the software itself rather than the system cursor. Therefore, in a VDI architecture, the CAD cursor is transmitted from the server via screen feed. Due to the large amount of data transmitted and the insufficient frame rate, there is a certain delay in CAD cursor usage. The CAD cursor optimization function mainly addresses the slow and delayed cursor movement issue in VDI architectures. It identifies and hides the CAD cursor itself, then transmits it to the client via a separate cursor channel for the client to draw itself.

[0056] Wayland is a display server protocol for UNIX-like systems. Linux-Wayland aims to replace the traditional Linux-X11 system, managing graphics displays through a simpler architecture and higher security.

[0057] 3D Protocol: 3D Protocol refers to 3D Remote Display Protocol, which is a dedicated communication protocol used in virtual desktop (VDI) and remote desktop environments for the efficient and smooth transmission and display of 3D graphics application screens.

[0058] API (Application Programming Interface): API refers to application programming interface.

[0059] The VDI architecture, based on data transmission flow, can be divided into server and client. The server is typically a virtual machine (VM) on a remote server, while the client can be a local client program used by the user. A relative cursor moves the cursor based on positional changes, while an absolute cursor moves the cursor based on precise coordinates.

[0060] In existing solutions, Linux-X11 has a clear implementation of relative cursor support in order to enable VDI to support some software that relies on relative cursors (such as Lumion (real-time 3D architectural visualization software), CS (games), etc.). However, in the Linux-Wayland architecture, the server-side active cursor movement function supported after enabling relative cursors basically needs to be implemented by moving the client cursor through the client software. But in the Wayland architecture, such behavior is completely blocked by the system, and the traditional Linux-X11 method cannot be used to move the client cursor.

[0061] The server-side active cursor movement functionality includes cursor snapping and server-side cursor return functionality. Here, we'll only use cursor snapping as an example to illustrate the problems in the existing solution: In the application scenario of cursor snapping, after the client accesses the server's Windows resources, the server's system mouse properties are configured to "Automatically move the pointer to the default button in the dialog box," thus enabling cursor snapping. After enabling cursor snapping, when a user clicks on a file's properties on the client to activate the properties dialog box, the mouse automatically moves to the default button position in the properties dialog box. However, since the user is using the client's cursor, not the actual server-side virtual machine cursor, the server needs to generate a cursor movement message to notify the client to change the client's cursor position so that the client's cursor is also moved to the corresponding position, achieving the snapping effect. However, in the Wayland architecture, the client is not allowed to actively modify the client's cursor position based on cursor movement messages.

[0062] Therefore, this application provides a cursor movement method, apparatus, device, and medium based on a VDI client, which actively moves the client cursor based on cursor movement messages sent by the server under display server protocol architectures such as Wayland, ensuring the accuracy of the client cursor displacement and solving the problem of the client cursor moving automatically under display server protocol architectures such as Wayland.

[0063] See Figure 1 The flowchart of a cursor movement method based on a VDI client provided in this application embodiment includes:

[0064] S101. Determine whether the VDI client is running on the target display server protocol architecture;

[0065] If yes, then execute S102; otherwise, end the process.

[0066] In this application, the target display server protocol architecture refers to a display server protocol architecture that cannot respond to cursor movement messages sent by the server. This target display server protocol architecture can be a Wayland architecture or other types of display server protocol architectures, and is not specifically limited here. In the following embodiments, only the Wayland architecture will be used as an example for illustration.

[0067] This application takes into account that in non-target display server protocol architectures, there will be no problem of the client being unable to actively move the client cursor position. Therefore, in this embodiment, it is first necessary to determine whether the VDI client is running in a target display server protocol architecture. If it is running in a target display server protocol architecture, the subsequent target virtual input device creation process is executed. If the VDI client is not running in a target display server protocol architecture, but is running in a non-target display server protocol architecture such as linux-X11, then there is no need to create a target virtual input device, and the process ends.

[0068] The Wayland architecture can be a Linux-Wayland architecture. Furthermore, the scenario in this application where the client actively moves the client cursor position refers to a scenario where the server actively moves the server cursor due to the cursor snapping function or the server cursor return function, and the client needs to synchronously move the client to the corresponding position according to the cursor movement message sent by the server. This does not include the scenario where the user actively moves the cursor position on the client side.

[0069] S102. Create the target virtual input device for the VDI client;

[0070] In this application, to enable the client to successfully move the client cursor under the Wayland architecture, a virtual input device (uinput device) is used to simulate the cursor movement. Furthermore, based on the target virtual input device, the client can also implement functions such as keyboard / cursor key simulation; this application refers to the virtual input device of the VDI client as the target virtual input device.

[0071] In this application, in order to create a target virtual input device for a VDI client, after the client connects, it is first necessary to initialize the virtual input device context and save necessary information such as the display handle. In addition, in order to ensure that each session has its own virtual input device in a multi-client session scenario, this application needs to pre-store the session unique ID. When creating the target virtual input device, it is necessary to construct the unique device name of the target virtual input device based on the different session IDs.

[0072] Specifically, the target virtual input device context initialized in this application includes the following:

[0073] 1. Virtual input device operation mutex: ensures thread safety when using virtual input devices;

[0074] 2. Session Unique ID: A unique identifier for the session between the VDI client and the VDI server. This session unique ID is used to determine the unique device name of the target virtual input device when it is created.

[0075] 3. Cursor boundary information: Determined by the current screen resolution to prepare for creating the target virtual input device;

[0076] 4. Display handle: In Linux, this is the handle used to establish a connection with the local graphics display server. The display interface of the client system needs to use this handle, such as to obtain the system resolution.

[0077] After initializing the context, this application calls the uinput device creation interface to set the corresponding capabilities and then creates the corresponding target virtual input device. The context initialization process can be performed before determining whether the VDI client is running on a Wayland architecture, or after confirming that the VDI client is running on a Wayland architecture; it is not specifically limited here. Furthermore, the capabilities set by calling the uinput device creation interface specifically include: absolute coordinate support, absolute coordinate event response, mouse button event support, and synchronization event support, etc. Only by setting the virtual input device to have these capabilities can absolute coordinate displacement operations be supported.

[0078] If a user exits the session, the corresponding virtual input device needs to be destroyed and the virtual input device context needs to be deinitialized. This deinitialization includes destroying the mutex lock and clearing other data in the initialization context.

[0079] S103. Detect whether there is a cursor movement message sent by the VDI server;

[0080] If yes, then execute S104; otherwise, continue executing S103.

[0081] In this application, the cursor movement message is a movement message sent by the server. If the server is a virtual machine, then the cursor movement message is a cursor movement message sent within the virtual machine. This cursor movement message can also be called a cursor movement message caused by the relative cursor. Here, the cursor movement message is referred to as a cursor movement message caused by the relative cursor because cursor movement messages arise from the server's requirements for functions such as cursor snapping and server-side cursor postback. In order to support the above functions, the server needs a relative cursor to actually process the cursor. Therefore, this application refers to the cursor movement messages sent by the server due to cursor snapping, server-side cursor postback, etc., as cursor movement messages caused by the relative cursor.

[0082] In this application, after the target virtual input device is successfully created, if there is a cursor movement message sent by the VDI server relative to the cursor, the target virtual input device is used to simulate the cursor position by executing subsequent steps in order to realize the movement of the client cursor; if the cursor movement message is not detected, S103 is executed to move the client cursor in a timely manner after the cursor movement message sent by the VDI server is detected.

[0083] S104. Determine the absolute coordinate position based on the cursor movement message;

[0084] It should be noted that traditional uinput devices move via relative displacement. However, in the Wayland architecture, internal cursor acceleration is unavoidable. This acceleration often results in the cursor not moving to the expected position when moving via relative displacement. Therefore, in this application, when simulating cursor movement using the target virtual input device, it is necessary to calculate the absolute coordinate position of the screen to ensure the accuracy of the displacement by simulating absolute coordinate displacement using the target virtual input device.

[0085] Specifically, the position in the cursor movement message is the cursor's position within the session window, while the absolute coordinate position is the cursor's position on the display screen. Therefore, this application needs to determine the absolute coordinate position based on the cursor movement message in order to achieve accurate cursor displacement on the client side.

[0086] In another embodiment of this application, determining the absolute coordinate position based on the cursor movement message includes: determining the coordinate position of the upper left corner of the session window; wherein, the session window is the session window between the VDI client and the VDI server; and determining the absolute coordinate position based on the upper left corner coordinate position and the movement coordinate position in the cursor movement message.

[0087] Specifically, the cursor coordinates are two-dimensional, with a reference coordinate system of two-dimensional x / y coordinates. In the Linux system, the top-left corner is the origin of the session window (0, 0), and the bottom-right corner is the current resolution position. Therefore, this application can convert the movement coordinates in the cursor movement message into the client's actual coordinates by using the top-left corner of the session window. The absolute coordinate conversion method is as follows: add the X-axis coordinate of the movement coordinate to the X-axis coordinate of the top-left corner to obtain the X-axis coordinate of the absolute coordinate. Add the Y-axis coordinate of the movement coordinate to the Y-axis coordinate of the top-left corner to obtain the Y-axis coordinate of the absolute coordinate.

[0088] For example, if the current window's top-left corner coordinates are (100, 100), and the server sends the desired location, the cursor needs to move to (50, 60). Therefore, the client's cursor needs to move to (100+50, 100+60), which is the absolute coordinates (150, 160).

[0089] It should be noted that when simulating specific absolute displacement, this application needs to calculate the absolute coordinates of the absolute cursor based on the coordinates of the top-left corner of the session window. Therefore, this application needs to acquire and save the coordinates of the top-left corner of the session window so that the absolute coordinates can be quickly calculated when there is a cursor movement message. Thus, if this application detects a change in the coordinates of the top-left corner of the session window, it needs to update the coordinates of the top-left corner of the session window in a timely manner so that the correct absolute coordinates can be calculated next time. The coordinates of the top-left corner of the session window will change when performing operations such as window dragging, resolution changes, and maximizing / shifting the window.

[0090] S105. Generate a cursor synchronization event using absolute coordinates and write the cursor synchronization event to the kernel through the target virtual input device so that the kernel can move the position of the client cursor by responding to the cursor synchronization event.

[0091] In this application, a cursor synchronization event that the kernel can recognize needs to be generated based on the absolute coordinate position so that the kernel can move the client cursor after responding to the cursor synchronization event. Specifically, the kernel is the operating system kernel of the local computer where the VDI client resides, such as the Linux kernel. After responding to the cursor synchronization event, the kernel will move the client cursor to the absolute coordinate position determined through the above process.

[0092] In summary, by creating a target virtual input device, this application can adjust the position of the client cursor in a timely manner after the cursor movement message is sent by the VDI server through cursor synchronization events and the target virtual input device. Furthermore, by determining the absolute coordinate position, this application can avoid inaccurate displacement caused by cursor acceleration, thereby improving the accuracy of cursor displacement.

[0093] In another embodiment of this application, generating a cursor synchronization event using the absolute coordinate position includes: constructing a cursor synchronization event using an event type, a processing operation, and an absolute displacement value; wherein the event type is an absolute movement event, the processing operation includes an X-axis displacement operation and a Y-axis displacement operation, and the absolute displacement value includes the X-axis coordinate value and the Y-axis coordinate value of the absolute coordinate position.

[0094] In this application, when simulating cursor position using the target virtual input device, a cursor synchronization event that the kernel can recognize needs to be generated. This cursor synchronization event includes: event type, processing operation, and specific absolute displacement value, so as to realize the movement of the client cursor. Specifically, the event type is an absolute movement event, and the processing operation includes: X-axis displacement operation and Y-axis displacement operation. The absolute displacement value includes X-axis displacement operation value and Y-axis displacement operation value. The X-axis displacement operation value is the X-axis coordinate value of the absolute coordinate position, and the Y-axis displacement operation value is the Y-axis coordinate value of the absolute coordinate position.

[0095] In another embodiment of this application, writing the cursor synchronization event into the kernel via the target virtual input device includes: writing the cursor synchronization event into the kernel via the device descriptor of the target virtual input device, so that the kernel moves the client cursor to the position corresponding to the absolute displacement value by responding to the cursor synchronization event.

[0096] In this application, when writing the cursor synchronization event to the kernel through the target virtual input device, the write operation can be used directly to write the cursor synchronization event to the descriptor of the target virtual input device. After receiving the cursor synchronization event, the kernel can respond to the cursor synchronization event and move the client cursor to the position corresponding to the absolute displacement value.

[0097] As can be seen, this application writes cursor synchronization events to the kernel, and the kernel can accurately perform displacement operations on the client cursor based on the event type, processing operation, and absolute displacement value of the cursor synchronization events, thereby achieving precise cursor positioning.

[0098] In this embodiment, when creating the target virtual input device, an absolute cursor movement range needs to be provided. This absolute cursor movement range includes the movement range of the X-axis coordinate values ​​and the movement range of the Y-axis coordinate values. This absolute cursor movement range is strongly correlated with the system resolution. For example, if the resolution is 1K, then the absolute cursor movement range is (0,0)->(1920,1080). In another embodiment of this application, before generating a cursor synchronization event using the absolute coordinate position, the method further includes: determining whether the absolute coordinate position is within the absolute cursor movement range of the target virtual input device; if so, then performing the step of generating a cursor synchronization event using the absolute coordinate position; if a change in system resolution is detected, then updating the absolute cursor movement range according to the changed system resolution, and recreating the target virtual input device of the VDI client using the updated absolute cursor movement range.

[0099] Specifically, if both the X-axis and Y-axis coordinates of the absolute coordinate position are within the movable range of the absolute cursor, then the absolute coordinate position is within a reasonable range. For example, if the movable range of the absolute cursor is (0,0)->(1920,1080), then the movable range of the X-axis coordinate is 0 to 1920, and the movable range of the Y-axis coordinate is 0 to 1080. If the absolute coordinate position is (300,500), it can be seen that the X-axis coordinate is within the movable range of the X-axis coordinate, and the movable range of the Y-axis coordinate is also within the movable range of the Y-axis coordinate. Therefore, it is determined that the absolute coordinate position is within the movable range of the absolute cursor of the target virtual input device.

[0100] If the absolute coordinates are outside the absolute cursor's movable range on the target virtual input device, you can either refuse to generate a cursor synchronization event, or modify the absolute coordinates to fit within the cursor's movable range and generate the synchronization event based on the modified position. One modification method is to change the coordinates exceeding the cursor's movable range to the corresponding threshold value. For example, if the X-axis coordinate of the absolute position exceeds the maximum value of the movable X-axis coordinate range, then modify the X-axis coordinate of the absolute position to the maximum value of the movable X-axis coordinate range, and so on.

[0101] Furthermore, in this application, it is also necessary to detect whether the system resolution has changed. If it has changed, the absolute cursor movement range is updated with the changed system resolution to generate a new cursor boundary. The target virtual input device is then recreated with the updated absolute cursor movement range to ensure the accuracy of the cursor position simulation.

[0102] For a detailed explanation of the completion of this solution, please refer to [link / reference]. Figure 2This application provides a flowchart of cursor movement based on the Wayland architecture, which mainly includes the following steps:

[0103] After the client connects, the uinput device context is initialized, and it is determined whether it is a Wayland architecture. If not, the client waits for the user to exit the client, the uinput device context is deinitialized, and the process ends.

[0104] If so, create a uinput device and determine if there is a cursor movement message relative to the cursor; if there is a cursor movement message, simulate cursor movement through the uinput device to achieve precise cursor displacement on the client side; if there is no cursor movement message, determine if the coordinates of the top left corner of the session window have changed; if they have changed, update the top left corner coordinates of the session window and continuously check if the top left corner coordinates of the session window keep changing.

[0105] If the top-left corner coordinates of the session have not changed, determine if the system resolution has changed. If it has changed, update the absolute coordinate boundaries and rebuild the uinput device. If it has not changed, or after rebuilding the uinput device, determine if the user has exited the client. If the user has not exited the client, return to the step of determining if there is a cursor movement message relative to the cursor. If the user has exited the client, destroy the uinput device, deinitialize the uinput device context, wait for the user to exit the client, and end the process.

[0106] In summary, this application addresses the issues of client-side active cursor movement and the inaccuracy of relative displacement using the uinput device in the Wayland architecture. It proposes a method where client software utilizes the uinput device to achieve cursor movement within the Wayland architecture. To mitigate the impact of cursor acceleration on relative displacement, absolute cursor calculation is used to achieve absolute cursor displacement, ensuring accuracy. This solution enables accurate implementation of the Windows cursor snapping function and server-side cursor feedback functionality for peripheral devices within the Wayland architecture. It also provides a good solution for simulating keyboard and cursor key presses.

[0107] In another embodiment of this application, the cursor movement method further includes: cropping a cursor shape area from the cursor window according to the cursor configuration information and absolute coordinate position of the cursor to be displayed; wherein the background color of the cursor window is the same as the color of the cursor to be displayed, the cursor shape area cropped from the cursor window has the same display style as the cursor to be displayed, and the cropped area in the cursor window other than the cursor shape area is hidden.

[0108] The display screen is rendered based on the image window and the cursor window with the cursor shape area cropped. The image window is the parent window, which is used to listen for all events related to the display screen. The cursor window is the child window of the image window, and the cursor window with the cursor shape area cropped is displayed on top of the image window.

[0109] This application provides a cursor display scheme. This scheme, through parent-child windows, background filling, and cursor window clipping, achieves better cursor optimization on domestically developed PCs (Personal Computers), enabling independent cursor / image processing in dual-window environments. The cursor to be displayed using this scheme can be a standard cursor from the client or a crosshair cursor from CAD software; it is not specifically limited here, but will be illustrated using a CAD software cursor as an example.

[0110] Currently, with the support of the 3D protocol by domestic IT innovation clients, the user experience of CAD software in 3D scenarios is very important. Before this, other clients that support the 3D protocol (such as Debian (operating system) and Windows PC (operating system personal computer)) all had CAD cursor optimization functions. However, domestic IT innovation clients have encountered considerable difficulties in implementing CAD cursor optimization. Compared with Windows (a mature operating system with rich APIs) and Debian (a fixed set-top box system with no windowed scenarios), it requires more complex considerations.

[0111] In the existing solutions, the following approach can be used to implement the CAD cursor optimization function in domestically developed PC clients:

[0112] Single-window solution: The CAD cursor drawn by the client needs to be rendered together with the display screen, which is cumbersome and requires differentiation between different protocol scenarios;

[0113] Multi-window solutions include parent-child window solutions: they share a single buffer and cannot combine a cursor window (requiring 32-bit color depth) with an image window (requiring 24-bit color depth); furthermore, the parent window cannot be adjusted to be above the child window, and the parent window cannot capture events when child windows exist; temporary window (temporarily adding parent-child attributes) / independent window (no parent-child relationship) solutions: the system window manager manages one window, and the software manages another window, simulating the effect of a parent-child window, but for some domestically developed systems, certain processes of dragging / displaying windows will still distinguish two independent windows, making it impossible to achieve complete consistency with a single window.

[0114] Therefore, in this application, in order to solve the defects of various solutions in the above-mentioned multi-window scenario and make the CAD cursor experience better, this embodiment proposes a multi-window solution compared with the single-window solution. The image window and the cursor window form a parent-child window. The cursor / image are processed independently in the dual window. The cursor software cursor is optimized by filling the background of the cursor window with the cursor color, cropping the cursor window to the cursor style, and refreshing after drawing separately.

[0115] Specifically, this application requires the addition of a cursor window. The cursor window and the image window have the same lifecycle and are in a parent-child relationship. The image window acts as the parent window, listening to all events, while the cursor window, as the child window of the image window, is only responsible for cursor optimization and does not handle any events. To achieve this functionality, the event receiving function of the cursor window needs to be disabled by setting the input area (ShapeInput) of the cursor window to 0. This way, even if the cursor window is cropped to the cursor shape, mouse clicks and other events, although occurring on the cursor window, will still be transmitted to the image window.

[0116] When displaying the cursor, the cursor window needs to be cropped to the cursor shape area according to the cursor configuration information and absolute coordinate position of the cursor to be displayed. The cursor configuration information includes the color, size, shape, etc. of the client cursor. The absolute coordinate position can be the absolute coordinate position determined by the above embodiments, or it can be the absolute cursor position generated after the user actively moves the mouse or other devices to move the cursor. It is not limited here.

[0117] This application, through cursor configuration information, can crop out a cursor shape area in the cursor window that has the same display style as the cursor to be displayed. The background color of the cursor window is the same as the color of the cursor to be displayed, so the color of the cropped cursor shape area is also the same as the color of the cursor to be displayed. This application can display the cropped cursor in the correct position by using absolute coordinates. In the display screen generated by rendering based on the image window and the cursor window with the cropped cursor shape area, the cursor window with the cropped cursor shape area is displayed on top of the image window, and only the cursor shape area is displayed in the cursor window, while the cropped area in the cursor window other than the cursor shape area is hidden. Therefore, from the overall display effect of the screen, only a cursor can be seen displayed on the normal screen.

[0118] Specifically, if the cursor to be displayed is a CAD software cursor, in order to support CAD cursor optimization, this application needs to draw the CAD software cursor independently. It also needs to add a cursor window under the domestically developed PC and set the cursor window to be hidden by default. The window is only displayed when using CAD cursor optimization. When not in use, the client cursor is displayed according to the original solution.

[0119] For example, when a user uses CAD software through a client, the CAD software cursor displayed in the client is a crosshair cursor. This crosshair cursor is the object to be optimized for the CAD cursor. At this time, it is necessary to hide the real CAD cursor and use the CAD cursor drawn by the client through the cursor window. The process of the client drawing the CAD cursor through the cursor window is as follows: set the background color of the cursor window to the CAD cursor color; when the client moves the cursor, hide the real client cursor; and trim the cursor window according to the cursor position and the cursor configuration information of the CAD cursor to simulate the cursor style. This application achieves a "transparent" effect outside the cursor area by trimming the cursor window, thus eliminating the need for 32-bit color depth and requiring only 24-bit (no need to set window transparency). Therefore, it can be used as a parent-child window with the image window and immediately trigger dirty domain update rendering. When the CAD cursor is no longer used, the cursor window is hidden and the actual client cursor is displayed.

[0120] In another embodiment of this application, the cursor movement method includes: receiving updated cursor configuration information sent by a VDI server; saving the updated cursor configuration information so as to crop the cursor window to a cursor shape area based on the updated cursor configuration information and absolute coordinate position.

[0121] Specifically, to accurately display the cursor through the cursor window, this application needs to promptly save the updated cursor configuration information after receiving it from the VDI server. This allows the cursor window to be cropped to the cursor shape area for display based on the updated configuration and absolute coordinates. Furthermore, similar to the uinput device solution described above, whenever the window size or position changes, the relevant position information of the window needs to be updated to ensure accurate cursor drawing. This position information includes the coordinates of the top-left corner of the window and the overall width and height of the window. The top-left corner coordinates are used to accurately convert the client's local coordinates into the actual coordinates within the virtual machine, and the overall width and height are used to crop the correct CAD cursor shape.

[0122] See Figure 3 The figure shows a schematic diagram of the cursor implementation provided in the embodiment of this application. As shown in the figure, vdi_session (VDI-linux client session process) is the client. After the user joins the session, vdi_session creates a session window and a cursor window. When the user leaves the session, vdi_session destroys the session window and the cursor window. When the user changes the size / position of the session window through vd_session, vd_session updates the corresponding position information.

[0123] See Figure 4This document describes the CAD cursor optimization implementation process provided in this application embodiment. The `vm&spice-server` in the figure represents the server, i.e., a virtual machine using the SPICE protocol (Remote Desktop Protocol). As shown, the process includes: when a user uses / updates the CAD cursor through `vm&spice-server`, `vm&spice-server` sends the CAD cursor information to `vdi_session`. `vdi_session` stores the cursor information and fills the cursor window with the cursor color according to the cursor color. This cursor information includes: information about starting to use the CAD cursor, modified cursor configuration information, etc. The cursor configuration information includes the CAD cursor size, color, and shape. In this way, the server can notify the client that the CAD cursor information has been updated, store the latest CAD cursor configuration information, and set the cursor window background color to the corresponding CAD cursor color. When the user actively moves the CAD cursor through `vdi_session`, `vdi_session` needs to crop the cursor shape and render the cursor image to display the CAD cursor.

[0124] In summary, this application achieves better CAD cursor optimization for domestically developed PCs through the above-described solution, reducing unnecessary overhead such as cursor drawing and enabling independent processing of dual-window cursors / images under domestically developed PCs. Specifically, due to the numerous window behavior issues with independent windows and the poor performance of self-control, this application delegates the parent-child window mode to the client. The color depth of the parent and child window images needs to be consistent, but when the color depth is consistent, the cursor window cannot be transparent. Therefore, this application simulates the transparency effect by cropping the cursor window. For example, when the CAD cursor is not used, the cursor window is completely cropped (invisible). When the CAD cursor is used, the cursor window is cropped to the corresponding cursor shape each time to achieve cursor display.

[0125] Furthermore, since parent and child windows share the same buffer, traditional cursor data rendering schemes would overwrite image data. However, in this scheme, when cursor display is achieved through clipping, the clipped area of ​​the cursor window becomes the cursor shape area, i.e., the area where the cursor is displayed. The remaining areas of other cursor windows, after being clipped, are not drawn in the buffer; image data can still be drawn in these areas, requiring only a notification to the screen to trigger rendering. Moreover, in the cursor window clipping scheme described in this application, the cursor does not need to be drawn frame by frame. For example, when setting the CAD cursor configuration, the background of the entire cursor window can be directly set to the color of the CAD cursor. When moving the cursor, only clipping of the cursor is needed, reducing overhead by 3%-6% compared to frame-by-frame drawing.

[0126] The cursor movement device provided in the embodiments of this application is described below. The cursor movement device described below and the cursor movement method described above can be referred to each other.

[0127] See Figure 5 , Figure 5 A schematic diagram of a cursor movement device based on a VDI client is provided for embodiments of this application. The device specifically includes:

[0128] The first judgment module 11 is used to determine whether the VDI client is running on the target display server protocol architecture; if it is running on the target display server protocol architecture, the creation module is triggered.

[0129] Module 12 is used to create the target virtual input device for the VDI client;

[0130] The detection module 13 is used to detect whether there is a cursor movement message sent by the VDI server; if there is a cursor movement message, the determination module is triggered.

[0131] The determination module 14 is used to determine the absolute coordinate position based on the cursor movement message;

[0132] Generation module 15 is used to generate a cursor synchronization event using the absolute coordinate position;

[0133] The writing module 16 is used to write the cursor synchronization event to the kernel through the target virtual input device, so that the kernel can move the position of the client cursor in response to the cursor synchronization event.

[0134] As an optional embodiment, the determining module is specifically used to: determine the coordinate position of the upper left corner of the session window; wherein, the session window is the session window between the VDI client and the VDI server; and determine the absolute coordinate position based on the upper left corner coordinate position and the movement coordinate position in the cursor movement message.

[0135] As an optional embodiment, the generation module is specifically used for:

[0136] A cursor synchronization event is constructed using event type, processing operation, and absolute displacement value; wherein, the event type is an absolute movement event, the processing operation includes: X-axis displacement operation and Y-axis displacement operation, and the absolute displacement value includes: the X-axis coordinate value and Y-axis coordinate value of the absolute coordinate position.

[0137] As an optional embodiment, the writing module is specifically used for:

[0138] The cursor synchronization event is written to the kernel through the device descriptor of the target virtual input device, so that the kernel can move the client cursor to the position corresponding to the absolute displacement value by responding to the cursor synchronization event.

[0139] As an optional embodiment, the device further includes:

[0140] The second judgment module is used to determine whether the absolute coordinate position is within the absolute cursor movement range of the target virtual input device; if so, the generation module is triggered to generate a cursor synchronization event using the absolute coordinate position.

[0141] The update module is used to update the absolute cursor movement range according to the changed system resolution when a change in system resolution is detected, so that the creation module can recreate the target virtual input device of the VDI client with the updated absolute cursor movement range.

[0142] As an optional embodiment, the device further includes:

[0143] The clipping module is used to clip the cursor window to a cursor shape area based on the cursor configuration information and absolute coordinate position of the cursor to be displayed; wherein, the background color of the cursor window is the same as the color of the cursor to be displayed, the cursor shape area clipped by the cursor window has the same display style as the cursor to be displayed, and the clipped area of ​​the cursor window other than the cursor shape area is hidden.

[0144] The rendering module is used to render and generate a display screen based on the image window and the cursor window with the cursor shape area cropped out. The image window is the parent window, which is used to listen for all events related to the display screen. The cursor window is the child window of the image window, and the cursor window with the cursor shape area cropped out is displayed on top of the image window.

[0145] As an optional embodiment, the device further includes:

[0146] The saving module is used to receive the updated cursor configuration information sent by the VDI server and save the updated cursor configuration information so that the cursor window can be cropped to the cursor shape area based on the updated cursor configuration information and absolute coordinate position.

[0147] Figure 6 A structural diagram of an electronic device provided in an embodiment of the present invention is shown in the figure, comprising:

[0148] Memory 20 is used to store computer programs;

[0149] The processor 21 is configured to implement the steps of the cursor movement method as described in the above embodiments when executing a computer program.

[0150] The electronic devices provided in this embodiment may include, but are not limited to, smartphones, tablets, laptops, or desktop computers.

[0151] The processor 21 may include one or more processing cores, such as a quad-core processor or an octa-core processor. The processor 21 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 21 may also include a main processor and a coprocessor. The main processor, also known as the Central Processing Unit (CPU), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, the processor 21 may integrate a Graphics Processing Unit (GPU), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, the processor 21 may also include an Artificial Intelligence (AI) processor, which handles computational operations related to machine learning.

[0152] The memory 20 may include one or more computer-readable storage media, which may be non-transitory. The memory 20 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In this embodiment, the memory 20 is used to store at least the following computer program 201, which, after being loaded and executed by the processor 21, is capable of implementing the relevant steps of the cursor movement method disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, and the storage method may be temporary storage or permanent storage. The operating system 202 may include Windows, Unix, Linux, etc.

[0153] In some embodiments, the electronic device may further include a display screen 22, an input / output interface 23, a communication interface 24, a power supply 25, and a communication bus 26.

[0154] Those skilled in the art will understand that Figure 6 The structures shown do not constitute a limitation on electronic devices and may include more or fewer components than those shown.

[0155] In another exemplary embodiment, a computer storage medium is also provided, wherein the program instructions, when executed by a processor, implement the steps of the data deduplication method described in any of the above method embodiments.

[0156] It is understood that if the cursor movement method in the above embodiments is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the current technology, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods in the various embodiments of the present invention. The aforementioned storage medium includes: USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), electrically erasable programmable ROM, register, hard disk, removable disk, CD-ROM, magnetic disk, or optical disk, and other media capable of storing program code.

[0157] The various embodiments described in this specification are presented in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” used herein may also mean the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a specific order described or illustrated, unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0158] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0159] The above are only some embodiments of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A cursor movement method based on a VDI client, characterized in that, include: Determine if the VDI client is running on the target display server protocol architecture; If running on the target display server protocol architecture, then create the target virtual input device for the VDI client; Detect whether a cursor movement message is sent by the VDI server; If a cursor movement message exists, the absolute coordinate position is determined based on the cursor movement message; A cursor synchronization event is generated using the absolute coordinate position, and the cursor synchronization event is written to the kernel through the target virtual input device, so that the kernel can move the position of the client cursor by responding to the cursor synchronization event.

2. The cursor movement method according to claim 1, characterized in that, Determining the absolute coordinate position based on the cursor movement message includes: Determine the coordinates of the top left corner of the session window; wherein, the session window is the session window between the VDI client and the VDI server; The absolute coordinate position is determined based on the coordinate position of the upper left corner and the movement coordinate position in the cursor movement message.

3. The cursor movement method according to claim 1, characterized in that, Generating cursor synchronization events using the absolute coordinates includes: Construct cursor synchronization events using event types, processing operations, and absolute displacement values; The event type is an absolute movement event, and the processing operation includes X-axis displacement operation and Y-axis displacement operation. The absolute displacement value includes the X-axis coordinate value and Y-axis coordinate value of the absolute coordinate position.

4. The cursor movement method according to claim 3, characterized in that, Writing the cursor synchronization event to the kernel via the target virtual input device includes: The cursor synchronization event is written to the kernel through the device descriptor of the target virtual input device, so that the kernel can move the client cursor to the position corresponding to the absolute displacement value by responding to the cursor synchronization event.

5. The cursor movement method according to claim 3, characterized in that, Before generating the cursor synchronization event using the absolute coordinate position, the process also includes: Determine whether the absolute coordinate position is within the absolute cursor movement range of the target virtual input device; If so, then execute the step of generating a cursor synchronization event using the absolute coordinate position; If a change in system resolution is detected, the absolute cursor movement range is updated according to the changed system resolution, and the target virtual input device of the VDI client is recreated according to the updated absolute cursor movement range.

6. The cursor movement method according to any one of claims 1 to 5, characterized in that, The cursor movement method further includes: Based on the cursor configuration information and absolute coordinate position of the cursor to be displayed, the cursor window is cropped to include a cursor shape area; wherein, the background color of the cursor window is the same as the color of the cursor to be displayed, the cursor shape area cropped by the cursor window has the same display style as the cursor to be displayed, and the cropped area of ​​the cursor window other than the cursor shape area is hidden. The display screen is rendered based on the image window and the cursor window with the cursor shape area cropped. The image window is the parent window, which is used to listen for all events related to the display screen. The cursor window is the child window of the image window, and the cursor window with the cursor shape area cropped is displayed on top of the image window.

7. The cursor movement method according to claim 6, characterized in that, The cursor movement method includes: Receive the updated cursor configuration information sent by the VDI server; Save the updated cursor configuration information so that the cursor window can be cropped to the cursor shape area based on the updated cursor configuration information and absolute coordinate position.

8. A cursor movement device based on a VDI client, characterized in that, include: The first judgment module is used to determine whether the VDI client is running on the target display server protocol architecture; If running on the target display server protocol architecture, module creation will be triggered; A module is created to create the target virtual input device for the VDI client. The detection module is used to detect whether there are cursor movement messages sent by the VDI server; If a cursor movement message exists, the confirmation module is triggered; The determination module is used to determine the absolute coordinate position based on the cursor movement message; The generation module is used to generate a cursor synchronization event using the absolute coordinate position; The writing module is used to write the cursor synchronization event to the kernel through the target virtual input device, so that the kernel can move the position of the client cursor in response to the cursor synchronization event.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the cursor movement method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the cursor movement method as described in any one of claims 1 to 7.

Images