Touch screen redirection method, device, electronic device and storage medium

Abstract

The present application provides a touch screen redirection method, device, electronic device, and storage medium. The method includes: reading touch data from the touch screen; determining whether the operation display form mapped by the current touch operation satisfies a preset gesture through the touch data; if so, switching the current display interface to the interface to be displayed; if not, determining whether the current display interface is a virtual machine interface; if so, sending the touch data to a server via a preset communication protocol, so that the virtual machine in the server responds to the touch operation and performs a corresponding action; if not, sending the touch data to a designated local program in the thin client, so that the designated local program responds to the touch operation and performs a corresponding action. The present application can solve the problem of not being able to freely switch between the virtual machine interface and the thin client graphical interface when using a touch screen device, so that the touch screen device can control both the virtual machine and the thin client, and switch at will.

Description

Technical Field

[0001] The present application relates to the technical field of electronic digital data processing, and in particular to a touch screen redirection method, device, electronic device and storage medium. Background Art

[0002] When using a thin client to implement a virtual cloud desktop using QEMU+KVM technology through the SPICE protocol, if a touch screen device is used as the display operation device of the thin client, the following technical problems will be encountered when the touch screen directly redirects data to the virtual machine through the USB port: the touch screen device is redirected to the virtual machine through the USB port and directly interacts using the general USB protocol, which makes it inconvenient to intercept and process the touch screen device data.

[0003] Therefore, after being redirected to the virtual machine, the local graphical interface of the thin client cannot obtain the touch screen device data, which results in the touch screen device being unable to operate the thin client when the thin client's own graphical interface needs to be operated. Summary of the Invention

[0004] In view of this, the embodiments of the present application provide a touch screen redirection method, device, electronic device and storage medium, which can solve the problem of not being able to freely switch between the virtual machine interface and the thin client graphical interface when using a touch screen device, so that the touch screen device can control both the virtual machine and the thin client, and switch at will.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a touch screen redirection method, comprising the following steps:

[0007] In response to a touch operation on a touch screen, reading touch data of the touch screen, wherein the touch screen is connected to a thin client via a USB interface;

[0008] Determining whether the operation display form of the current touch operation mapping satisfies a preset gesture through the touch data; the preset gesture is used to switch the display interface of the thin client; the display interface includes a local interface or a virtual machine interface, and the virtual machine interface is a cloud desktop;

[0009] If satisfied, determine the interface to be displayed according to the current display interface of the touch screen, and switch the current display interface to the interface to be displayed;

[0010] If not, determine whether the current display interface is a virtual machine interface. If so, send the touch data to the server through a preset communication protocol so that the virtual machine in the server responds to the touch operation and performs a corresponding action; if not, send the touch data to a specified local program in the thin client so that the specified local program responds to the touch operation and performs a corresponding action.

[0011] In a possible implementation, the preset communication protocol is an extended SPICE protocol, which is used to encapsulate the touch data. The extended SPICE protocol is established in the following manner:

[0012] defining a target command and a target data structure in a SPICE protocol, wherein the target command is used to identify and send the target data structure, and the target data structure includes the touch data;

[0013] The SPICE protocol that defines the target command and the target data structure is used as the extended SPICE protocol.

[0014] In one possible implementation, the server is preset with a simulated touch screen module, the simulated touch screen module simulates the touch screen based on a USB device framework in QEMU, the virtual machine is preset with a touch screen driver, and the touch data is sent to the server via a preset communication protocol so that the virtual machine in the server can perform corresponding actions based on the touch data, including:

[0015] Encapsulating the touch data using a SPICE client in the thin client to obtain encapsulated data;

[0016] Sending the encapsulated data to the SPICE server in the server through the extended SPICE protocol;

[0017] The packaged data is parsed and processed by the simulated touch screen module to obtain a touch instruction;

[0018] The touch screen instruction is executed by the touch screen driver, so that the virtual machine performs corresponding actions based on the touch screen instruction.

[0019] In a possible implementation, before reading touch data of the touch screen in response to a touch operation on the touch screen, the method further includes:

[0020] In response to a startup operation of the virtual machine, mounting the simulated touch screen module to the server so that the virtual machine can recognize the touch screen;

[0021] The touch screen driver is loaded to enable the virtual machine to interact with the simulated touch screen module.

[0022] In a possible implementation, before reading touch data of the touch screen in response to a touch operation on the touch screen, the method further includes:

[0023] Determining, by the thin client, the type of device connected to the USB interface;

[0024] If the device type is the touch screen, the touch data is intercepted and processed so that the thin client forwards the touch data.

[0025] In a possible implementation, determining whether the operation display form of the current touch operation mapping satisfies a preset gesture through the touch data includes:

[0026] The touch data is compared with the gesture data corresponding to the preset gesture. If the similarity between the touch data and the gesture data is greater than or equal to a preset matching threshold, the operation display form mapped to the current touch operation is determined to be the preset gesture.

[0027] In a possible implementation, the touch screen has a multi-touch function, and the method further includes:

[0028] In response to a multi-touch operation on the touch screen, reading multi-touch data of the touch screen;

[0029] The local interface and local program of the virtual machine and / or the thin client are operated based on the multi-touch data.

[0030] In a second aspect, an embodiment of the present application further provides a touch screen redirection device, the device comprising:

[0031] a reading module, configured to read touch data of the touch screen in response to a touch operation on the touch screen, wherein the touch screen is connected to the thin client via a USB interface;

[0032] a judgment module, configured to judge, based on the touch data, whether the operation display form of the current touch operation mapping satisfies a preset gesture; the preset gesture is used to switch the display interface of the thin client; the display interface includes a local interface or a virtual machine interface, and the virtual machine interface is a cloud desktop;

[0033] If satisfied, determine the interface to be displayed according to the current display interface of the touch screen, and switch the current display interface to the interface to be displayed;

[0034] If not, determine whether the current display interface is a virtual machine interface. If so, send the touch data to the server through a preset communication protocol so that the virtual machine in the server responds to the touch operation and performs a corresponding action; if not, send the touch data to a specified local program in the thin client so that the specified local program responds to the touch operation and performs a corresponding action.

[0035] In a third aspect, an embodiment of the present application further provides an electronic device comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the storage medium through the bus, and the processor executes the machine-readable instructions to execute the touch screen redirection method described in any one of the first aspects.

[0036] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the touch screen redirection method described in any one of the first aspects is executed.

[0037] The embodiments of the present application have the following beneficial effects:

[0038] 1. You can freely switch and control the local graphical interface and the remote virtual machine interface through the touch screen without an external mouse or keyboard.

[0039] 2. Since the software simulates a universal touch screen device, it is compatible with a variety of virtual machines.

[0040] 3. Support for multi-touch operations: Since the simulated touch screen supports multi-touch data transmission, the corresponding specific content is the device descriptor marking the multi-touch device, and data supports batch transmission. Therefore, the connected physical touch screen supports multi-touch and, after redirection, also supports multi-touch operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0042] Figure 1 101 - 102 is a flowchart of steps S101-S102 provided in an embodiment of the present application;

[0043] Figure 2 It is a flowchart of steps S201-S202 provided in an embodiment of the present application;

[0044] Figure 3 Schematic diagram of the process of steps S301-S304 provided in the embodiment of the present application;

[0045] Figure 4 4 is a flow chart of steps S401-S402 provided in an embodiment of the present application;

[0046] Figure 5 It is a flowchart of steps S501-S502 provided in an embodiment of the present application;

[0047] Figure 6 Schematic diagram of the process of steps S601-S602 provided in an embodiment of the present application;

[0048] Figure 7 This is a block diagram of the touch screen redirection principle provided by an embodiment of the present application;

[0049] Figure 8 This is a touch screen redirection flow chart provided in an embodiment of the present application;

[0050] Figure 9 This is a schematic diagram of the structure of a touch screen redirection device provided in an embodiment of the present application;

[0051] Figure 10 It is a schematic diagram of the composition structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of illustration and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowcharts can be implemented out of sequence, and steps without logical context can be reversed or implemented simultaneously. In addition, those skilled in the art, under the guidance of the contents of this application, can add one or more other operations to the flowchart, or remove one or more operations from the flowchart.

[0053] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it will be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0054] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0055] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0056] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0057] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. The terms used herein are for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0058] Before further explaining the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0059] QEMU: is a general-purpose open-source computer emulator and virtualizer.

[0060] SPICE: is a virtual desktop transmission protocol.

[0061] KVM: is an open source system virtual machine module.

[0062] Redirection: In a private cloud desktop, the client's device is passed to the virtual machine to which it is connected through the network.

[0063] Thin Client: A desktop computer without a hard drive that is powered by a thin client operating system rather than another operating system like Windows or Linux. These fanless computers do not store sensitive data and applications locally, but instead store them locally or in the cloud.

[0064] See also Figure 1 , Figure 1This is a flow chart of steps S101-S102 of the touch screen redirection method provided in the embodiment of the present application, which will be combined with Figure 1 Steps S101-S102 are shown for explanation.

[0065] Step S101, in response to a touch operation on a touch screen, reading touch data of the touch screen, wherein the touch screen is connected to a thin client via a USB interface;

[0066] Step S102: determining, based on the touch data, whether the operation display form of the current touch operation mapping satisfies a preset gesture; the preset gesture is used to switch the display interface of the thin client; the display interface includes a local interface or a virtual machine interface, and the virtual machine interface is a cloud desktop;

[0067] If satisfied, determine the interface to be displayed according to the current display interface of the touch screen, and switch the current display interface to the interface to be displayed;

[0068] If not, determine whether the current display interface is a virtual machine interface. If so, send the touch data to the server through a preset communication protocol so that the virtual machine in the server responds to the touch operation and performs a corresponding action; if not, send the touch data to a specified local program in the thin client so that the specified local program responds to the touch operation and performs a corresponding action.

[0069] The above touch screen redirection method has the following beneficial effects:

[0070] 1. You can freely switch and control the local graphical interface and the remote virtual machine interface through the touch screen without an external mouse or keyboard.

[0071] 2. Since the software simulates a universal touch screen device, it is compatible with a variety of virtual machines.

[0072] 3. Support for multi-touch operations: Since the simulated touch screen supports multi-touch data transmission, the corresponding specific content is the device descriptor marking the multi-touch device, and data supports batch transmission. Therefore, the connected physical touch screen supports multi-touch and, after redirection, also supports multi-touch operations.

[0073] The above exemplary steps of the embodiment of the present application are described below respectively.

[0074] In step S101, in response to a touch operation on a touch screen, touch data of the touch screen is read, wherein the touch screen is connected to a thin client via a USB interface.

[0075] In step S102, it is determined whether the operation display form of the current touch operation mapping satisfies a preset gesture based on the touch data; the preset gesture is used to switch the display interface of the thin client; the display interface includes a local interface or a virtual machine interface, and the virtual machine interface is a cloud desktop;

[0076] If satisfied, determine the interface to be displayed according to the current display interface of the touch screen, and switch the current display interface to the interface to be displayed;

[0077] If not, determine whether the current display interface is a virtual machine interface. If so, send the touch data to the server through a preset communication protocol so that the virtual machine in the server responds to the touch operation and performs a corresponding action; if not, send the touch data to a specified local program in the thin client so that the specified local program responds to the touch operation and performs a corresponding action.

[0078] See also Figure 7 and Figure 8 , Figure 7 This is a block diagram of the touch screen reset principle provided by an embodiment of the present application. Figure 8 This is a touch screen redirection flow chart provided by an embodiment of the present application, such as Figure 7 As shown, this system is divided into a server host and a thin client. The thin client is connected to a physical touch screen device through a USB interface. The thin client is equipped with a touch screen reading module and a gesture processing and forwarding module. The gesture processing and forwarding module can directly control the local graphical interface (it can switch the display interface or operate the local program), or it can send the touch data to the SPICE client in the server host through the extended SPICE protocol after switching to the virtual machine interface, and then operate the application in the client OS system (that is, the virtual machine) through the simulated touch screen module and the touch screen driver.

[0079] like Figure 8 As shown, after reading the touch screen data (i.e., touch data), gesture determination is performed. If it is a gesture (the gesture here refers to "switch display interface", such as sliding to the right), the corresponding action is performed to switch the display interface; if not, it is determined again whether it is a virtual machine interface. If it is a virtual machine interface, the virtual machine OS (i.e., the guest OS system) is controlled by simulating the touch screen module. If it is not a virtual machine interface, it is considered to be an operation on the local interface. Accordingly, the touch data is sent to the local graphical interface receiving module (i.e. Figure 7 ), and perform corresponding graphical interface operations.

[0080] In some embodiments, see Figure 2 , Figure 2This is a flow chart of steps S201-S202 provided in an embodiment of the present application. The preset communication protocol is the extended SPICE protocol, which is used to encapsulate the touch data. The extended SPICE protocol is established through steps S201-S202 and will be described in conjunction with each step.

[0081] In step S201 , a target command and a target data structure are defined in the SPICE protocol, wherein the target command is used to identify and send the target data structure, and the target data structure includes the touch data.

[0082] In step S202 , the SPICE protocol that defines the target command and the target data structure is used as the extended SPICE protocol.

[0083] Here, new commands and new data structures are defined in the SPICE protocol to enable the transmission of encapsulated touch data between the thin client and the server Host through the Input channel. For example:

[0084] New command Id: SPICE_MSGC_INPUT_TOUCHSCREEN_MOTION (115).

[0085] New data structure:

[0086]

[0087] Here, unit represents an unsigned integer, and the number after unit represents the number of bits, for example, 32 represents 32 bits, num_of_motions represents the number of touch operation actions, data_size represents the size of the touch data, and data represents the specific touch data. For specific touch data, touch represents whether there is contact (1 for yes, 0 for no), valid represents whether it is valid (1 for yes, 0 for no), inrange represents whether it is within the range (1 for yes, 0 for no), reseve represents the reserved number, id is the id of the touch data, x represents the touch value on the x-axis, and y represents the touch value on the y-axis.

[0088] In some embodiments, see Figure 3 , Figure 3This is a flow chart of steps S301-S304 provided in an embodiment of the present application. A simulated touch screen module is preset in the server, and the simulated touch screen module simulates the touch screen based on the USB device framework in QEMU. A touch screen driver is preset in the virtual machine. The touch data is sent to the server through a preset communication protocol so that the virtual machine in the server can perform corresponding actions based on the touch data. This can be achieved through steps S301-S304, which will be explained in combination with each step.

[0089] In step S301, the touch data is packaged and processed by the SPICE client in the thin client to obtain packaged data.

[0090] In step S302, the encapsulated data is sent to the SPICE server in the server via the extended SPICE protocol.

[0091] In step S303, the packaged data is parsed and processed by the simulated touch screen module to obtain a touch instruction.

[0092] In step S304, the touch screen instruction is executed by the touch screen driver, so that the virtual machine performs corresponding actions based on the touch screen instruction.

[0093] like Figure 7 As shown, the encapsulated data is sent to the SPICE server by extending the SPICE protocol, and then the simulated touch screen module can read the data and pass it to the touch screen driver, thereby executing corresponding actions in the client OS system (virtual machine).

[0094] In some embodiments, see Figure 4 , Figure 4 This is a flow chart of steps S401-S402 provided in an embodiment of the present application. In response to a touch operation on the touch screen, steps S401-S402 may be further executed before reading the touch data of the touch screen, which will be described in conjunction with specific steps.

[0095] In step S401, in response to the startup operation of the virtual machine, the simulated touch screen module is mounted to the server, so that the virtual machine can recognize the touch screen.

[0096] In step S402, the touch screen driver is loaded to enable the virtual machine to interact with the simulated touch screen module.

[0097] Here, based on the USB device framework in QEMU, a standard HID touch screen device is simulated (i.e., a simulated touch screen module is obtained), and it is mounted into the system when the virtual machine starts, so that the virtual machine can recognize the simulated touch screen and automatically load the standard touch screen driver to interact with the simulated touch screen module.

[0098] In some embodiments, see Figure 5 , Figure 5 This is a flow chart of steps S501-S502 provided in an embodiment of the present application. In response to a touch operation on the touch screen, steps S501-S502 may be further executed before reading the touch data of the touch screen, which will be described in conjunction with specific steps.

[0099] In step S501, the thin client determines the type of device connected to the USB interface.

[0100] In step S502, if the device type is the touch screen, the touch data is intercepted and processed so that the thin client forwards the touch data.

[0101] Here, the thin client intercepts devices such as touch screens by determining the specific USB device type, allowing the thin client to read local data from the touch screen, and then forwards the touch data according to the current display content.

[0102] In some embodiments, determining whether the operation display form of the current touch operation mapping satisfies a preset gesture through the touch data includes:

[0103] The touch data is compared with the gesture data corresponding to the preset gesture. If the similarity between the touch data and the gesture data is greater than or equal to a preset matching threshold, the operation display form mapped to the current touch operation is determined to be the preset gesture.

[0104] Here, the preset gesture can be a screen switching gesture, a page turning gesture, or a zoom in / out gesture. The preset gesture corresponds to a set of standard gesture data. When judging whether the operation display form mapped by the previous touch operation satisfies the preset gesture, the standard gesture data is compared with the touch data to determine whether the current touch operation satisfies the corresponding preset gesture. For example, if the similarity between the gesture data and the touch data is greater than 90%, it is considered that the operation display form mapped by the current touch operation satisfies the preset gesture.

[0105] In some embodiments, the touch data includes X-axis data and Y-axis data, and the gesture is defined by the following rules:

[0106] performing detection processing on the X-axis data and the Y-axis data in the touch data;

[0107] If the fluctuation range of the Y-axis data is smaller than a preset fluctuation threshold, and the X-axis data is in an increasing state, the touch data is determined to be the gesture.

[0108] Here, the touch screen device is operated, and the touch screen reading module in the thin client can read the corresponding touch data, which is composed of some point coordinates, press identification, data point ID and other information;

[0109] A specific gesture is composed of a set of regular touch data. Multiple sets of touch data are recorded and then compared to form a gesture if all the rules are met. For example, a right swipe gesture can be understood as a set of touch data with basically equal Y values ​​and increasing X values.

[0110] In some embodiments, see Figure 6 , Figure 6 This is a flow chart of steps S601-S602 provided in an embodiment of the present application. The touch screen has a multi-touch function, and the method can also execute steps S601-S602, which will be explained in conjunction with specific steps.

[0111] In step S601, in response to a multi-touch operation on the touch screen, multi-touch data of the touch screen is read.

[0112] In step S602, the local interface and local program of the virtual machine and / or the thin client are operated based on the multi-touch data.

[0113] Here, since the simulated touch screen supports multi-touch data transmission, the corresponding specific content is the device descriptor marking the multi-touch device, and the data supports batch transmission. Therefore, the connected physical touch screen supports multi-touch and, after redirection, also supports multi-touch operations.

[0114] In summary, the embodiments of the present application have the following beneficial effects:

[0115] 1. You can freely switch and control the local graphical interface and the remote virtual machine interface through the touch screen without an external mouse or keyboard.

[0116] 2. Since the software simulates a universal touch screen device, it is compatible with a variety of virtual machines.

[0117] 3. Support for multi-touch operations: Since the simulated touch screen supports multi-touch data transmission, the corresponding specific content is the device descriptor marking the multi-touch device, and data supports batch transmission. Therefore, the connected physical touch screen supports multi-touch and, after redirection, also supports multi-touch operations.

[0118] Based on the same inventive concept, the embodiments of the present application also provide a touch screen redirection device corresponding to the touch screen redirection method in the first embodiment. Since the principle of solving the problem by the device in the embodiments of the present application is similar to that of the above-mentioned touch screen redirection method, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0119] like Figure 9 As shown, Figure 9 FIG1 is a schematic diagram of the structure of a touch screen redirection device 900 provided in an embodiment of the present application. The touch screen redirection device 900 includes:

[0120] The reading module 901 is configured to read touch data of the touch screen in response to a touch operation on the touch screen, wherein the touch screen is connected to the thin client via a USB interface.

[0121] A judgment module 902 determines, based on the touch data, whether the operation display form of the current touch operation mapping satisfies a preset gesture; the preset gesture is used to switch the display interface of the thin client; the display interface includes a local interface or a virtual machine interface, and the virtual machine interface is a cloud desktop;

[0122] If satisfied, determine the interface to be displayed according to the current display interface of the touch screen, and switch the current display interface to the interface to be displayed;

[0123] If not, determine whether the current display interface is a virtual machine interface. If so, send the touch data to the server through a preset communication protocol so that the virtual machine in the server responds to the touch operation and performs a corresponding action; if not, send the touch data to a specified local program in the thin client so that the specified local program responds to the touch operation and performs a corresponding action.

[0124] Those skilled in the art should understand that Figure 9 The functions implemented by each unit in the touch screen redirection device 900 can be understood by referring to the related description of the touch screen redirection method mentioned above. Figure 9 The functions of the various units in the touch screen redirection device 900 may be implemented by a program running on a processor, or by a specific logic circuit.

[0125] In a possible implementation, the preset communication protocol is an extended SPICE protocol, which is used to encapsulate the touch data. The determination module 902 establishes the extended SPICE protocol in the following manner:

[0126] defining a target command and a target data structure in a SPICE protocol, wherein the target command is used to identify and send the target data structure, and the target data structure includes the touch data;

[0127] The SPICE protocol that defines the target command and the target data structure is used as the extended SPICE protocol.

[0128] In one possible implementation, a simulated touch screen module is preset in the server, and the simulated touch screen module simulates the touch screen based on the USB device framework in QEMU. A touch screen driver is preset in the virtual machine. The judgment module 902 sends the touch data to the server through a preset communication protocol, so that the virtual machine in the server can perform corresponding actions based on the touch data, including:

[0129] Encapsulating the touch data using a SPICE client in the thin client to obtain encapsulated data;

[0130] Sending the encapsulated data to the SPICE server in the server through the extended SPICE protocol;

[0131] The packaged data is parsed and processed by the simulated touch screen module to obtain a touch instruction;

[0132] The touch screen instruction is executed by the touch screen driver, so that the virtual machine performs corresponding actions based on the touch screen instruction.

[0133] In a possible implementation, before the reading module 901 reads touch data of the touch screen in response to a touch operation on the touch screen, the method further includes:

[0134] In response to a startup operation of the virtual machine, mounting the simulated touch screen module to the server so that the virtual machine can recognize the touch screen;

[0135] The touch screen driver is loaded to enable the virtual machine to interact with the simulated touch screen module.

[0136] In a possible implementation, before the reading module 901 reads touch data of the touch screen in response to a touch operation on the touch screen, the method further includes:

[0137] Determining, by the thin client, the type of device connected to the USB interface;

[0138] If the device type is the touch screen, the touch data is intercepted and processed so that the thin client forwards the touch data.

[0139] In a possible implementation, the determination module 902 determines whether the operation display form of the current touch operation mapping satisfies a preset gesture based on the touch data, including:

[0140] The touch data is compared with the gesture data corresponding to the preset gesture. If the similarity between the touch data and the gesture data is greater than or equal to a preset matching threshold, the operation display form mapped to the current touch operation is determined to be the preset gesture.

[0141] In a possible implementation, the touch screen has a multi-touch function, and the determination module 902 further includes:

[0142] In response to a multi-touch operation on the touch screen, reading multi-touch data of the touch screen;

[0143] The local interface and local program of the virtual machine and / or the thin client are operated based on the multi-touch data.

[0144] The above touch screen redirection device has the following beneficial effects:

[0145] 1. You can freely switch and control the local graphical interface and the remote virtual machine interface through the touch screen without an external mouse or keyboard.

[0146] 2. Since the software simulates a universal touch screen device, it is compatible with a variety of virtual machines.

[0147] 3. Support for multi-touch operations: Since the simulated touch screen supports multi-touch data transmission, the corresponding specific content is the device descriptor marking the multi-touch device, and data supports batch transmission. Therefore, the connected physical touch screen supports multi-touch and, after redirection, also supports multi-touch operations.

[0148] like Figure 10 As shown, Figure 10 This is a schematic diagram of the structure of an electronic device 1000 provided in an embodiment of the present application. The electronic device 1000 includes:

[0149] A processor 1001, a storage medium 1002 and a bus 1003, wherein the storage medium 1002 stores machine-readable instructions executable by the processor 1001. When the electronic device 1000 is running, the processor 1001 communicates with the storage medium 1002 via the bus 1003, and the processor 1001 executes the machine-readable instructions to perform the steps of the touch screen redirection method described in the embodiment of the present application.

[0150] In actual application, the various components in the electronic device 1000 are coupled together via the bus 1003. It is understood that the bus 1003 is used to realize the connection and communication between these components. In addition to the data bus, the bus 1003 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 10 Various buses are labeled as bus 1003.

[0151] The electronic device has the following beneficial effects:

[0152] 1. You can freely switch and control the local graphical interface and the remote virtual machine interface through the touch screen without an external mouse or keyboard.

[0153] 2. Since the software simulates a universal touch screen device, it is compatible with a variety of virtual machines.

[0154] 3. Support for multi-touch operations: Since the simulated touch screen supports multi-touch data transmission, the corresponding specific content is the device descriptor marking the multi-touch device, and data supports batch transmission. Therefore, the connected physical touch screen supports multi-touch and, after redirection, also supports multi-touch operations.

[0155] The embodiment of the present application further provides a computer-readable storage medium, which stores executable instructions. When the executable instructions are executed by at least one processor 1001, the touch screen redirection method described in the embodiment of the present application is implemented.

[0156] In some embodiments, the storage medium can be a magnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface storage, an optical disc, or a compact disc read-only memory (CD-ROM); it can also be various devices including one or any combination of the above memories.

[0157] In some embodiments, executable instructions may be in the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0158] As an example, executable instructions may, but do not necessarily, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (for example, files storing one or more modules, subroutines, or code portions).

[0159] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0160] The computer-readable storage medium has the following advantages:

[0161] 1. You can freely switch and control the local graphical interface and the remote virtual machine interface through the touch screen without an external mouse or keyboard.

[0162] 2. Since the software simulates a universal touch screen device, it is compatible with a variety of virtual machines.

[0163] 3. Support for multi-touch operations: Since the simulated touch screen supports multi-touch data transmission, the corresponding specific content is the device descriptor marking the multi-touch device, and data supports batch transmission. Therefore, the connected physical touch screen supports multi-touch and, after redirection, also supports multi-touch operations.

[0164] In the several embodiments provided in this application, it should be understood that the disclosed methods and electronic devices can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0165] The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed across multiple network elements. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0166] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0167] If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, platform server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage media include various media that can store program code, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0168] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A touch screen redirection method, characterized in that: The following steps are involved: Determining, by the thin client, the type of device connected to the USB interface; If the device type is the touch screen, intercepting and processing the touch data so that the thin client forwards the touch data; In response to a touch operation on a touch screen, reading touch data of the touch screen, wherein the touch screen is connected to a thin client via a USB interface; Determining whether the operation display form of the current touch operation mapping satisfies a preset gesture through the touch data; the preset gesture is used to switch the display interface of the thin client; the display interface includes a local interface or a virtual machine interface, and the virtual machine interface is a cloud desktop; If satisfied, determine the interface to be displayed according to the current display interface of the touch screen, and switch the current display interface to the interface to be displayed; If not, determining whether the current display interface is a virtual machine interface; if so, sending the touch data to a server via a preset communication protocol, so that the virtual machine in the server performs a corresponding action in response to the touch operation; if not, sending the touch data to a designated local program in the thin client, so that the designated local program performs a corresponding action in response to the touch operation; The preset communication protocol is an extended SPICE protocol, which is used to encapsulate the touch data. The extended SPICE protocol is established in the following manner: defining a target command and a target data structure in a SPICE protocol, wherein the target command is used to identify and send the target data structure, and the target data structure includes the touch data; Using the SPICE protocol that defines the target command and the target data structure as the extended SPICE protocol; The server is provided with a simulated touch screen module, which simulates the touch screen based on the USB device framework in QEMU. The virtual machine is provided with a touch screen driver. The touch data is sent to the server via a preset communication protocol so that the virtual machine in the server can perform corresponding actions based on the touch data, including: Encapsulating the touch data using a SPICE client in the thin client to obtain encapsulated data; Sending the encapsulated data to the SPICE server in the server through the extended SPICE protocol; The packaged data is parsed and processed by the simulated touch screen module to obtain a touch instruction; The touch screen instruction is executed by the touch screen driver, so that the virtual machine performs corresponding actions based on the touch screen instruction.

2. The method according to claim 1, characterized in that Before reading touch data of the touch screen in response to a touch operation on the touch screen, the method further includes: In response to a startup operation of the virtual machine, mounting the simulated touch screen module to the server so that the virtual machine can recognize the touch screen; The touch screen driver is loaded to enable the virtual machine to interact with the simulated touch screen module.

3. The method according to claim 1, characterized in that Determining whether the operation display form of the current touch operation mapping satisfies the preset gesture through the touch data includes: The touch data is compared with the gesture data corresponding to the preset gesture. If the similarity between the touch data and the gesture data is greater than or equal to a preset matching threshold, the operation display form mapped to the current touch operation is determined to be the preset gesture.

4. The method according to claim 1, wherein The touch screen has a multi-touch function, and the method further includes: In response to a multi-touch operation on the touch screen, reading multi-touch data of the touch screen; The local interface and local program of the virtual machine and / or the thin client are operated based on the multi-touch data.

5. A touch screen redirection device, characterized in that: The device comprises: a reading module configured to determine, through the thin client, the type of device connected to the USB interface; if the device type is the touch screen, intercept and process the touch data so that the thin client forwards the touch data; and read the touch data from the touch screen in response to a touch operation on the touch screen, wherein the touch screen is connected to the thin client via the USB interface; a judgment module, configured to judge, based on the touch data, whether the operation display form of the current touch operation mapping satisfies a preset gesture; the preset gesture is used to switch the display interface of the thin client; the display interface includes a local interface or a virtual machine interface, and the virtual machine interface is a cloud desktop; If satisfied, determine the interface to be displayed according to the current display interface of the touch screen, and switch the current display interface to the interface to be displayed; If not, determining whether the current display interface is a virtual machine interface; if so, sending the touch data to a server via a preset communication protocol, so that the virtual machine in the server performs a corresponding action in response to the touch operation; if not, sending the touch data to a designated local program in the thin client, so that the designated local program performs a corresponding action in response to the touch operation; The preset communication protocol is an extended SPICE protocol, which is used to encapsulate the touch data. The extended SPICE protocol is established in the following manner: defining a target command and a target data structure in the SPICE protocol, wherein the target command is used to identify and send the target data structure, which includes the touch data; using the SPICE protocol that defines the target command and the target data structure as the extended SPICE protocol; a simulated touch screen module is preset in the server, which simulates the touch screen based on the USB device framework in QEMU; a touch screen driver is preset in the virtual machine; and sending the touch data to the server via the preset communication protocol so that the virtual machine in the server can perform corresponding actions based on the touch data includes: encapsulating the touch data by a SPICE client in the thin client to obtain encapsulated data; sending the encapsulated data to a SPICE server in the server via the extended SPICE protocol; parsing the encapsulated data by the simulated touch screen module to obtain a touch instruction; and executing the touch screen instruction by the touch screen driver so that the virtual machine performs corresponding actions based on the touch screen instruction.

6. An electronic device, characterized in that: include: A processor, a storage medium, and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the electronic device is running, the processor and the storage medium communicate via the bus, and the processor executes the machine-readable instructions to perform the touch screen redirection method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the touch screen redirection method according to any one of claims 1 to 4 is executed.

Images