Touch simulation method and device, storage medium and equipment

Abstract

The invention relates to a touch simulation method and device, a storage medium and equipment. A preset mapping table used for recording the mapping relation between each multi-finger touch operation and each touch operation event of a touchpad is pre-stored in a first data transmission application; in this way, after the first data transmission application responds to the target multi-finger touch data sent by the second data transmission application and determines the target multi-finger touch operation according to the target multi-finger touch data, the target multi-finger touch operation event corresponding to the target multi-finger touch operation can be determined according to the preset mapping table; according to the method and the device, the target multi-finger touch operation event is acquired, the response to the target multi-finger touch operation event is realized by calling the system interface corresponding to the target multi-finger touch operation event, so that the simulation of the target multi-finger touch operation is realized, the multi-finger touch requirement of a user can be met, and the user experience is improved.

Description

Technical Field

[0001] This invention relates to the field of touch technology, and in particular to a touch simulation method, apparatus, storage medium, and device. Background Technology

[0002] Touch operation simulation refers to simulating the real touch behavior of human fingers on a touch screen through software or hardware means, so that the device can make corresponding responses.

[0003] Traditional touch simulation methods simulate touch operations by receiving touch data and mapping it to mouse-related operations. However, this method can only simulate single-finger touch operations and cannot simulate multi-finger touch operations. Summary of the Invention

[0004] This application provides a touch simulation method, apparatus, storage medium, and device that can simulate multi-finger touch operations and improve user experience.

[0005] In a first aspect, embodiments of this application provide a touch simulation method applied to a first data transmission application of a first terminal device, wherein a second terminal device has a touchpad, the first data transmission application stores a preset mapping table, the preset mapping table being used to record the mapping relationship between each multi-finger touch operation of the touchpad and each multi-finger touch operation event, and the first data transmission application is communicatively connected to a second data transmission application of the second terminal device; the method includes:

[0006] In response to target multi-finger touch data sent by the second data transmission application, a target multi-finger touch operation is determined based on the target multi-finger touch data;

[0007] According to the preset mapping table, determine the target multi-finger touch operation event corresponding to the target multi-finger touch operation;

[0008] The system interface corresponding to the target multi-finger touch operation event is invoked in response to the target multi-finger touch operation event.

[0009] Optionally, the target multi-finger touch data includes information on the number of fingers and the positional change information of each finger;

[0010] Determining the target multi-finger touch operation based on the target multi-finger touch data specifically includes:

[0011] The target multi-finger touch operation is determined based on the preset multi-finger touch operation corresponding information, the number of fingers, and the position change information of each finger.

[0012] In this embodiment, based on preset multi-finger touch operation corresponding information, the target multi-finger touch operation is determined according to the number of fingers and the position change information of each finger. The correspondence between the target multi-finger touch data and the multi-finger touch operation can be set according to the actual application, so that the multi-finger touch input of the first terminal device can better fit the user's habits and improve the user experience.

[0013] Optionally, the method further includes:

[0014] In response to the target single-finger touch data sent by the second data transmission application, a target single-finger touch operation is determined based on the target single-finger touch data;

[0015] Call the system event creation interface corresponding to the target single-finger touch operation to create the target single-finger touch operation event;

[0016] The system interface corresponding to the target single-finger touch operation event is invoked in response to the target single-finger touch operation event.

[0017] By responding to the target single-finger touch data sent by the second data transmission application, a single-finger touch operation event is created by calling the system event creation interface of the first terminal device. This allows the first terminal device to simulate a single-finger touch operation by executing the single-finger touch operation event. The embodiments of this application can simulate single-finger touch operation and multi-finger touch operation on the first terminal device, meet different touch usage needs of users, and improve user experience.

[0018] Optionally, the target multi-finger touch data includes a first protocol identifier, and the target single-finger touch data includes a second protocol identifier. Determining the target multi-finger touch operation based on the target multi-finger touch data includes:

[0019] The first communication protocol is obtained based on the first protocol identifier;

[0020] The target multi-finger touch data is parsed according to the first communication protocol, and the target multi-finger touch operation is determined according to the parsed target multi-finger touch data;

[0021] Determining the target single-finger touch operation based on the target single-finger touch data includes:

[0022] The second communication protocol is obtained based on the second protocol identifier;

[0023] The target single-finger touch data is parsed according to the second communication protocol, and the target single-finger touch operation is determined based on the parsed target single-finger touch data.

[0024] In this embodiment, after the first data transmission application receives the touch data, it determines whether the received data is target multi-finger touch data or target single-finger touch data based on the protocol identifier included in the touch data. It then obtains the corresponding communication protocol based on the protocol identifier to parse the touch data. This embodiment can quickly determine whether the received data is target multi-finger touch data or target single-finger touch data based on the protocol identifier, thereby simulating the corresponding touch operation. Furthermore, this embodiment uses different communication protocols for different touch data, which can improve the security of touch data transmission.

[0025] Secondly, embodiments of this application provide a touch simulation method applied to a second data transmission application of a second terminal device. The second terminal device has a touchpad, and a first data transmission application stores a preset mapping table. The preset mapping table records the mapping relationship between each multi-finger touch operation of the touchpad and each multi-finger touch operation event. The second data transmission application is communicatively connected to the first data transmission application of the first terminal device. The method includes:

[0026] In response to a touch operation on the touchpad of the second terminal device, determine whether the touch operation is a multi-finger touch operation;

[0027] When the touch operation is a multi-finger touch operation, target multi-finger touch data is sent to the first data transmission application so that the first terminal device determines the target multi-finger touch operation based on the target multi-finger touch data; according to the preset mapping table, a target multi-finger touch operation event corresponding to the target multi-finger touch operation is determined; and the system interface corresponding to the target multi-finger touch operation event is called to respond to the target multi-finger touch operation event.

[0028] Thirdly, embodiments of this application provide a touch simulation device applied to a first terminal device, a second terminal device having a touchpad, a first data transmission application storing a preset mapping table, the preset mapping table being used to record the mapping relationship between each multi-finger touch operation of the touchpad and each multi-finger touch operation event, the first terminal device being communicatively connected to a second data transmission application of the second terminal device through the first data transmission application; the device includes:

[0029] A target multi-finger touch operation determination module is used to determine a target multi-finger touch operation based on the target multi-finger touch data sent by a second data transmission application in response to the target multi-finger touch data.

[0030] The multi-finger touch operation event determination module is used to determine the target multi-finger touch operation event corresponding to the target multi-finger touch operation according to the preset mapping table;

[0031] The first system interface calling module is used to call the system interface corresponding to the target multi-finger touch operation event in response to the target multi-finger touch operation event.

[0032] Fourthly, embodiments of this application provide a touch simulation device applied to a second terminal device. The second terminal device has a touchpad, and a first data transmission application stores a preset mapping table. The preset mapping table records the mapping relationship between each multi-finger touch operation of the touchpad and each multi-finger touch operation event. The second terminal device is communicatively connected to the first data transmission application of the first terminal device through a second data transmission application. The device includes:

[0033] A multi-finger touch operation determination module is used to determine whether the touch operation is a multi-finger touch operation in response to a touch operation on the touchpad of the second terminal device.

[0034] The touch data sending module is configured to send target multi-finger touch data to the first data transmission application when the touch operation is a multi-finger touch operation, so that the first terminal device determines the target multi-finger touch operation based on the target multi-finger touch data; determine the target multi-finger touch operation event corresponding to the target multi-finger touch operation according to the preset mapping table; and call the system interface corresponding to the target multi-finger touch operation event to respond to the target multi-finger touch operation event.

[0035] Fifthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the touch simulation method as described in any of the preceding claims.

[0036] Sixthly, embodiments of this application provide a first terminal device, including:

[0037] At least one processor; and

[0038] A memory communicatively connected to the at least one processor; wherein,

[0039] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform any of the touch simulation methods described above.

[0040] In a seventh aspect, embodiments of this application provide a second terminal device, including:

[0041] At least one processor; and

[0042] A memory communicatively connected to the at least one processor; wherein,

[0043] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the touch simulation method described above.

[0044] Eighthly, embodiments of this application provide a computer-readable storage medium storing computer instructions that cause a processor to execute and implement the touch simulation method described above, or to implement the touch simulation method described above.

[0045] In this embodiment, by pre-storing a preset mapping table in the first data transmission application to record the mapping relationship between each multi-finger touch operation and each touch operation event of the touchpad, the first data transmission application, in response to the target multi-finger touch data sent by the second data transmission application, determines the target multi-finger touch operation based on the target multi-finger touch data, and then determines the target multi-finger touch operation event corresponding to the target multi-finger touch operation according to the preset mapping table. Furthermore, by calling the system interface corresponding to the target multi-finger touch operation event, the application responds to the target multi-finger touch operation event, thereby simulating the target multi-finger touch operation. This application can meet the user's multi-finger touch needs and improve the user experience.

[0046] To better understand and implement this invention, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description

[0047] Figure 1 This is a schematic diagram illustrating an application scenario of a touch simulation method according to one embodiment of the present invention;

[0048] Figure 2 This is a flowchart of a touch simulation method in one embodiment of the present invention;

[0049] Figure 3 This is a flowchart of a touch simulation method in another embodiment of the present invention;

[0050] Figure 4 This is a flowchart of step S110 in one embodiment of the present invention;

[0051] Figure 5 This is a flowchart of step S210 in one embodiment of the present invention;

[0052] Figure 6 This is a flowchart of a touch simulation method in another embodiment of the present invention;

[0053] Figure 7 This is a schematic diagram of the structure of a touch simulation device in one embodiment of the present invention;

[0054] Figure 8This is a schematic diagram of the structure of a touch simulation device according to another embodiment of the present invention;

[0055] Figure 9 This is a schematic diagram of the structure of a first terminal device in one embodiment of the present invention;

[0056] Figure 10 This is a schematic diagram of the structure of a second terminal device in one embodiment of the present invention. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0058] It should be understood that the described embodiments are merely some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of the embodiments of this application.

[0059] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the embodiments of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0060] In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims. In the description of this application, it should be understood that the terms "first," "second," "third," etc., are used only to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0061] Furthermore, in the description of this application, unless otherwise stated, "several" refers to two or more. "And / or" describes the correspondence between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0062] Example 1

[0063] Touch operation simulation refers to simulating the real touch behavior of human fingers on a touch screen through software or hardware means without actual touch, so that the device can make a corresponding response. Touch simulation technology can be used to detect the response and touch effect of touch input without physical devices.

[0064] Traditional touch simulation methods simulate touch operations by receiving touch data and mapping it to mouse-related operations. However, this method can only simulate single-finger touch operations and cannot simulate multi-finger touch operations.

[0065] Therefore, to address the aforementioned problems, this application provides a touch simulation method. This method, through a first data transmission application, pre-stores a preset mapping table recording the mapping relationship between each multi-finger touch operation and each touch operation event on the touchpad. This allows the first data transmission application, in response to target multi-finger touch data sent by a second data transmission application, to determine the target multi-finger touch operation based on the target multi-finger touch data. Then, it can determine the target multi-finger touch operation event corresponding to the target multi-finger touch operation based on the preset mapping table. Furthermore, it responds to the target multi-finger touch operation event by calling the system interface corresponding to the target multi-finger touch operation event, thereby simulating the target multi-finger touch operation. This application can meet the user's multi-finger touch needs and improve the user experience.

[0066] Please see Figure 1 This is a schematic diagram illustrating an application scenario of the touch simulation method in one embodiment of this application; for example... Figure 1 As shown, the application scenario includes a first terminal device 110 and a second terminal device 120. The first terminal device 110 is equipped with a first data transmission application 111, and the second terminal device 120 is equipped with a second data transmission application 121.

[0067] The first terminal device 110 is communicatively connected to the second data transmission application 121 of the second terminal device 120 through the first data transmission application 111. The first data transmission application 111 and the second data transmission application 121 are used to realize data transmission between the first terminal device 110 and the second terminal device 120.

[0068] The first terminal device 110 and the second terminal device 120 can be mobile phones, wearable devices, tablets, personal digital assistants (PDAs), laptops, mobile computers, interactive whiteboards, and other terminal devices.

[0069] In one embodiment, the second terminal device 120 may have a touchpad, through which the user can perform single-finger touch input or multi-finger touch input. The second terminal device 120 can transmit touch data to the first data transmission application 111 through the second data transmission application 121, so that the first data transmission application 111 can simulate the corresponding touch operation on the first terminal device 110 according to the touch data.

[0070] The first terminal device 110 and the second terminal device 120 can be terminal devices with displays. Optionally, the second terminal device 120 can be a large-sized interactive flat panel.

[0071] Users can perform touch operations on the second terminal device 120. The second terminal device 120 can transmit touch data to the first data transmission application 111 through the second data transmission application 121. The first data transmission application 111 simulates the corresponding touch operation on the first terminal device 110 based on the touch data.

[0072] Please see Figure 2 This application provides a touch simulation method, which can be applied to a first data transmission application of a first terminal device. The method includes the following steps:

[0073] S110: In response to the target multi-finger touch data sent by the second data transmission application, determine the target multi-finger touch operation based on the target multi-finger touch data;

[0074] Target multi-finger touch data is touch data generated based on multi-finger touch operations performed by the user on the touchpad of a second terminal device. Target multi-finger touch data is used to determine the target multi-finger touch operations that need to be simulated.

[0075] The target multi-finger touch operation can be the touchpad operation that needs to be simulated based on the target multi-finger touch data. The target multi-finger touch operation can include operations such as swiping left, swiping right, swiping up, swiping down, rotating, zooming in, zooming out, returning to the dispatch center, switching screens to the left, switching screens to the right, returning to the desktop, and returning to the launcher.

[0076] Specifically, the target multi-finger touch operation can be determined based on the correspondence between multi-finger touch data and multi-finger touch operations. This correspondence can be preset by the user.

[0077] For example, when the target multi-finger touch operation data is the touch data generated by the user swiping the touchpad of the second terminal device with two fingers to the left, the target multi-finger touch operation is determined to be scrolling the page to the left based on the corresponding information.

[0078] S120: Determine the target multi-finger touch operation event corresponding to the target multi-finger touch operation according to the preset mapping table;

[0079] A preset mapping table is used to determine the target multi-finger touch operation event corresponding to the target multi-finger touch operation. The preset mapping table records the mapping relationship between each multi-finger touch operation and each multi-finger touch operation event on the touchpad. For example, swiping up or down or left or right with two fingers corresponds to scrolling the page in the direction corresponding to the swipe; pinching or spreading two fingers zooms in or out on images, web pages, etc.; swiping left or right with four fingers corresponds to switching between full-screen applications. Specifically, in presentation mode, swiping left or right with two fingers switches to the previous or next slide.

[0080] After determining the target multi-finger touch operation, the corresponding target multi-finger touch operation event can be obtained through a preset mapping table.

[0081] S130: Call the system interface corresponding to the target multi-finger touch operation event to respond to the target multi-finger touch operation event;

[0082] The system interface can be used to respond to target multi-finger touch operation events, thereby simulating the target multi-finger touch operation.

[0083] In this embodiment, by pre-storing a preset mapping table in the first data transmission application to record the mapping relationship between each multi-finger touch operation and each touch operation event of the touchpad, the first data transmission application, in response to the target multi-finger touch data sent by the second data transmission application, determines the target multi-finger touch operation based on the target multi-finger touch data, and then determines the target multi-finger touch operation event corresponding to the target multi-finger touch operation according to the preset mapping table. Furthermore, by calling the system interface corresponding to the target multi-finger touch operation event, the application responds to the target multi-finger touch operation event, thereby simulating the target multi-finger touch operation. This application can meet the user's multi-finger touch needs and improve the user experience.

[0084] In step S110, the target multi-finger touch data may include information on the number of fingers and the position change information of each finger;

[0085] The quantity information is used to determine the number of fingers that touch the second terminal device.

[0086] Position change information is used to determine the position change of the finger that is touching the second terminal device.

[0087] In this embodiment, the position change information can be the coordinate change information of the finger. The coordinate change information can include information such as an increase in x-coordinate, an increase in y-coordinate, a change in coordinate by radians, a decrease in x-coordinate, a decrease in y-coordinate, or a change in coordinate along a straight line. The finger's coordinates can be coordinates on a coordinate system established based on the touch plane of the second terminal device, and these coordinates can be used to determine the position of the finger on the touch plane of the second terminal device.

[0088] In one embodiment, determining the target multi-finger touch operation based on the target multi-finger touch data specifically includes:

[0089] The target multi-finger touch operation is determined based on the preset target multi-finger touch operation corresponding information, the number of fingers, and the position change information of each finger.

[0090] As shown in Table 1, in one embodiment, the target multi-finger touch operation correspondence information can be a multi-finger touch operation correspondence table pre-set by the user.

[0091]

[0092]

[0093] When there are two fingers and the x-coordinate changes, the target multi-finger touch operation is determined to be a left or right swipe based on the increase or decrease of the x-coordinate. When there are two fingers and the y-coordinate changes, the target multi-finger touch operation is determined to be an up or down swipe based on the increase or decrease of the y-coordinate. When there are two fingers and the coordinates (x1, y1) and (x2, y2) of the two fingers can change along a straight line, the target multi-finger touch operation is determined to be zoom in or zoom out based on the direction of the change in the coordinates of the two fingers. When there are three or four fingers and the y-coordinate increases, the target multi-finger touch operation is determined to be entering the dispatch center. When there are three or four fingers and the x-coordinate changes, the target multi-finger touch operation is determined to be a left or right screen swipe based on the increase or decrease of the x-coordinate. When there are four or five fingers and the fingers are spread outwards, the target multi-finger touch operation is determined to be returning to the desktop. When there are four or five fingers and the fingers are gathered inwards, the target multi-finger touch operation is determined to be returning to the launcher.

[0094] In this embodiment, based on preset multi-finger touch operation corresponding information, the target multi-finger touch operation is determined according to the number of fingers and the position change information of each finger. The correspondence between the target multi-finger touch data and the multi-finger touch operation can be set according to the actual application, so that the multi-finger touch input of the first terminal device can better fit the user's habits and improve the user experience.

[0095] In one embodiment, the first data transmission application and the second data transmission application can transmit multi-finger touch data through a pre-set communication protocol.

[0096] The communication protocol can be pre-stored in the second data transmission application. When the second data transmission application recognizes a multi-finger touch operation on the second terminal device, it calls the communication protocol to generate multi-finger touch data and sends it to the first data transmission application.

[0097] Communication protocols can be used to determine the content and format of the multi-finger touch data to be generated. For example, multi-finger touch data can include the number, identifier, and position of fingers. Multi-finger touch data can be in array format, with different data stored in different rows or columns of the array.

[0098] Optionally, multi-finger touch data may include information on the number of fingers, finger position, finger identification, and touch behavior. Finger identification information is used to identify the corresponding finger and may include information such as the finger's ID. Touch behavior information may include up and down events. An up event occurs when a finger that has been touching the second terminal device leaves the second terminal device; a down event occurs when a finger that has not been touching the second terminal device touches the second terminal device.

[0099] In step S120, the preset mapping table may also include event information for each multi-finger touch operation event. The event information is used to create multi-finger touch operation events and may include information related to the creation of multi-finger touch operation events, such as event type, event parameters, and creation method.

[0100] The system interface can be the system interface of the operating system of the first terminal device.

[0101] Event information can be obtained by the user by reversing a database of touch operation events stored in the operating system of the first terminal device. Event information may include event type and target offset.

[0102] Specifically, when the operating system of the first terminal device is a MAC system, the event information can be obtained by reverse engineering the Io Kit (Input / Output Toolbox) database of the MAC system, which stores the event information of each touch operation event.

[0103] The Io Kit (Input / Output Toolkit) library stores event information for various touch events on the Mac system, including publicly available touch event information and hidden touch event information. Hidden touch event information refers to touch event information that is not publicly available and cannot be directly obtained.

[0104] This application embodiment confirms by studying the event information of hidden touch operation events that when the operating system of the first terminal device is a MAC system, the event type can be the IOHID event type.

[0105] IOHID (Input / Output Human Interface Device) is a framework provided by Apple for interacting with human interface devices such as keyboards, mice, trackpads, touchscreens, and accelerometers. The IOHID framework is part of the I / O Kit framework in macOS and iOS, enabling developers to write applications that respond to and handle events from various input devices. The I / O Kit framework is a low-level framework in macOS and iOS that allows developers to access and control hardware devices such as disks, network interfaces, and printers, enabling the operating system to interact with these devices.

[0106] Event types may include undisclosed event types such as the swipe event type kIOHIDEventTypeDockSwipe corresponding to the swipe operation, the scroll event type kIOHIDEventTypeScroll corresponding to the scroll operation, and the zoom event type kIOHIDEventTypeZoom corresponding to the zoom operation.

[0107] The target multi-finger touch operation event includes at least one CGEventRef, and the event types of each CGEventRef can be different.

[0108] The target offset is used to determine the location of a multi-finger touch operation event on the screen of the first terminal device.

[0109] The target offset can be determined based on the position change information of each finger in the multi-finger touch data, the position correspondence between the first terminal device and the second terminal device, and the multi-finger touch operation event. For example, for a rotation operation event, the target offset can be used to determine the rotation angle; for a swipe operation event, the target offset can determine the amount of position change during the swipe.

[0110] When the operating system of the first terminal device is a MAC system, the target multi-finger touch operation event may include at least one CGEventRef that can be recognized and processed by the MAC system. The CGEventRef is a low-level hardware event of the MAC system generated when the user operates the input device, such as mouse movement, keyboard key press or other input device events. The CGEventRef can be recognized by the MAC system and processed like an event generated by actual hardware.

[0111] The target multi-touch operation event can include multiple CGEventRefs of different event types. The relationship between the target multi-touch operation and the CGEventRef, as well as the event type of the CGEventRef, can be pre-stored in a preset mapping table.

[0112] Specifically, you can call the CGEventCreate function to create a CGEventRef of the corresponding event type, and then use the CGEventSetDoubleValueField function to set the offset of the CGEventRef.

[0113] The CGEventCreate function is a function in the Core Graphics framework used to create Core Graphics Events (CGEvents). This function allows developers to simulate various input device events in the system, such as mouse movement and keyboard key presses.

[0114] The CGEventSetDoubleValueField function is a Core Graphics framework function used to set the value of a specific field in the Core Graphics Event (CGEvent) structure. This function allows developers to customize the properties of the CGEvent object, such as the position of a mouse event or the key value of a keyboard event.

[0115] In one optional embodiment, the CGEventCreate function and the CGEventSetDoubleValueField function can be pre-stored in the first data transmission application. Alternatively, in another optional embodiment, the first data transmission application can also call the external CGEventCreate function and CGEventSetDoubleValueField function, for example, by calling the CGEventCreate function and CGEventSetDoubleValueField function of the operating system of the first terminal device.

[0116] For example, when the target multi-finger touch operation is a rotation operation, first create a CGEventRef with the event type of gesture (NSEventTypeGesture), then set the subtype to rotation (kIOHIDEventTypeRotation) corresponding to the rotation operation to obtain the rotation operation event. Then, set the offset of the rotation operation event according to the target offset to obtain the rotation operation event corresponding to the rotation operation.

[0117] When the target multi-finger touch operation is a swipe operation, first create a CGEventRef with event type NSEventTypeGesture, then create a CGEventRef with event type NSEventTypeMagnify, and set its subtype to swipe (kIOHIDEventTypeDockSwipe). Combine the above CGEventRefs to obtain the swipe operation event. Set the offset according to the target offset and set the swipe operation event to a horizontal offset to generate the swipe operation event.

[0118] In this embodiment, by creating multiple CGEventRefs of different event types and setting the offset of the CGEventRefs, multiple CGEventRefs are combined to form a target multi-finger touch operation event, thereby simulating multi-finger touch operation and improving user experience.

[0119] In step S130, the system interface can be a system interface used to respond to target multi-finger touch operation events. The target multi-finger touch operation event can be executed by the operating system of the first terminal device.

[0120] Specifically, when the operating system of the first terminal device is a MAC system, the target multi-finger touch operation event can be sent to the operating system of the first terminal device by calling the CGEventPost function, so that the operating system of the first terminal device can respond to the target multi-finger touch operation event and perform the corresponding target multi-finger touch operation.

[0121] The CGEventPost function is a Core Graphics framework function in the macOS system. It is used to post Core Graphics events (CGEvents) to the operating system's event queue, allowing the operating system to process the events.

[0122] like Figure 3 As shown, in one embodiment, the method further includes:

[0123] S210: In response to the target single-finger touch data sent by the second data transmission application, determine the target single-finger touch operation based on the target single-finger touch data;

[0124] S220: Call the system event creation interface corresponding to the target single-finger touch operation to create a target single-finger touch operation event;

[0125] S230: Call the system interface corresponding to the target single-finger touch operation event to respond to the target single-finger touch operation event.

[0126] The system event creation interface can be a pre-configured interface for the first terminal device to create single-finger touch operation events.

[0127] In this embodiment of the application, the target single-finger touch operation event can be a mouse operation event, and the system event creation interface can be the event creation interface of the operating system of the first terminal device.

[0128] In this embodiment of the application, the system event creation interface can create a target single-finger touch operation event by using the CGEventCreateMouseEvent function, which is used to create mouse operation events related to mouse operations.

[0129] In this embodiment, a single-finger touch operation event is created by calling the system event creation interface of the first terminal device in response to the target single-finger touch data sent by the second data transmission application. This allows the first terminal device to simulate a single-finger touch operation by executing the single-finger touch operation event. This embodiment can simulate single-finger touch operation and multi-finger touch operation on the first terminal device, meet different touch usage needs of users, and improve user experience.

[0130] like Figure 4 As shown, in one embodiment, the target multi-finger touch data includes a first protocol identifier, and determining the target multi-finger touch operation based on the target multi-finger touch data includes:

[0131] S111: Obtain the first communication protocol based on the first protocol identifier;

[0132] S112: Parse the target multi-finger touch data according to the first communication protocol, and determine the target multi-finger touch operation based on the parsed target multi-finger touch data;

[0133] The first protocol identifier is the protocol identifier corresponding to the first communication protocol.

[0134] After receiving the target multi-finger touch data, the first data transmission application obtains the first communication protocol based on the first protocol identifier included in the target multi-finger touch data. It then parses the target multi-finger touch data according to the first communication protocol to obtain its content. For example, if the target multi-finger touch data is an array, the application reads the corresponding elements from the array using the first communication protocol to obtain the relevant data. The content of the target multi-finger touch data may include information such as the number of fingers and the positional changes of each finger.

[0135] like Figure 5 As shown, determining the target single-finger touch operation based on the target single-finger touch data includes:

[0136] S211: Obtain the second communication protocol based on the second protocol identifier;

[0137] S212: Parse the target single-finger touch data according to the second communication protocol, and determine the target single-finger touch operation based on the parsed target single-finger touch data.

[0138] The second protocol identifier is the protocol identifier corresponding to the second communication protocol.

[0139] After receiving the target single-finger touch data, the first data transmission application obtains the second communication protocol based on the second protocol identifier included in the target single-finger touch data.

[0140] The first data transmission application can pre-store multiple protocol identifiers, multiple communication protocols, and the correspondence between protocol identifiers and communication protocols, and obtain the corresponding communication protocol based on the protocol identifier.

[0141] Multiple protocol identifiers may include a first protocol identifier and a second protocol identifier;

[0142] Multiple communication protocols may include a first communication protocol identifier and a second communication protocol, and the first communication protocol and the second communication protocol may be different communication protocols.

[0143] The communication protocol is used to determine the content and data format of the multi-finger touch data to be generated. For example, the target multi-finger touch data is determined according to the communication protocol, including the number, identification, and position of the fingers. The target multi-finger touch data can be in the data format of an array, with different data stored in different rows or columns of the array.

[0144] It should be noted that the first data transmission application can quickly determine whether the received data is target multi-finger touch data or target single-finger touch data based on the protocol identifier, thereby realizing the simulation of the corresponding touch operation.

[0145] In this embodiment, after the first data transmission application receives the touch data, it determines whether the received data is target multi-finger touch data or target single-finger touch data based on the protocol identifier included in the touch data. It then obtains the corresponding communication protocol based on the protocol identifier to parse the touch data. This embodiment can quickly determine whether the received data is target multi-finger touch data or target single-finger touch data based on the protocol identifier, thereby simulating the corresponding touch operation. Furthermore, this embodiment uses different communication protocols for different touch data, which can improve the security of touch data transmission.

[0146] In an optional embodiment, the touch simulation method of this application can be applied in a screen projection scenario, such as... Figure 1As shown, the first terminal device 110 and the second terminal device 120 can be terminal devices with displays. The second terminal device 120 can be a large-sized interactive flat panel, which can have a touchpad.

[0147] The first data transmission application 111 and the second data transmission application 121 can be screen mirroring applications.

[0148] The first terminal device 110 sends media data to the second data transmission application 121 through the first data transmission application 111 and displays it on the second terminal device 120.

[0149] The media data may include audio and / or video data from the first terminal device 110. After receiving the media data, the second data transmission application 121 displays the media data on the second terminal device 120, thereby realizing the projection of the media data from the first terminal device 110 to the second terminal device 120.

[0150] Through the touch simulation method of this application embodiment, in the screen projection scenario, the user can perform multi-finger touch operation through the touchpad of the second terminal device 120, thereby realizing multi-finger touch control of the first terminal device 110. This application can improve the convenience of touch control of the first terminal device 110.

[0151] Example 2

[0152] Please see Figure 6 This application also provides a touch simulation method applied to a second data transmission application of a second terminal device. The second terminal device has a touchpad, and a first data transmission application stores a preset mapping table. The preset mapping table records the mapping relationship between each multi-finger touch operation of the touchpad and each multi-finger touch operation event. The second data transmission application is communicatively connected to the first data transmission application of the first terminal device. The method includes:

[0153] S310: In response to a touch operation on the second terminal device, determine whether the touch operation is a multi-finger touch operation;

[0154] Touch operations can include single-finger touch operations and multi-finger touch operations.

[0155] The second data transmission application can be pre-set with a multi-finger touch operation recognition algorithm to identify multi-finger touch operations. When a touch operation on the second terminal device is detected, the multi-finger touch operation recognition algorithm is used to identify whether the touch operation is a multi-finger touch operation.

[0156] Alternatively, the second data transmission application may be pre-configured with a touch operation recognition algorithm for identifying single-finger touch operations and multi-finger touch operations. When a touch operation on the second terminal device is detected, the touch operation recognition algorithm identifies the touch operation as a single-finger touch operation or a multi-finger touch operation.

[0157] S320: When the touch operation is a multi-finger touch operation, send multi-finger touch data to the first data transmission application so that the first terminal device determines the target multi-finger touch operation based on the target multi-finger touch data; determine the target multi-finger touch operation event corresponding to the target multi-finger touch operation according to the preset mapping table; call the system interface corresponding to the target multi-finger touch operation event to respond to the target multi-finger touch operation event.

[0158] When the touch operation is not a multi-finger touch operation, the second data transmission application may not send the target multi-finger touch data to the first data transmission application, so the first data transmission application does not need to respond; or, when the touch operation is not a multi-finger touch operation, for example when the touch operation is a single-finger touch operation, the second data transmission application may send the target single-finger touch data to the first data transmission application, so that the second data transmission application can simulate the target single-finger touch operation based on the target single-finger touch data.

[0159] In this embodiment, the second data transmission application, in response to a touch operation on the second terminal device, determines whether the touch operation is a multi-finger touch operation. If the touch operation is a multi-finger touch operation, it sends target multi-finger touch data to the first data transmission application, enabling the first terminal device to simulate the target multi-finger touch operation based on the target multi-finger touch data.

[0160] The target multi-touch data can be generated by a second data transmission application according to a pre-set communication protocol.

[0161] Communication protocols can be used to determine the content and data format of the multi-finger touch data to be generated. For example, according to the communication protocol, the target multi-finger touch data can include the number, identification, and position of fingers. The target multi-finger touch data can be in the format of an array, with different data stored in different rows or columns of the array.

[0162] The communication protocol can be pre-stored in the second data transmission application. When the second data transmission application recognizes a multi-finger touch operation on the second terminal device, it calls the communication protocol to generate multi-finger touch data.

[0163] Optionally, in one embodiment, after determining whether the touch operation is a multi-finger touch operation, the method further includes:

[0164] When the touch operation is a multi-finger touch operation, a first communication protocol corresponding to the multi-finger touch operation is obtained, and multi-finger touch data is generated according to the first communication protocol; the multi-finger touch data includes a first communication protocol identifier.

[0165] When the touch operation is a single-finger touch operation, the second communication protocol corresponding to the single-finger touch operation is obtained, and single-finger touch data is generated according to the second communication protocol; the single-finger touch data includes the second communication protocol identifier.

[0166] The data content and format of multi-finger touch data and single-finger touch data can be different.

[0167] The first communication protocol and the second communication protocol are different communication protocols. The first communication protocol and the second communication protocol can be preset by the user and stored in the first data transmission application and the second data transmission application.

[0168] In this embodiment, by calling different communication protocols based on whether the touch operation is a multi-finger touch operation or a single-finger touch operation, multi-finger touch data or single-finger touch data including the corresponding communication protocol identifier is generated. This enables the first data transmission application to quickly determine whether the touch data is generated by a multi-finger touch operation or a single-finger touch operation based on the corresponding communication protocol identifier when it receives multi-finger touch data or single-finger touch data, and to obtain the corresponding communication protocol for data parsing, thereby improving data acquisition efficiency and data transmission security.

[0169] For details not disclosed in this embodiment, please refer to Embodiment 1 of this application.

[0170] Example 3

[0171] like Figure 7 As shown, this application embodiment also provides a touch simulation device applied to a first terminal device. The second terminal device has a touchpad, and a first data transmission application stores a preset mapping table. The preset mapping table is used to record the mapping relationship between each multi-finger touch operation of the touchpad and each multi-finger touch operation event. The first terminal device is communicatively connected to the second data transmission application of the second terminal device through the first data transmission application. The device includes:

[0172] The target multi-finger touch operation determination module 210 is used to determine a target multi-finger touch operation based on the target multi-finger touch data sent by the second data transmission application in response to the target multi-finger touch data sent by the second data transmission application.

[0173] The multi-finger touch operation event determination module 220 is used to determine the target multi-finger touch operation event corresponding to the target multi-finger touch operation according to the preset mapping table;

[0174] The first system interface calling module 230 is used to call the system interface corresponding to the target multi-finger touch operation event in response to the target multi-finger touch operation event.

[0175] Optionally, the target multi-finger touch data includes information on the number of fingers and the positional change information of each finger;

[0176] The target multi-finger touch operation determination module 210 includes:

[0177] The touch operation determination unit is used to determine the target multi-finger touch operation based on preset multi-finger touch operation corresponding information, the number of fingers, and the position change information of each finger.

[0178] Optionally, the device includes:

[0179] A single-finger touch operation determination module is used to determine a target single-finger touch operation based on the target single-finger touch data sent by the second data transmission application in response to the target single-finger touch data.

[0180] The single-finger touch operation event creation module is used to call the system event creation interface corresponding to the target single-finger touch operation to create the target single-finger touch operation event.

[0181] The second system interface calling module is used to call the system interface corresponding to the target single-finger touch operation event in response to the target single-finger touch operation event. Optionally, the target multi-finger touch operation determination module 210 includes:

[0182] The first communication protocol acquisition unit is configured to acquire a first communication protocol based on the first protocol identifier.

[0183] The target multi-finger touch operation determination unit is used to parse the target multi-finger touch data according to the first communication protocol, and determine the target multi-finger touch operation based on the parsed target multi-finger touch data;

[0184] The target single-finger touch operation determination module includes:

[0185] The second communication protocol acquisition unit is used to acquire the second communication protocol according to the second protocol identifier;

[0186] The target single-finger touch operation determination module unit is used to parse the target single-finger touch data according to the second communication protocol, and determine the target single-finger touch operation based on the parsed target single-finger touch data.

[0187] It should be noted that the touch simulation device provided in the above embodiments is only illustrated by the division of the above functional modules when executing the touch simulation method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the touch simulation device provided in the above embodiments and the touch simulation method in Embodiment 1 belong to the same concept, and its implementation process can be found in the method embodiments, which will not be repeated here.

[0188] Example 4

[0189] like Figure 8 As shown, this embodiment provides a touch simulation device applied to a second terminal device. The second terminal device is communicatively connected to a first data transmission application of a first terminal device through a second data transmission application. The device includes:

[0190] The multi-finger touch operation determination module 310 is used to determine whether the touch operation is a multi-finger touch operation in response to a touch operation on the touchpad of the second terminal device.

[0191] The touch data sending module 320 is used to send multi-finger touch data to the first data transmission application when the touch operation is a multi-finger touch operation, so that the first terminal device determines the target multi-finger touch operation based on the target multi-finger touch data; determine the target multi-finger touch operation event corresponding to the target multi-finger touch operation according to the preset mapping table; and call the system interface corresponding to the target multi-finger touch operation event to respond to the target multi-finger touch operation event.

[0192] In one embodiment, the apparatus includes:

[0193] The first touch data generation module is used to, when the touch operation is a multi-finger touch operation, obtain a first communication protocol corresponding to the multi-finger touch operation, and generate multi-finger touch data according to the first communication protocol; the multi-finger touch data includes a first communication protocol identifier.

[0194] The second touch data generation module is used to, when the touch operation is a single-finger touch operation, obtain the second communication protocol corresponding to the single-finger touch operation, and generate single-finger touch data according to the second communication protocol; the single-finger touch data includes a second communication protocol identifier.

[0195] It should be noted that the touch simulation device provided in the above embodiments is only illustrated by the division of the above functional modules when executing the touch simulation method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the touch simulation device provided in the above embodiments and the touch simulation method in Embodiment 2 above belong to the same concept, and its implementation process is detailed in the method embodiments, which will not be repeated here.

[0196] Example 5

[0197] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the touch simulation method as described in any of the preceding claims.

[0198] This application may take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-readable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information may be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to: phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0199] Example 6

[0200] like Figure 9 As shown in the figure, this application embodiment also provides a first terminal device 400, including:

[0201] At least one processor 410; and

[0202] Memory 420 communicatively connected to the at least one processor 410; wherein,

[0203] The memory 420 stores a computer program that can be executed by the at least one processor 410 to enable the at least one processor 410 to perform any of the touch simulation methods described above.

[0204] The memory 420 includes read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0205] The processor 410 is the control unit of the first terminal device 400. It connects to various components of the first terminal device 400 via various interfaces and lines. By running or executing programs or modules stored in the memory 420, and by calling data stored in the memory 420, it performs various functions and processes data for the first terminal device 400. For example, when the processor 410 executes a computer program stored in the memory 420, it implements all or part of the steps of the touch simulation method described in this application embodiment; or it implements all or part of the functions of the touch simulation device. The processor 410 can be composed of integrated circuits, such as a single packaged integrated circuit, or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips.

[0206] Example 7

[0207] like Figure 10 As shown in the illustration, this application also provides a second terminal device 500, including:

[0208] At least one processor 510; and

[0209] Memory 420 communicatively connected to the at least one processor 510; wherein,

[0210] The memory 520 stores a computer program that can be executed by the at least one processor 410 to enable the at least one processor 410 to perform any of the touch simulation methods described above.

[0211] The memory 520 includes read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0212] The processor 510 is the control unit of the second terminal device 500. It connects to various components of the second terminal device 500 via various interfaces and lines. By running or executing programs or modules stored in the memory 520, and by calling data stored in the memory 520, it performs various functions and processes data for the second terminal device 500. For example, when the processor 510 executes a computer program stored in the memory 520, it implements all or part of the steps of the touch simulation method described in this application embodiment; or it implements all or part of the functions of the touch simulation device. The processor 510 can be composed of integrated circuits, such as a single packaged integrated circuit, or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips.

[0213] This invention is not limited to the above-described embodiments. If any modifications or variations to this invention do not depart from the spirit and scope of this invention, and if such modifications and variations fall within the scope of the claims and equivalent technologies of this invention, then this invention also intends to include such modifications and variations.

Claims

1. A touch simulation method, characterized in that, A first data transmission application is applied to a first terminal device, and a second terminal device has a touchpad. The first data transmission application stores a preset mapping table, which records the mapping relationship between each multi-finger touch operation and each multi-finger touch operation event on the touchpad. The first data transmission application is communicatively connected to a second data transmission application of the second terminal device. The method includes: In response to target multi-finger touch data sent by the second data transmission application, a target multi-finger touch operation is determined based on the target multi-finger touch data; According to the preset mapping table, determine the target multi-finger touch operation event corresponding to the target multi-finger touch operation; The system interface corresponding to the target multi-finger touch operation event is invoked in response to the target multi-finger touch operation event.

2. The touch simulation method according to claim 1, characterized in that, The target multi-finger touch data includes information on the number of fingers and the positional change information of each finger; Determining the target multi-finger touch operation based on the target multi-finger touch data specifically includes: The target multi-finger touch operation is determined based on the preset multi-finger touch operation corresponding information, the number of fingers, and the position change information of each finger.

3. The touch simulation method according to claim 1, characterized in that, The method further includes: In response to the target single-finger touch data sent by the second data transmission application, a target single-finger touch operation is determined based on the target single-finger touch data; Call the system event creation interface corresponding to the target single-finger touch operation to create the target single-finger touch operation event; The system interface corresponding to the target single-finger touch operation event is invoked in response to the target single-finger touch operation event.

4. The touch simulation method according to claim 3, characterized in that, The target multi-finger touch data includes a first protocol identifier, and the target single-finger touch data includes a second protocol identifier. Determining the target multi-finger touch operation based on the target multi-finger touch data includes: The first communication protocol is obtained based on the first protocol identifier; The target multi-finger touch data is parsed according to the first communication protocol, and the target multi-finger touch operation is determined according to the parsed target multi-finger touch data; Determining the target single-finger touch operation based on the target single-finger touch data includes: The second communication protocol is obtained based on the second protocol identifier; The target single-finger touch data is parsed according to the second communication protocol, and the target single-finger touch operation is determined based on the parsed target single-finger touch data.

5. A touch simulation method, characterized in that, A second data transmission application is applied to a second terminal device, the second terminal device having a touchpad. A first data transmission application stores a preset mapping table, the preset mapping table being used to record the mapping relationship between each multi-finger touch operation and each multi-finger touch operation event on the touchpad. The second data transmission application is communicatively connected to the first data transmission application of the first terminal device. The method includes: In response to a touch operation on the touchpad of the second terminal device, determine whether the touch operation is a multi-finger touch operation; When the touch operation is a multi-finger touch operation, target multi-finger touch data is sent to the first data transmission application so that the first terminal device determines the target multi-finger touch operation based on the target multi-finger touch data; according to the preset mapping table, a target multi-finger touch operation event corresponding to the target multi-finger touch operation is determined; and the system interface corresponding to the target multi-finger touch operation event is called to respond to the target multi-finger touch operation event.

6. A touch simulation device, characterized in that, The device is applied to a first terminal device, and a second terminal device has a touchpad. A first data transmission application stores a preset mapping table, which records the mapping relationship between each multi-finger touch operation and each multi-finger touch operation event on the touchpad. The first terminal device communicates with a second data transmission application of the second terminal device through the first data transmission application. The device includes: A target multi-finger touch operation determination module is used to determine a target multi-finger touch operation based on the target multi-finger touch data sent by a second data transmission application in response to the target multi-finger touch data. The multi-finger touch operation event determination module is used to determine the target multi-finger touch operation event corresponding to the target multi-finger touch operation according to the preset mapping table; The first system interface calling module is used to call the system interface corresponding to the target multi-finger touch operation event in response to the target multi-finger touch operation event.

7. A touch simulation device, characterized in that, An apparatus is applied to a second terminal device, the second terminal device having a touchpad, and a first data transmission application storing a preset mapping table. The preset mapping table records the mapping relationship between each multi-finger touch operation and each multi-finger touch operation event on the touchpad. The second terminal device is communicatively connected to the first data transmission application of the first terminal device via a second data transmission application. The apparatus includes: A multi-finger touch operation determination module is used to determine whether the touch operation is a multi-finger touch operation in response to a touch operation on the touchpad of the second terminal device. The touch data sending module is configured to send target multi-finger touch data to the first data transmission application when the touch operation is a multi-finger touch operation, so that the first terminal device determines the target multi-finger touch operation based on the target multi-finger touch data; determine the target multi-finger touch operation event corresponding to the target multi-finger touch operation according to the preset mapping table; and call the system interface corresponding to the target multi-finger touch operation event to respond to the target multi-finger touch operation event.

8. A first terminal device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the touch simulation method according to any one of claims 1-4.

9. A second terminal device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the touch simulation method of claim 5.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute and implement the touch simulation method of any one of claims 1-4, or the touch simulation method of claim 5.

Images