Touch simulation method and device, electronic equipment and storage medium

Abstract

The invention provides a touch simulation method and device, electronic equipment and a storage medium. The method comprises the following steps: reading a touch event equipment file of original equipment based on a system debugging tool so as to detect a touch event occurring in the original equipment; when it is detected that a touch event occurs in the original device, an original operation set is updated according to the touch event, the original operation set is used for recording touch data, and the touch data is used for representing touch operation received by the original device; according to display configuration information of a target device and the original device, correcting touch point coordinates contained in touch data recorded by the original operation set; a touch instruction is determined according to the corrected original operation set, and the touch instruction is used for simulating the touch operation received by the original device on the target device.

Description

Technical Field

[0001] The present disclosure relates to the field of software testing technology, and in particular to a touch simulation method, device, electronic device and storage medium. Background Art

[0002] Touch operation testing is often an important part of the software testing process. The same set of touch test processes often needs to be executed on multiple devices. Therefore, it is particularly important to generate touch commands to automatically complete the test process. At present, it is not uncommon to have relatively closed software systems or software systems that are specialized for a certain type of product. Generally speaking, there are few system interfaces that can be called (resulting in some software functions being unable to be implemented) and the permissions open to users are low (for example, third-party applications are not allowed to be installed), which results in performance statistics software and automatic testing software being unable to run normally on them, or even being unable to be installed on the system.

[0003] In the related technology, faced with the above situation, usually only the method of manually writing the underlying instruction set is used to realize automated testing. However, the writing of the underlying instruction set is difficult to complete in time during the software development process. At the same time, this development process is also very unintuitive, and the labor and time costs are high. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides a touch simulation method, device, electronic device and storage medium.

[0005] A first aspect of the present disclosure provides a touch simulation method, the method comprising:

[0006] Reading a touch event device file of an original device based on a system debugging tool to detect a touch event occurring in the original device;

[0007] When a touch event is detected in the original device, an original operation set is updated according to the touch event, wherein the original operation set is used to record touch data, and the touch data is used to represent the touch operation received by the original device;

[0008] According to the display configuration information of the target device and the original device, the touch point coordinates contained in the touch data recorded in the original operation set are corrected;

[0009] A touch instruction is determined according to the modified original operation set, wherein the touch instruction is used to simulate, on the target device, the touch operation received by the original device.

[0010] Optionally, updating the original operation set according to the touch event includes:

[0011] In the case where the touch control event is a contact tracking event or a touch pressing event, creating a data structure for storing contact point coordinates having a time sequence relationship;

[0012] In the case where the touch event is a touch point coordinate event, recording the coordinate value indicated by the touch point coordinate event;

[0013] In the case where the touch event is a synchronous reporting event, pairing the recorded coordinate values ​​to obtain current touch point coordinates, and storing the current touch point coordinates in the data structure;

[0014] When the touch event is a touch-up event, the data structure is used as touch data and recorded in the original operation set.

[0015] Optionally, the pairing of the recorded coordinate values ​​to obtain the current touch point coordinates includes:

[0016] In the case where the coordinate values ​​corresponding to some coordinate axes are not recorded, the current touch point coordinates are compensated based on at least one historical touch point coordinate stored in the data structure.

[0017] Optionally, the method further includes:

[0018] When there are a plurality of touch point coordinates in the data structure, the coordinate spacing between each two of which is less than a spacing threshold and which are adjacent in time sequence, the plurality of touch point coordinates are unified or part of the plurality of touch point coordinates are discarded.

[0019] Optionally, the modifying the coordinates contained in the touch data recorded in the original operation set according to the display configuration information of the target device and the original device includes:

[0020] Determining a coordinate scaling factor between the original device and the target device according to display configuration information of the target device and the original device, wherein the display configuration information includes a screen size and / or a display resolution;

[0021] The touch point coordinates included in the touch data recorded in the original operation set are scaled according to the coordinate scaling factor.

[0022] Optionally, determining the touch instruction according to the modified original operation set includes:

[0023] For each touch data in the corrected original operation set, determining the touch point coordinates contained in the touch data;

[0024] When the number of touch point coordinates included in the touch operation is less than or equal to the number threshold, determining a single-click instruction according to the touch point coordinates included in the touch operation, and determining the single-click instruction as the touch instruction corresponding to the touch operation;

[0025] When the number of touch point coordinates included in the touch operation is greater than the quantity threshold, a touch press instruction is determined according to the first touch point coordinate in the timing relationship included in the touch operation, a touch lift instruction is determined according to the last touch point coordinate in the timing relationship included in the touch operation, and at least one touch move instruction is determined according to the other touch point coordinates included in the touch operation, and the touch press instruction, the at least one touch move instruction and the touch lift instruction are determined as touch instructions corresponding to the touch operation.

[0026] Optionally, the method further includes:

[0027] Based on a system debugging tool, inputting the touch control instruction into the target device to simulate the touch control operation received by the original device on the target device;

[0028] Alternatively, the touch control instruction is input to a robotic arm so that the robotic arm simulates, on the target device, the touch control operation received by the original device.

[0029] A second aspect of the present disclosure provides a touch simulation device, the device comprising:

[0030] An event detection module, used for reading a touch event device file of an original device based on a system debugging tool to detect a touch event occurring in the original device;

[0031] A touch recording module, configured to update an original operation set according to a touch event detected in the original device, wherein the original operation set is used to record touch data, and the touch data is used to represent the touch operation received by the original device;

[0032] A coordinate correction module, used to correct the touch point coordinates contained in the touch data recorded in the original operation set according to the display configuration information of the target device and the original device;

[0033] The instruction generation module is used to determine a touch instruction according to the modified original operation set, wherein the touch instruction is used to simulate, on the target device, the touch operation received by the original device.

[0034] Optionally, when the touch recording module is used to update the original operation set according to the touch event, it is specifically used to:

[0035] In the case where the touch control event is a contact tracking event or a touch pressing event, creating a data structure for storing contact point coordinates having a time sequence relationship;

[0036] In the case where the touch event is a touch point coordinate event, recording the coordinate value indicated by the touch point coordinate event;

[0037] In the case where the touch event is a synchronous reporting event, pairing the recorded coordinate values ​​to obtain current touch point coordinates, and storing the current touch point coordinates in the data structure;

[0038] When the touch event is a touch-up event, the data structure is used as touch data and recorded in the original operation set.

[0039] Optionally, when the touch recording module is used to pair the recorded coordinate values ​​to obtain the current touch point coordinates, it is specifically used to:

[0040] In the case where the coordinate values ​​corresponding to some coordinate axes are not recorded, the current touch point coordinates are compensated based on at least one historical touch point coordinate stored in the data structure.

[0041] Optionally, the device further comprises:

[0042] The filtering module is used to unify the multiple touch point coordinates or discard part of the multiple touch point coordinates when there are multiple touch point coordinates in the data structure whose coordinate spacing between each two is less than the spacing threshold and are adjacent in time sequence.

[0043] Optionally, when the coordinate correction module is used to correct the coordinates contained in the touch data recorded in the original operation set according to the display configuration information of the target device and the original device, it is specifically used to:

[0044] Determining a coordinate scaling factor between the original device and the target device according to display configuration information of the target device and the original device, wherein the display configuration information includes a screen size and / or a display resolution;

[0045] The touch point coordinates included in the touch data recorded in the original operation set are scaled according to the coordinate scaling factor.

[0046] Optionally, when the instruction determination module is used to determine the touch instruction according to the modified original operation set, it is specifically used to:

[0047] For each touch data in the corrected original operation set, determining the touch point coordinates contained in the touch data;

[0048] When the number of touch point coordinates included in the touch operation is less than or equal to the number threshold, determining a single-click instruction according to the touch point coordinates included in the touch operation, and determining the single-click instruction as the touch instruction corresponding to the touch operation;

[0049] When the number of touch point coordinates included in the touch operation is greater than the quantity threshold, a touch press instruction is determined according to the first touch point coordinate in the timing relationship included in the touch operation, a touch lift instruction is determined according to the last touch point coordinate in the timing relationship included in the touch operation, and at least one touch move instruction is determined according to the other touch point coordinates included in the touch operation, and the touch press instruction, the at least one touch move instruction and the touch lift instruction are determined as touch instructions corresponding to the touch operation.

[0050] Optionally, the device further includes a touch simulation module, which is used to:

[0051] Based on a system debugging tool, inputting the touch control instruction into the target device to simulate the touch control operation received by the original device on the target device;

[0052] Alternatively, the touch control instruction is input to a robotic arm so that the robotic arm simulates, on the target device, the touch control operation received by the original device.

[0053] A third aspect of the present disclosure provides a computer program product, including a computer program / instruction, which implements the method described in the first aspect when executed by a processor.

[0054] A fourth aspect of the present disclosure provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the first aspect when executing the program.

[0055] A fifth aspect of the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in the first aspect.

[0056] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects:

[0057] The disclosed embodiment adopts the method of recording touch operations to intuitively realize the generation of test instructions, which is not easily restricted by system permissions and can still be applied in some relatively closed software systems that are specialized for a certain type of product, greatly improving the usability and universality of the touch-based software testing method. At the same time, the method provided by the disclosure provides developers with an extremely intuitive testing method, which greatly reduces the development workload of operators, and further reduces the manpower cost and time cost of software testing.

[0058] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0060] Figure 1 It is an application scenario diagram of a touch simulation method shown in some exemplary embodiments.

[0061] Figure 2 The figure is a flow chart of a touch simulation method shown in some exemplary embodiments.

[0062] Figure 3 is a flow chart of another touch simulation method shown in some exemplary embodiments.

[0063] Figure 4 is a flow chart of yet another touch simulation method shown in some exemplary embodiments.

[0064] Figure 5 is a block diagram of a touch simulation device shown in some exemplary embodiments.

[0065] Figure 6 It is a hardware structure diagram of an electronic device shown in some exemplary embodiments. DETAILED DESCRIPTION

[0066] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0067] The same product is usually suitable for devices with different screen sizes and screen resolutions (hereinafter, screen size, screen resolution and other parameters that may affect the layout of software controls are referred to as the display configuration information of the device). In order to truly simulate the operations of users in their daily lives and avoid possible interference factors as much as possible, touch testing is an important test acceptance link. This test will simulate the user's real touch operations on different devices, that is, make the tested process believe that these touch operations are received at an adaptive position. For touch testing, an exemplary implementation is to generate a system-level instruction set and input these instructions into the system console, or run these instructions through a debugging bridge to simulate these touch operations; another exemplary implementation is to generate an instruction set for a robotic arm (or other equipment independent of the device under test), and then control the movement of the robotic arm based on these instruction sets, so that the robotic arm simulates these touch operations on the device under test.

[0068] As described in the background technology, it is not uncommon to have relatively closed software systems or software systems specialized for a certain type of product. Generally speaking, they have few callable system interfaces (resulting in some software functions being unable to be implemented) and low permissions open to users (for example, installation of third-party applications is not allowed), resulting in performance statistics software and automatic testing software being unable to run normally on them, or even being unable to be installed on the system.

[0069] It should be understood that, unless otherwise specified, the units used in describing parameters such as touch point coordinates, control positions, and screen resolution in this disclosure are all based on the number of pixels, with the origin of the coordinates being the upper left corner.

[0070] In the related technology, faced with the above situation, usually only the method of manually writing the underlying instruction set is used to realize automated testing. However, the writing of the underlying instruction set is difficult to complete in time during the software development process. At the same time, this development process is also very unintuitive, and the labor and time costs are high.

[0071] In view of this, the present disclosure provides a touch simulation method, device, electronic device and storage medium.

[0072] The following briefly introduces exemplary events that occur in the present disclosure (taking the Android system as an example):

[0073] Contact tracking event: EV_ABS ABS_MT_TRACKING_ID, usually indicates the start of tracking a new contact ID (contact identification), which can be understood as the device identifying the contact at this time;

[0074] Touch point coordinate events: EV_ABS ABS_MT_POSITION_X (X-axis coordinate event) and EV_ABS ABS_MT_POSITION_Y (Y-axis coordinate event), etc., usually represent the current coordinate value of the touch point on a certain coordinate axis;

[0075] Touch press event: EV_KEY BTN_TOUCH DOWN, usually indicates the start of touch;

[0076] Touch up event: EV_KEY BTN_TOUCH UP, usually indicates the end of a touch process, that is, the touch point disappears;

[0077] Synchronous report event: EV_SYN SYN_REPORT, usually represents the end of an event. The event here is a logical concept, such as representing the event that occurs in a touch frame (or scan frame, that is, a complete touch scan process). For example, if the X-axis coordinate of the touch point has not changed in the current frame, the X-axis coordinate event may not have occurred, but the current touch frame has ended and the next touch scan has begun, then only the Y-axis coordinate event may appear (that is, only the touch coordinate event corresponding to part of the coordinate axis occurs), and then a synchronous report event occurs. Then, after the synchronous report event is detected, it can be considered that a scan frame has ended.

[0078] The following briefly introduces the exemplary instructions that appear in this disclosure (taking the Android Debug Bridge command line as an example):

[0079] Single-click command: For example, the input tap command is used to simulate a single click on a touch screen;

[0080] Touch press command: For example, input motionevent DOWN command is used to simulate the press operation on the touch screen;

[0081] Touch movement commands: such as input motionevent MOVE command, which is used to simulate sliding operation on the touch screen;

[0082] Touch up command: For example, the input motionevent UP command is used to simulate a touch up operation on the touch screen.

[0083] Next, the embodiments of the present disclosure are described in detail.

[0084] The first aspect of the present disclosure provides a touch simulation method. Figure 1 , which may include steps S101 to S104.

[0085] Step S101 : reading a touch event device file of an original device based on a system debugging tool to detect a touch event occurring in the original device.

[0086] Among them, system debugging tools are software tools used to monitor, analyze and solve various problems in computer systems, such as Android Debug Bridge (ADB) in Android system and WinDbg (Windows Debugger) in Windows system.

[0087] Exemplarily, the above method can be applied to a device independent of the original device (i.e., the device under test) (of course, the device to which the above method is applied and the device under test can be physically the same device, for example, if the device under test is a virtual machine running on a physical device, then the above method can also be applied to the operating system of the physical device itself, software for supporting the virtual machine, or other virtual machines), for example Figure 2 The debugging device in the system. The device applying the above method can establish a connection with the original device through a system debugging tool. In addition, the above method can be executed by one software module or by multiple software modules asynchronously, which is not limited in the present disclosure.

[0088] Based on this, touch events can be monitored without writing scripts on the original device, thus avoiding interference of system installation permissions on the software testing process. It also reduces the development difficulty for developers because there is no need to optimize the compatibility of test scripts for different devices (instead, the touch information of the original device is directly recorded on another device based on the system debug bridge).

[0089] Touch Event Device File (generally referred to as TouchEvent file) can refer to the device file in the operating system used to capture (and process) input events of the touch module (of course, it can also be peripherals such as mouse). These files can be regarded as an interface. They can usually interact with the hardware through the device driver and convert touch events into input signals that the system can understand. It is worth noting that there can be one or more touch event device files. The paths of touch event device files in different systems and different versions of the same system may be different. For example, in the Android system (taking Android 13 as an example), the TouchEvent file is usually located in the / dev / input / event6 path (this will be explained later as an example).

[0090] The above process is based on the system debugging tool to read the touch event device file in the device and directly obtain the touch event information, thereby avoiding the problem of high development difficulty and large system permission restrictions caused by the development of scripts and the acquisition of system bottom permissions to monitor event information. In other words, the above process is a simple read-only process. Through this simple read-only process, the acquisition of touch events can be completed without affecting the normal operation of the operating system and processes on the original device and without being restricted by the permissions of the operating system.

[0091] In some embodiments, the above touch events may include touch tracking events, touch pressing events, touch coordinate events, synchronous reporting events, touch lifting events, etc.

[0092] Step S102, when a touch event is detected in the original device, an original operation set is updated according to the touch event, wherein the original operation set is used to record touch data, and the touch data is used to represent the touch operation received by the original device.

[0093] The original operation set in the above steps can be a string directly used to record the event content (for example, directly record the data stream in the touch event device file one by one), or it can be an array used to record the touch point coordinates in chronological order, for example, in the form of "[{x:1001,y:200},{x:1101,y:200},...]", the coordinates of the X axis and Y axis in different frames are recorded in sequence, where {x:1001,y:200} and {x:1101,y:200} are respectively an element in the array. This can maximize the savings in storage space, because information other than the touch point coordinates can usually be determined based on reference quantities of other dimensions. For example, when the original operation set contains only one element, the user performs a single-click operation, and the touch point coordinates of the single-click operation are the touch point coordinate values ​​recorded in the element. This will be explained later. The original operation set will be explained later as an array that records the touch point coordinates in chronological order.

[0094] As an exemplary implementation, updating the original operation set according to the touch event may include:

[0095] In the case where the touch control event is a contact tracking event or a touch pressing event, creating a data structure for storing contact point coordinates having a time sequence relationship;

[0096] In the case where the touch event is a touch point coordinate event, recording the coordinate value indicated by the touch point coordinate event;

[0097] In the case where the touch event is a synchronous reporting event, pairing the recorded coordinate values ​​to obtain current touch point coordinates, and storing the current touch point coordinates in the data structure;

[0098] When the touch event is a touch-up event, the data structure is used as touch data and recorded in the original operation set.

[0099] The following is an example data stream of a touch event device file that appears on an Android 13 device:

[0100] / dev / input / event6:EV_ABS ABS_MT_TRACKING_ID 00000253

[0101] …

[0102] / dev / input / event6:EV_ABS ABS_MT_POSITION_X 00000298

[0103] / dev / input / event6:EV_ABS ABS_MT_POSITION_Y 00000926

[0104] / dev / input / event6:EV_KEY BTN_TOUCH DOWN

[0105] / dev / input / event6:EV_SYN SYN_REPORT 00000000

[0106] …

[0107] / dev / input / event6:EV_ABS ABS_MT_POSITION_X 00000296

[0108] / dev / input / event6:EV_SYN SYN_REPORT 00000000

[0109] / dev / input / event6:EV_ABS ABS_MT_TRACKING_ID ffffffff

[0110] …

[0111] / dev / input / event6:EV_KEY BTN_TOUCH UP

[0112] / dev / input / event6:EV_SYN SYN_REPORT 00000000.

[0113] Each data row in the above example can be regarded as a touch event. Among them, / dev / input / event6 is the touch event device file, EV_ABS is the event type, ABS_MT_TRACING_ID is the event instruction, and 00000395 is the value corresponding to the event. The above data row can be described as "{device file}:{event type}{event instruction}{instruction value}" format. Exemplarily, the above data row can be obtained by monitoring the device event based on the "adb-s{deviceId}shell getevent-l" command line of the Android Debug Bridge, and then filtering out the data of the " / dev / input / event6" device file.

[0114] First, you can get the start flag of the operation from the monitored real-time data. Figure 3 In the aforementioned Android13 example, it can generally be considered that when the value of the "ABT_MT_TRACKING_ID" event is not equal to "ffffffff" (that is, the event has a valid value, which usually refers to the ID of the contact), it means that a contact tracking event has occurred. For example, in the above data row, the data event type is "EV_ABS", the event instruction is "ABS_MT_TRACKING_ID", and the instruction value is not equal to "ffffffff" represents the start of an operation behavior.

[0115] Next, subsequent data rows with data event types of "EV_ABS" and event instructions of "ABS_MT_POSITION_X" (X coordinate) or "ABS_MT_POSITION_Y" (Y coordinate) can indicate changes in touch point coordinates during the touch process. These events are touch point coordinate events. At this time, the coordinate values ​​indicated by the touch point coordinate events can be recorded, for example, by converting the hexadecimal instruction value into a decimal instruction value, and then pairing the X coordinate and the Y coordinate, and finally storing them in an array.

[0116] The above pairing and storage process can be performed when a "SYN_REPORT" event (synchronous report event) occurs. The synchronous report event usually indicates that the X or Y coordinate has changed while the user is completing a touch track. At this time, the X and Y coordinates indicated by the touch point coordinate event that occurred before the synchronous report event can be paired. For example, before the first "SYN_REPORT" event in the above data row, the recorded X coordinate is 00000298 and the Y coordinate is 00000926, then the decimal coordinate pair {x:664,y:2334} can be determined and added to the array as the current touch point coordinates.

[0117] Before the "SYN_REPORT" instruction, a single coordinate change instruction may be received according to the change of the user's finger operation path. At this time, it is necessary to match the monitored X or Y coordinate with the last Y or X coordinate. In other words, the pairing of the recorded coordinate values ​​to obtain the current touch point coordinates may include: when the coordinate values ​​corresponding to some coordinate axes are not recorded, compensating the current touch point coordinates based on at least one historical touch point coordinate stored in the data structure.

[0118] For example, before the second "SYN_REPORT" event in the above data line, there is only an "ABS_MT_POSITION_X" instruction with a value of "00000296" (indicating that the X-axis coordinate is 662 in decimal), but no "ABS_MT_POSITION_Y" instruction (that is, the Y coordinate is missing). Then the X coordinate can be paired with the Y coordinate (historical contact coordinate) in the previous frame (or other previous frames) (for example, the array element "{x:662,y:2334}") and stored in the array.

[0119] Finally, when the event command "BTN_TOUCH" is monitored and the value is "UP" (i.e. a touch up event occurs), it usually indicates the end of a touch operation. At this time, the array (original operation set) recorded in the above process can be used to indicate a complete touch track (i.e. the single event independent array described in the figure). At this time, the array can be stored in other data structures or databases.

[0120] In some embodiments, during the above process (that is, the process of creating and updating the original operation set) or after the above process is completed (that is, after the original operation set corresponding to a touch track is obtained), the touch point coordinates recorded in the original operation set can also be corrected. For example, the method can also include: when there are multiple touch point coordinates in the data structure whose coordinate spacing between each two is less than the spacing threshold and are adjacent in time relationship, unify the multiple touch point coordinates or discard part of the multiple touch point coordinates.

[0121] The above process can be regarded as a similar filtering of the generated original operation set (considering small changes in the operation process as no change, and unifying them into a stable coordinate value). The purpose of this is to consider that the coordinates do not actually change between the adjacent frames when the change of X or Y generated in the adjacent frames of the operation process is less than a preset threshold (this threshold can usually be adjusted according to actual conditions), thereby avoiding the problem of "single click operation being misjudged as drag operation" due to touch module error.

[0122] During the execution process, illustratively, the execution logic of the above process may be: read in a touch point coordinate, determine the distance between the touch point coordinate and the touch point coordinate in the previous frame, and take the touch point coordinate in the previous frame as the touch point coordinate of the current frame when the distance is very small. Another illustrative example is that the execution logic of the above process may be: select a touch point coordinate as the reference coordinate, and then determine whether there are several frames before and after the reference coordinate whose average inter-frame displacement is less than a preset threshold. If so, the average value of the touch point coordinates of these frames may be taken as their corrected touch point coordinates, or the coordinates of the reference coordinates or other frames (such as the middle frame in the time sequence) may be directly used as the touch point coordinates of these frames, or part of these touch point coordinates may be directly discarded.

[0123] Step S103: correcting the touch point coordinates contained in the touch data recorded in the original operation set according to the display configuration information of the target device and the original device.

[0124] Specifically, since these tests may involve devices with different display configuration information, the size and layout of controls also need to be considered when simulating the same user touch operations on different devices.

[0125] For example, in Figure 2In the example shown, assuming that device A and device B have the same screen size, but the horizontal and vertical pixel numbers of the screen of device B are both twice that of device A, then in a common scaling logic, for the same software interface, when the interface runs full screen on device A, the position of the button may be (100,100) and the size may be (20,20), that is, a single click event within this range can trigger the button; and when the interface runs full screen on device B, its position may change based on the adaptation algorithm deployed in the system layer of device B or the application under test. For example, in a common logic, the position of the button changes to (200,200) and the size changes to (40,40). Similarly, a single click event within this range can trigger the button.

[0126] Then, if you need to complete the touch test on both device A and device B, you need to map the touch operation on the button on device A (touch coordinates are in the range of (100,100) to (120,120)) to the touch operation on the button on device B (touch coordinates are in the range of (200,200) to (240,240)). The concept of "targeting" above refers to targeting at the user operation level, that is, for the user, the touch operation on A should theoretically correspond to which position of the display module of device B.

[0127] That is to say, optionally, correcting the coordinates contained in the touch data recorded by the original operation set according to the display configuration information of the target device and the original device may include: determining a coordinate scaling factor between the original device and the target device according to the display configuration information of the target device and the original device, wherein the display configuration information includes a screen size and / or a display resolution; and scaling the touch point coordinates contained in the touch data recorded by the original operation set according to the coordinate scaling factor.

[0128] Generally speaking, for the current common software interface scaling logic, the touch data (the touch point coordinates) recorded in the original operation set can be corrected in the following way to adapt it to the target device: assuming that the pixel density of the original device in the X-axis direction is w1 and the pixel density in the Y-axis direction is h1, and the pixel density of the target device in the X-axis direction is w2 and the pixel density in the Y-axis direction is h2, then each X coordinate recorded in the original operation set can be converted as X=POSITION_X*(w2 / w1), and each X coordinate can be converted as Y=POSITION_Y*(h2 / h1), where POSITION_X and POSITION_Y are respectively the X value, Y value and before the conversion, and w2 / w1 and h2 / h1 can be regarded as coordinate scaling coefficients.

[0129] It should be understood that if the software or operating system has its own unique software interface scaling logic, then the above correction logic can be adapted to this unique software interface scaling logic so that the same touch operation can be simulated on device B. The "same touch operation" described here refers to a touch operation that can theoretically lead to the same operation result. Since the touch operation is for the interface for the touch module, the above correction of the original operation set can be based on the display configuration information of the original device (such as the size, resolution, and pixel density of the display module). It should be understood that the present disclosure does not exclude input devices such as a mouse, a pointing stick (such as TrackPoint or TrackStick), and a touch pad independent of the display module (commonly found in laptops) (that is, the above touch operation for the original device is input by these input devices), but no matter which device it is, the cursor it controls (the cursor can correspond to the touch point) has a corresponding relationship with the screen coordinate system of the original device, therefore, the coordinate conversion process of these input devices is also applicable to the above principle, and some additional conversion logic related to peripherals will not be repeated.

[0130] Step S104: determining a touch instruction according to the modified original operation set, wherein the touch instruction is used to simulate, on the target device, the touch operation received by the original device.

[0131] After the original operation set is modified, the touch operations recorded therein can be determined to determine the touch instructions, and the specific instruction correspondence can be determined according to the devices that execute these instructions.

[0132] In some embodiments, determining the touch instruction according to the modified original operation set may include:

[0133] For each touch data in the corrected original operation set, determining the touch point coordinates contained in the touch data;

[0134] When the number of touch point coordinates included in the touch operation is less than or equal to the number threshold, determining a single-click instruction according to the touch point coordinates included in the touch operation, and determining the single-click instruction as the touch instruction corresponding to the touch operation;

[0135] When the number of touch point coordinates included in the touch operation is greater than the quantity threshold, a touch press instruction is determined according to the first touch point coordinate in the timing relationship included in the touch operation, a touch lift instruction is determined according to the last touch point coordinate in the timing relationship included in the touch operation, and at least one touch move instruction is determined according to the other touch point coordinates included in the touch operation, and the touch press instruction, the at least one touch move instruction and the touch lift instruction are determined as touch instructions corresponding to the touch operation.

[0136] See also Figure 4 , where the "filtering" in the "filtered array" refers to the correction process and the similar filtering process exemplarily described above. Since the number of touch coordinates contained in the original operation set actually reflects the time consumed by this touch operation (for the device, the reporting of touch coordinates is usually related to the number of scan frames and the specific changes in the touch points), when the array length is 1 (or other values ​​less than the quantity threshold), it can be considered that the user (that is, the developer who enters the touch operation on the original device) actually wants to execute a single-click instruction in this touch process (because the duration is very short or the change in the coordinates is very small), then a single-click instruction can be generated based on these coordinates (the coordinates of this instruction can be determined by mean filtering or direct value acquisition), such as the input tap{}{} instruction (taking the ADB instruction line as an example).

[0137] When the number of touch point coordinates in the original operation set is large, it can be considered that the user performs a sliding (dragging) operation in this process, and this process is usually divided into three processes: finger pressing, finger moving, and finger lifting. In this case, the 0th data can be taken (the "0" here is the number of the data item in the array, and the 0th data is the first data; in addition, the first data here can also be the first N data, and N can be related to the above-mentioned quantity threshold or the characteristics of the original device and the target device, for example, the above-mentioned quantity threshold is directly taken as N), and a touch press instruction is generated based on the data.

[0138] Next, the touch lift instruction can be generated based on the last touch point coordinate (i.e., the element number is len-2, where len is the array length; similar to the previous text, the "one" here can also be M, and M can also be related to the above-mentioned quantity threshold or the characteristics of the original device or the target device, or can be other preset values), and the remaining data in the array is the touch point movement process contained in the touch trajectory. Therefore, based on these remaining data (that is, the data remaining in the original operation set after removing the data used to generate the touch press instruction and the data used to generate the touch lift instruction), at least one touch movement instruction can be generated according to the timing relationship. After the above instructions are arranged according to the timing relationship, they can be input into the original device or another device (such as a target device or a robotic arm), so as to complete the playback of the touch trajectory and automatically complete the touch test on different devices. The operations that the operator needs to complete in this process are only to perform instance operations on the original device (such as Figure 2 ), extremely efficient.

[0139] For example, in some embodiments, as described above, the method may further include: based on a system debugging tool, inputting the touch command to the target device to simulate the touch operation received by the original device on the target device. Or, inputting the touch command to a robotic arm so that the robotic arm simulates the touch operation received by the original device on the target device. Both of the above exemplary methods can achieve complete decoupling of the target device and the test script, that is, in addition to not needing to install the test script on the original device, there is no need to install the test script on the target device (in the case of the robotic arm, it is not even necessary to establish a connection with the target device), further improving the usability of the method. For the introduction to the scenario in this regard, please refer to the previous text and will not be repeated here.

[0140] The disclosed embodiment adopts the method of recording touch operation to intuitively realize the generation of test instructions. Since the test instructions are generated based on the detected touch events, and the touch events are realized based on the system debugging tool and the method of reading the touch event device file, this process does not need to install the application and run the script on the original device (this is a read-only process for the file), so it is not easily restricted by system permissions, and can still be applied in some relatively closed software systems that are specialized for a certain type of product, which greatly improves the availability and universality of the touch-based software testing method. At the same time, based on the above implementation method, the operator only needs to simulate the real user on the original device to perform the touch operation, and then obtain the touch instruction for replaying the operation process on the target device, which is very intuitive and has a good user experience, greatly reducing the development workload of the operator, which also further reduces the manpower cost and time cost of software testing.

[0141] Corresponding to the embodiments of the aforementioned method, the present disclosure also provides embodiments of a device and a terminal to which the device is applied.

[0142] The second aspect of the present disclosure provides a touch simulation device, see Figure 5 , the device comprises:

[0143] An event detection module 501 is used to read a touch event device file of an original device based on a system debugging tool to detect a touch event occurring in the original device;

[0144] A touch recording module 502, configured to update an original operation set according to a touch event detected in the original device, wherein the original operation set is used to record touch data, and the touch data is used to represent the touch operation received by the original device;

[0145] A coordinate correction module 503, used to correct the touch point coordinates contained in the touch data recorded in the original operation set according to the display configuration information of the target device and the original device;

[0146] The instruction generation module 504 is used to determine a touch instruction according to the modified original operation set, wherein the touch instruction is used to simulate, on the target device, the touch operation received by the original device.

[0147] Optionally, when the touch recording module is used to update the original operation set according to the touch event, it is specifically used to:

[0148] In the case where the touch control event is a contact tracking event or a touch pressing event, creating a data structure for storing contact point coordinates having a time sequence relationship;

[0149] In the case where the touch event is a touch point coordinate event, recording the coordinate value indicated by the touch point coordinate event;

[0150] In the case where the touch event is a synchronous reporting event, pairing the recorded coordinate values ​​to obtain current touch point coordinates, and storing the current touch point coordinates in the data structure;

[0151] When the touch event is a touch-up event, the data structure is used as touch data and recorded in the original operation set.

[0152] Optionally, when the touch recording module is used to pair the recorded coordinate values ​​to obtain the current touch point coordinates, it is specifically used to:

[0153] In the case where the coordinate values ​​corresponding to some coordinate axes are not recorded, the current touch point coordinates are compensated based on at least one historical touch point coordinate stored in the data structure.

[0154] Optionally, the device further comprises:

[0155] The filtering module is used to unify the multiple touch point coordinates or discard part of the multiple touch point coordinates when there are multiple touch point coordinates in the data structure whose coordinate spacing between each two is less than the spacing threshold and are adjacent in time sequence.

[0156] Optionally, when the coordinate correction module is used to correct the coordinates contained in the touch data recorded in the original operation set according to the display configuration information of the target device and the original device, it is specifically used to:

[0157] Determining a coordinate scaling factor between the original device and the target device according to display configuration information of the target device and the original device, wherein the display configuration information includes a screen size and / or a display resolution;

[0158] The touch point coordinates included in the touch data recorded in the original operation set are scaled according to the coordinate scaling factor.

[0159] Optionally, when the instruction determination module is used to determine the touch instruction according to the modified original operation set, it is specifically used to:

[0160] For each touch data in the corrected original operation set, determining the touch point coordinates contained in the touch data;

[0161] When the number of touch point coordinates included in the touch operation is less than or equal to the number threshold, determining a single-click instruction according to the touch point coordinates included in the touch operation, and determining the single-click instruction as the touch instruction corresponding to the touch operation;

[0162] When the number of touch point coordinates included in the touch operation is greater than the quantity threshold, a touch press instruction is determined according to the first touch point coordinate in the timing relationship included in the touch operation, a touch lift instruction is determined according to the last touch point coordinate in the timing relationship included in the touch operation, and at least one touch move instruction is determined according to the other touch point coordinates included in the touch operation, and the touch press instruction, the at least one touch move instruction and the touch lift instruction are determined as touch instructions corresponding to the touch operation.

[0163] Optionally, the device further includes a touch simulation module, which is used to:

[0164] Based on a system debugging tool, inputting the touch control instruction into the target device to simulate the touch control operation received by the original device on the target device;

[0165] Alternatively, the touch control instruction is input to a robotic arm so that the robotic arm simulates, on the target device, the touch control operation received by the original device.

[0166] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.

[0167] A third aspect of the present disclosure provides a computer program product, including a computer program / instruction, which implements the method described in the first aspect when executed by a processor.

[0168] For the device embodiments and computer program product embodiments, since they basically correspond to the method embodiments, the relevant parts can refer to the partial description of the method embodiments. In addition, the device embodiments described above are merely schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the disclosed solution. Those of ordinary skill in the art can understand and implement it without paying creative labor.

[0169] In a fourth aspect, the embodiments of the touch simulation device provided by the present disclosure can be applied to electronic devices. Figure 6 , which exemplarily shows a hardware schematic diagram of an electronic device. For example, the device 600 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0170] The device 600 may include one or more of the following components: a processing component 601 , a memory 602 , a power component 603 , a multimedia component 604 , an audio component 605 , an input / output (I / O) interface 606 , a sensor component 607 , and a communication component 608 .

[0171] The processing component 601 generally controls the overall operation of the device 600, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 601 may include one or more processors 609 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 601 may include one or more modules to facilitate the interaction between the processing component 601 and other components. For example, the processing component 601 may include a multimedia module to facilitate the interaction between the multimedia component 604 and the processing component 601.

[0172] The memory 602 is configured to store various types of data to support operations on the device 600. Examples of such data include instructions for any application or method operating on the device 600, contact data, phone book data, messages, pictures, videos, etc. The memory 602 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0173] The power component 603 provides power to the various components of the device 600. The power component 603 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 600.

[0174] The multimedia component 604 includes a screen providing an output interface between the device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 604 includes a front camera and / or a rear camera. When the device 600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0175] The audio component 605 is configured to output and / or input audio signals. For example, the audio component 605 includes a microphone (MIC), and when the device 600 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 602 or sent via the communication component 608. In some embodiments, the audio component 605 also includes a speaker for outputting audio signals.

[0176] I / O interface 606 provides an interface between processing component 601 and peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0177] The sensor assembly 607 includes one or more sensors for providing various aspects of status assessment for the device 600. For example, the sensor assembly 607 can detect the open / closed state of the device 600, the relative positioning of components, such as the display and keypad of the device 600, and the sensor assembly 607 can also detect the position change of the device 600 or a component of the device 600, the presence or absence of user contact with the device 600, the orientation or acceleration / deceleration of the device 600, and the temperature change of the device 600. The sensor assembly 607 can also include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 607 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 607 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0178] The communication component 608 is configured to facilitate wired or wireless communication between the device 600 and other devices. The device 600 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G or 5G or a combination thereof. In an exemplary embodiment, the communication component 608 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 608 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0179] In an exemplary embodiment, the device 600 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the touch simulation method of the above-mentioned electronic device.

[0180] In a fifth aspect, the present disclosure, in an exemplary embodiment, further provides a non-temporary computer-readable storage medium including instructions, such as a memory 602 including instructions, and the instructions can be executed by a processor 609 of a device 600 to complete the touch simulation method of the electronic device. For example, the non-temporary computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0181] The above describes specific embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0182] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the inventions claimed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not claimed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0183] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

[0184] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the scope of protection of the present disclosure.

Claims

1. A touch simulation method, characterized in that: The method comprises: Reading a touch event device file of an original device based on a system debugging tool to detect a touch event occurring in the original device; When a touch event is detected in the original device, an original operation set is updated according to the touch event, wherein the original operation set is used to record touch data, and the touch data is used to represent the touch operation received by the original device; According to the display configuration information of the target device and the original device, the touch point coordinates contained in the touch data recorded in the original operation set are corrected; A touch instruction is determined according to the modified original operation set, wherein the touch instruction is used to simulate, on the target device, the touch operation received by the original device.

2. The touch simulation method according to claim 1, characterized in that: The updating of the original operation set according to the touch event includes: In the case where the touch control event is a contact tracking event or a touch pressing event, creating a data structure for storing contact point coordinates having a time sequence relationship; In the case where the touch event is a touch point coordinate event, recording the coordinate value indicated by the touch point coordinate event; In the case where the touch event is a synchronous reporting event, pairing the recorded coordinate values ​​to obtain current touch point coordinates, and storing the current touch point coordinates in the data structure; When the touch event is a touch-up event, the data structure is used as touch data and recorded in the original operation set.

3. The touch simulation method according to claim 2, characterized in that: The pairing of the recorded coordinate values ​​to obtain the current touch point coordinates includes: In the case where the coordinate values ​​corresponding to some coordinate axes are not recorded, the current touch point coordinates are compensated based on at least one historical touch point coordinate stored in the data structure.

4. The touch simulation method according to claim 2, characterized in that: The method further comprises: When there are a plurality of touch point coordinates in the data structure, the coordinate spacing between each two of which is less than a spacing threshold and which are adjacent in time sequence, the plurality of touch point coordinates are unified or part of the plurality of touch point coordinates are discarded.

5. The touch simulation method according to claim 1, characterized in that: The step of correcting the coordinates contained in the touch data recorded in the original operation set according to the display configuration information of the target device and the original device includes: Determining a coordinate scaling factor between the original device and the target device according to display configuration information of the target device and the original device, wherein the display configuration information includes a screen size and / or a display resolution; The touch point coordinates included in the touch data recorded in the original operation set are scaled according to the coordinate scaling factor.

6. The touch simulation method according to claim 1, characterized in that: The step of determining the touch control instruction according to the modified original operation set includes: For each touch data in the corrected original operation set, determining the touch point coordinates contained in the touch data; When the number of touch point coordinates included in the touch operation is less than or equal to the number threshold, determining a single-click instruction according to the touch point coordinates included in the touch operation, and determining the single-click instruction as the touch instruction corresponding to the touch operation; When the number of touch point coordinates included in the touch operation is greater than the quantity threshold, a touch press instruction is determined according to the first touch point coordinate in the timing relationship included in the touch operation, a touch lift instruction is determined according to the last touch point coordinate in the timing relationship included in the touch operation, and at least one touch move instruction is determined according to the other touch point coordinates included in the touch operation, and the touch press instruction, the at least one touch move instruction and the touch lift instruction are determined as touch instructions corresponding to the touch operation.

7. The touch simulation method according to claim 1, characterized in that: The method further comprises: Based on a system debugging tool, input the touch control instruction to the target device to simulate the touch control operation received by the original device on the target device; or, The touch control instruction is input to the robotic arm, so that the robotic arm simulates the touch control operation received by the original device on the target device.

8. A touch simulation device, characterized in that: The device comprises: An event detection module, used for reading a touch event device file of an original device based on a system debugging tool to detect a touch event occurring in the original device; A touch recording module, configured to update an original operation set according to a touch event detected in the original device, wherein the original operation set is used to record touch data, and the touch data is used to represent the touch operation received by the original device; A coordinate correction module, used to correct the touch point coordinates contained in the touch data recorded in the original operation set according to the display configuration information of the target device and the original device; The instruction generation module is used to determine a touch instruction according to the modified original operation set, wherein the touch instruction is used to simulate, on the target device, the touch operation received by the original device.

9. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images