Device control method, device, control terminal, main controlled device and medium

Abstract

The present invention provides a device control method, apparatus, control terminal, master controlled device and medium, and relates to the field of data processing technology. The method comprises: sending a preset operation acquisition script to the master controlled device, so that the master controlled device acquires the input screen touch operation instruction based on the operation acquisition script, and performs the screen operation corresponding to the screen touch operation instruction; receiving the screen touch operation instruction sent by the master controlled device; sending the screen touch operation instruction to at least one slave controlled device, so that each slave controlled device performs the corresponding operation based on the screen touch operation instruction. The control terminal is used to send the screen touch operation instruction to at least one slave controlled device, and the slave controlled device is controlled to perform the corresponding operation, so that the control of multiple controlled devices can be realized, and the delay of the control of the master controlled device and the slave controlled device can be avoided, so as to improve the real-time performance of the control, and make the control of the slave controlled device and the master controlled device more accurate.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a device control method, device, control terminal, main controlled device and medium. Background Art

[0002] With the development of science and technology, there are more and more electronic devices. Testing of electronic devices is also very important. Testing of applications installed on electronic devices is becoming more and more common. Therefore, testing based on the content displayed on the screen of electronic devices has also become a hot topic in research.

[0003] In the related art, the screen of a controlled device is displayed on a control device by remote screen projection, and the controlled device can be controlled by inputting touch operations on the screen image of the controlled device displayed by the control device.

[0004] However, in the related art, only one controlled device can be controlled by screen projection, and a delay will occur, thereby reducing the accuracy of controlling the controlled device. Summary of the invention

[0005] The purpose of the present invention is to provide a device control method, apparatus, control terminal, main controlled device and medium to address the deficiencies in the above-mentioned prior art, so as to solve the problem in the related art that only one controlled device can be controlled by screen projection, and delays will occur, thereby reducing the accuracy of control of the controlled device.

[0006] To achieve the above purpose, the technical solution adopted by the embodiment of the present invention is as follows:

[0007] In a first aspect, an embodiment of the present invention provides a device control method, including:

[0008] Sending a preset operation acquisition script to the master controlled device, so that the master controlled device acquires the input screen touch operation instruction based on the operation acquisition script, and performs the screen operation corresponding to the screen touch operation instruction;

[0009] Receiving the screen touch operation instruction sent by the master controlled device;

[0010] The screen touch operation instruction is sent to at least one slave controlled device, so that each slave controlled device performs a corresponding operation based on the screen touch operation instruction.

[0011] Optionally, the method further includes:

[0012] Sending a file acquisition command to the master controlled device;

[0013] Receiving paths of multiple device files returned by the master controlled device based on the file acquisition command;

[0014] Determining a path of a device file for touch operation from the paths of the multiple device files as a target file path;

[0015] The target file path is sent to the master controlled device, so that the master controlled device acquires the screen touch operation instruction from the device file under the target file path by executing the operation acquisition script.

[0016] Optionally, the method further includes:

[0017] A script execution command is sent to the master controlled device, so that the master controlled device executes the operation acquisition script through the script execution command, and acquires the screen touch operation instruction from the device file under the target file path.

[0018] Optionally, if there are multiple screen touch operation instructions;

[0019] Before sending the screen touch operation instruction to at least one slave controlled device, the method further includes:

[0020] Combining the plurality of the screen touch operation instructions to obtain at least one operation command, each operation command including: at least two of the screen touch operation instructions;

[0021] The sending the screen touch operation instruction to at least one slave controlled device includes:

[0022] The at least one operation command is sent to the at least one slave controlled device.

[0023] Optionally, before sending the at least one operation command to the at least one slave controlled device, the method further includes:

[0024] Convert each operation command into corresponding screen operation information;

[0025] According to the screen operation information, using the touch operation library corresponding to each slave controlled device, generating a screen touch operation instruction corresponding to each slave controlled device;

[0026] The sending the at least one operation command to the at least one slave controlled device comprises:

[0027] Sending a screen touch operation instruction corresponding to each slave controlled device to each slave controlled device.

[0028] Optionally, the step of generating, according to the screen operation information, a screen touch operation library corresponding to each slave controlled device, and generating a screen touch operation instruction corresponding to each slave controlled device includes:

[0029] adjusting the operation coordinates in the screen operation information according to the resolution of each slave controlled device;

[0030] According to the adjusted screen operation information, the touch operation library corresponding to each slave controlled device is used to generate a screen touch operation instruction corresponding to each slave controlled device.

[0031] In a second aspect, an embodiment of the present invention further provides a device control method, including:

[0032] Receive the operation acquisition script sent by the control terminal;

[0033] Based on the operation acquisition script, the input screen touch operation instruction is acquired, and the operation corresponding to the screen touch operation instruction is executed;

[0034] The screen touch operation instruction is sent to the control terminal, so that the control terminal sends the screen touch operation instruction to at least one slave controlled device, so that each slave controlled device performs a corresponding operation based on the screen touch operation instruction.

[0035] Optionally, before acquiring the input screen touch operation instruction based on the operation acquisition script, the method further includes:

[0036] Receiving a file acquisition command sent by the control terminal;

[0037] Sending multiple device file paths based on the file acquisition command, so that the control terminal determines the path of the device file for touch operation from the multiple device file paths as the target file path;

[0038] Receiving the target file path sent by the control terminal;

[0039] The step of acquiring the input screen touch operation instruction based on the operation acquisition script includes:

[0040] The screen touch operation instruction is obtained from the device file under the target file path by executing the operation acquisition script.

[0041] Optionally, before acquiring the screen touch operation instruction from the device file under the target file path by executing the operation acquisition script, the method further includes:

[0042] Receiving a script execution command sent by the control terminal;

[0043] The step of executing the operation acquisition script to acquire the screen touch operation instruction from the device file under the target file path includes:

[0044] The operation acquisition script is executed through the script execution command to obtain the screen touch operation instruction from the device file under the target file path.

[0045] In a third aspect, an embodiment of the present invention provides a device control apparatus, including:

[0046] A first sending module, configured to send a preset operation acquisition script to a master controlled device, so that the master controlled device acquires an input screen touch operation instruction based on the operation acquisition script, and performs a screen operation corresponding to the screen touch operation instruction;

[0047] A receiving module, used for receiving the screen touch operation instruction sent by the master controlled device;

[0048] The second sending module is used to send the screen touch operation instruction to at least one slave controlled device, so that each slave controlled device performs a corresponding operation based on the screen touch operation instruction.

[0049] Optionally, the device further comprises:

[0050] A third sending module, used for sending a file acquisition command to the master controlled device;

[0051] A first receiving module, configured to receive paths of multiple device files returned by the master controlled device based on the file acquisition command;

[0052] A determination module, configured to determine a path of a device file for touch operation from the paths of the plurality of device files as a target file path;

[0053] The fourth sending module is used to send the target file path to the master controlled device, so that the master controlled device obtains the screen touch operation instruction from the device file under the target file path by executing the operation acquisition script.

[0054] Optionally, the device further comprises:

[0055] The fifth sending module is used to send a script execution command to the master controlled device, so that the master controlled device executes the operation acquisition script through the script execution command and acquires the screen touch operation instruction from the device file under the target file path.

[0056] Optionally, if there are multiple screen touch operation instructions;

[0057] The device also includes:

[0058] A combining module, used for combining the plurality of the screen touch operation instructions to obtain at least one operation command, each operation command including: at least two of the screen touch operation instructions;

[0059] The second sending module is further configured to send the at least one operation command to the at least one slave controlled device.

[0060] Optionally, the device further comprises:

[0061] A conversion module, used for converting each operation command into corresponding screen operation information;

[0062] A generating module, configured to generate, according to the screen operation information, a screen touch operation instruction corresponding to each slave controlled device by using a touch operation library corresponding to each slave controlled device;

[0063] The second sending module is further configured to send the screen touch operation instruction corresponding to each slave controlled device to each slave controlled device.

[0064] Optionally, the generating module is further used to adjust the operation coordinates in the screen operation information according to the resolution of each slave controlled device; and based on the adjusted screen operation information, use the touch operation library corresponding to each slave controlled device to generate screen touch operation instructions corresponding to each slave controlled device.

[0065] In a fourth aspect, an embodiment of the present invention provides a device control apparatus, including:

[0066] A receiving module, used for receiving an operation acquisition script sent by a control terminal;

[0067] An acquisition module, used to acquire a script based on the operation, acquire an input screen touch operation instruction, and execute an operation corresponding to the screen touch operation instruction;

[0068] The sending module is used to send the screen touch operation instruction to the control terminal, so that the control terminal sends the screen touch operation instruction to at least one slave controlled device, so that each slave controlled device performs a corresponding operation based on the screen touch operation instruction.

[0069] Optionally, the device further comprises:

[0070] A first receiving module, used for receiving a file acquisition command sent by the control terminal;

[0071] A first sending module, configured to send paths of multiple device files based on the file acquisition command, so that the control terminal determines the path of the device file for touch operation from the paths of the multiple device files as the target file path;

[0072] A second receiving module, used for receiving the target file path sent by the control terminal;

[0073] The acquisition module is used to acquire the screen touch operation instruction from the device file under the target file path by executing the operation acquisition script.

[0074] Optionally, the device further comprises:

[0075] A third receiving module, used to receive a script execution command sent by the control terminal;

[0076] The acquisition module is further used to execute the operation acquisition script through the script execution command to acquire the screen touch operation instruction from the device file under the target file path.

[0077] In a fifth aspect, an embodiment of the present invention provides a control terminal, comprising: a memory and a processor, wherein the memory stores a computer program executable by the processor, and when the processor executes the computer program, the method described in any one of the first aspects is implemented.

[0078] In a sixth aspect, an embodiment of the present invention provides a master controlled device, comprising: a memory and a processor, wherein the memory stores a computer program executable by the processor, and when the processor executes the computer program, the method described in any one of the second aspects above is implemented.

[0079] In a seventh aspect, an embodiment of the present invention provides a storage medium having a computer program stored thereon. When the computer program is read and executed, the method described in any one of the first and second aspects above is implemented.

[0080] The beneficial effect of the present invention is as follows: an embodiment of the present invention provides a device control method, including: sending a preset operation acquisition script to a master controlled device, so that the master controlled device acquires an input screen touch operation instruction based on the operation acquisition script, and performs a screen operation corresponding to the screen touch operation instruction; receiving the screen touch operation instruction sent by the master controlled device; sending the screen touch operation instruction to at least one slave controlled device, so that each slave controlled device performs a corresponding operation based on the screen touch operation instruction. By acquiring the input screen touch operation instruction based on the operation acquisition script by the master controlled device, using a control terminal to send the screen touch operation instruction to at least one slave controlled device, and controlling the slave controlled device to perform the corresponding operation, control of multiple controlled devices can be achieved, and delays in controlling the master controlled device and the slave controlled device can be avoided, thereby improving the real-time performance of the control and making the control of the slave controlled device and the master controlled device more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0082] Figure 1 A schematic diagram of the structure of a device control system provided by an embodiment of the present invention;

[0083] Figure 2 A schematic diagram of a flow chart of a device control method provided by an embodiment of the present invention;

[0084] Figure 3 A schematic diagram of a flow chart of a device control method provided by an embodiment of the present invention;

[0085] Figure 4 A schematic diagram of a flow chart of a device control method provided by an embodiment of the present invention;

[0086] Figure 5 A schematic diagram of a flow chart of a device control method provided by an embodiment of the present invention;

[0087] Figure 6 A schematic diagram of a flow chart of a device control method provided by an embodiment of the present invention;

[0088] Figure 7 A schematic diagram of a flow chart of a device control method provided by an embodiment of the present invention;

[0089] Figure 8 A schematic diagram of a flow chart of a device control method provided by an embodiment of the present invention;

[0090] Fig. 9 A schematic diagram of a flow chart of a device control method provided by an embodiment of the present invention;

[0091] Fig.10 A schematic diagram of an interactive flow of a device control method provided by an embodiment of the present invention;

[0092] Fig.11 A schematic diagram of the structure of a device control device provided by an embodiment of the present invention;

[0093] Fig.12 A schematic diagram of the structure of a device control device provided by an embodiment of the present invention;

[0094] Fig.13 A schematic diagram of the structure of a control terminal provided by an embodiment of the present invention;

[0095] Fig.14 A schematic diagram of the structure of a master controlled device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0096] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0097] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0098] In the description of the present application, it should be noted that if the terms "upper", "lower", etc. appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the drawings, or is the orientation or position relationship in which the product of the application is usually placed when used. It is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application.

[0099] In addition, the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0100] It should be noted that, in the absence of conflict, the features in the embodiments of the present application may be combined with each other.

[0101] Figure 1 A schematic diagram of a device control system provided by an embodiment of the present invention is shown in FIG. Figure 1 As shown, the device control system may include: a control terminal 101 , a master controlled device 102 , and a slave controlled device 103 .

[0102] The control terminal 101 is respectively connected to the master controlled device 102 and the slave controlled device 103, which can be collectively referred to as controlled devices. The communication connection can be wired communication connection or wireless communication connection.

[0103] In an embodiment of the present application, the controlled device can be connected to the computer using USB (Universal Serial Bus), and then the USB debugging mode is turned on. The control terminal can execute the controlled device acquisition command to obtain the information of the connected controlled device. If the controlled device needs to be connected wirelessly, the controlled device can be connected to the same network as the control terminal. The control terminal executes the open port command to open the port on the controlled device. The control terminal executes the connect command to connect the controlled device from the control terminal. After the wireless connection is successful, the wired connection between the controlled device and the control terminal can be disconnected.

[0104] It should be noted that the controlled device acquisition command can be the adb devices command; the port opening command can be adb tcpip [port], where port is the port number; the connection command can be adb connect [device_ip]: [port], where device_ip is the ip (Internet Protocol Address) of the Android device, and port is the opened port number.

[0105] In some embodiments, the control terminal 101 can send a preset operation acquisition script to the master controlled device 102; the master controlled device 102 can receive the operation acquisition script sent by the control terminal 101, and based on the operation acquisition script, obtain the input screen touch operation instruction, and execute the operation corresponding to the screen touch operation instruction, and send the screen touch operation instruction to the control terminal 101; the control terminal 101 can receive the screen touch operation instruction sent by the master controlled device 102, and send the screen touch operation instruction to at least one slave controlled device 103, so that each slave controlled device 103 performs the corresponding operation based on the screen touch operation instruction.

[0106] It should be noted that the master controlled device 102 and the slave controlled device 103 can be movable devices, for example, the master controlled device 102 and the slave controlled device 103 can be mobile phones, tablet computers, etc. Of course, the master controlled device 102 and the slave controlled device 103 can be other devices with display, touch and processing functions, and the embodiments of the present application do not impose specific restrictions on this.

[0107] In addition, the control terminal 101 may be a device with processing functions, for example, the control terminal 101 may be a desktop computer, a laptop computer, and the like.

[0108] In the embodiment of the present application, the master controlled device 102 and the slave controlled device 103 may also be Android devices, and the master controlled device 102 and the slave controlled device 103 may be devices of different models or devices of the same model. For example, the master controlled device 102 and the slave controlled device 103 may have different brands, system versions, hardware configurations, etc.

[0109] In actual applications, the master controlled device 102 and the slave controlled device 103 may be installed with developed applications, which may be game applications. In order to ensure that the game applications run normally on different controlled devices, the game applications installed on the controlled devices may be tested. During the test, the user may input a touch operation for the game application on the master controlled device 102, and the screen touch operation instruction may also be an instruction for the game application, and the screen touch operation instruction is also sent to at least one slave controlled device 103. Precise control of the game applications installed on the master controlled device 102 and the slave controlled device 103 may be achieved, thereby achieving precise testing.

[0110] Optional, such as Figure 1 As shown, the device control system may further include: a server 104. The control terminal may also query information from the database of the server 104.

[0111] The device control method provided in the embodiment of the present application is explained below with the control terminal 101 as the execution subject.

[0112] Figure 2 A flow chart of a device control method provided by an embodiment of the present invention is as follows: Figure 2 As shown, the method may include:

[0113] S201: Send a preset operation acquisition script to a master controlled device, so that the master controlled device acquires an input screen touch operation instruction based on the operation acquisition script, and executes a screen operation corresponding to the screen touch operation instruction.

[0114] In some implementations, after the control terminal and the master controlled device are connected in communication, the control terminal can automatically send a preset operation acquisition script to the master controlled device. The user can input a touch operation on the master controlled device, and when the master controlled device executes the operation acquisition script, it can obtain the screen touch operation instruction corresponding to the input touch operation and execute the screen operation corresponding to the screen touch operation instruction.

[0115] In addition, the operation acquisition script can be a script written in a preset programming language, for example, the operation acquisition script can be a C++ script. Of course, the operation acquisition script can also be a script written in other preset programming languages ​​executable by the main controlled device, and the embodiment of the present application does not specifically limit this.

[0116] S202: Receive a screen touch operation instruction sent by a master controlled device.

[0117] In some implementations, the master controlled device executes the operation acquisition script to acquire the screen touch operation instruction in real time, and sends the screen touch operation instruction to the control terminal in real time, and the control terminal can receive the screen touch operation instruction in real time.

[0118] S203: Send a screen touch operation instruction to at least one slave controlled device, so that each slave controlled device performs a corresponding operation based on the screen touch operation instruction.

[0119] In an embodiment of the present application, the control terminal can send a screen touch operation instruction to at least one slave controlled device, each slave controlled device can receive the screen touch operation instruction, and perform a corresponding operation according to the screen touch operation instruction, thereby realizing control of the master controlled device and the slave controlled device.

[0120] It should be noted that the controlled device and the control terminal can both be in remote communication, so that the remote control of the controlled device can be realized. Of course, the controlled device and the control terminal can also be in short-distance communication.

[0121] In summary, an embodiment of the present invention provides a device control method, including: sending a preset operation acquisition script to a master controlled device, so that the master controlled device acquires the input screen touch operation instruction based on the operation acquisition script, and performs the screen operation corresponding to the screen touch operation instruction; receiving the screen touch operation instruction sent by the master controlled device; sending the screen touch operation instruction to at least one slave controlled device, so that each slave controlled device performs the corresponding operation based on the screen touch operation instruction. By acquiring the input screen touch operation instruction based on the operation acquisition script by the master controlled device, using a control terminal to send the screen touch operation instruction to at least one slave controlled device, and controlling the slave controlled device to perform the corresponding operation, control of multiple controlled devices can be achieved, and delay in controlling the master controlled device and the slave controlled device can be avoided, thereby improving the real-time performance of the control and making the control of the slave controlled device and the master controlled device more accurate.

[0122] It should be noted that when users perceive real-time, they mainly focus on control feedback. In the related art, control feedback is to perform operations on the master device, send commands to the slave device, the slave device executes and returns the execution results to the master device, and the user perceives the control feedback from the master device; while in the embodiment of the present application, screen operations are performed on the master-controlled device, the master-controlled device can execute corresponding screen touch operation instructions, the user perceives control feedback on the master-controlled device, the screen touch operation instructions are sent to the slave device, and the slave device and the master device are executed synchronously, which can improve the real-time control mainly in that it can bring better control feedback time and improve the user's operating experience.

[0123] In the embodiment of the present application, when the touch operation is a script recording operation, the corresponding operation command can be stored in the control terminal to form a script file, and the control terminal can use the script file to control the controlled device.

[0124] Optional, Figure 3 A flow chart of a device control method provided by an embodiment of the present invention is as follows: Figure 3 As shown, the method may also include:

[0125] S301: Send a file acquisition command to a master controlled device.

[0126] The terminal may send a file acquisition command to the master controlled device, and the master controlled device may receive the file acquisition command. Optionally, the file acquisition command may be an adb shell getevent -i command.

[0127] S302: Receive paths of multiple device files returned by the master controlled device based on a file acquisition command.

[0128] In the embodiment of the present application, the master controlled device can return the paths of multiple device files to the control terminal according to the file acquisition command, and the control terminal can receive the paths of multiple device files.

[0129] S303: Determine the path of the device file for the touch operation as the target file path from the paths of the plurality of device files.

[0130] In some real-time modes, the control terminal may use a preset target file path determination rule to determine the path of the device file for touch operation from multiple device file paths as the target file path. The target file path determination rule includes: analyzing the legal input rule of the device file.

[0131] S304: Send the target file path to the master controlled device, so that the master controlled device acquires the screen touch operation instruction from the device file under the target file path by executing the operation acquisition script.

[0132] In an embodiment of the present application, the control terminal can send a target file path to the master controlled device, the master controlled device can receive the target file path and execute an operation acquisition script, and then obtain the screen touch operation instructions from the device file under the target file path.

[0133] Optionally, the method may further include:

[0134] A script execution command is sent to the master controlled device, so that the master controlled device executes the operation acquisition script through the script execution command, and acquires the screen touch operation instruction from the device file under the target file path.

[0135] The script execution command may be an adb shell command.

[0136] In some implementations, the control terminal may send an adb shell command to the master controlled device, and the master controlled device may execute an operation acquisition script according to the adb shell command and obtain the screen touch operation instruction from the device file under the target file path.

[0137] Optional, if there are multiple screen touch operation instructions. Figure 4 A flow chart of a device control method provided by an embodiment of the present invention is as follows: Figure 4 As shown, before sending the screen touch operation instruction to at least one slave controlled device in the above S203, the method may further include:

[0138] S401: Combine multiple screen touch operation instructions to obtain at least one operation command.

[0139] Wherein, each operation command includes: at least two screen touch operation instructions.

[0140] The sending of the screen touch operation instruction to at least one slave controlled device in S203 may include:

[0141] S402: Send at least one operation command to at least one slave controlled device.

[0142] In some implementations, the control terminal may separate multiple screen touch operation instructions based on the instruction corresponding to the instruction code to obtain at least one operation command. Optionally, the instruction code may be SYN_REPORT (0x0000), the instruction code may be part of the instruction, and the instruction may consist of an instruction type, an instruction code, and an instruction value.

[0143] Optional, Figure 5 A flow chart of a device control method provided by an embodiment of the present invention is as follows: Figure 5 As shown, before sending at least one operation command to at least one slave controlled device in the above S203, the method further includes:

[0144] S501: Convert each operation command into corresponding screen operation information.

[0145] The screen operation information may include the screen operation type and the operation coordinates.

[0146] S502: Generate a screen touch operation instruction corresponding to each slave controlled device using a touch operation library corresponding to each slave controlled device according to the screen operation information.

[0147] Optionally, the touch operation library may be an airtest library.

[0148] In an embodiment of the present application, at least one touch operation library corresponding to a slave controlled device may include: a minitouch library, a maxtouch library, and an adbtouch library. A mapping relationship between each slave controlled device and a touch operation library may be stored in the control terminal, and the control terminal may determine the touch operation library corresponding to each slave controlled device according to the mapping relationship.

[0149] The sending of at least one operation command to at least one slave controlled device in S203 includes:

[0150] S503: Sending a screen touch operation instruction corresponding to each slave controlled device to each slave controlled device.

[0151] It should be noted that the control terminal sends the screen touch operation instruction corresponding to each slave controlled device to each slave controlled device, and correspondingly, each slave controlled device can receive the screen touch operation instruction corresponding to each slave controlled device.

[0152] Optional, Figure 6A flow chart of a device control method provided by an embodiment of the present invention is as follows: Figure 6 As shown, in the above S502, according to the screen operation information, the touch operation library corresponding to each slave controlled device is used to generate the screen touch operation instruction corresponding to each slave controlled device, which may include:

[0153] S601. Adjust the operation coordinates in the screen operation information according to the resolution of each slave controlled device.

[0154] The resolution of each slave controlled device may be different. Of course, there may be slave controlled devices with the same resolution among at least one slave controlled device.

[0155] In addition, the control terminal can also obtain the value range of the legal input according to the file acquisition command.

[0156] In some implementations, the control terminal may transform the operation coordinates in the screen operation information according to the resolution of each slave controlled device and the value range of legal input to obtain the transformed operation coordinates, ie, the operation coordinates at the resolution of each slave controlled device.

[0157] S602: According to the adjusted screen operation information, a touch operation library corresponding to each slave controlled device is used to generate a screen touch operation instruction corresponding to each slave controlled device.

[0158] The adjusted screen operation information may include: the screen operation type and the operation coordinates of each slave controlled device at a resolution.

[0159] In an embodiment of the present application, the control terminal may generate a screen touch operation instruction corresponding to each slave controlled device using a touch operation library corresponding to each slave controlled device according to the screen operation type and the operation coordinates at the resolution of each slave controlled device.

[0160] The device control method provided in the embodiment of the present application is explained below with the main controlled device as the execution subject.

[0161] Optional, Figure 7 A flow chart of a device control method provided by an embodiment of the present invention is as follows: Figure 7 As shown, the method may include:

[0162] S701, receiving an operation acquisition script sent by a control terminal.

[0163] The operation acquisition script may be a script written in a preset programming language, for example, the operation acquisition script may be a C++ script.

[0164] S702: Based on the operation acquisition script, the input screen touch operation instruction is acquired, and the operation corresponding to the screen touch operation instruction is executed.

[0165] In some implementations, when the master controlled device and the slave controlled device need to be controlled, the user can input a touch operation on the master controlled device, and the master controlled device executes an operation acquisition script to acquire a screen touch operation instruction corresponding to the touch operation.

[0166] It should be noted that the present embodiment does not specifically limit the execution order of the two steps of executing the operation corresponding to the screen touch operation instruction and obtaining the input screen touch operation instruction. Of course, they can also be executed synchronously.

[0167] S703: Send a screen touch operation instruction to the control terminal, so that the control terminal sends the screen touch operation instruction to at least one slave controlled device, so that each slave controlled device performs a corresponding operation based on the screen touch operation instruction.

[0168] Among them, the master controlled device can send screen touch operation instructions to the control terminal, the control terminal can obtain the screen touch operation instructions and send the screen touch operation instructions to at least one slave controlled device; each slave controlled device can receive the screen touch operation instructions and perform corresponding operations according to the screen touch operation instructions.

[0169] In summary, the embodiment of the present application provides a device control method, including: receiving an operation acquisition script sent by a control terminal; based on the operation acquisition script, acquiring an input screen touch operation instruction, and executing an operation corresponding to the screen touch operation instruction; sending a screen touch operation instruction to the control terminal, so that the control terminal sends a screen touch operation instruction to at least one slave controlled device, so that each slave controlled device executes a corresponding operation based on the screen touch operation instruction. By having the master controlled device acquire the input screen touch operation instruction based on the operation acquisition script, using the control terminal to send the screen touch operation instruction to at least one slave controlled device, and controlling the slave controlled device to execute the corresponding operation, control of multiple controlled devices can be achieved, and delays in controlling the master controlled device and the slave controlled device can be avoided, thereby improving the real-time performance of the control and making the control of the slave controlled device and the master controlled device more accurate.

[0170] Optional, Figure 8 A flow chart of a device control method provided by an embodiment of the present invention is as follows: Figure 8 As shown, before obtaining the input screen touch operation instruction based on the operation acquisition script in the above S702, the method may further include:

[0171] S801. Receive a file acquisition command sent by a control terminal.

[0172] Optionally, the file acquisition command may be an adb shellgetevent -i command.

[0173] S802: Send multiple device file paths based on the file acquisition command, so that the control terminal determines the path of the device file for the touch operation as the target file path from the multiple device file paths.

[0174] Optionally, the path to multiple device files can be / dev / input on Android devices.

[0175] S803: Receive the target file path sent by the control terminal.

[0176] Among them, the master controlled device can store the target file path.

[0177] In the above S702, based on the operation acquisition script, the input screen touch operation instruction is acquired, which may include:

[0178] S804: Get the screen touch operation instruction from the device file under the target file path by executing the operation acquisition script.

[0179] In some implementations, the control terminal may send a target file path to the master controlled device, the master controlled device may receive the target file path, and execute an operation acquisition script, and then may acquire the screen touch operation instruction from the device file under the target file path.

[0180] Optional, Fig. 9 A flow chart of a device control method provided by an embodiment of the present invention is as follows: Fig. 9 As shown, before the process of acquiring the screen touch operation instruction from the device file under the target file path by executing the operation acquisition script in the above S804, the method also includes:

[0181] S901. Receive a script execution command sent by a control terminal.

[0182] The script execution command may be an adb shell command.

[0183] In the above S804, the screen touch operation instruction is obtained from the device file under the target file path by executing the operation acquisition script, which may include:

[0184] S902: Execute the operation acquisition script through the script execution command to obtain the screen touch operation instruction from the device file under the target file path.

[0185] Optionally, the master controlled device may execute an operation acquisition script according to an adb shell command, and acquire the screen touch operation instruction from the device file under the target file path according to the target file path.

[0186] In the embodiment of the present application, the screen touch operation instruction can be binary content, and the master controlled device can divide the binary content into multiple instructions consisting of a timestamp, an instruction type, an instruction code, and an instruction value, and store them in long double (long floating point), short (short integer), short (short integer), and int (integer). Based on the instructions shown in the parsing table 1, the required part of the binary content is preliminarily parsed.

[0187] Table 1

[0188]

[0189]

[0190] In addition, the master controlled device can send the screen touch operation instructions after preliminary analysis to the control terminal.

[0191] Fig.10 FIG10 is a schematic diagram of an interactive flow chart of a device control method provided by an embodiment of the present invention. The method may include:

[0192] S1001. The control terminal sends a preset operation acquisition script to the main controlled device.

[0193] S1002: The master controlled device obtains an input screen touch operation instruction based on the operation acquisition script, and executes an operation corresponding to the screen touch operation instruction.

[0194] S1003: The master controlled device sends a screen touch operation instruction to the control terminal.

[0195] S1004: The control terminal sends a screen touch operation instruction to at least one slave controlled device.

[0196] S1005: The controlled device performs a corresponding operation based on the screen touch operation instruction.

[0197] It should be noted that S1001 and S1002 are the preparation stages. S1001 and S1002 only need to be executed once before S1003. After executing S1001 and S1002 once, that is, after the preparation stage is completed, S1003 and S1004 can be executed once or multiple times to realize the sending of at least one screen touch operation instruction.

[0198] In an embodiment of the present application, as another way to control the controlled device, the controlled device can also send a video stream of the screen to the control terminal. The control terminal can decode the video stream and find the screen occlusion information of the device corresponding to the video stream from the server database, and add the occlusion information to the decoded video stream. The controlled device can display the decoded video streams of all controlled devices on the same page. The screen multi-control process is performed using an external device of the control terminal, wherein the external device includes: a mouse and / or a keyboard. Among them, the screen occlusion information may include screen bangs, holes, rounded corners, etc.

[0199] In some implementations, the control terminal executes the adb shell getprop command to obtain the device information of the connected controlled device. The screen projection method is automatically selected according to the device information of the controlled device, and the video stream of the Android device is obtained according to the selected screen projection scheme, parsed into a picture, and the control terminal searches the database for the corresponding screen occlusion information of the device through the serial number of the controlled device, and the occlusion information is added to the screen projection picture of the controlled device, and the screen projection pictures of each controlled device are displayed on the same page.

[0200] The device information includes: brand information, model information, Android version information, etc. The brand information is the ro.product.brand line in the command output, the model information is the ro.product.model line in the command output, and the Android version information is the ro.build.version.release line in the command output.

[0201] It should be noted that the screen projection method can include: minicap and javacap. Minicap has better frame rate performance, but minicap may be incompatible with some brands of controlled devices and newer Android versions. If minicap is not supported, it is necessary to use the javacap solution with better compatibility but poor performance such as frame rate for screen projection. Both screen projection solutions are integrated in the airtest library, but the present invention adopts a strategy to actively choose to use the touch operation library.

[0202] In addition, the control terminal can obtain the tapping operation or sliding operation on the projection screen input by the external device, process the operation, obtain the processed operation command, send the processed operation command to the controlled device, and the controlled device can execute the processed operation command. The control terminal can replace the user's direct operation on the projection screen according to the pre-recorded script to realize the script playback function.

[0203] In an embodiment of the present application, as another control method for a controlled device, the control terminal can obtain an input non-touch operation command; parse the non-touch operation command to obtain the screen coordinates on which the non-touch operation command acts; adjust the screen coordinates according to the resolution of each controlled device to obtain the target screen coordinates of each controlled device; based on the target screen coordinates, use the operation library corresponding to each slave controlled device to generate a screen operation instruction corresponding to each controlled device; send the screen operation instruction to each controlled device so that each controlled device performs the screen operation based on the screen operation instruction.

[0204] The non-touch operation may be an operation inputted by an external device, for example, the external device may be a mouse. The control terminal may also obtain the type of the non-touch operation, which may include: a press type, a lift type, a move type, and the like.

[0205] The above process of generating screen operation instructions corresponding to each controlled device according to the target screen coordinates and using the operation library corresponding to each slave controlled device may include: generating screen operation instructions corresponding to each controlled device according to the target screen coordinates and the type of non-touch operation and using the operation library corresponding to each slave controlled device.

[0206] Optionally, the control terminal may convert the non-touch operation into an ADB command, that is, the screen operation instruction may be an ADB command, and the controlled device may perform the screen operation according to the ADB command.

[0207] The following describes the device control apparatus, control terminal, main controlled device and medium used to execute the device control method provided in this application. Its specific implementation process and technical effects refer to the relevant content of the above-mentioned device control method, which will not be repeated below.

[0208] Fig.11 A schematic diagram of a device control device provided by an embodiment of the present invention is shown in FIG. Fig.11 As shown, the device comprises:

[0209] The first sending module 1101 is used to send a preset operation acquisition script to the master controlled device, so that the master controlled device acquires the input screen touch operation instruction based on the operation acquisition script and performs the screen operation corresponding to the screen touch operation instruction;

[0210] The receiving module 1102 is used to receive the screen touch operation instruction sent by the master controlled device;

[0211] The second sending module 1103 is configured to send a screen touch operation instruction to at least one slave controlled device, so that each slave controlled device performs a corresponding operation based on the screen touch operation instruction.

[0212] Optionally, the device further comprises:

[0213] A third sending module, used for sending a file acquisition command to the main controlled device;

[0214] A first receiving module, configured to receive paths of multiple device files returned by the master controlled device based on a file acquisition command;

[0215] A determination module, used for determining a path of a device file for touch operation from paths of multiple device files as a target file path;

[0216] The fourth sending module is used to send the target file path to the master controlled device, so that the master controlled device obtains the screen touch operation instruction from the device file under the target file path by executing the operation acquisition script.

[0217] Optionally, the device further comprises:

[0218] The fifth sending module is used to send a script execution command to the master controlled device, so that the master controlled device executes the operation acquisition script through the script execution command and obtains the screen touch operation instruction from the device file under the target file path.

[0219] Optionally, if there are multiple screen touch operation instructions;

[0220] The device also includes:

[0221] A combination module, used for combining multiple screen touch operation instructions to obtain at least one operation command, each operation command including: at least two screen touch operation instructions;

[0222] The second sending module is further used to send at least one operation command to at least one slave controlled device.

[0223] Optionally, the device further comprises:

[0224] A conversion module, used to convert each operation command into corresponding screen operation information;

[0225] A generation module, used to generate a screen touch operation instruction corresponding to each slave controlled device using a touch operation library corresponding to each slave controlled device according to the screen operation information;

[0226] The second sending module is further used to send the screen touch operation instruction corresponding to each slave controlled device to each slave controlled device.

[0227] Optionally, the generating module is also used to adjust the operation coordinates in the screen operation information according to the resolution of each slave controlled device; based on the adjusted screen operation information, the touch operation library corresponding to each slave controlled device is used to generate the screen touch operation instructions corresponding to each slave controlled device.

[0228] Fig.12 A schematic diagram of a device control device provided by an embodiment of the present invention is shown in FIG. Fig.12 As shown, the device comprises:

[0229] Receiving module 1201, used for receiving the operation acquisition script sent by the control terminal;

[0230] The acquisition module 1202 is used to acquire the input screen touch operation instruction based on the operation acquisition script, and perform the operation corresponding to the screen touch operation instruction;

[0231] The sending module 1203 is used to send the screen touch operation instruction to the control terminal, so that the control terminal sends the screen touch operation instruction to at least one slave controlled device, so that each slave controlled device performs a corresponding operation based on the screen touch operation instruction.

[0232] Optionally, the device further comprises:

[0233] A first receiving module, used for receiving a file acquisition command sent by a control terminal;

[0234] A first sending module, configured to send paths of multiple device files based on a file acquisition command, so that the control terminal determines the path of the device file for touch operation from the paths of the multiple device files as the target file path;

[0235] The second receiving module is used to receive the target file path sent by the control terminal;

[0236] The acquisition module 1202 is used to acquire the screen touch operation instruction from the device file under the target file path by executing the operation acquisition script.

[0237] Optionally, the device further comprises:

[0238] A third receiving module is used to receive a script execution command sent by the control terminal;

[0239] The acquisition module 1202 is further configured to execute an operation acquisition script through a script execution command, and acquire a screen touch operation instruction from a device file under a target file path.

[0240] The above-mentioned device is used to execute the method provided by the aforementioned embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0241] The above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more digital singnal processors (DSP), or one or more field programmable gate arrays (FPGA). For another example, when a module is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0242] Fig.13 A schematic diagram of a control terminal provided by an embodiment of the present invention is shown in FIG. Fig.13 As shown, the control terminal includes: a processor 1301 and a memory 1302.

[0243] The memory 1302 is used to store programs, and the processor 1301 calls the programs stored in the memory 1302 to execute the above method embodiment. The specific implementation method and technical effect are similar and will not be repeated here.

[0244] Fig.14 A schematic diagram of the structure of a master controlled device provided by an embodiment of the present invention is shown in FIG. Fig.14 As shown, the master controlled device includes: a processor 1401 and a memory 1402 .

[0245] The memory 1402 is used to store programs, and the processor 1401 calls the programs stored in the memory 1402 to execute the above method embodiment. The specific implementation method and technical effect are similar and will not be repeated here.

[0246] Optionally, the present invention also provides a program product, such as a computer-readable storage medium, comprising a program, which is used to execute the above method embodiment when executed by a processor.

[0247] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0248] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0249] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0250] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to perform some steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (English: Read-Only Memory, abbreviated: ROM), random access memory (English: Random Access Memory, abbreviated: RAM), disk or optical disk and other media that can store program codes.

[0251] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A device control method, characterized in that: include: Sending a preset operation acquisition script to the master controlled device, so that the master controlled device acquires the input screen touch operation instruction based on the operation acquisition script, and performs the screen operation corresponding to the screen touch operation instruction; Receiving the screen touch operation instruction sent by the master controlled device; Sending the screen touch operation instruction to at least one slave controlled device, so that each slave controlled device performs a corresponding operation based on the screen touch operation instruction; The method further comprises: Sending a file acquisition command to the master controlled device; Receiving paths of multiple device files returned by the master controlled device based on the file acquisition command; Using a preset target file path determination rule, determining the path of the device file for the touch operation from the paths of the multiple device files as the target file path, wherein the target file path determination rule includes: analyzing a legal input rule of the device file; The target file path is sent to the master controlled device, so that the master controlled device acquires the screen touch operation instruction from the device file under the target file path by executing the operation acquisition script.

2. The method according to claim 1, characterized in that The method further comprises: A script execution command is sent to the master controlled device, so that the master controlled device executes the operation acquisition script through the script execution command, and acquires the screen touch operation instruction from the device file under the target file path.

3. The method according to claim 1, characterized in that If there are multiple screen touch operation instructions; Before sending the screen touch operation instruction to at least one slave controlled device, the method further includes: Combining the plurality of the screen touch operation instructions to obtain at least one operation command, each operation command including: at least two of the screen touch operation instructions; The sending the screen touch operation instruction to at least one slave controlled device includes: The at least one operation command is sent to the at least one slave controlled device.

4. The method according to claim 3, characterized in that Before sending the at least one operation command to the at least one slave controlled device, the method further includes: Convert each operation command into corresponding screen operation information; According to the screen operation information, using the touch operation library corresponding to each slave controlled device, generating a screen touch operation instruction corresponding to each slave controlled device; The sending the at least one operation command to the at least one slave controlled device comprises: Sending a screen touch operation instruction corresponding to each slave controlled device to each slave controlled device.

5. The method according to claim 4, characterized in that The step of generating, according to the screen operation information, a screen touch operation library corresponding to each slave controlled device and generating a screen touch operation instruction corresponding to each slave controlled device comprises: adjusting the operation coordinates in the screen operation information according to the resolution of each slave controlled device; According to the adjusted screen operation information, the touch operation library corresponding to each slave controlled device is used to generate a screen touch operation instruction corresponding to each slave controlled device.

6. A device control method, characterized in that: include: Receive the operation acquisition script sent by the control terminal; Based on the operation acquisition script, the input screen touch operation instruction is acquired, and the operation corresponding to the screen touch operation instruction is executed; Sending the screen touch operation instruction to the control terminal, so that the control terminal sends the screen touch operation instruction to at least one slave controlled device, so that each slave controlled device performs a corresponding operation based on the screen touch operation instruction; Before acquiring the input screen touch operation instruction based on the operation acquisition script, the method further includes: Receiving a file acquisition command sent by the control terminal; Sending multiple device file paths based on the file acquisition command so that the control terminal uses a preset target file path determination rule to determine the path of the device file for touch operation from the multiple device file paths as the target file path; wherein the target file path determination rule includes: analyzing a legal input rule of the device file; Receiving the target file path sent by the control terminal; The step of acquiring the input screen touch operation instruction based on the operation acquisition script includes: The screen touch operation instruction is obtained from the device file under the target file path by executing the operation acquisition script.

7. The method according to claim 6, characterized in that Before acquiring the screen touch operation instruction from the device file under the target file path by executing the operation acquisition script, the method further includes: Receiving a script execution command sent by the control terminal; The step of executing the operation acquisition script to acquire the screen touch operation instruction from the device file under the target file path includes: The operation acquisition script is executed through the script execution command to obtain the screen touch operation instruction from the device file under the target file path.

8. A device control device, characterized in that: include: A first sending module, configured to send a preset operation acquisition script to a master controlled device, so that the master controlled device acquires an input screen touch operation instruction based on the operation acquisition script, and performs a screen operation corresponding to the screen touch operation instruction; A receiving module, used for receiving the screen touch operation instruction sent by the master controlled device; A second sending module, used to send the screen touch operation instruction to at least one slave controlled device, so that each slave controlled device performs a corresponding operation based on the screen touch operation instruction; The device also includes: A third sending module, used for sending a file acquisition command to the master controlled device; A first receiving module, configured to receive paths of multiple device files returned by the master controlled device based on the file acquisition command; A determination module is used to determine the path of the device file for the touch operation as the target file path from the paths of the multiple device files by using a preset target file path determination rule, wherein the target file path determination rule includes: analyzing a legal input rule of the device file; The fourth sending module is used to send the target file path to the master controlled device, so that the master controlled device obtains the screen touch operation instruction from the device file under the target file path by executing the operation acquisition script.

9. A device control device, characterized in that: include: A receiving module, used for receiving an operation acquisition script sent by a control terminal; An acquisition module, used to acquire a script based on the operation, acquire an input screen touch operation instruction, and execute an operation corresponding to the screen touch operation instruction; A sending module, configured to send the screen touch operation instruction to the control terminal, so that the control terminal sends the screen touch operation instruction to at least one slave controlled device, so that each slave controlled device performs a corresponding operation based on the screen touch operation instruction; The device also includes: A first receiving module, used for receiving a file acquisition command sent by the control terminal; A first sending module is used to send the paths of multiple device files based on the file acquisition command, so that the control terminal adopts a preset target file path determination rule to determine the path of the device file for touch operation from the paths of the multiple device files as the target file path; wherein the target file path determination rule includes: analyzing a legal input rule of the device file; A second receiving module, used for receiving the target file path sent by the control terminal; The acquisition module is used to acquire the screen touch operation instruction from the device file under the target file path by executing the operation acquisition script.

10. A control terminal, characterized in that: include: A memory and a processor, wherein the memory stores a computer program executable by the processor, and when the processor executes the computer program, the method described in any one of claims 1 to 5 is implemented.

11. A master controlled device, characterized in that: include: A memory and a processor, wherein the memory stores a computer program executable by the processor, and the processor implements the method described in claim 6 or 7 when executing the computer program.

12. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is read and executed, the method described in any one of claims 1 to 7 is implemented.

Images