Terminal device control device and method, electronic device, and storage medium

Abstract

The present disclosure is about a terminal device control device and method, an electronic device, and a storage medium, and belongs to the field of intelligent device technology. The terminal device control device includes: an acquisition module, configured to acquire the user's action; a control module, configured to determine the terminal device operation of the user according to the user's action, and determine the target terminal device of the terminal device operation and the corresponding instruction; and a transmission module, configured to transmit the instruction to the target terminal device so that the target terminal device executes the instruction. The user's action is acquired by the acquisition module, the terminal device operation, the target terminal device of the terminal device operation and the corresponding instruction are determined by the control module, and the instruction is transmitted to the target terminal device by the transmission module so that it executes the instruction, thereby realizing the control of multiple target terminal devices according to the user's action.

Description

Technical Field

[0001] The present disclosure relates to the technical field of intelligent devices, and in particular to a terminal device control device and method, an electronic device, and a storage medium. Background Art

[0002] Currently, intelligent hardware is used to control computers, but this control is highly dependent on the software ecosystem, and each intelligent hardware can usually only be used with one computer. Therefore, there is still room for improvement in the design of intelligent hardware used to control computers. Summary of the invention

[0003] In order to overcome the problems existing in the related art, the present disclosure provides a terminal device control device and method, an electronic device, and a storage medium.

[0004] According to a first aspect of an exemplary embodiment of the present disclosure, a terminal device control apparatus is provided, the terminal device control apparatus comprising: an acquisition module configured to acquire a user's action; a control module configured to determine a terminal device operation of the user according to the user's action, and determine a target terminal device of the terminal device operation and a corresponding instruction; and a transmission module configured to transmit the instruction to the target terminal device so that the target terminal device executes the instruction.

[0005] In one embodiment, the transmission module includes a transmission unit based on a human interface device protocol.

[0006] In one embodiment, the transmission module includes: a first transmission module, which is communicatively connected to a first terminal device and configured to transmit a first instruction corresponding to the first terminal device to the first terminal device; and a second transmission module, which is communicatively connected to a second terminal device and configured to transmit a second instruction corresponding to the second terminal device to the second terminal device.

[0007] In one embodiment, the first transmission module includes: a Bluetooth unit based on a human-machine interface device protocol, configured to transmit the first instruction to the first terminal device.

[0008] In one embodiment, the second transmission module includes: a first Bluetooth transparent transmission unit, which is communicatively connected to the control module and configured to receive the second instruction from the control module; a second Bluetooth transparent transmission unit, which is configured to pair with the first Bluetooth transparent transmission unit and receive the second instruction from the first Bluetooth transparent transmission unit; and a serial port to universal serial bus unit based on the human-machine interface device protocol, which is communicatively connected to the second Bluetooth transparent transmission unit and configured to receive the second instruction from the second Bluetooth transparent transmission unit.

[0009] In one embodiment, the acquisition module includes: a first acquisition module including a first infrared diffuse reflection sensor and configured to determine whether the user exists or not; and a second acquisition module including a second infrared diffuse reflection sensor and configured to acquire the user's gesture.

[0010] In an embodiment, when the first acquisition module determines that the user does not exist, the terminal device operation is determined to be an operation of performing screen locking on the target terminal device.

[0011] In one embodiment, when the second acquisition module acquires the first gesture of the user, the terminal device operation is determined to be an operation of performing screen unlocking on the target terminal device.

[0012] In an embodiment, when the second acquisition module acquires the second gesture of the user, the terminal device operation is determined to be an operation of executing a screenshot of the target terminal device.

[0013] In an embodiment, when the second acquisition module acquires the third gesture of the user, the terminal device operation is determined to be an operation of performing, on the target terminal device, an operation of selecting the target terminal device.

[0014] In one embodiment, the terminal device control apparatus further includes: a setting module configured to set the terminal device operation corresponding to the action of the user.

[0015] In one embodiment, the control module includes a first serial port and a second serial port; the setting module is connected to the first serial port of the control module; and the setting module is configured to reuse the second serial port.

[0016] According to a second aspect of an exemplary embodiment of the present disclosure, a terminal device control method is provided, the terminal device control method comprising: acquiring a user's action; determining the user's terminal device operation according to the user's action, and determining a target terminal device of the terminal device operation and a corresponding instruction; and transmitting the instruction to the target terminal device so that the target terminal device executes the instruction.

[0017] In one embodiment, transmitting the instruction to the target terminal device so that the target terminal device executes the instruction includes: transmitting the instruction to the target terminal device based on a human-machine interface device protocol.

[0018] In one embodiment, transmitting the instruction to the target terminal device so that the target terminal device executes the instruction includes: transmitting a first instruction corresponding to the first terminal device to the first terminal device through a first transmission module that is communicatively connected to the first terminal device; and transmitting a second instruction corresponding to the second terminal device to the second terminal device through a second transmission module that is communicatively connected to the second terminal device.

[0019] In one embodiment, the first transmission module connected to the first terminal device through communication transmits the first instruction corresponding to the first terminal device to the first terminal device, including: transmitting the first instruction to the first terminal device through a Bluetooth unit based on the human-machine interface device protocol.

[0020] In one embodiment, the second instruction corresponding to the second terminal device is transmitted to the second terminal device through the second transmission module that is communicatively connected to the second terminal device, including: receiving the second instruction through the first Bluetooth transparent transmission unit; receiving the second instruction from the first Bluetooth transparent transmission unit through the second Bluetooth transparent transmission unit paired with the first Bluetooth transparent transmission unit; and receiving the second instruction from the second Bluetooth transparent transmission unit through a serial port to universal serial bus unit based on the human-machine interface device protocol that is communicatively connected to the second Bluetooth transparent transmission unit.

[0021] In one embodiment, acquiring the action of the user includes: determining the presence or absence of the user through a first infrared diffuse reflection sensor; and acquiring the gesture of the user through a second infrared diffuse reflection sensor.

[0022] In one embodiment, when it is determined that the user does not exist, the terminal device operation is determined to be an operation of performing a screen lock on the target terminal device.

[0023] In one embodiment, when acquiring the first gesture of the user, it is determined that the operation of the terminal device is an operation of performing screen unlocking on the target terminal device.

[0024] In one embodiment, when acquiring the second gesture of the user, it is determined that the terminal device operation is an operation of executing a screenshot of the target terminal device.

[0025] In one embodiment, when acquiring the third gesture of the user, it is determined that the terminal device operation is to perform an operation of selecting the target terminal device on the target terminal device.

[0026] In one embodiment, before determining the terminal device operation of the user according to the action of the user, and determining the target terminal device of the terminal device operation and the corresponding instruction, the terminal device control method further includes: setting the terminal device operation corresponding to the action of the user.

[0027] According to a third aspect of an exemplary embodiment of the present disclosure, there is provided an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to: execute any one of the aforementioned terminal device control methods.

[0028] According to a fourth aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of a terminal, the terminal is enabled to execute any one of the terminal device control methods described above.

[0029] The technical solutions provided by the exemplary embodiments of the present disclosure may include the following beneficial technical effects:

[0030] Exemplary embodiments of the present disclosure provide a terminal device control device and method, an electronic device, and a storage medium. According to the exemplary embodiments of the present disclosure, the terminal device control device includes: an acquisition module configured to acquire the user's action; a control module configured to determine the terminal device operation of the user according to the user's action, and determine the target terminal device of the terminal device operation and the corresponding instruction; and a transmission module configured to transmit the instruction to the target terminal device so that the target terminal device executes the instruction. The user's action is acquired by the acquisition module, the terminal device operation, the target terminal device of the terminal device operation and the corresponding instruction are determined by the control module, and the instruction is transmitted to the target terminal device by the transmission module so that it executes the instruction, thereby realizing the control of multiple target terminal devices according to the user's action.

[0031] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0033] Figure 1 It is a schematic block diagram of a terminal device control apparatus according to an exemplary embodiment.

[0034] Figure 2 It is a schematic block diagram of a terminal device control apparatus according to an exemplary embodiment.

[0035] Figure 3 It is a schematic block diagram of a terminal device control apparatus according to an exemplary embodiment.

[0036] Figure 4 It is a schematic block diagram of an acquisition module in a terminal device control apparatus according to an exemplary embodiment.

[0037] Figure 5 The present invention is a schematic block diagram of a power module in a terminal device control device according to an exemplary embodiment.

[0038] Figure 6 The present invention is a flowchart of a terminal device control method according to an exemplary embodiment.

[0039] Figure 7 It is a block diagram of a device according to an exemplary embodiment. DETAILED DESCRIPTION

[0040] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0041] At present, smart hardware can realize linkage control of computers. Wearable devices such as smart bracelets can realize linkage control of computers through Bluetooth. However, this linkage control is highly dependent on the software ecosystem. For example, when a smart bracelet is used to lock a computer, Bluetooth is used to determine the distance between the computer and the user, and the computer is locked when the distance exceeds a preset value. Smart bracelets require the support of the computer's system software to automatically lock the computer, and each smart bracelet can only be used with one computer. When a smart bracelet unlocks a computer, the computer software needs to verify the identity. In terms of linkage control of computers, smart bracelets do not support multiple computers or multiple operating systems. In general, the functions of smart bracelets are relatively single, dependent on system software, and the operation is relatively slow.

[0042] The Human Interface Device (HID) protocol is applied to the mouse and keyboard of the computer. The HID protocol is also applied to application scenarios such as answer buttons. The answer button simulates the keys on the keyboard or mouse through the HID protocol and is applied to answering questions, drawing prizes, etc. When the user presses the answer button, the answer button sends the corresponding set key value to the computer. For example, when the answer software uses the Enter (ENTER) key as a stop command, the key value of the answer button is set to ENTER, so that it serves as a control key for answering questions. However, the answer button can usually only be used as a button, and the user usually cannot modify the pre-set keys by himself after leaving the factory. In addition, the answer button usually does not support combination keys and does not support the connection of multiple computers.

[0043] In view of this, the inventors have proposed a terminal device control device and method that can control multiple terminal devices. As described above, the technical solution of the present disclosure can be applied to the field of smart device technology. However, it should be noted that the technical solution of the present disclosure can also be applied to the field of smart office technology and smart home technology, for example, it can be integrated into a desk or can be combined with devices such as a smart speaker.

[0044] For specific implementations of the terminal device control apparatus and method, electronic device, and storage medium according to the exemplary embodiments of the present disclosure, please refer to the exemplary embodiments described below.

[0045] Figure 1 It is a schematic block diagram of a terminal device control apparatus according to an exemplary embodiment.

[0046] refer to Figure 1 , shows a terminal device control apparatus 100 according to an exemplary embodiment of the present disclosure. The terminal device control apparatus 100 includes: an acquisition module 110, the acquisition module 110 is configured to acquire a user's action; a control module 130, the control module 130 is configured to determine the user's terminal device operation according to the user's action, and determine the target terminal device of the terminal device operation (for example, in Figure 2 and a transmission module 140, the transmission module 140 is configured to transmit the instruction to the target terminal device so that the target terminal device executes the instruction.

[0047] In such Figure 1In the terminal device control device 100 shown in , the user's action is acquired by the acquisition module 110, the terminal device operation, the target terminal device of the terminal device operation and the corresponding instruction are determined by the control module 130, and the instruction is transmitted to the target terminal device by the transmission module 140 so that it executes the instruction, thereby realizing the control of multiple target terminal devices according to the user's action. As mentioned above, the answer button and smart devices such as smart bracelets cannot support multiple computers. In contrast, the terminal device control device 100 according to the exemplary embodiment of the present disclosure can realize the control of multiple target terminal devices.

[0048] In an exemplary embodiment of the present disclosure, the transmission module 140 may include a transmission unit based on the HID protocol. That is to say, the exemplary embodiment of the present disclosure proposes intelligent hardware designed based on the HID protocol, which is compatible with multiple operating systems and can control multiple terminal devices. Since the HID protocol is natively compatible with operating systems such as Windows, Mac OS, Unix, Linux, Ubuntu, FreeBSD, etc., the terminal device control device 100 can achieve multi-operating system support. On the other hand, the support of the host computer is not required. The design based on the HID protocol does not require computer host software support, so the terminal device control device 100 will not cause incompatibility problems with the upgrade of the computer system.

[0049] In an exemplary embodiment of the present disclosure, the control module 130 may include a storage unit such as a flash memory, and the storage unit stores a correspondence relationship between a user's action and an operation of the terminal device.

[0050] In some embodiments, the correspondence between the user's actions and the terminal device operations may be pre-stored in the storage unit of the control module 130 .

[0051] refer to Figure 1 The terminal device control apparatus 100 may further include a setting module 120, and the setting module 120 is configured to set the terminal device operation corresponding to the user's action. In other words, the user can set the corresponding relationship between the user's action and the terminal device operation through the setting module 120, thereby realizing the customizability of the control of the terminal device.

[0052] refer to Figure 1The terminal device control device 100 may further include a power module 150, and the power module 150 is configured to provide the power required for the control module 130 to work. It should be noted that the power module 150 may also be configured to provide the power required for the other modules in the terminal device control device 100 to work, for example, configured to provide the power required for the acquisition module 110, the setting module 120 or the transmission module 140 to work. It should also be noted that the power module 150 may be provided separately from the control module 130, or may be provided integrally with the control module 130.

[0053] Figure 2 It is a schematic block diagram of a terminal device control apparatus according to an exemplary embodiment.

[0054] refer to Figure 2 , shows a terminal device control apparatus 100 according to an exemplary embodiment of the present disclosure, and also shows a terminal device 200 as a target terminal device of the terminal device control apparatus 100 for the convenience of explanation.

[0055] In an exemplary embodiment of the present disclosure, the transmission module 140 of the terminal device control apparatus 100 may include one or more transmission modules. Figure 2 In the embodiment shown in , the transmission module 140 may include multiple transmission modules, for example, a first transmission module 140a, a second transmission module 140b, ... and an nth transmission module 140n, where n may be a natural number greater than or equal to 2. Accordingly, the terminal device 200 may include multiple terminal devices, for example, a first terminal device 200a, a second terminal device 200b, ... and an nth terminal device 200n, where n may be a natural number greater than or equal to 2. Therefore, the number of transmission modules included in the transmission module 140 is the same as the number of target terminal devices included in the terminal device 200. Figure 2 As shown in, the first transmission module 140a is communicatively connected to the first terminal device 200a, and is configured to transmit the first instruction corresponding to the first terminal device 200a to the first terminal device 200a; the second transmission module 140b is communicatively connected to the second terminal device 200b, and is configured to transmit the second instruction corresponding to the second terminal device 200b to the second terminal device 200b; and the nth transmission module 140n is communicatively connected to the nth terminal device 200n, and is configured to transmit the nth instruction corresponding to the nth terminal device 200n to the nth terminal device 200n. Through the configuration as described above, the terminal device control apparatus 100 can control the first terminal device 200a, the second terminal device 200b, ... and the nth terminal device 200n respectively.

[0056] Figure 3It is a schematic block diagram of a terminal device control apparatus according to an exemplary embodiment.

[0057] Combination Figure 3 , a terminal device control apparatus according to an exemplary embodiment of the present disclosure is described in a scenario of controlling two terminal devices. Figure 2 similar, Figure 3 A terminal device control apparatus 100 according to an exemplary embodiment of the present disclosure is shown, and a terminal device 200 which is a target terminal device of the terminal device control apparatus 100 is also shown for convenience of explanation.

[0058] Specifically, in an exemplary embodiment according to the present disclosure, the transmission module 140 of the terminal device control device 100 may include two transmission modules, namely, a first transmission module 140a and a second transmission module 140b. Accordingly, the first transmission module 140a is communicatively connected to the first terminal device 200a, and is configured to transmit a first instruction corresponding to the first terminal device 200a to the first terminal device 200a; and the second transmission module 140b is communicatively connected to the second terminal device 200b, and is configured to transmit a second instruction corresponding to the second terminal device 200b to the second terminal device 200b. In this way, the terminal device control device 100 can control the first terminal device 200a and the second terminal device 200b respectively. In other words, the terminal device control device 100 supports connection control of two target terminal devices. This can meet most of the needs of users in daily application scenarios.

[0059] In an exemplary embodiment of the present disclosure, the first terminal device 200a can be a target terminal device that supports wireless transmission such as Bluetooth, for example, a notebook computer operating on a Windows operating system, and the second terminal device 200b can be a target terminal device that does not support wireless transmission such as Bluetooth, for example, a desktop computer operating on an Ubuntu operating system.

[0060] In an exemplary embodiment of the present disclosure, the first transmission module 140a may include a Bluetooth unit (Bluetooth HID unit) 140a1 based on a human interface device protocol. The Bluetooth HID unit 140a1 is configured to transmit the first instruction to the first terminal device 200a.

[0061] In an exemplary embodiment of the present disclosure, the second transmission module 140b may include a first Bluetooth transparent transmission unit 140b1, a second Bluetooth transparent transmission unit 140b2, and a serial port to universal serial bus unit (serial port to USB HID unit) 140b3 based on the human-machine interface device protocol. The first Bluetooth transparent transmission unit 140b1 is communicatively connected to the control module 130 and is configured to receive a second instruction from the control module 130. The second Bluetooth transparent transmission unit 140b2 is configured to pair with the first Bluetooth transparent transmission unit 140b1 and receive a second instruction from the first Bluetooth transparent transmission unit 140b1. The serial port to USB HID unit 140b3 is communicatively connected to the second Bluetooth transparent transmission unit 140b2 and is configured to receive a second instruction from the second Bluetooth transparent transmission unit 140b2.

[0062] In combination Figure 3 In the exemplary embodiment described, the first terminal device 200a is a target terminal device that supports wireless transmission such as Bluetooth, and the second terminal device 200b is a target terminal device that does not support wireless transmission such as Bluetooth. However, it should be noted that the terminal device control device 100 according to the embodiment of the present disclosure can be applied to scenarios where the terminal device partially supports wireless transmission, scenarios where the terminal device fully supports wireless transmission, and scenarios where the terminal device does not support wireless transmission at all. It should also be noted that in the exemplary embodiment according to the present disclosure, the wireless transmission between the control module 130 of the terminal device control device 100 and the terminal device 200 is not limited to the Bluetooth transmission method as described above. For example, in principle, wireless transmission methods such as WiFi and ZigBee can also be used. It should also be noted that the target terminal device is not limited to the notebook computer or desktop computer as described above, and can also be a mobile terminal such as a tablet computer, a mobile phone, and a smart watch.

[0063] Figure 4 It is a schematic block diagram of an acquisition module in a terminal device control apparatus according to an exemplary embodiment.

[0064] refer to Figure 4 In an exemplary embodiment of the present disclosure, the acquisition module 110 may include a first acquisition module 110a and a second acquisition module 110b. For example, the first acquisition module 110a may include a first infrared diffuse reflection sensor and is configured to determine whether a user exists or not. For example, the second acquisition module 110b may include a second infrared diffuse reflection sensor and is configured to acquire a user's gesture.

[0065] In an exemplary embodiment, the first infrared diffuse reflection sensor may be installed on the side of the terminal device control device 100 facing the user, so as to detect whether the user is standing or sitting in front of the terminal device control device 100. For example, when the user is standing or sitting in front of the terminal device control device 100, the first infrared diffuse reflection sensor is blocked, indicating that the user exists. For example, when the user is not standing or sitting in front of the terminal device control device 100, the first infrared diffuse reflection sensor is not blocked, indicating that the user does not exist.

[0066] In an exemplary embodiment, the second infrared diffuse reflection sensor can be installed on the top side of the terminal device control device 100 to facilitate the detection of user gesture input. For example, when the user shakes a finger above the terminal device control device 100, the second infrared diffuse reflection sensor can detect the user's gesture input and transmit the gesture input to the control module 130.

[0067] In an exemplary embodiment of the present disclosure, when the first acquisition module 110a determines that the user does not exist, the terminal device operation is determined to be an operation of performing screen locking on the target terminal device.

[0068] In an exemplary embodiment of the present disclosure, when the second acquisition module 110 b acquires the first gesture of the user, the terminal device operation is determined to be an operation of performing screen unlocking on the target terminal device.

[0069] In an exemplary embodiment of the present disclosure, when the second acquisition module 110 b acquires the second gesture of the user, the terminal device operation is determined to be an operation of executing a screenshot on the target terminal device.

[0070] In an exemplary embodiment of the present disclosure, when the second acquisition module 110 b acquires the third gesture of the user, the terminal device operation is determined to be an operation of performing a selection of a target terminal device on the target terminal device.

[0071] In an exemplary embodiment of the present disclosure, the control module 130 may include n+1 serial ports, where n is the number of target terminal devices controlled by the terminal device control apparatus 100. In this case, one serial port is reserved for connecting the setting module 120 to implement the corresponding function setting. In some embodiments, the control module 130 may include a first serial port and a second serial port, the setting module 120 may be connected to the first serial port of the control module 130, and the setting module 120 may be configured to reuse the second serial port. Serial port multiplexing saves serial port modules, reduces hardware requirements, and thus reduces costs.

[0072] In terms of cost, when the control module 130 uses a 51-core single-chip microcomputer (i.e., a single-chip microcomputer compatible with the Intel 8051 instruction system) as the core controller, and through serial port multiplexing, for example, using one serial port to support dual Bluetooth connections, the hardware cost of the prototype can be reduced. When the control module 130 uses the Bluetooth chip CC2541 as the core controller, the hardware cost can also be reduced.

[0073] Combination Figure 3 and Figure 4 , briefly describe the operation process of the terminal device control device 100.

[0074] When used for computer control in an environment where multiple computers are used, for example, a first terminal device 200a (e.g., a Windows laptop) and a second terminal device 200b (e.g., an Ubuntu desktop computer), the user's behavior and gestures are collected by the first infrared diffuse reflection sensor included in the first acquisition module 110a and the second infrared diffuse reflection sensor included in the second acquisition module 110b, and after being processed by the control module 130 (e.g., a single-chip microcomputer), the instructions are sent to the first terminal device 200a and the second terminal device 200b connected thereto respectively through the first transmission module 140a including the Bluetooth HID unit 140a1 and the second transmission module 140b including the serial port to USB HID unit 140b3, thereby realizing the set control.

[0075] like Figure 3 As shown in , firstly, the first terminal device 200a (Windows laptop) is connected via Bluetooth, and the USB receiving head is plugged into the second terminal device 200b (Ubuntu desktop computer). The Bluetooth transparent transmission modules such as the first Bluetooth transparent transmission unit 140b1 and the second Bluetooth transparent transmission unit 140b2 will automatically connect after being powered on. At this time, the control module 130 starts to work normally.

[0076] The setting module 120 can be used to set the serial port to interact with the control module 130, and set the unlock password and gesture function. For example, the setting module 120 uses a data cable to connect to the first serial port of the control module 130, and sends a setting instruction to set the gesture function and the unlock password of the two computers. The control module 130 saves the instruction to the memory and reads it every time it is turned on. This realizes the customization of the function.

[0077] The control module 130 collects information through the first infrared diffuse reflection sensor and the second infrared diffuse reflection sensor, analyzes and determines the function of the user operation and generates corresponding instructions, thereby realizing complex logic judgment and functions.

[0078] For example, when the first infrared diffuse reflection sensor used to identify occlusion is not blocked, the control module 130 recognizes and determines that the user has left, and sends a screen lock instruction to the two HID units, the Bluetooth HID unit 140a1 and the serial port to USB HID unit 140b3. The Bluetooth HID unit 140a1 simulates the keyboard to execute a screen lock instruction such as a combination of keys WIN+L (taking Windows 10 system as an example), and the serial port to USB HID unit 140b3 simulates the keyboard to execute a screen lock instruction such as a combination of keys Ctrl+Alt+L, thereby completing the automatic screen locking of the first terminal device 200a (Windows notebook computer) and the second terminal device 200b (Ubuntu desktop computer) respectively.

[0079] For example, when the first infrared diffuse reflection sensor for identifying gestures is blocked, the control module 130 analyzes the user's gestures, such as unlocking gestures, by collecting data collected by the second infrared diffuse reflection sensor. The control module 130 sends a wake-up instruction to the two HID units, the Bluetooth HID unit 140a1 and the serial port to USB HID unit 140b3, to wake up the computer, and then sends the unlock password to complete the input of the password in the password box. In this way, the control operation of multiple computers and multiple operating systems is realized.

[0080] The control module 130 multiplexes the second serial port to the Bluetooth HID unit 140a1, sends corresponding control instructions according to the instructions obtained above, and uses the Bluetooth HID unit 140a1 to control the first terminal device 200a to complete corresponding operations.

[0081] The control module 130 multiplexes the second serial port to the first Bluetooth transparent transmission unit 140b1, and sends corresponding control instructions according to the above-mentioned instructions. The first Bluetooth transparent transmission unit 140b1 transmits the instructions to the serial port to USB HID unit 140b3, and the serial port to USB HID unit 140b3 controls the second terminal device 200b to complete the corresponding operation.

[0082] For operations that need to be executed by two computers, such as unlocking and locking the screen, the control module 130 automatically sends instructions to the two computers.

[0083] For operations that require only one computer to perform, you can use gestures to switch and select the computer on which to perform the operation.

[0084] The solution provided by this technology supports all keyboard-related operations and can wirelessly connect two computers to meet the needs of daily scenarios. It also has a programmable core controller and sensors that can implement complex control logic, such as screen lock, unlock, screenshot, etc.

[0085] Figure 5The present invention is a schematic block diagram of a power module in a terminal device control device according to an exemplary embodiment.

[0086] In an exemplary embodiment of the present disclosure, the power module 150 includes a charging unit 150 a , a battery unit 150 b , and a DC-DC boost unit 150 c .

[0087] Figure 6 The present invention is a flowchart of a terminal device control method according to an exemplary embodiment.

[0088] refer to Figure 6 , showing a terminal device control method according to an exemplary embodiment of the present disclosure.

[0089] In an exemplary embodiment of the present disclosure, a terminal device control method includes: step S510, obtaining a user's action; step S530, determining the user's terminal device operation according to the user's action, and determining a target terminal device of the terminal device operation and a corresponding instruction; and step S540, transmitting the instruction to the target terminal device so that the target terminal device executes the instruction.

[0090] In an exemplary embodiment of the present disclosure, transmitting an instruction to a target terminal device so that the target terminal device executes the instruction includes: transmitting the instruction to the target terminal device based on a human-machine interface device protocol.

[0091] In an exemplary embodiment of the present disclosure, an instruction is transmitted to a target terminal device so that the target terminal device executes the instruction, including: transmitting a first instruction corresponding to the first terminal device to the first terminal device through a first transmission module that is communicatively connected to the first terminal device; and transmitting a second instruction corresponding to the second terminal device to the second terminal device through a second transmission module that is communicatively connected to the second terminal device.

[0092] In an exemplary embodiment of the present disclosure, a first instruction corresponding to the first terminal device is transmitted to the first terminal device through a first transmission module that is communicatively connected to the first terminal device, including: transmitting the first instruction to the first terminal device through a Bluetooth unit based on a human-machine interface device protocol.

[0093] In an exemplary embodiment of the present disclosure, a second instruction corresponding to the second terminal device is transmitted to the second terminal device through a second transmission module that is communicatively connected to the second terminal device, including: receiving the second instruction through a first Bluetooth transparent transmission unit; receiving the second instruction from the first Bluetooth transparent transmission unit through a second Bluetooth transparent transmission unit paired with the first Bluetooth transparent transmission unit; and receiving the second instruction from the second Bluetooth transparent transmission unit through a serial port to universal serial bus unit based on the human-machine interface device protocol that is communicatively connected to the second Bluetooth transparent transmission unit.

[0094] In an exemplary embodiment of the present disclosure, acquiring the user's action includes: determining the presence or absence of the user through a first infrared diffuse reflection sensor; and acquiring the user's gesture through a second infrared diffuse reflection sensor.

[0095] In an exemplary embodiment of the present disclosure, when it is determined that the user does not exist, the terminal device operation is determined to be an operation of performing a screen lock on the target terminal device.

[0096] In an exemplary embodiment of the present disclosure, when acquiring the first gesture of the user, it is determined that the terminal device operation is an operation of performing screen unlocking on the target terminal device.

[0097] In an exemplary embodiment of the present disclosure, when acquiring the second gesture of the user, it is determined that the terminal device operation is an operation of performing a screenshot on the target terminal device.

[0098] In an exemplary embodiment of the present disclosure, when the third gesture of the user is acquired, it is determined that the terminal device operation is to perform an operation of selecting a target terminal device on the target terminal device.

[0099] In an exemplary embodiment of the present disclosure, before determining the terminal device operation of the user according to the user's action, and determining the target terminal device of the terminal device operation and the corresponding instruction, the terminal device control method also includes: step S520, setting the terminal device operation corresponding to the user's action.

[0100] Figure 7 600 is a block diagram of a device according to an exemplary embodiment. The device 600 can be used for remote control or for writing device information. For example, the device 600 can be a mobile phone, a computer, a digital broadcast terminal, a message transceiver, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0101] refer to Figure 7 , the device 600 may include one or more of the following components: a processing component 602 , a memory 604 , a power component 606 , a multimedia component 608 , an audio component 610 , an input / output (I / O) interface 612 , a sensor component 614 , and a communication component 616 .

[0102] The processing component 602 generally controls the overall operation of the device 600, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 602 may include one or more modules to facilitate the interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602.

[0103] The memory 604 is configured to store various types of data to support operations on the device 600. Examples of such data include instructions for any application or method operating on the device 600, contact data, phone book data, messages, pictures, videos, etc. The memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0104] The power component 606 provides power to the various components of the device 600. The power component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 600.

[0105] The multimedia component 608 includes a screen that provides an output interface between the device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. When the device 600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0106] The audio component 610 is configured to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC), and when the device 600 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 604 or sent via the communication component 616. In some embodiments, the audio component 610 also includes a speaker for outputting audio signals.

[0107] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0108] The sensor assembly 614 includes one or more sensors for providing various aspects of status assessment for the device 600. For example, the sensor assembly 614 can detect the open / closed state of the device 600, the relative positioning of components, such as the display and keypad of the device 600, and the sensor assembly 614 can also detect the position change of the device 600 or a component of the device 600, the presence or absence of user contact with the device 600, the orientation or acceleration / deceleration of the device 600, and the temperature change of the device 600. The sensor assembly 614 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 614 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0109] The communication component 616 is configured to facilitate wired or wireless communication between the device 600 and other devices. The device 600 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0110] In an exemplary embodiment, the apparatus 600 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0111] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, and the instructions can be executed by a processor 620 of the device 600 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0112] It is to be understood that in the present disclosure, "plurality" refers to two or more than two, and other quantifiers are similar. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. The singular forms "a", "the" and "the" are also intended to include plural forms, unless the context clearly indicates other meanings.

[0113] It is further understood that the terms "first", "second", etc. are used to describe various information, but such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other, and do not indicate a specific order or degree of importance. In fact, the expressions "first", "second", etc. can be used interchangeably. For example, without departing from the scope of the present disclosure, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information.

[0114] It will be further understood that the terms “center”, “longitudinal”, “lateral”, “lower”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside”, “outside”, etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the present embodiment and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation.

[0115] It can be further understood that, unless otherwise specified, “connection” includes a direct connection without other components between the two, and also includes an indirect connection with other components between the two.

[0116] It is further understood that, although the operations are described in a specific order in the drawings in the embodiments of the present disclosure, it should not be understood as requiring the operations to be performed in the specific order shown or in a serial order, or requiring the execution of all the operations shown to obtain the desired results. In certain environments, multitasking and parallel processing may be advantageous.

[0117] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0118] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A terminal device control device, characterized in that: include: An acquisition module, configured to acquire user actions; A control module, configured to determine a terminal device operation of the user according to the action of the user, and determine a target terminal device of the terminal device operation and a corresponding instruction; as well as A transmission module, configured to transmit the instruction to the target terminal device so that the target terminal device executes the instruction; Wherein, the transmission module includes: a first transmission module, communicatively connected to the first terminal device and configured to transmit a first instruction corresponding to the first terminal device to the first terminal device; and The second transmission module is communicatively connected to the second terminal device and is configured to transmit a second instruction corresponding to the second terminal device to the second terminal device.

2. The terminal device control device according to claim 1, characterized in that: The transmission module includes a transmission unit based on a human-machine interface device protocol.

3. The terminal device control device according to claim 1, characterized in that: The first transmission module includes: The Bluetooth unit based on the human-machine interface device protocol is configured to transmit the first instruction to the first terminal device.

4. The terminal device control device according to claim 3, characterized in that: The second transmission module includes: A first Bluetooth transparent transmission unit, communicatively connected to the control module and configured to receive the second instruction from the control module; a second Bluetooth transparent transmission unit, configured to pair with the first Bluetooth transparent transmission unit and receive the second instruction from the first Bluetooth transparent transmission unit; and The serial port to universal serial bus unit based on the human-machine interface device protocol is communicatively connected to the second Bluetooth transparent transmission unit and is configured to receive the second instruction from the second Bluetooth transparent transmission unit.

5. The terminal device control device according to any one of claims 1 to 4, characterized in that: The acquisition module comprises: a first acquisition module, comprising a first infrared diffuse reflection sensor, and configured to determine whether the user exists or not; and The second acquisition module includes a second infrared diffuse reflection sensor and is configured to acquire the user's gesture.

6. The terminal device control device according to claim 5, characterized in that: When the first acquisition module determines that the user does not exist, the terminal device operation is determined to be an operation of performing screen locking on the target terminal device.

7. The terminal device control device according to claim 5, characterized in that: When the second acquisition module acquires the first gesture of the user, the terminal device operation is determined to be an operation of performing screen unlocking on the target terminal device.

8. The terminal device control device according to claim 5, characterized in that: When the second acquisition module acquires the second gesture of the user, the terminal device operation is determined to be an operation of executing a screenshot of the target terminal device.

9. The terminal device control device according to claim 5, characterized in that: When the second acquisition module acquires the third gesture of the user, the terminal device operation is determined to be an operation of performing, on the target terminal device, an operation of selecting the target terminal device.

10. The terminal device control device according to claim 1, characterized in that: Also includes: A setting module is configured to set the terminal device operation corresponding to the action of the user.

11. The terminal device control device according to claim 10, characterized in that: The control module includes a first serial port and a second serial port; The setting module is connected to the first serial port of the control module; and The setting module is configured to reuse the second serial port.

12. A terminal device control method, characterized in that: include: Get the user's actions; Determine the terminal device operation of the user according to the action of the user, and determine the target terminal device of the terminal device operation and the corresponding instruction; as well as transmitting the instruction to the target terminal device so that the target terminal device executes the instruction; The step of transmitting the instruction to the target terminal device so that the target terminal device executes the instruction includes: transmitting a first instruction corresponding to the first terminal device to the first terminal device through a first transmission module communicatively connected to the first terminal device; and The second instruction corresponding to the second terminal device is transmitted to the second terminal device through the second transmission module which is communicatively connected to the second terminal device.

13. The terminal device control method according to claim 12, characterized in that: The step of transmitting the instruction to the target terminal device so that the target terminal device executes the instruction includes: Based on a human-machine interface device protocol, the instruction is transmitted to the target terminal device.

14. The terminal device control method according to claim 12, characterized in that: The first transmission module connected to the first terminal device through communication transmits the first instruction corresponding to the first terminal device to the first terminal device, including: The first instruction is transmitted to the first terminal device through a Bluetooth unit based on a human-machine interface device protocol.

15. The terminal device control method according to claim 14, characterized in that: The method of transmitting the second instruction corresponding to the second terminal device to the second terminal device through the second transmission module that is communicatively connected to the second terminal device includes: Receiving the second instruction through the first Bluetooth transparent transmission unit; receiving the second instruction from the first Bluetooth transparent transmission unit through a second Bluetooth transparent transmission unit paired with the first Bluetooth transparent transmission unit; and The second instruction is received from the second Bluetooth transparent transmission unit through a serial port to universal serial bus unit based on a human-machine interface device protocol that is communicatively connected to the second Bluetooth transparent transmission unit.

16. The terminal device control method according to any one of claims 12 to 15, characterized in that: The action of obtaining the user includes: Determining the presence or absence of the user by means of a first infrared diffuse reflection sensor; and The user's gesture is acquired through a second infrared diffuse reflection sensor.

17. The terminal device control method according to claim 16, characterized in that: When it is determined that the user does not exist, the terminal device operation is determined to be an operation of performing screen locking on the target terminal device.

18. The terminal device control method according to claim 16, characterized in that: When acquiring the first gesture of the user, it is determined that the terminal device operation is an operation of performing screen unlocking on the target terminal device.

19. The terminal device control method according to claim 16, characterized in that: When acquiring the second gesture of the user, it is determined that the terminal device operation is an operation of executing a screenshot of the target terminal device.

20. The terminal device control method according to claim 16, characterized in that: When acquiring the third gesture of the user, it is determined that the terminal device operation is to perform an operation of selecting the target terminal device on the target terminal device.

21. The terminal device control method according to claim 12, characterized in that: Before determining the terminal device operation of the user according to the action of the user, and determining the target terminal device of the terminal device operation and the corresponding instruction, the terminal device control method further includes: Set the terminal device operation corresponding to the action of the user.

22. An electronic device, characterized in that: include: processor; as well as a memory for storing instructions executable by the processor, Wherein, the processor is configured to: execute the terminal device control method described in any one of claims 12 to 21.

23. A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a terminal, enables the terminal to execute the terminal device control method described in any one of claims 12 to 21.

Images