Equipment control method and device, display equipment and computer readable storage medium

Abstract

The invention provides a device control method and device, a display device and a computer readable storage medium, and the method comprises the steps: displaying a terminal display interface of an external terminal device on a television display interface of a target television if it is determined that the external terminal device is accessed; registering a virtual input device; converting a first control instruction aiming at the television display interface into a second control instruction corresponding to the virtual input equipment; and sending the second control instruction to an external terminal device to control a terminal display interface. According to the equipment control method provided by the invention, after the external terminal equipment is accessed, a user can control the external terminal equipment through the television, and the user does not need to move to the external terminal equipment.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of electrical appliances, in particular to a device control method and device, display device and computer readable storage medium. BACKGROUND

[0002] DisplayPort (DP) is a digital video interface standard, which is currently mainly applied to PCs and mobile devices with type-C interfaces. With more digital products (such as TVs, video boxes, etc.) introducing DisplayPort functions, the control experience of DisplayPort sources affects users' selection of the function. The current problems faced by DisplayPort sources include: after a mobile phone or computer outputs a DP signal to a TV display device through a type-C, it still needs to be operated on the device. For example, when a mobile phone or computer is connected to a TV for picture sharing, if a user wants to control the playing picture of the computer or mobile phone through the TV, the user needs to control the computer or mobile phone. Under normal circumstances, the TV is far away from the user, and when the computer and the mobile phone are connected to the TV through a type-C or DP interface, the mobile phone or computer is near the TV, and the user needs to move to the mobile phone or computer to control the picture of the mobile phone or computer, so as to change the picture displayed on the TV. This way is extremely inconvenient. SUMMARY

[0003] The present application provides a device control method, which can control the picture of an external terminal device by controlling the picture of a TV after the external terminal device is connected to the TV.

[0004] In a first aspect, the present application provides a device control method applied to a target TV, the method comprising:

[0005] If it is determined that an external terminal device is connected, displaying a terminal display interface of the external terminal device on a TV display interface of the target TV;

[0006] Registering a virtual input device;

[0007] Converting a first control instruction for the TV display interface into a second control instruction corresponding to the virtual input device;

[0008] Sending the second control instruction to the external terminal device to control the terminal display interface.

[0009] In some embodiments of the present application, the conversion of the first control instruction for the TV display interface into the second control instruction corresponding to the virtual input device comprises:

[0010] Obtaining the first control instruction through a remote control device of the target TV.

[0011] convert the first control instruction into a second control instruction corresponding to the virtual input device.

[0012] In some embodiments of the present application, the first control instruction comprises a direction key click instruction and / or a confirmation key click instruction of the remote control device, the virtual input device comprises a virtual mouse device, and the second control instruction comprises a third control instruction for controlling a cursor corresponding to the virtual mouse device.

[0013] The conversion of the first control instruction into the second control instruction corresponding to the virtual input device comprises:

[0014] obtaining a direction key click instruction and / or a confirmation key click instruction of the remote control device;

[0015] converting the direction key click instruction and / or the confirmation key click instruction into a third control instruction for a cursor corresponding to the virtual mouse device.

[0016] In some embodiments of the present application, the conversion of the direction key click instruction and / or the confirmation key click instruction into the third control instruction for the cursor corresponding to the virtual mouse device comprises:

[0017] determining a current screen resolution of the external terminal device;

[0018] determining a first target moving step length of the cursor corresponding to the virtual mouse device according to the screen resolution;

[0019] converting and obtaining a third control instruction for moving the cursor corresponding to the virtual mouse device according to a click number, direction information corresponding to the direction key click instruction, and the first target moving step length; and / or

[0020] converting and obtaining a third control instruction for clicking the cursor corresponding to the virtual mouse device according to the confirmation key click instruction.

[0021] In some embodiments of the present application, after the conversion of the third control instruction for moving the cursor corresponding to the virtual mouse device according to the click number, the direction information corresponding to the direction key click instruction, and the first target moving step length, the method further comprises:

[0022] if a step length switching instruction for switching the first target moving step length to a second target moving step length is obtained, switching the first target moving step length to the second target moving step length, the second target moving step length being smaller than the first target moving step length.

[0023] According to the re-acquired direction key click instruction, the click times, and the second target moving step, a third control instruction for moving of the cursor corresponding to the virtual mouse device is converted and obtained.

[0024] In some embodiments of the present application, the conversion of the direction key click instruction and / or the confirmation key click instruction into the third control instruction of the cursor corresponding to the virtual mouse device comprises:

[0025] According to the click duration of the direction key click instruction, a target moving speed of the cursor corresponding to the virtual mouse device is determined;

[0026] According to the target moving speed and corresponding direction information, a third control instruction for moving of the cursor corresponding to the virtual mouse device is determined; and / or,

[0027] According to the confirmation key click instruction, a third control instruction for clicking of the cursor corresponding to the virtual mouse device is determined.

[0028] In some embodiments of the present application, the first control instruction comprises a character input instruction for the remote control device, the virtual input device comprises a virtual keyboard device, and the second control instruction comprises a fourth control instruction for controlling the virtual keyboard device;

[0029] The conversion of the first control instruction into the second control instruction corresponding to the virtual input device comprises:

[0030] An character input instruction for the remote control device is acquired;

[0031] The character input instruction is converted into a fourth control instruction corresponding to the virtual keyboard device.

[0032] In some embodiments of the present application, the sending of the second control instruction to the external terminal device comprises:

[0033] The second control instruction is converted into a fifth control instruction of a communication protocol matched with the external terminal device;

[0034] The fifth control instruction is sent to the external terminal device.

[0035] In some embodiments of the present application, the target television comprises a target management module, a non-USB interface, and a target conduction interface, the target management module is configured to control a target switch according to a user instruction, so that the non-USB interface is electrically connected or disconnected with the target conduction interface;

[0036] The determination of the access to the external terminal device comprises:

[0037] If the non-USB interface is in electrical connection with the target on interface, and if the external terminal device is accessed through the non-USB interface, it is determined that an external terminal device is accessed.

[0038] In some embodiments of the present application, the target television further comprises a USB interface, and the target management module is further configured to control the target switch to electrically connect or disconnect the USB interface with the target on interface according to a user instruction, and the method further comprises:

[0039] If the USB interface is in electrical connection with the target on interface, and if it is detected that the USB interface accesses other external devices, a control instruction is exchanged through the USB interface to control the other external devices.

[0040] In a second aspect, the present application further provides a device control apparatus applied to a target television, and the apparatus comprises:

[0041] A display module is configured to display a terminal display interface of the external terminal device on a television display interface of the target television if it is determined that the external terminal device is accessed.

[0042] A registration module is configured to register a virtual input device.

[0043] A conversion module is configured to convert a first control instruction for the television display interface into a second control instruction corresponding to the virtual input device.

[0044] A sending module is configured to send the second control instruction to the external terminal device to control the terminal display interface.

[0045] In a third aspect, the present application further provides a display device, which comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to implement the steps of any of the device control methods.

[0046] In a fourth aspect, the present application further provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the steps of any of the device control methods.

[0047] The device control method provided in this application can synchronize the terminal display interface of an external terminal device to the television display interface after the external terminal device is connected. Since a normal input device can directly control the terminal device after it is connected, the television can register a virtual input device to simulate a normal input device connected to the external terminal device. Next, if the user issues a first control command to the television display interface, the television can convert it into a second control command corresponding to the virtual input device and send it to the external terminal device. This allows the external terminal device to obtain the same control command as a normal physical input device, enabling the user to control the external terminal device simply by controlling the television, without requiring the user to move to the external terminal device. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a schematic diagram of a scenario for the device control system provided in the embodiments of this application;

[0050] Figure 2 This is a schematic flowchart of one embodiment of the device control method in this application;

[0051] Figure 3 This is a schematic diagram of an embodiment of the device control method in this application;

[0052] Figure 4 This is a schematic diagram of an embodiment of the device control method in this application.

[0053] Figure 5 This is a schematic diagram of a functional module of the device control device in an embodiment of this application;

[0054] Figure 6 This is a schematic diagram of the structure of the display device in the embodiments of this application. Detailed Implementation

[0055] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0056] In the description of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0057] In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. Furthermore, it is understood that in the specific embodiments of this application, user information, user data, and other related data are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0058] To enable any person skilled in the art to implement and use this application, the following description is provided. In this description, details are set forth for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be implemented without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0059] This application provides a device control method, apparatus, device, and storage medium, which are described in detail below.

[0060] Please see Figure 1 , Figure 1 This is a schematic diagram of a device control system provided in an embodiment of this application. The device control system may include a television 101 and a terminal device 102, which can transmit data to the television 101. Figure 1 The television 101 in the middle can obtain the logs stored in the terminal device 102 to execute the device control method in this application.

[0061] In this embodiment, the television 101 may include, but is not limited to, a cathode ray tube television, a liquid crystal display television, a light-emitting diode television, etc., and this embodiment does not limit it.

[0062] In this embodiment, the terminal device 102 may include, but is not limited to, desktop computers, portable computers, network servers, PDAs (personal digital assistants), tablet computers, wireless terminal devices, embedded devices, mobile phones, etc., and this embodiment does not limit it.

[0063] In the embodiments of this application, the television 101 and the terminal device 102 can be connected by a wired method, such as a Type-C data cable, a DP data cable, etc., and the embodiments of this application do not limit it.

[0064] It should be noted that, Figure 1 The schematic diagram of the equipment control system shown is merely an example. The equipment control system and scenario described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of equipment control systems and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0065] like Figure 2 As shown, Figure 2 This is a schematic flowchart of an embodiment of the device control method in this application. The device control method can be applied to a target television and includes the following steps 201-204:

[0066] 201. If it is determined that an external terminal device is connected, display the terminal display interface of the external terminal device on the TV display interface of the target TV.

[0067] In this embodiment, the target television may include multiple external device expansion interfaces, such as HDMI, DP, and Type-C interfaces. When an external terminal device, such as a mobile phone or computer, is connected to the television through these expansion interfaces, the television can determine that the external terminal device has been connected.

[0068] Then, according to the corresponding signal transmission protocol and the user's permission (e.g., the user can click a dialog box to allow external device access), the connection is completed. The television can then display the external terminal device's interface on its own screen, achieving image sharing and synchronization.

[0069] 202. Register a virtual input device.

[0070] Since external input devices like computers or mobile phones can directly control computers and mobile phones without user authorization after connecting a mouse or keyboard, once the TV detects an external terminal device connected, it can register the USB-HID device. At this point, other modules in the TV can communicate with the external device through this USB-HID device. Because the signal received by the external terminal device is the input signal from the virtual input device on the TV, the external terminal device will treat it as a normal physical input device and respond to the input signal.

[0071] Registering virtual input devices can be implemented at the software level, including creating virtual mouse operations using pyautogui or pynput, and then registering the virtual input device through its driver, such as WinIo or HID driver. Alternatively, virtual input devices can be registered using scripts, such as Free Mouse Clicker or AutoHotkey. Therefore, there are multiple ways to register virtual input devices, and this application does not limit the specific methods used.

[0072] Furthermore, in this embodiment, the virtual input device may include any peripheral device, such as a virtual mouse, virtual keyboard, virtual touchpad, etc., and this embodiment does not limit it.

[0073] 203. Convert the first control command for the TV display interface into the second control command corresponding to the virtual input device.

[0074] In this embodiment, any operation command performed by the user on the TV display interface can be considered as a first control command. Examples include: zooming in / out, pausing, dragging the progress bar, and turning off the screen. After registering the virtual input device, if the user manipulates the TV display interface, for example, if the TV display interface shows a focus frame and the user moves the focus frame using the arrow keys, the first control command to move the focus frame needs to be converted into a second control command for the virtual input device. For example, assuming the virtual input device is a virtual touchpad, the corresponding focus frame movement command is converted into a swipe command on the virtual touchpad. For instance, if the user moves the focus frame one step, it is converted into a swipe on the virtual touchpad. Alternatively, the user can also control the TV display to pause via voice input. For example, if the user inputs "pause playback," the TV can recognize that the pause button needs to be clicked, and the TV converts this first control command into a single click command on the virtual touchpad. Since the TV display interface is synchronized with the terminal display interface of the external terminal device, the click location is the same, i.e., the location of the pause button is the same. Therefore, the first control command can be directly converted into the second control command.

[0075] 204. Send the second control command to the external terminal device to control the terminal display interface.

[0076] Based on the steps described above, the first control command can be converted into a second control command. Since the external terminal device can directly respond to the control commands from the input device, the television can send the converted second control command to the external terminal device through its access interface. This allows the external terminal device to receive the control signal from the analog input device, thereby enabling control of the external terminal device. Examples include clicking, moving, zooming, etc.

[0077] The device control method provided in this application can synchronize the terminal display interface of an external terminal device to the television display interface after the external terminal device is connected. Since a normal input device can directly control the terminal device after it is connected, the television can register a virtual input device to simulate a normal input device connected to the external terminal device. Next, if the user issues a first control command to the television display interface, the television can convert it into a second control command corresponding to the virtual input device and send it to the external terminal device. This allows the external terminal device to obtain the same control command as a normal physical input device, enabling the user to control the external terminal device simply by controlling the television, without requiring the user to move to the external terminal device.

[0078] To better implement the embodiments of this application, in one embodiment, the first control command for the television display interface is converted into a second control command corresponding to the virtual input device, including:

[0079] The first control command is obtained through the remote control of the target TV; the first control command is converted into a second control command corresponding to the virtual input device.

[0080] The above embodiments provide a solution for issuing a first control command via voice input. In practical applications, the television control device may also include a television remote control, i.e., a remote control device. Therefore, the user can issue the first control command via the remote control, including using the directional keys to move the screen, using the confirmation key to click on the screen, etc.

[0081] If a corresponding first control command is obtained from the remote control, the remote control button command can be converted into a corresponding second control command. For example, the first control command of clicking the directional key can be converted into a second control command of moving, and the first control command of clicking the confirmation key can be converted into a second control command of clicking, etc. This application embodiment will not elaborate on this.

[0082] To better implement the embodiments of this application, in one embodiment, the first control instruction includes a directional key click instruction and / or an confirmation key click instruction for the remote control device, the virtual input device includes a virtual mouse device, and the second control instruction includes a third control instruction for controlling the cursor corresponding to the virtual mouse device; converting the first control instruction into a second control instruction corresponding to the virtual input device includes:

[0083] Obtain directional key click commands and / or confirmation key click commands for the remote control device; convert the directional key click commands and / or confirmation key click commands into third control commands for the corresponding cursor on the virtual mouse device.

[0084] The above embodiments provide a solution where the virtual input device can be a virtual touchpad. In practice, the virtual input device can also include a virtual mouse. Therefore, if it is a virtual mouse, after the TV completes the registration of the virtual mouse, a cursor corresponding to the virtual mouse will be generated on the display interface. Thus, when the user uses the directional keys on the remote control, they can operate the cursor corresponding to the virtual mouse. For example, clicking the directional keys controls the movement of the virtual mouse cursor; clicking the confirmation key controls the clicking of the virtual mouse cursor, and so on.

[0085] Similar to the above embodiments, user operation methods on the remote control device can still include directional key clicks and / or confirmation key clicks. However, in this embodiment, the virtual input device includes a virtual mouse device; therefore, the first control commands of directional key clicks and / or confirmation key clicks need to be converted into third control commands for virtual mouse operations. For example, if a first control command of clicking a directional key once is obtained, this command is converted into a third control command of moving the virtual mouse one step. The specific step size of the virtual mouse movement can be set according to actual conditions and is not limited here. Similarly, if a first control command of clicking the confirmation key once (or twice, simulating a double-click of the left mouse button, depending on the actual settings) is obtained, this command can be converted into a third control command of clicking the virtual mouse once.

[0086] To better implement the embodiments of this application, in one embodiment, the arrow key click command and / or confirmation key click command are converted into a third control command corresponding to the cursor of the virtual mouse device, including:

[0087] Determine the current screen resolution of the external terminal device; based on the screen resolution, determine the first target movement step size of the cursor corresponding to the virtual mouse device; based on the number of clicks, direction information, and the first target movement step size corresponding to the arrow key click command, convert and obtain the third control command for moving the cursor corresponding to the virtual mouse device; and / or, based on the confirmation key click command, convert and obtain the third control command for clicking the cursor corresponding to the virtual mouse device.

[0088] The above embodiments provide a scheme where the mouse movement step size can be set according to actual conditions. To improve the accuracy of virtual mouse control, this application also provides a scheme for determining the virtual mouse's unit movement step size, and converting the third control command based on the unit movement step size.

[0089] In this embodiment, assuming the current screen resolution of the external terminal device is 1920*1080, when the external terminal device connects to the target TV, the target TV can determine its specific resolution information based on the transmitted video signal. Then, a movement step size coefficient can be set, for example, 50. The method for determining the first target movement step size based on the resolution can include 1920 / 50 = 38, 1080 / 50 = 21. If the virtual mouse moves horizontally, it means the user clicks and presses the horizontal direction key on the remote control 50 times to move the mouse cursor from one side to the other. Similarly, if the virtual mouse moves vertically, it means the user clicks and presses the vertical direction key on the remote control 50 times to move the mouse cursor from one side to the other. It should be noted that the value of this step size coefficient is the same as the current definition for the left-right and up-down directions. In actual applications, it will be defined separately according to the signal resolution, and this embodiment does not limit it. Therefore, if the first control information is a single click of the horizontal arrow key, it is converted into a third control instruction to move the virtual mouse horizontally by 38 pixels; or, if the first control information is a single click of the vertical arrow key, it is converted into a third control instruction to move the virtual mouse vertically by 21 pixels. Of course, the more clicks, the greater the movement distance; in this case, multiplying the number of clicks by the unit movement distance yields the final movement distance.

[0090] Furthermore, the first control information obtained by clicking the confirmation button can be converted into the second control information of clicking the virtual left mouse button. For example, clicking the confirmation button once can be converted into the third control information of clicking the left mouse button once. The specific number of times the confirmation button is clicked can be determined according to the actual situation, and this application embodiment does not limit it.

[0091] It should also be noted that this application uses 1920*1080 as an example for illustration, but the actual situation also includes 2K resolution, 4K resolution, etc. The specific unit movement step size can be determined by referring to the above description method, and this application will not elaborate further.

[0092] To better implement the embodiments of this application, in one embodiment, after converting and obtaining the third control command for moving the cursor corresponding to the virtual mouse device based on the number of clicks, direction information, and first target movement step size corresponding to the arrow key click command, the method further includes:

[0093] If a step size switching instruction is obtained to switch the first target movement step size to the second target movement step size, the first target movement step size is switched to the second target movement step size, where the second target movement step size is smaller than the first target movement step size; based on the obtained arrow key click instruction, the number of clicks, and the second target movement step size, an additional third control instruction for moving the cursor of the virtual mouse device is converted and obtained.

[0094] The above embodiments provide a scheme for determining the unit step size based on resolution. However, controlling the virtual mouse using only one unit movement step size may result in poor virtual mouse control and imprecise control. For example, if the user clicks an arrow key once and the mouse moves one unit step, the clicked target may be skipped. Therefore, to address this issue, this application also provides an embodiment.

[0095] Specifically, when using a remote control to control the virtual mouse, the user can first control the movement or clicking of the virtual mouse as described in the above embodiments. When a scenario requires more precise virtual mouse control, the user can choose to switch the command for the unit target movement step size. For example, according to the above embodiments, if clicking the confirmation button once corresponds to clicking the left mouse button once, the target movement step size can be modified by double-clicking the confirmation button. Alternatively, if double-clicking the confirmation button corresponds to clicking the left mouse button once, the target movement step size can be modified by clicking the confirmation button once. The specific settings can be adjusted according to the actual situation.

[0096] The second target movement step size needs to be smaller than the first target movement step size, and the specific size can be set according to the actual situation. For example, if the lateral movement step size of the first target movement step size is 38, then the lateral movement step size of the second target movement step size can be 30, 20, 15, 10, etc.; if the vertical movement step size of the first target movement step size is 21, then the vertical movement step size of the second target movement step size can be 15, 10, 5, etc. It can be seen that it is only necessary to determine that the second target movement step size is smaller than the first target movement step size, and the specific embodiments of this application do not limit it.

[0097] Once the target movement step size is switched, if the user subsequently issues first control commands such as directional key clicks or confirmation key clicks to the remote control device, the second target movement step size can be used to convert these commands into the corresponding third control commands for the virtual mouse.

[0098] Furthermore, in this embodiment of the application, after the user performs the step size switching operation, a window information on whether to switch the step size can be displayed on the TV display interface. If the user clicks the button to agree to switch, then the step size will be switched; if the user clicks the button to cancel switching, it means that the user accidentally touched it, and the step size will not be switched.

[0099] To better implement the embodiments of this application, in one embodiment, the arrow key click command and / or confirmation key click command are converted into a third control command corresponding to the cursor of the virtual mouse device, including:

[0100] Based on the click duration of the arrow key click command, determine the target movement speed of the cursor corresponding to the virtual mouse device; based on the target movement speed and the corresponding direction information, determine the third control command for moving the cursor corresponding to the virtual mouse device; and / or, based on the confirmation key click command, determine the third control command for clicking the cursor corresponding to the virtual mouse device.

[0101] The above embodiments provide a scheme for determining the corresponding third control command based on the number of times the directional keys on the remote control device are clicked. This application also provides other control methods and conversion methods to ensure adaptability to different application scenarios or different user operating habits.

[0102] Specifically, users can press and hold the directional keys on the remote control; the longer the directional key press is held, the faster the virtual mouse cursor moves. For example... Figure 3 As shown. Figure 3 In the graph, the horizontal axis T represents the duration of the key press, and the vertical axis V represents the movement speed of the mouse cursor.

[0103] To better implement the embodiments of this application, in one embodiment, the first control instruction includes a character input instruction for the remote control device, the virtual input device includes a virtual keyboard device, and the second control instruction includes a fourth control instruction corresponding to the virtual keyboard device; converting the first control instruction into a second control instruction corresponding to the virtual input device includes:

[0104] Acquire character input commands for the remote control device; convert the character input commands into the fourth control commands corresponding to the virtual keyboard device.

[0105] The above embodiments provide a solution where the virtual input device can be a virtual mouse. To meet different application scenarios, this application also provides a solution where the virtual input device can be a virtual keyboard.

[0106] Building upon this foundation, if a user manipulates the remote control device, such as adjusting the input method or using keys with character functions, the user's character input command for the remote control device is converted into a fourth control command corresponding to the virtual keyboard device. For example, if the user clicks the key on the remote control device to input the character 'A', the first control command is converted into a fourth control command on the virtual keyboard corresponding to clicking the 'A' character button. Of course, this also includes the conversion of other character keys, which is the same and will not be elaborated further in this embodiment.

[0107] To better implement the embodiments of this application, in one embodiment, sending a second control command to an external terminal device includes:

[0108] The second control command is converted into a fifth control command that matches the communication protocol of the external terminal device; the fifth control command is then sent to the external terminal device.

[0109] To ensure that the second control command, converted from the first control command, can successfully control the external input device, this application also provides a scheme that utilizes a communication protocol to assist in the conversion.

[0110] Specifically, the communication protocol can be the USB-HID protocol. The corresponding second control command is encoded and transmitted through the USB-HID protocol, which ensures that the external terminal device can perform the operation of the corresponding first control command issued by the user according to the fifth control command, and complete the control of the external terminal device.

[0111] To better implement the embodiments of this application, in one embodiment, the target television includes a target management module, a non-USB interface, and a target connection interface. The target management module is used to control a target switch according to user instructions, so that the non-USB interface and the target connection interface are electrically connected or disconnected. If it is determined that an external terminal device is connected, the method includes: if the non-USB interface and the target connection interface are electrically connected, and if the external terminal device is connected through the non-USB interface, then it is determined that an external terminal device is connected. Simultaneously, the target television also includes a USB interface. The target management module is further used to control the target switch according to user instructions, so that the USB interface and the target connection interface are electrically connected or disconnected. The method further includes: if the USB interface and the target connection interface are electrically connected, and if it is detected that the USB interface is connected to another external device, control commands are exchanged through the USB interface to control the other external device.

[0112] This application also provides a port structure for detecting whether a target television is connected to an external terminal device. See details below. Figure 4 .like Figure 4As shown, the target switch can be a single-switch switch, which can connect the USB interface to the target conductive interface or the non-USB interface to the target conductive interface according to user commands. The target conductive interface is used for electrical connection to the TV's display module, allowing user commands to be executed on the TV display interface and fed back to external terminal devices. When the USB interface is connected to the target conductive interface, the user can connect a USB device to an external USB port for use.

[0113] In summary, the overall solution of this application embodiment may include:

[0114] USB Port Control Module: This module controls the connection or disconnection of the USB port on the Type-C port from the TV's USB port. When connected, the TV can communicate with external devices, such as mobile phones, via USB.

[0115] Virtual Mouse Event Handling Module: This module is one of the core modules of this solution. It mainly converts the TV remote control button operation information into mouse events, allowing users to operate the mouse of external devices through the remote control and perform operations such as mouse movement and clicking.

[0116] Virtual keyboard event handling module: This module is one of the core modules of this solution. The remote control operation brings up the TV input method and converts the operation information on the virtual keyboard of the input method into keyboard events, allowing users to realize the key input function of the external device keyboard through the remote control.

[0117] Virtual Mouse and Keyboard Event Sending Module: This module converts mouse or keyboard events sent from the virtual mouse and keyboard processing module into USB-HID events, and then sends them to external devices through the USB-HID processing module.

[0118] USB-HID processing module: When the USB port control module connects the USB of the Type-C port to the USB of the TV, this module registers the USB-HID device with the TV system, and deregisters the USB-HID device when disconnected.

[0119] Type-C Hardware Management Module: This is a hardware module controlled by the USB port control module. It enables the physical connection and disconnection of the Type-C port's USB and TV USB ports.

[0120] To better implement the device control method in the embodiments of this application, in addition to the device control method, the embodiments of this application also provide a device control apparatus, which is applied to a target television, such as... Figure 5 As shown, the device 300 includes:

[0121] Display module 301 is used to display the terminal display interface of the external terminal device on the TV display interface of the target TV if it is determined that an external terminal device is connected.

[0122] Registration module 302 is used to register virtual input devices;

[0123] The conversion module 303 is used to convert the first control command for the TV display interface into the second control command corresponding to the virtual input device.

[0124] The sending module 304 is used to send the second control command to an external terminal device to control the terminal display interface.

[0125] The device control apparatus provided in this application can synchronize the terminal display interface of an external terminal device to the television display interface via the display module 301 after the external terminal device is connected. Since a normal input device can directly control the terminal device after it is connected, the television can register a virtual input device via the registration module 302 to simulate a normal input device connecting to the external terminal device. Next, if the user issues a first control command to the television display interface, the television can use the conversion module 303 to convert it into a second control command corresponding to the virtual input device, and send it to the external terminal device via the sending module 304. This allows the external terminal device to obtain the same control command as a normal physical input device, enabling the user to control the external terminal device simply by controlling the television, without requiring the user to move to the external terminal device.

[0126] In some embodiments of this application, the conversion module 303 is specifically used for:

[0127] Obtain the first control command through the remote control device of the target TV;

[0128] The first control command is converted into a second control command corresponding to the virtual input device.

[0129] In some embodiments of this application, the first control instruction includes a directional key click instruction and / or an confirmation key click instruction for the remote control device, the virtual input device includes a virtual mouse device, and the second control instruction includes a third control instruction for controlling the cursor corresponding to the virtual mouse device. The conversion module 303 is further specifically used for:

[0130] Obtain directional key click commands and / or confirmation key click commands for the remote control device;

[0131] Translate arrow key click commands and / or confirmation key click commands into third control commands for the corresponding cursor on the virtual mouse device.

[0132] In some embodiments of this application, the conversion module 303 is further configured to:

[0133] Determine the current screen resolution of the external terminal device;

[0134] Based on the screen resolution, determine the first target movement step of the virtual mouse device corresponding to the cursor;

[0135] Based on the number of clicks, direction information, and first target movement step size corresponding to the arrow key click command, a third control command for moving the cursor corresponding to the virtual mouse device is converted and obtained; and / or,

[0136] Based on the confirmation key click instruction, the third control instruction for clicking the corresponding cursor of the virtual mouse device is converted and obtained.

[0137] In some embodiments of this application, after converting and obtaining the third control command for moving the cursor corresponding to the virtual mouse device based on the number of clicks, direction information, and first target movement step size corresponding to the arrow key click command, the conversion module 303 is further used for:

[0138] If a step size switching instruction is obtained to switch the first target movement step size to the second target movement step size, the first target movement step size is switched to the second target movement step size, and the second target movement step size is smaller than the first target movement step size;

[0139] Based on the acquired arrow key click commands, the number of clicks, and the second target movement step, additional third control commands for moving the virtual mouse device's corresponding cursor are converted and obtained.

[0140] In some embodiments of this application, the conversion module 303 is further configured to:

[0141] The target movement speed of the cursor on the virtual mouse device is determined based on the duration of the arrow key click command.

[0142] Based on the target's movement speed and corresponding direction information, determine the third control command for moving the cursor on the virtual mouse device; and / or,

[0143] Based on the confirmation button click command, determine the third control command for clicking the cursor corresponding to the virtual mouse device.

[0144] In some embodiments of this application, the first control instruction includes a character input instruction for the remote control device, the virtual input device includes a virtual keyboard device, the second control instruction includes a fourth control instruction corresponding to the virtual keyboard device, and the conversion module 303 is further specifically used for:

[0145] Obtain character input commands for the remote control device;

[0146] Translate character input commands into the fourth control command corresponding to the virtual keyboard device.

[0147] In some embodiments of this application, the sending module 304 is further configured to:

[0148] The second control command is converted into a fifth control command that matches the communication protocol of the external terminal device;

[0149] The fifth control command is sent to the external terminal device.

[0150] In some embodiments of this application, the target television includes a target management module, a non-USB interface, and a target connection interface. The target management module is used to control a target switch according to user instructions, so that the non-USB interface is electrically connected or disconnected from the target connection interface. The device also includes a detection module, which is specifically used for:

[0151] If the non-USB interface and the target conductive interface are electrically connected, and if the external terminal device is connected through the non-USB interface, then the external terminal device is confirmed to be connected.

[0152] In some embodiments of this application, the target television also includes a USB interface, and the target management module is further configured to control the target switch according to user instructions, so that the USB interface is electrically connected or disconnected from the target conduction interface. The detection module is further specifically configured to:

[0153] If the USB interface is electrically connected to the target conductive interface, and if other external devices are detected connected to the USB interface, control commands can be exchanged through the USB interface to control the other external devices.

[0154] This application also provides a display device, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor. The processor executes the computer program to implement the steps of the device control method according to any one of the embodiments of this application. The display device integrates any one of the device control methods provided in the embodiments of this application, such as... Figure 6 As shown, it illustrates a schematic diagram of the display device involved in the embodiments of this application. Specifically:

[0155] The display device may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will understand that... Figure 6 The display device structure shown does not constitute a limitation on the display device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0156] The processor 401 is the control center of the display device. It connects various parts of the display device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 402, and by calling data stored in the memory 402, it performs various functions and processes data of the display device, thereby providing overall monitoring of the display device. Optionally, the processor 401 may include one or more processing cores. The processor 401 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. Preferably, the processor 401 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 401.

[0157] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the display device, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0158] The display device also includes a power supply 403 that supplies power to the various components. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 403 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0159] The display device may also include an input unit 404, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0160] Although not shown, the display device may also include display units, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the display device loads the executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 runs the application programs stored in the memory 402 to realize various functions, such as:

[0161] If it is determined that an external terminal device is connected, the terminal display interface of the external terminal device will be displayed on the TV display interface of the target TV.

[0162] Register a virtual input device;

[0163] The first control command for the TV display interface is converted into a second control command corresponding to the virtual input device;

[0164] The second control command is sent to the external terminal device to control the terminal display interface.

[0165] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0166] Therefore, embodiments of this application provide a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any of the device control methods provided in embodiments of this application. For example, the computer program loaded by the processor can execute the following steps:

[0167] If it is determined that an external terminal device is connected, the terminal display interface of the external terminal device will be displayed on the TV display interface of the target TV.

[0168] Register a virtual input device;

[0169] The first control command for the TV display interface is converted into a second control command corresponding to the virtual input device;

[0170] The second control command is sent to the external terminal device to control the terminal display interface.

[0171] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed descriptions of other embodiments above, which will not be repeated here.

[0172] In practice, each of the above units or structures can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units or structures, please refer to the previous method embodiments, which will not be repeated here.

[0173] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0174] The above provides a detailed description of a device control method and apparatus provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A device control method, characterized in that, Applied to a target television, the method includes: If it is determined that an external terminal device is connected, the terminal display interface of the external terminal device will be displayed on the TV display interface of the target TV. Register a virtual input device; The first control command for the television display interface is converted into a second control command corresponding to the virtual input device; The second control command is sent to the external terminal device to control the terminal display interface.

2. The equipment control method according to claim 1, characterized in that, The step of converting the first control command for the television display interface into a second control command corresponding to the virtual input device includes: The first control command is obtained through the remote control device of the target television; The first control command is converted into a second control command corresponding to the virtual input device.

3. The equipment control method according to claim 2, characterized in that, The first control instruction includes the directional key click instruction and / or confirmation key click instruction of the remote control device, the virtual input device includes a virtual mouse device, and the second control instruction includes a third control instruction to control the cursor corresponding to the virtual mouse device; The step of converting the first control command into a second control command corresponding to the virtual input device includes: Obtain directional key click commands and / or confirmation key click commands for the remote control device; The arrow key click command and / or confirmation key click command are converted into the third control command of the cursor corresponding to the virtual mouse device.

4. The equipment control method according to claim 3, characterized in that, The step of converting the arrow key click command and / or confirmation key click command into a third control command corresponding to the cursor of the virtual mouse device includes: Determine the current screen resolution of the external terminal device; Based on the screen resolution, determine the first target movement step of the cursor corresponding to the virtual mouse device; Based on the number of clicks, direction information, and the first target movement step size corresponding to the arrow key click command, a third control command for moving the cursor of the virtual mouse device is converted and obtained; and / or, Based on the confirmation key click instruction, a third control instruction for clicking the cursor corresponding to the virtual mouse device is converted and obtained.

5. The equipment control method according to claim 4, characterized in that, After converting and obtaining the third control command for moving the cursor of the virtual mouse device based on the number of clicks, direction information, and the first target movement step size corresponding to the directional key click command, the method further includes: If a step size switching instruction is obtained to switch the first target movement step size to the second target movement step size, the first target movement step size is switched to the second target movement step size, where the second target movement step size is smaller than the first target movement step size; Based on the acquired directional key click command, the number of clicks, and the second target movement step size, an additional third control command for moving the cursor corresponding to the virtual mouse device is converted and obtained.

6. The equipment control method according to claim 3, characterized in that, The step of converting the arrow key click command and / or confirmation key click command into a third control command corresponding to the cursor of the virtual mouse device includes: The target movement speed of the cursor corresponding to the virtual mouse device is determined based on the click duration of the arrow key click command. Based on the target movement speed and corresponding direction information, a third control command for moving the cursor of the virtual mouse device is determined; and / or, Based on the confirmation button click instruction, a third control instruction for clicking the cursor corresponding to the virtual mouse device is determined.

7. The equipment control method according to claim 2, characterized in that, The first control instruction includes a character input instruction for the remote control device, the virtual input device includes a virtual keyboard device, and the second control instruction includes a fourth control instruction corresponding to the virtual keyboard device; The step of converting the first control command into a second control command corresponding to the virtual input device includes: Obtain character input commands for the remote control device; The character input instructions are converted into the fourth control instructions corresponding to the virtual keyboard device.

8. The equipment control method according to claim 1, characterized in that, Sending the second control command to the external terminal device includes: The second control command is converted into a fifth control command that matches the communication protocol of the external terminal device; The fifth control command is sent to the external terminal device.

9. The equipment control method according to claim 1, characterized in that, The target TV includes a target management module, a non-USB interface, and a target connection interface. The target management module is used to control the target switch according to user instructions, so that the non-USB interface is electrically connected or disconnected from the target connection interface. If it is determined that an external terminal device is connected, it includes: If the non-USB interface is electrically connected to the target conductive interface, and if the external terminal device is connected through the non-USB interface, then it is determined that the external terminal device is connected.

10. The equipment control method according to claim 9, characterized in that, The target television also includes a USB interface. The target management module is further configured to control a target switch according to user instructions, so that the USB interface is electrically connected or disconnected from the target conduction interface. The method further includes: If the USB interface is electrically connected to the target conductive interface, and if it is detected that the USB interface is connected to another external device, control commands can be exchanged through the USB interface to control the other external device.

11. A device control apparatus, characterized in that, Applied to a target television, the device includes: The display module is used to display the terminal display interface of the external terminal device on the TV display interface of the target TV if it is determined that an external terminal device is connected. The registration module is used to register virtual input devices; The conversion module is used to convert the first control command for the TV display interface into the second control command corresponding to the virtual input device; The sending module is used to send the second control command to the external terminal device to control the terminal display interface.

12. A display device, characterized in that, The display device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the device control method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is executed by a processor to implement the steps of the device control method according to any one of claims 1 to 10.

Images