Device control method and electronic device

Abstract

The application discloses a device control method and an electronic device. The method applied to a first electronic device comprises the following steps: obtaining target display data of a second electronic device; sending the target display data to a third electronic device; the first electronic device is connected with the second electronic device; obtaining control data sent by the third electronic device; the control data is generated in response to a user's interactive operation on the target display data output in the third electronic device; based on the control data, a corresponding input event is generated, and the input event is sent to the second electronic device.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a device control method and apparatus. Background Technology

[0002] Currently, docking stations can synchronize the display signal of one connected electronic device to other connected electronic devices, achieving display synchronization.

[0003] However, users can only view the displays of other electronic devices through their own electronic devices, but cannot control them, resulting in a poor user experience. Summary of the Invention

[0004] In view of the above, this application provides a device control method and an electronic device, as follows:

[0005] A device control method, applied to a first electronic device, the method comprising:

[0006] Acquire target display data from the second electronic device;

[0007] The target display data is sent to a third electronic device; the first electronic device is connected to the second electronic device;

[0008] Acquire control data sent by the third electronic device; the control data is generated in response to an interactive operation by a user on the target display data output in the third electronic device;

[0009] Based on the control data, a corresponding input event is generated, and the input event is sent to the second electronic device.

[0010] Optionally, the above method generates corresponding input events based on the control data, including:

[0011] Based on the target transmission rules between the first electronic device and the second electronic device, the control data is converted into corresponding input events;

[0012] The input event enables the second electronic device to recognize the first electronic device as an input device based on the target transmission rule.

[0013] Optionally, the above method involves acquiring the target display data of the second electronic device, including:

[0014] Obtain the raw display data of the second electronic device;

[0015] Based on the data specifications supported by the third electronic device, the original display data is processed to obtain the target display data.

[0016] Optionally, the above method involves processing the original display data based on the data specifications supported by the third electronic device to obtain target display data, including at least one of the following:

[0017] Based on the channel type of the data transmission channel between the first electronic device and the third electronic device, the data format of the original display data is processed to obtain the target display data, wherein the data format of the target display data matches the channel type;

[0018] Based on the channel bandwidth of the data transmission channel, the amount of data in the original display data is processed, and the amount of data in the target display data is matched with the channel bandwidth.

[0019] Optionally, the above method may further include:

[0020] Determine the operating status of the target control function of the first electronic device;

[0021] If the target control function of the first electronic device is in the activated state, the target display data is sent to the third electronic device.

[0022] Optionally, the above method involves determining the operating state of the target control function of the first electronic device, including at least one of the following:

[0023] In response to a control start command from the third electronic device, the operating state of the target control function of the first electronic device is determined to be the start state;

[0024] In response to the user's trigger operation on the first electronic device, the operating state of the target control function of the first electronic device is determined to be the activated state, and the trigger operation is used to trigger the target control function.

[0025] Optionally, the above method may further include:

[0026] If the target transmission function of the first electronic device is in the activated state, the target data in the storage device is obtained, and the storage device is connected to the first electronic device.

[0027] The target data is stored in the third electronic device.

[0028] A device control method, applied to a third electronic device, the method comprising:

[0029] Obtain the target display data that the first electronic device receives from the second electronic device;

[0030] The target display data is output; the third electronic device establishes a connection with the second electronic device through the first electronic device;

[0031] Control data is generated in response to an interactive operation by a user on the target display data output in the third electronic device;

[0032] The control data is sent to the first electronic device, and the control data is used to trigger the first electronic device to generate a corresponding input event and send the input event to the second electronic device.

[0033] An electronic device, the electronic device being a first electronic device, includes:

[0034] A communication module is used to acquire target display data from a second electronic device and send the target display data to a third electronic device; the first electronic device is connected to the second electronic device; and receives control data sent by the third electronic device; the control data is generated in response to a user's interactive operation on the target display data output in the third electronic device;

[0035] The processor is configured to generate corresponding input events based on the control data, and send the input events to the second electronic device via the communication module.

[0036] An electronic device, wherein the electronic device is a third electronic device, comprises:

[0037] The communication module is used to obtain target display data from the second electronic device obtained by the first electronic device;

[0038] A display for outputting the target display data; the third electronic device establishes a connection with the second electronic device through the first electronic device;

[0039] The processor is configured to generate control data in response to an interactive operation by a user on the target display data output in the third electronic device, and to send the control data to the first electronic device via the communication module. The control data is used to trigger the first electronic device to generate a corresponding input event and send the input event to the second electronic device. Attached Figure Description

[0040] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 A flowchart of a device control method provided in an embodiment of this application;

[0042] Figure 2 This is an example diagram illustrating the connection between the first electronic device, the second electronic device, and the third electronic device in an embodiment of this application.

[0043] Figure 3 This is a partial flowchart of a device control method provided in an embodiment of this application;

[0044] Figure 4 This is an example diagram illustrating the triggering of the target control function in an embodiment of this application;

[0045] Figure 5 This is another example diagram illustrating the triggering of the target control function in an embodiment of this application;

[0046] Figure 6 A flowchart illustrating another device control method provided in this application embodiment;

[0047] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0048] Figure 8 This is a schematic diagram of the structure of another electronic device provided in an embodiment of this application;

[0049] Figure 9 This is an example diagram illustrating the application of this application to cloud-based operation and maintenance scenarios. Detailed Implementation

[0050] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0051] refer to Figure 1 The diagram shown is a flowchart illustrating the implementation of a device control method according to an embodiment of this application. This method can be applied to a first electronic device, which can be an Internet of Things docking station (IoT dock). The first electronic device is connected to a second electronic device, as shown below. Figure 2As shown, the second electronic device can be a laptop computer, and the third electronic device can be a cloud server or a remote control terminal (a display device with keyboard and mouse input devices). The device control method in this embodiment is mainly used for display synchronization and remote control between the third electronic device and the second electronic device, thereby improving the user's experience with the electronic devices.

[0052] Specifically, the method in this embodiment may include the following steps:

[0053] Step 101: Obtain the target display data of the second electronic device.

[0054] Specifically, the first electronic device is connected to the second electronic device via a physical wired connection. For example, the first electronic device has a built-in multi-stream transport hub (MST Hub) and a universal serial bus hub (USB Hub). The MST Hub and USB Hub are physically connected to a physical interface such as a Type-C interface in the second electronic device, sharing the four pairs of high-speed signal transmission paths within the Type-C interface. This allows for the multiplexing and isolation of display data streams and control data streams within the same physical interface, preventing interference between them.

[0055] Specifically, the MST Hub can be used for the distribution and expansion of display data. The MST Hub can capture raw display data from a second electronic device, distribute and expand a single display output link to multiple devices simultaneously, and distribute the target display data to each device for synchronized screen output. Simultaneously, the MST Hub distributes raw display data to the processor or processing chip inside the first electronic device (such as a docking station) for subsequent compression, encoding, conversion, and other processing before uploading it to a third electronic device as the source data for remote control.

[0056] A USB hub can be used for the transparent transmission of input events. On one hand, a USB hub can expand the external USB interface of a first electronic device (such as a docking station) to multiple devices, allowing connection of conventional USB peripherals such as USB flash drives or keyboards and mice. On the other hand, the USB hub establishes a physical connection with the processing chip inside the first electronic device (such as a docking station), carries the human interface device (HID) events generated by the processing chip, and transmits the HID events to a second electronic device, enabling the second electronic device to recognize the first electronic device as a virtual keyboard and mouse input device.

[0057] In one implementation, this embodiment utilizes the MST Hub in the first electronic device to collect the raw display data of the second electronic device and obtain the corresponding target display data accordingly.

[0058] It should be noted that the target display data is compatible with a third electronic device. For example, compatibility of the target display data with a third electronic device includes at least the following: the data format of the target display data enables the target display data to be transmitted from the first electronic device to the third electronic device, and to be processed by the third electronic device and displayed to the user.

[0059] For example, taking the connection of an IoT docking station to a laptop as an example, the laptop is directly connected to the IoT docking station via a Type-C interface. The MST Hub in the IoT docking station intercepts the display screen data stream of the laptop's computer desktop in real time and sends it to the IoT docking station, so that the IoT docking station can obtain the target display data based on the display screen data stream of the computer desktop.

[0060] Step 102: Send the target display data to the third electronic device.

[0061] The first electronic device incorporates a built-in Internet of Things (IoT) communication module, which establishes a wired or wireless communication link between the first and third electronic devices. The third electronic device can be a local display device or a remote cloud server. Based on this, the first electronic device can upload acquired target display data to the third electronic device via the IoT communication module for image rendering and display.

[0062] It should be noted that after the target display data is rendered and displayed on the third electronic device, the user can interact with the output target display data on the third electronic device. Interactive operations can include at least one of the following: mouse movement, mouse click, keyboard text input, window manipulation, and keyboard shortcuts. Correspondingly, control data is generated on the third electronic device in response to the interactive operations.

[0063] Step 103: Obtain control data sent by the third electronic device.

[0064] The control data is generated in response to the user's interactive operation on the target display data output in the third electronic device.

[0065] In one implementation, the third electronic device encapsulates each interaction into a standardized control data packet, i.e., control data, and sends it to the first electronic device through a communication link. Thus, the first electronic device can receive the control data sent by the third electronic device through the IoT communication module.

[0066] In another type of electronic device, the third electronic device merges and encapsulates multiple interactive operations, and then sends them to the first electronic device through a communication link. As a result, the first electronic device can receive the control data sent by the third electronic device through the IoT communication module, which can reduce the frequency of interaction between the third electronic device and the first electronic device.

[0067] Step 104: Based on the control data, generate the corresponding input event and send the input event to the second electronic device.

[0068] The first electronic device can have a built-in processing chip, which is physically soldered or connected to a USB hub inside the first electronic device via board traces. The processing chip can convert the received control data into standard HID events and send the HID events to the second electronic device. Based on this, the second electronic device can directly respond to HID events without installing additional drivers or software, and perform actions such as remote clicking, input, and window manipulation.

[0069] In one implementation, the processing chip in the first electronic device can be set up independently, communicating only with the USB Hub and not occupying the transmission resources of the MST Hub. For example, the processing chip is located separately on the motherboard side of the IoT docking station and is connected to the USB Hub only through differential signal lines.

[0070] In another implementation, the processing chip can be integrated into the MST Hub package, reusing the internal power supply and motherboard wiring structure of the first electronic device, thus reducing the overall size and hardware cost. For example, in an IoT docking station, a system-on-chip (SoC) can simultaneously handle MST processing, USB virtualization, and IoT networking functions.

[0071] As can be seen from the above technical solutions, in the device control method provided in this application embodiment, while achieving display synchronization in the second electronic device and the third electronic device, an input event can be generated based on the control data generated on the third electronic device and sent to the second electronic device, thereby realizing remote control from the third electronic device to the second electronic device. It is evident that this embodiment relies on the bidirectional transparent transmission of display data and control data in the first electronic device, eliminating the need to install dedicated software on the second electronic device to achieve display synchronization and remote control between the first and third electronic devices. Therefore, in operation and maintenance scenarios, this simplifies remote operation and maintenance processes, improves equipment fault diagnosis efficiency, and ultimately enhances the user experience of electronic devices.

[0072] Based on the above implementation scheme, in one implementation method, in step 104, when generating the corresponding input event based on the control data, the control data is converted into the corresponding input event based on the target transmission rules between the first electronic device and the second electronic device.

[0073] The input event enables the second electronic device to recognize the first electronic device as an input device based on the target transmission rules.

[0074] It should be noted that the target transmission rule follows the data transmission protocol between the first electronic device and the second electronic device, such as the Universal Serial Bus transmission protocol. Based on this, after the first electronic device generates an input event based on the target transmission rule, it transmits it to the second electronic device through the USB Hub. The corresponding input event enables the second electronic device to recognize the first electronic device as an external virtual keyboard and mouse input device based on the Universal Serial Bus transmission protocol.

[0075] In one implementation, the processing chip in the first electronic device can encapsulate control data such as mouse click coordinates and keyboard key commands sent by the third electronic device (such as a cloud server) into standard HID events, such as standard USB HID report descriptors, according to the data transmission protocol corresponding to the target transmission rules. Then, it can be transparently transmitted to the second electronic device through the USB Hub, and the second electronic device can perform corresponding actions according to the HID events.

[0076] For example, a user views the laptop desktop displayed on the cloud server's monitor and interacts with the laptop desktop by moving the mouse. The cloud server responds to the user's interaction by generating control data and sending it to the IoT docking station. The IoT docking station encapsulates the mouse movement coordinates and other data from the control data sent by the cloud server into HID events for USB mouse movement and sends the HID events to the laptop. The laptop recognizes the device connected to the IoT docking station as a USB mouse based on the HID events, and thus the laptop can perform corresponding mouse movement operations in response to the HID events.

[0077] In another implementation, the processing chip in the first electronic device can cache multiple consecutive control data received from the third electronic device within a certain time period, and package them into a set of HID events to be sent to the second electronic device in batches, thereby reducing the number of interactions between the first and second electronic devices.

[0078] For example, a user views a laptop desktop displayed on a cloud server's monitor and repeatedly presses multiple keys on the keyboard to input multiple characters. The cloud server responds to the user's interaction by generating control data and sending it to an IoT docking station. The IoT docking station encapsulates the multiple characters in the control data sent by the cloud server into HID events representing multiple consecutive inputs from a USB keyboard and sends these HID events to the laptop. The laptop recognizes the device connected to the IoT docking station as a USB keyboard based on the HID events, thus enabling the laptop to perform corresponding continuous input operations in response to the HID events.

[0079] Based on the above implementation scheme, in one implementation method, when acquiring the target display data of the second electronic device in step 101, it can be achieved in the following way, such as... Figure 3 As shown:

[0080] Step 301: Obtain the raw display data of the second electronic device.

[0081] The raw display data output by the second electronic device can be acquired on the first electronic device via MST Hub.

[0082] Step 302: Based on the data specifications supported by the third electronic device, process the raw display data to obtain the target display data.

[0083] It should be noted that the raw display data is an uncompressed, high-bandwidth native high-definition image data stream. Based on this, the first electronic device processes the raw display data according to the data rules supported by the third electronic device, such as adapting at least one of the following: data format, data volume, resolution, and encoding bitstream, to obtain the target display data adapted for transmission to the third electronic device.

[0084] Specifically, step 302, based on the data specifications supported by the third electronic device, processes the original display data to obtain the target display data, which can be achieved through at least one of the following processing methods:

[0085] In one processing method, step 302 can be based on the channel type of the data transmission channel between the first electronic device and the third electronic device to process the data format of the original display data in order to obtain the target display data.

[0086] In this embodiment, the data format of the target display data is matched with the channel type. The channel type represents the data transmission protocol used in the data transmission channel. For example, the channel type can be Ethernet wired channel, wireless LAN channel, cellular 5G / 4G channel, enterprise intranet leased line channel, etc. Correspondingly, the data format can be binary format, message pack format, JavaScript object notation (JSON) format, etc., with each data matching its corresponding data transmission protocol, i.e., the corresponding channel type. Based on this, in this embodiment, the original display data is converted according to the data transmission protocol between the first electronic device and the third electronic device to obtain target display data whose data format matches the channel type of the data transmission channel.

[0087] For example, a cellular network channel is used between the cloud server and the IoT docking station. Based on this, a lightweight binary format is used on the processing chip of the IoT docking station to convert the data format of the original display data. Alternatively, a dedicated enterprise intranet line is used between the cloud server and the IoT docking station. Based on this, a lossless original frame format is used on the processing chip of the IoT docking station to convert the data format of the original display data, and so on.

[0088] As can be seen, this embodiment can achieve adaptive matching between the data format of the displayed data and the channel type of the data transmission channel, enabling remote viewing while ensuring transmission reliability in different network environments (channel types).

[0089] In another processing method, step 302 can be based on the channel bandwidth of the data transmission channel to process the amount of data in the original display data to obtain the target display data.

[0090] The target data volume is matched with the channel bandwidth. Channel bandwidth characterizes the transmission performance of the data transmission channel, such as the amount of data transmitted per second.

[0091] Specifically, in this embodiment, the resolution and encoding bitstream of the original display data can be dynamically compressed and adjusted so that the amount of data in the target display data matches the current bandwidth of the data transmission channel.

[0092] As can be seen, this embodiment can achieve adaptive adaptation between the data format of the displayed data and the channel bandwidth of the data transmission channel, enabling remote viewing while taking into account the transmission performance of different network environments (channel bandwidth) to ensure the smooth transmission of the displayed data.

[0093] For example, the original high-definition screen data stream of a laptop desktop is 1080P high bitrate. When the bandwidth in the data transmission channel is sufficient, the IoT docking station can keep the original high-definition screen data stream of the laptop desktop at 1080P high bitrate, thereby transmitting the 1080P high bitrate high-definition screen data stream to the cloud server; when the bandwidth in the data transmission channel is congested, the IoT docking station can reduce the original high-definition screen data stream of the laptop desktop to 480P low bitrate, thereby transmitting the 480P low bitrate low-definition screen data stream to the cloud server.

[0094] For example, the original high-definition screen data stream of a laptop desktop is 4K. During peak office hours, when the wireless network bandwidth between the cloud server and the IoT docking station decreases, the IoT docking station can compress the original high-definition screen data stream of the laptop desktop from 4K to 720P standard definition resolution. This allows the standard definition resolution screen data stream to be transmitted to the cloud server, saving bandwidth and ensuring that the remote screen on the cloud server does not lag.

[0095] For example, the original high-definition screen data stream of a laptop desktop is a lossless red, green, and blue pixel image (RGB). During peak office hours, when the wireless network bandwidth between the cloud server and the IoT docking station decreases, the IoT docking station can convert the original high-definition screen data stream of the laptop desktop from the lossless RGB image data stream to the H.265 encoded bitstream. This allows the converted image data stream to be transmitted to the cloud server, saving more than 60% of bandwidth and ensuring that the remote image on the cloud server does not lag.

[0096] For example, the original high-definition screen data stream of a laptop desktop is in 444 sampling luminance chrominance format (YUV444). When the wireless network bandwidth between the cloud server and the IoT docking station decreases during peak office hours, the IoT docking station can simplify the original high-definition screen data stream of the laptop desktop to a lightweight 420 sampling luminance chrominance format (YUV420), thereby reducing the amount of data per frame in the screen data stream.

[0097] As can be seen, in this embodiment, the format of the original display data can be converted based on the channel type of the transmission channel, and the amount of the original display data can also be changed based on the channel bandwidth. Thus, in this embodiment, through dual adaptive adaptation of data format and data amount, it is possible to achieve remote viewing while taking into account the transmission reliability and transmission smoothness under different network environments.

[0098] Based on the above implementation scheme, this embodiment can also add control logic for the status detection and conditional execution of the target control function (i.e., the remote control function from the third electronic device to the second electronic device). Specifically, in this embodiment, before step 102, the running status of the target control function of the first electronic device can be determined first. If the running status of the target control function of the first electronic device is the start state, the target display data is sent to the third electronic device.

[0099] The first electronic device can monitor the operating status of its target control function in real time, which can include three states: active, sleep, and off. If the first electronic device detects that the target control function is active, it triggers the step of sending target display data to the third electronic device. If the first electronic device detects that the target control function is in sleep or off state, it at least suspends the uploading of target display data to the third electronic device to reduce the power consumption of the first electronic device and the network bandwidth occupied between the first and third electronic devices.

[0100] Furthermore, if the first electronic device detects that the target control function is in a sleep or off state, it also suspends the acquisition of target display data, further reducing the power consumption of the first electronic device.

[0101] It should be noted that after the first electronic device is powered on, it continuously monitors the on / off status of the target control function in the background and is ready to be triggered at any time.

[0102] For example, in an enterprise office setting, an IoT docking station that connects a local laptop to a cloud server and other display devices is always powered on, ready to be activated by the remote control function and await remote control of the laptop by the cloud server.

[0103] In specific implementation, the operating status of the target control function in the first electronic device can be determined through at least one of the following processing methods:

[0104] In one processing method, in this embodiment, the operating state of the target control function of the first electronic device can be determined to be in the start state in response to a control start command from a third electronic device.

[0105] Specifically, a control interface can be output on the third electronic device. The control interface displays the trigger controls corresponding to at least one second electronic device connected to the first electronic device. The user can select one or more second electronic devices in the control interface and click on their corresponding trigger controls. In response to the click operation, the third electronic device generates a corresponding control start command and sends it to the first electronic device. The first electronic device responds to the control start command and determines that the target control function enters the start state.

[0106] It should be noted that the control start command corresponds to the second electronic device connected to the first electronic device. This can also be understood as the third electronic device's control over each second electronic device through the first electronic device being independent. In the control interface output by the third electronic device, the trigger controls corresponding to each second electronic device are independent. Specifically, the user can click on a trigger control corresponding to one of the second electronic devices. After the third electronic device sends the generated control start command to the first electronic device, the first electronic device can send target display data to the third electronic device. Upon receiving the control data corresponding to that second electronic device from the third electronic device, the first electronic device generates a corresponding input event and sends it to the corresponding second electronic device, achieving remote control of a single device. Alternatively, the user can click on the trigger controls corresponding to two or more second electronic devices. These second electronic devices can be connected to the same first electronic device or to multiple different first electronic devices. The third electronic device generates a corresponding control start command for each selected second electronic device and sends these commands to the first electronic devices to which they are connected. The first electronic device receiving the control start command sends the target display data from each second electronic device to the third electronic device. Upon receiving the control data corresponding to that second electronic device from the third electronic device, the first electronic device generates a corresponding input event and sends it to the corresponding second electronic device. The second electronic device then executes the operation corresponding to the input event, achieving remote control of multiple devices.

[0107] For example, such as Figure 4As shown in the diagram, the control interface of the cloud server displays trigger controls for each of the four employee devices (i.e., second electronic devices) connected to the two IoT docks 1 and 2, such as "Start Remote Control," for employee devices 1, 2, 3, and 4. The cloud server's operations administrator selects the trigger control for one or more employee devices to indicate that the remote control function is enabled. The cloud server then sends a control start command to the corresponding IoT dock. The IoT dock directly starts the remote control function according to the received control start command. For example, IoT dock 1 responds to the control start command for employee device 1 by sending the desktop screen data stream of employee device 1 to the cloud server, and IoT dock 2 responds to the control start command for employee device 3... The control start command sends the desktop screen data stream of employee device 3 to the cloud server. As a result, the desktops of employee devices 1 and 3 can be displayed synchronously on the cloud server. The operation and maintenance administrator can interact with the desktops displayed on the cloud server and send corresponding control data to the corresponding IoT expansion docks 1 and 2. IoT expansion dock 1 generates corresponding HID events based on the control data of employee device 1 and sends them to employee device 1. Employee device 1 executes the operation corresponding to the HID event. IoT expansion dock 2 generates corresponding HID events based on the control data of employee device 3 and sends them to employee device 3. Employee device 3 executes the operation corresponding to the HID event, thereby realizing remote control of multiple devices.

[0108] In another approach, in this embodiment, the operating state of the target control function of the first electronic device can be determined to be in the "started" state in response to the user's triggering operation on the first electronic device.

[0109] The trigger operation is used to activate the target control function. Specifically, the trigger operation can be a user's action on a corresponding physical button in the first electronic device. For example, the trigger operation can be a single-click trigger operation, a long-press trigger operation, or a combination trigger operation of multiple buttons.

[0110] For example, such as Figure 5As shown, after receiving a message about a malfunction in employee device 4, the operations and maintenance administrator presses a button on the IoT docking station 2 that is dedicated to the remote control function of employee device 4, such as "Remote Control". While the button indicator light remains on, the IoT docking station 2 confirms that the remote control function corresponding to employee device 4 has been activated. The IoT docking station 2 sends the desktop screen data stream of employee device 4 to the cloud server. As a result, the desktop of employee device 4's laptop can be displayed synchronously on the cloud server. The operations and maintenance administrator can perform interactive operations on the desktop displayed on the cloud server and send corresponding control data to the corresponding IoT docking station 2. The IoT docking station 2 generates a corresponding HID event based on the control data of employee device 4 and sends it to employee device 4. Employee device 4 executes the operation corresponding to the HID event, thereby realizing remote control.

[0111] As can be seen, the two methods of enabling the target control function in this embodiment can be simultaneously compatible, serve as backups for each other, adapt to different application scenarios such as cloud-based remote operation and maintenance, and on-site manual activation, further improving the user experience.

[0112] Based on the above implementation scheme, this embodiment can also add a multi-layered implementation method for external storage device identification, data reading, and data backup. Specifically, in this embodiment, the operating status of the target transmission function of the first electronic device can be monitored. If the operating status of the target transmission function of the first electronic device is in the activated state, then the target data in the storage device can be obtained and the target data can be stored in the third electronic device.

[0113] The storage device is connected to the first electronic device. The target transfer function refers to the function of uploading target data from the storage device connected to the first electronic device to the third electronic device for backup. The target transfer function can be triggered by the user on the third electronic device, or it can be triggered by the user on the first electronic device, or it can be automatically triggered when the first electronic device is connected to the storage device.

[0114] In one implementation, the storage device can be an external, independent storage device connected to the first electronic device, such as a USB flash drive, portable hard drive, solid-state drive, memory card reader, or other USB peripheral. The storage device is connected to the first electronic device via a port in the first electronic device's USB hub.

[0115] For example, when a portable hard drive is plugged into the USB expansion port of an IoT docking station, the IoT docking station detects that the cloud backup function is enabled, identifies the drive letter and file target of the portable hard drive, reads the target data such as office documents, business materials, photos, and videos stored on it, and then packages the target data and uploads it to the cloud server for backup.

[0116] In another implementation, the storage device can be a built-in storage device in a second electronic device with computing capabilities connected to the first electronic device, such as a built-in hard drive in a laptop, a solid-state drive in a laptop, or a mechanical hard drive in a laptop.

[0117] For example, a laptop connects to an IoT docking station via a Type-C interface. When the IoT docking station detects that the cloud backup function is enabled, it can access local file data in the laptop's built-in hard drive, such as target data in the desktop directory and the system disk working directory. After reading this target data, it sends it to the cloud server through the IoT communication module to complete the remote cloud backup of the laptop's local data.

[0118] Optionally, the first electronic device supports individual backup, time-sharing backup, or batch backup of the above two types of storage devices. Understandably, the first electronic device also converts control data into input events that are recognized by the second electronic device as input devices based on target transmission rules. This allows it to be compatible with both data acquisition from external mobile storage devices and data reading from the built-in storage of computing storage devices (laptops with computing capabilities), eliminating the need for manual copying and improving the convenience of data maintenance and disaster recovery.

[0119] Specifically, a processing chip, such as a System-on-a-Chip (SoC), can be deployed in the first electronic device to monitor the target transmission function, read the target data, and upload it. For example, when the SoC deployed in the IoT dock detects that the backup function has been activated, it sends a data backup request to the connected laptop. The application deployed on the laptop can respond to the data backup request, collect the target data from the laptop's internal hard drive, and send it to the SoC in the IoT dock. The SoC in the IoT dock then uploads this target data to a cloud server for storage, thus achieving data backup for the laptop.

[0120] It should be noted that the target data can be all the data in the storage device, or it can be incremental data of the data already stored in the corresponding third electronic device, such as newly created files or modified files, in order to save on backup processes and storage space.

[0121] In addition, the first electronic device can acquire the target data in the storage device according to the time period and upload it to the third electronic device.

[0122] The time period can be set according to business needs. For example, the IoT docking station can perform a process of reading and uploading target data once a day at 2:00 AM to achieve regular data backup.

[0123] As can be seen, in this embodiment, the first electronic device, as a unified relay carrier, can simultaneously realize remote reading and remote backup of local electronic devices and external storage devices, which can reduce the threshold and adaptation cost of operation and maintenance of local electronic devices.

[0124] refer to Figure 6 This is a flowchart illustrating the implementation of a device control method provided in this application. This method can be applied to a third electronic device, such as a cloud server or other display devices connected to a first electronic device, such as a local maintenance server. The third electronic device establishes a connection with the second electronic device through the first electronic device, such as... Figure 2 As shown in the figure. Specifically, the method in this embodiment may include the following steps:

[0125] Step 601: Obtain the target display data obtained by the first electronic device from the second electronic device.

[0126] In this process, the first electronic device receives raw display data from the second electronic device and processes the raw display data in terms of data format and data volume to obtain target display data. Then, when the target control function is activated, the first electronic device sends the target display data to the third electronic device through the IoT communication module. Based on this, the third electronic device can receive the target display data sent by the first electronic device through the corresponding communication link.

[0127] In one implementation, the third electronic device can actively retrieve the target display data. For example, after logging into the cloud management and control platform (i.e., the application deployed on the cloud server), the operations and maintenance administrator actively selects the laptop connected to the IoT docking station to be maintained. The cloud management and control platform sends a control start command to the IoT docking station, which then sends the screen data stream from the specified laptop to the cloud management and control platform.

[0128] In another implementation, the first electronic device can passively push the target display data corresponding to the second electronic device to the third electronic device. For example, after the user triggers a physical button to enable the remote control function, the IoT docking station automatically and continuously pushes the screen data stream of the connected laptop to the bound cloud management and control platform.

[0129] Step 602: Output the target display data.

[0130] The third electronic device can decode and render the received target display data and output it visually in the display interface.

[0131] It is evident that the third electronic device does not directly network and communicate with the second electronic device, but instead uses the first electronic device as a hardware relay gateway to build a cross-network remote control link.

[0132] In one implementation, the third electronic device can output the target display data in full-screen mode. For example, in an operations and maintenance scenario, when remote troubleshooting is required, displaying the laptop desktop in full screen allows the operations and maintenance administrator to fully observe the laptop's system interface and error pop-ups.

[0133] In another implementation, the third electronic device can output the target display data through split-screen mode or windowed mode. For example, in an operations and maintenance scenario, an operations and maintenance administrator can operate multiple employee devices simultaneously and view the real-time images of multiple devices side by side through multiple windows.

[0134] Step 603: In response to the user's interactive operation on the target display data output in the third electronic device, generate control data.

[0135] It should be noted that the implementation method of the third electronic device generating control data in step 603 can refer to the corresponding content above, and will not be described in detail here.

[0136] Step 604: Send control data to the first electronic device.

[0137] The control data is used to trigger the first electronic device to generate a corresponding input event and send the input event to the second electronic device.

[0138] It should be noted that the implementation method of the first electronic device generating input events and sending them to the second electronic device can be referred to the corresponding content above, and will not be elaborated here.

[0139] As can be seen from the above technical solutions, in the device control method provided in this application embodiment, while achieving display synchronization in the second electronic device and the third electronic device, an input event can be generated on the first device based on the control data generated on the third electronic device and sent to the second electronic device, thereby realizing remote control from the third electronic device to the second electronic device. It is evident that this embodiment relies on the bidirectional transparent transmission of display data and control data in the first electronic device, eliminating the need to install dedicated software on the second electronic device. This allows for display synchronization and remote control between the first and third electronic devices, simplifying remote maintenance processes and improving equipment fault diagnosis efficiency in operation and maintenance scenarios, thereby enhancing the user experience of electronic devices.

[0140] refer to Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device serves as... Figure 2The first electronic device in the process may include the following structure:

[0141] The communication module 701 is used to acquire target display data of the second electronic device and send the target display data to the third electronic device; the first electronic device is connected to the second electronic device; and receives control data sent by the third electronic device; the control data is generated in response to the user's interactive operation on the target display data output in the third electronic device;

[0142] The processor 702, i.e. the processing chip mentioned above, is used to generate corresponding input events based on the control data and send the input events to the second electronic device through the communication module 701.

[0143] Specifically, the communication module 701 can be divided into a sub-module 711 (such as an MST Hub and a USB Hub) for interacting with a second electronic device and a sub-module 712 (such as an IoT communication module) for interacting with a third electronic device. The sub-module 711 can be used to acquire target display data of the second electronic device and send the input event to the second electronic device. The sub-module 712 can be used to send the target display data to the third electronic device and receive control data sent by the third electronic device.

[0144] It should be noted that the communication module 701 can be implemented in hardware. Specifically, the sub-modules 711 and 712 in the communication module 701 are implemented through hardware communication chips.

[0145] As can be seen from the above technical solutions, in the first electronic device provided in this application embodiment, while display synchronization is achieved in the second and third electronic devices, the first electronic device can generate input events based on control data generated on the third electronic device and send them to the second electronic device, thereby realizing remote control from the third electronic device to the second electronic device. It is evident that this embodiment relies on the bidirectional transparent transmission of display and control data in the first electronic device, eliminating the need to install dedicated software on the second electronic device. This allows for display synchronization and remote control between the first and third electronic devices, simplifying remote maintenance processes and improving equipment fault diagnosis efficiency in operation and maintenance scenarios, thus enhancing the user experience of the electronic devices.

[0146] It should be noted that the specific implementation of the communication module 701 and the processor 702 in this embodiment can be referred to the corresponding content above, and will not be described in detail here.

[0147] refer to Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device serves as... Figure 2The third electronic device in the system may include the following structures:

[0148] The communication module 801 is used to obtain target display data obtained by the first electronic device from the second electronic device, such as wireless communication module or wired communication module; it should be noted that the communication module 801 can be implemented in hardware structure.

[0149] Display 802 is used to output the target display data; the third electronic device establishes a connection with the second electronic device through the first electronic device;

[0150] The processor 803 is configured to generate control data in response to an interactive operation by a user on the target display data output in the third electronic device, and send the control data to the first electronic device via the communication module. The control data is used to trigger the first electronic device to generate a corresponding input event and send the input event to the second electronic device.

[0151] As can be seen from the above technical solutions, in the third electronic device provided in this application embodiment, while display synchronization is achieved between the second and third electronic devices, input events can be generated on the first device based on control data generated on the third electronic device and sent to the second electronic device, thereby realizing remote control from the third electronic device to the second electronic device. It is evident that this embodiment relies on the bidirectional transparent transmission of display and control data within the first electronic device, eliminating the need to install dedicated software on the second electronic device. This allows for display synchronization and remote control between the first and third electronic devices, simplifying remote maintenance processes and improving equipment fault diagnosis efficiency in operation and maintenance scenarios, ultimately enhancing the user experience of the electronic devices.

[0152] by Figure 9 Taking the scenario shown as an example, the technical solution of this application is illustrated below:

[0153] First, in this application, the IoT docking station uses its processing chip to present the acquired display signal (i.e., target display data) from the laptop to the maintenance engineer via a cloud server. The maintenance engineer then performs operations on the cloud server. After the control data generated by the cloud server is transmitted to the IoT docking station, the IoT docking station virtualizes the control data into HID events and sends them to the laptop.

[0154] The specific plan is as follows:

[0155] 1. When the remote control function of the IoT docking station is activated, a processing chip inside the IoT docking station starts up. It captures the computer's display signal obtained by MST Hub, processes and compresses it into a data format suitable for network transmission, and sends it to the cloud server.

[0156] 2. The operations and maintenance engineer can view the real-time computer screen by opening the management and control platform on the cloud server.

[0157] 3. The operations engineer opens the remote control interface and operates the employee's computer screen using their mouse and keyboard. The management control platform generates control data from these operations and sends it to the IoT docking station.

[0158] 4. After receiving this control data, the processing chip in the IoT dock converts it into HID events to virtualize the IoT dock as a USB device (the processing chip is physically connected to the USB hub). The HID events are then sent to the USB hub. The employee's computer can then receive these HID events and execute corresponding operations, thereby achieving remote control.

[0159] As can be seen, the IoT docking station has the ability to acquire computer display signals and can also directly establish a USB connection with the computer. Based on this, this application can realize the function of remote control by leveraging these two basic capabilities of the IoT docking station.

[0160] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0161] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0162] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0163] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A device control method applied to a first electronic device, the method comprising: Acquire target display data from the second electronic device; The target display data is sent to a third electronic device; The first electronic device is connected to the second electronic device; Acquire control data sent by the third electronic device; The control data is generated in response to the user's interactive operation on the target display data output in the third electronic device; Based on the control data, a corresponding input event is generated, and the input event is sent to the second electronic device.

2. The method according to claim 1, wherein generating a corresponding input event based on the control data includes: Based on the target transmission rules between the first electronic device and the second electronic device, the control data is converted into corresponding input events; The input event enables the second electronic device to recognize the first electronic device as an input device based on the target transmission rule.

3. The method according to claim 1 or 2, acquiring target display data of the second electronic device, comprising: Obtain the raw display data of the second electronic device; Based on the data specifications supported by the third electronic device, the original display data is processed to obtain the target display data.

4. The method according to claim 3, wherein the original display data is processed based on the data specifications supported by the third electronic device to obtain target display data, including at least one of the following: Based on the channel type of the data transmission channel between the first electronic device and the third electronic device, the data format of the original display data is processed to obtain the target display data, wherein the data format of the target display data matches the channel type; Based on the channel bandwidth of the data transmission channel, the amount of data in the original display data is processed, and the amount of data in the target display data is matched with the channel bandwidth.

5. The method according to claim 1, further comprising: Determine the operating status of the target control function of the first electronic device; If the target control function of the first electronic device is in the activated state, the target display data is sent to the third electronic device.

6. The method according to claim 5, determining the operating state of the target control function of the first electronic device, includes at least one of the following: In response to a control start command from the third electronic device, the operating state of the target control function of the first electronic device is determined to be the start state; In response to the user's trigger operation on the first electronic device, the operating state of the target control function of the first electronic device is determined to be in the activated state, and the trigger operation is used to trigger the target control function.

7. The method according to claim 1, further comprising: If the target transmission function of the first electronic device is in the activated state, the target data in the storage device is obtained, and the storage device is connected to the first electronic device. The target data is stored in the third electronic device.

8. A device control method applied to a third electronic device, the method comprising: Obtain the target display data that the first electronic device receives from the second electronic device; Output the target display data; The third electronic device establishes a connection with the second electronic device through the first electronic device; Control data is generated in response to an interactive operation by a user on the target display data output in the third electronic device; The control data is sent to the first electronic device, and the control data is used to trigger the first electronic device to generate a corresponding input event and send the input event to the second electronic device.

9. An electronic device, wherein the electronic device, as a first electronic device, comprises: The communication module is used to acquire target display data from the second electronic device and send the target display data to the third electronic device; The first electronic device is connected to the second electronic device and receives control data sent by the third electronic device; the control data is generated in response to the user's interactive operation on the target display data output in the third electronic device; The processor is configured to generate corresponding input events based on the control data, and send the input events to the second electronic device via the communication module.

10. An electronic device, wherein the electronic device, as a third electronic device, comprises: The communication module is used to obtain target display data from the second electronic device obtained by the first electronic device; A display, used to output the target display data; The third electronic device establishes a connection with the second electronic device through the first electronic device; The processor is configured to generate control data in response to an interactive operation by a user on the target display data output in the third electronic device, and to send the control data to the first electronic device via the communication module. The control data is used to trigger the first electronic device to generate a corresponding input event and send the input event to the second electronic device.

Images