Remote control method, remote control device, program product, and electronic device

Abstract

The invention provides a remote control method, a remote control device, a program product and electronic equipment, and relates to the technical field of computers. The method comprises the following steps: a controlled device generates a virtual display in response to a first preset instruction; wherein the first preset instruction is an instruction triggered under the condition that a first display parameter is not matched with a second display parameter of the controlled equipment, or an instruction triggered based on a preset function starting operation on the control equipment, and the first display parameter is a display parameter set for a remote control display area on the control equipment; setting a display parameter of the virtual display as a first display parameter; generating first display data of the virtual display based on the first display parameter; sending the first display data to the control equipment, so that the control equipment displays a picture corresponding to the first display data in the remote control display area by adopting the first display parameter; and receiving and executing a control instruction sent by the control equipment. The remote control display effect can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to remote control methods, remote control devices, program products, and electronic devices. Background Technology

[0002] Remote control refers to the technology that allows users to operate and control one device (the controlled device) on one device (the controlling device) via a network. It enables cross-regional device control and management, bringing convenience to scenarios such as remote work, technical support, operation and maintenance, and education.

[0003] In related technologies, due to the limitations of the display performance of the controlled device itself, when displaying remote control screens on the control device, it may be difficult to set the display parameters of the screen according to the display performance of the control device, which may lead to problems such as inappropriate display parameters, affecting the display effect and reducing the comfort and efficiency of users performing remote operations. Summary of the Invention

[0004] This disclosure provides remote control methods, remote control devices, program products, and electronic devices to at least partially solve the problem that the display effect of remote control is limited by the display performance of the controlled device.

[0005] According to a first aspect of this disclosure, a remote control method is provided, the method comprising: a controlled device generating a virtual display in response to a first preset instruction; wherein the first preset instruction is an instruction triggered when a first display parameter and a second display parameter of the controlled device do not match, or an instruction triggered based on a preset function activation operation on a control device, the first display parameter being a display parameter set for a remote control display area on the control device; setting the display parameter of the virtual display to the first display parameter; generating first display data of the virtual display based on the first display parameter; sending the first display data to the control device, so that the control device displays the image corresponding to the first display data in the remote control display area using the first display parameter; and receiving and executing a control instruction sent by the control device.

[0006] According to a second aspect of this disclosure, a remote control method is provided, the method comprising: a control device sending a first preset instruction to a controlled device in response to a preset function activation operation; or sending the first preset instruction to the controlled device when a first display parameter and a second display parameter of the controlled device do not match; or sending the first display parameter to the controlled device to cause the controlled device to generate the first preset instruction when the first display parameter and the second display parameter do not match; the first display parameter is a display parameter set for a remote control display area on the control device; the first preset instruction is used to trigger the controlled device to generate a virtual display, the virtual display being configured to use the first display parameter; receiving first display data of the virtual display sent by the controlled device, and displaying a screen corresponding to the first display data in the remote control display area using the first display parameter; and sending a corresponding control instruction to the controlled device in response to a control operation on the remote control display area to control the controlled device.

[0007] According to a third aspect of this disclosure, a remote control device is provided, the device comprising: a virtual display generation module configured to generate a virtual display in response to a first preset instruction; wherein the first preset instruction is an instruction triggered when a first display parameter and a second display parameter of the controlled device do not match, or an instruction triggered based on a preset function activation operation on the control device, and the first display parameter is a display parameter set for a remote control display area on the control device; a virtual display setting module configured to set the display parameter of the virtual display to the first display parameter; a display data generation module configured to generate first display data of the virtual display based on the first display parameter; a display data sending module configured to send the first display data to the control device, so that the control device displays the image corresponding to the first display data in the remote control display area using the first display parameter; and a remote controlled module configured to receive and execute control instructions sent by the control device.

[0008] According to a fourth aspect of this disclosure, a remote control device is provided, the device comprising: a display setting interaction module configured to send a first preset instruction to a controlled device in response to a preset function activation operation; or to send the first preset instruction to the controlled device when a first display parameter and a second display parameter of the controlled device do not match; or to send the first display parameter to the controlled device so that the controlled device generates the first preset instruction when the first display parameter and the second display parameter do not match; the first display parameter is a display parameter set for a remote control display area on the control device; the first preset instruction is used to trigger the controlled device to generate a virtual display, the virtual display being configured to use the first display parameter; a display data receiving module configured to receive first display data of the virtual display sent by the controlled device, and display the image corresponding to the first display data in the remote control display area using the first display parameter; and a remote control module configured to send a corresponding control instruction to the controlled device in response to a control operation on the remote control display area to control the controlled device.

[0009] According to a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method of the first or second aspect described above and its possible implementations.

[0010] According to a sixth aspect of this disclosure, an electronic device is provided, comprising: a processor and a memory, the memory being used to store executable instructions of the processor; wherein the processor is configured to perform the methods of the first or second aspect described above and possible implementations thereof by executing the executable instructions.

[0011] The technical solution disclosed herein has the following beneficial effects:

[0012] When the first display parameter and the second display parameter do not match, a first preset command is triggered, or a first preset command is triggered based on a preset function activation operation on the control device. This causes the controlled device to create a virtual display and set its display parameters to the first display parameter. During remote control, first display data matching the first display parameter is generated, enabling the control device to display the image corresponding to the first display data in the remote control display area using the first display parameter. Even if the controlled device itself does not support the first display parameter, it can still display the image in the remote control display area according to the first display parameter through the virtual display. This overcomes the limitations of the controlled device's display performance, improves the remote control display effect, and enhances the comfort and efficiency of remote operation for users. Attached Figure Description

[0013] Figure 1A schematic diagram of a system architecture according to an embodiment of this disclosure is shown;

[0014] Figure 2 A flowchart illustrating a remote control method according to an embodiment of this disclosure is shown;

[0015] Figure 3 A flowchart illustrating another remote control method according to an embodiment of this disclosure is shown;

[0016] Figure 4 This diagram illustrates a flowchart of sending a first preset command according to an embodiment of the present disclosure;

[0017] Figure 5 This diagram illustrates a flowchart of receiving second display data and displaying a corresponding screen according to an embodiment of the present disclosure;

[0018] Figure 6 A schematic diagram of the structure of a remote control device according to an embodiment of the present disclosure is shown;

[0019] Figure 7 A schematic diagram of another remote control device according to an embodiment of this disclosure is shown;

[0020] Figure 8 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0021] Exemplary embodiments of this disclosure will be described more fully below with reference to the accompanying drawings.

[0022] The accompanying drawings are schematic illustrations of this disclosure and are not necessarily drawn to scale. Some block diagrams shown in the drawings may be functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in hardware modules or integrated circuits, or in networks, processors, or microcontrollers. Implementations can be carried out in various forms and should not be construed as limited to the examples set forth herein. The features, structures, or characteristics described in this disclosure can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough description of embodiments of this disclosure. However, those skilled in the art will recognize that one or more specific details may be omitted when implementing the technical solutions of this disclosure, or other methods, components, apparatuses, steps, etc., may be used to replace one or more specific details.

[0023] In related technologies, due to the limitations of the controlled device's display performance, it may be difficult to set the display parameters of the screen according to the display performance of the control device when displaying remote control images on the control device, leading to problems such as inappropriate display parameters. For example, if the controlled device does not support the control device's resolution, the remote control image cannot be displayed using the control device's resolution. Typically, the controlled device's resolution is used, which may be too low relative to the control device's monitor, affecting the display effect. This, in turn, reduces the user's comfort and efficiency in remote operation.

[0024] In view of this, the present disclosure provides a remote control method to solve the problem that the display effect of remote control is limited by the display performance of the controlled device.

[0025] Figure 1 A system architecture diagram of an embodiment of this disclosure is shown. The system architecture includes a controlled device 110 and a control device 120. Both the controlled device 110 and the control device 120 can be electronic devices such as personal computers, mobile phones, tablets, wearable devices, and game consoles. The controlled device 110 and the control device 120 are connected via a wired or wireless network for information exchange and transmission. The controlled device 110 transmits its display data (such as image or video data generated after a screenshot) to the control device 120, allowing the user to see the screen of the controlled device 110 on the control device 120 and perform control operations based on the screen to generate control commands for the controlled device 110. The control device 120 transmits control commands and other related data to the controlled device 110 to execute the control commands on the controlled device 110, thereby realizing remote control functionality.

[0026] In one embodiment, the system architecture may further include a server for providing remote control services to the controlled device 110 and the control device 120. Specifically, display data from the controlled device 110 can be transmitted to the control device 120 via the server. Control commands from the control device 120 can be transmitted to the controlled device 110 via the server. Furthermore, the server can also process the data. For example, the controlled device 110 transmits initial display data to the server, which processes it into data suitable for display on the control device 120 and sends it to the control device 120. The control device 120 transmits control operation-related data or user setting data to the server, which generates control commands for the controlled device 110 based on this data and sends the control commands to the controlled device 110.

[0027] Remote control software can be installed on both the controlled device 110 and the control device 120. For example, the controlled device 110 can have the controlled end program of the remote control software installed, and the control device 120 can have the control end program of the remote control software installed. Alternatively, the controlled device 110 and the control device 120 can have the client program of the remote control software installed, which can function as both the controlled end program and the control end program. In this way, under certain circumstances, the controlled device 110 and the control device 120 can switch roles, meaning that a user can remotely control the control device 120 through the controlled device 110.

[0028] The remote control method of this disclosure will be described below from the aspects of the controlled device and the control device.

[0029] Figure 2 An exemplary flow of a remote control method performed by a controlled device is shown, including the following steps:

[0030] In step S210, the controlled device responds to the first preset instruction and generates a virtual display; wherein, the first preset instruction is an instruction triggered when the first display parameters and the second display parameters of the controlled device do not match, or an instruction triggered based on a preset function activation operation on the control device, and the first display parameters are display parameters set for the remote control display area on the control device;

[0031] Step S220: Set the display parameters of the virtual display to the first display parameter;

[0032] Step S230: Generate first display data for the virtual display based on the first display parameters;

[0033] Step S240: Send first display data to the control device so that the control device displays the screen corresponding to the first display data in the remote control display area using the first display parameters;

[0034] Step S250: Receive and execute control commands sent by the control device.

[0035] based on Figure 2The method involves triggering a first preset command when the first display parameter and the second display parameter do not match, or triggering the first preset command based on a preset function activation operation on the control device. This causes the controlled device to create a virtual display and set its display parameters to the first display parameter. During remote control, first display data matching the first display parameter is generated, enabling the control device to display the image corresponding to the first display data in the remote control display area using the first display parameter. Even if the controlled device itself does not support the first display parameter, it can still display the image in the remote control display area according to the first display parameter through the virtual display. This overcomes the limitations of the controlled device's display performance, improves the remote control display effect, and enhances the comfort and efficiency of remote operation for users.

[0036] The following is about Figure 2 Each step is explained in detail.

[0037] In step S210, the controlled device responds to the first preset instruction to generate a virtual display; wherein, the first preset instruction is an instruction triggered when the first display parameter and the second display parameter of the controlled device do not match, or an instruction triggered based on a preset function activation operation on the control device, and the first display parameter is a display parameter set for the remote control display area on the control device.

[0038] The remote control display area is the area used to display the remote control screen. For example, if a user sets the remote control screen to full-screen display on the control device, the remote control display area is the entire display area of ​​the control device's physical monitor. Alternatively, if a user sets the remote control screen to be displayed in a window, the remote control display area is the window area.

[0039] Display parameters are those that affect the display effect, such as resolution, color mode, contrast ratio, and refresh rate. The primary display parameter is the display parameter set for the remote control display area, i.e., the display parameters of the remote control screen. For example, a user can set parameters such as resolution, color mode, contrast ratio, and refresh rate of the remote control screen in the remote control software, and use these user-set parameters as the primary display parameter.

[0040] In one implementation, in response to a remote display matching operation on the control device, the current display parameters of the control device are used as the first display parameters. The remote display matching operation is used to set the display parameters of the remote control screen to be consistent with the current display parameters of the control device. For example, the remote control software provides a "follow local resolution" setting; when the user selects this setting, it constitutes a remote display matching operation, triggering the setting of the current display parameters of the control device as the first display parameter. This ensures that the remote control screen has the same display parameters as the original screen of the control device, thus adapting the remote control screen to the display performance of the control device and achieving a better display effect.

[0041] The second display parameter can be a display parameter supported by the controlled device. For example, if the physical display of the controlled device supports a maximum resolution of 1080P (1920×1080), then the second display parameter includes resolutions of 1920×1080 and below. Alternatively, the second display parameter can be the current display parameter of the controlled device. For example, if the physical display of the controlled device currently uses a resolution of 1920×1080, then the second display parameter includes a resolution of 1920×1080.

[0042] In this embodiment of the disclosure, matching of the first display parameter and the second display parameter means that the first display parameter and the second display parameter are the same, or that the second display parameter includes the first display parameter. For example, the second display parameter is a display parameter supported by the controlled device. If it does not include the first display parameter, then the two are not matched, which means that the physical display of the controlled device does not support the first display parameter.

[0043] In one embodiment, the control device can acquire the second display parameters of the controlled device and determine whether the first display parameters and the second display parameters match. If they do not match, the control device triggers the generation of a first preset instruction and sends the first preset instruction to the controlled device.

[0044] In one implementation, the control device can send a first display parameter to the controlled device. The controlled device determines whether the first display parameter and the second display parameter match. If they do not match, it triggers the generation of a first preset instruction.

[0045] In one implementation, the server can determine whether the first display parameter and the second display parameter match. If they do not match, the server can trigger the generation of a first preset instruction and send the first preset instruction to the controlled device.

[0046] The above explains how to trigger the first preset command when the first display parameter and the second display parameter do not match. In addition, the first preset command can also be triggered based on a preset function activation operation on the control device. A preset function refers to the function of remote control using a virtual display (or a specific virtual display, such as one supporting various common resolutions). Users can activate this function on the control device to trigger the first preset command. For example, if the remote control software provides a "Use Virtual Screen" setting, the user's selection of this setting is a preset function activation operation, triggering the generation of the first preset command. The control device can then send the first preset command triggered based on the preset function activation operation to the controlled device. Alternatively, the control device can send the relevant data for the preset function activation operation to a server, which will then generate the first preset command and send it to the controlled device.

[0047] The first preset command is an instruction to the controlled device to activate the virtual display. In response to the first preset command, the controlled device generates a virtual display, and subsequently processes and transmits relevant display data for remote control based on the virtual display.

[0048] In one implementation, the above-described generation of a virtual display includes the following steps:

[0049] A virtual monitor is created by installing a virtual display driver on the controlled device.

[0050] The virtual display driver can simulate the behavior of a physical display, generate virtual display data (such as display parameters supported by the virtual display), and report it to the system, so that the system believes that a real display exists and interacts with the display, thereby realizing the creation of a virtual display.

[0051] In one implementation, the controlled device installs a virtual display driver in response to initiating a remote control process. The remote control process, used to establish a remote control connection between the controlled device and the controlling device, can be initiated when the controlled device is running remote control software or when the controlling device requests control of the controlled device. The remote control software may contain code for the virtual display driver; the controlled device executes this code in response to initiating the remote control process to install the virtual display driver. Furthermore, during operation, the remote control process can monitor the status of the virtual display driver; if an abnormality is detected, the virtual display driver can be reinstalled.

[0052] Based on virtual display drivers, virtual displays can be created and used without connecting to or using a physical display.

[0053] Continue to refer to Figure 2In step S220, the display parameters of the virtual display are set to the first display parameters.

[0054] Virtual displays are not limited by the display performance of the physical display of the controlled device and can support various common display parameters. In one embodiment, when the controlled device creates a virtual display through a virtual display driver, it can specify the display parameters it supports, such as: display parameters supported by the physical display of the controlling device or the current display parameters; display parameters supported by the physical display of the controlled device; and preset common display parameters, such as resolutions including 1600*900, 1920*1080, 1920*1200, 2560*1440, and 3840*2160. Therefore, the virtual display can support a first display parameter, and setting the display parameters of the virtual display to the first display parameter conforms to the user's settings for the remotely controlled display area.

[0055] For example, the physical display of the control device currently has a resolution of 2560*1440. When a user performs a remote display matching operation, it means setting the resolution of the remotely controlled display area to be the same as the physical display of the control device. However, the physical display of the controlled device only supports a maximum resolution of 1920*1080 and does not support 2560*1440. That is, the first display parameter and the second display parameter do not match, triggering the controlled device to create a virtual display. This virtual display supports a resolution of 2560*1440. Setting it to 2560*1440 resolution overcomes the display performance limitations of the physical display of the controlled device.

[0056] Continue to refer to Figure 2 In step S230, first display data for the virtual display is generated based on the first display parameters.

[0057] The controlled device renders data that can be displayed in the virtual display based on the first display parameters; this data is called the first display data.

[0058] In one implementation, when the controlled device generates a virtual display, it sets the virtual display as the sole display. This means that other displays (including physical displays and other virtual displays) can be turned off, allowing the virtual display to acquire all or part of the current interface data of the controlled device. For example, when the virtual display is set as the sole display, the content of all the original displays can be aggregated onto the virtual display, allowing the virtual display to acquire all current interface data. Alternatively, when the virtual display is set as the sole display, the content originally displayed on other displays can be turned off, leaving only the content originally displayed on the virtual display, allowing the virtual display to acquire a portion of the current interface data.

[0059] In one implementation, the above-mentioned generation of first display data for a virtual display based on first display parameters includes the following steps:

[0060] The interface data is rendered based on the first display parameters to generate the first display data.

[0061] The controlled device can execute a graphics rendering pipeline, such as a GPU (Graphics Processing Unit), to render interface data. During rendering, pixel information in screen space is determined based on first display parameters. The resulting first display data then matches these first display parameters. For example, the first display parameters might include a first resolution, and the first display data might be image or video data based on that first resolution.

[0062] Continue to refer to Figure 2 In step S240, first display data is sent to the control device so that the control device displays the screen corresponding to the first display data in the remote control display area using the first display parameters.

[0063] The controlled device sends the first display data of the virtual display to the control device. Since the first display data matches the first display parameters, the control device can directly use the first display parameters to display the screen corresponding to the first display data in the remote control display area. This meets the user's needs, reduces the processing of display data, and improves efficiency.

[0064] In one implementation, if the virtual display of the controlled device is the only display, the first display data may include display data corresponding to all current interface data of the controlled device. The control device displays the corresponding screen based on the first display data, and this screen can aggregate the display content of all the original displays on the controlled device. In this way, the user can see all the interface content of the controlled device through the remote control display area, facilitating remote control.

[0065] Furthermore, if a mismatch is detected between the first and second display parameters, a first preset command can be automatically triggered, thereby creating and using a virtual display to provide remote control display functionality, simplifying the user's operation process. Moreover, the virtual display process can be made seamless for the user, reducing the learning and understanding costs and improving the practicality of the solution.

[0066] Continue to refer to Figure 2 In step S250, control commands sent by the control device are received and executed.

[0067] When a user performs a control operation on the control device, the control device or server can be triggered to send a corresponding control command to the controlled device. The controlled device receives and executes the control command to realize the remote control function.

[0068] In one implementation, the remote control method further includes the following steps:

[0069] The controlled device responds to the second preset command and shuts down the virtual display.

[0070] The second preset command is triggered by a preset function shutdown operation on the control device and is used to instruct the virtual display to be turned off. For example, the remote control software on the control device provides a "Use Virtual Screen" setting, which is currently checked, and a virtual display has been created and is being used on the controlled device. If the user unchecks this setting, this is a preset function shutdown operation, triggering the second preset command. The control device can send the second preset command triggered by the preset function shutdown operation to the controlled device. Alternatively, the control device can send the relevant data of the preset function shutdown operation to the server, which will then generate the second preset command and send it to the controlled device.

[0071] Upon receiving the second preset command, the controlled device shuts down the virtual display. Specifically, this can be done by uninstalling the virtual display driver, such as removing the loaded virtual display data. The controlled device can then revert to using its original display (such as a physical display) to provide display functionality.

[0072] In one implementation, a second preset command is triggered if the fourth display parameter matches the second display parameter. The fourth display parameter is a display parameter reset for the remote control display area when the virtual display is enabled, and it may differ from the first display parameter. If the fourth display parameter matches the second display parameter, it indicates that the physical display of the controlled device supports or is suitable for the fourth display parameter. In this case, the second preset command is triggered to disable the virtual display and use the physical display to provide display functionality.

[0073] In one implementation, the remote control method further includes the following steps:

[0074] After the controlled device turns off the virtual display, it generates second display data based on the third display parameters and sends the second display data to the control device, so that the control device can use the third display parameters to display the screen corresponding to the second display data in the remote control display area.

[0075] The third display parameter is the display parameter used by the controlled device before generating the virtual display; it can be the second display parameter. After the virtual display is turned off, the controlled device reverts to using the original third display parameter. In the graphics rendering pipeline, the pixel information of the screen space is determined based on the third display parameter, and then image or video data is generated as the second display data. The second display data matches the third display parameter, so the control device can use the third display parameter to display the image corresponding to the second display data in the remote control display area.

[0076] In one implementation, when creating a virtual display, the controlled device first saves the current display layout information by calling system screen management APIs (Application Programming Interfaces), such as GetDisplayConfigBufferSizes and QueryDisplayConfig. Next, it notifies the virtual display driver (such as a Virtual Display Adapter) to create the virtual display and then calls the system screen management APIs to set the virtual display as the sole display. When closing the virtual display, the controlled device notifies the virtual display driver to unload the created virtual display and calls the system screen management APIs to restore the original screen display layout based on the saved display layout information. This enables quick and flexible switching between using and not using a virtual display.

[0077] Figure 3 An exemplary flow of a remote control method performed by a control device is shown, including the following steps S310 to S330:

[0078] In step S310, the control device responds to the preset function activation operation by sending a first preset command to the controlled device; or, if the first display parameter and the second display parameter of the controlled device do not match, it sends a first preset command to the controlled device; or it sends the first display parameter to the controlled device so that the controlled device generates the first preset command if the first display parameter and the second display parameter do not match; the first display parameter is a display parameter set for the remote control display area on the control device; the first preset command is used to trigger the controlled device to generate a virtual display, and the virtual display is configured to use the first display parameter.

[0079] In this system, the user can set the first display parameters on the control device. For example, the user can set parameters such as resolution, color mode, contrast, and refresh rate of the remote control screen in the remote control software, and use these user-set parameters as the first display parameters. In one implementation, in response to a remote display matching operation, the current display parameters of the control device are used as the first display parameters. For example, the remote control software provides a "follow local resolution" setting; the user's selection of this setting constitutes a remote display matching operation, triggering the setting of the current display parameters of the control device as the first display parameters. This ensures that the remote control screen and the original screen of the control device have the same display parameters, thus adapting the remote control screen to the display performance of the control device and achieving a better display effect.

[0080] The control device can acquire the second display parameters of the controlled device and determine whether the first and second display parameters match. If they do not match, it triggers the generation of a first preset command and sends the first preset command to the controlled device. Alternatively, the control device can send the first display parameters to the controlled device, and the controlled device can determine whether the first and second display parameters match. If they do not match, it triggers the generation of the first preset command. Alternatively, the server can determine whether the first and second display parameters match. If they do not match, it triggers the generation of the first preset command and sends the first preset command to the controlled device.

[0081] Furthermore, the control device can also respond to a preset function activation operation by triggering a first preset command and sending it to the control device. For example, the remote control software may provide a "Use Virtual Screen" setting; the user's selection of this setting is a preset function activation operation, triggering the generation of a first preset command. The control device can then send the first preset command triggered by the preset function activation operation to the controlled device. Alternatively, the control device can send the relevant data for the preset function activation operation to a server, which will then generate the first preset command and send it to the controlled device.

[0082] The controlled device responds to the first preset command, generates a virtual display, and can set the display parameters of the virtual display to the first display parameters. Subsequent processing and transmission of display data related to the remote control screen are then performed based on the virtual display.

[0083] Step S320: Receive the first display data of the virtual display sent by the controlled device, and display the screen corresponding to the first display data in the remote control display area using the first display parameters.

[0084] The controlled device generates first display data for the virtual display based on first display parameters, and the first display data matches the first display parameters. The control device receives the first display data sent by the controlled device and can directly display the image corresponding to the first display data in the remote control display area using the first display parameters. This eliminates the need to adjust the display parameters of the first display data, improving processing efficiency.

[0085] In step S330, in response to a control operation targeting the remote control display area, a corresponding control command is sent to the controlled device to control the controlled device.

[0086] Users can perform control operations in the remote control display area, such as manipulating the interface of the controlled device displayed there. These control operations generate corresponding control commands, which are then sent to the controlled device to execute the control commands, thus achieving the remote control function.

[0087] based on Figure 3 The method involves the control device sending a first preset command to the controlled device in response to a preset function activation operation, or sending the first preset command when the first display parameters and the second display parameters do not match, or sending the first display parameters, causing the controlled device to trigger the first preset command when the first display parameters and the second display parameters do not match. This triggers the controlled device to create and use a virtual display. Even if the controlled device itself does not support the first display parameters, it can still generate first display data matching the first display parameters through the virtual display. The control device then displays the image in the remote control display area according to the first display parameters based on the received first display data. This overcomes the limitations of the controlled device's display performance, improves the remote control display effect, and enhances the comfort and efficiency of remote operation for users.

[0088] In one implementation, reference Figure 4 As shown, when the first display parameter and the second display parameter of the controlled device do not match, sending a first preset command to the controlled device includes the following steps S410 and S420:

[0089] Step S410: If the first display parameter and the second display parameter do not match, a preset function setting control is provided;

[0090] In step S420, in response to the activation operation of the preset function setting control, a first preset command is sent to the controlled device.

[0091] The preset function setting control can be used to enable or disable preset functions. For example, on the control device, if the first display parameter and the second display parameter do not match, a prompt message is displayed to indicate that the virtual display function can be enabled. This prompt message is the preset function setting control. If the user confirms the enable, i.e. performs an operation to enable the preset function setting control, it triggers the sending of a first preset command to the controlled device to instruct the controlled device to enable the virtual display.

[0092] based on Figure 4 The method automatically provides preset function setting controls when the first display parameters and the second display parameters do not match, so that users can quickly start the virtual monitor without having to search for the relevant settings of the virtual monitor in the remote control software. This simplifies the user operation process and reduces the user's understanding cost.

[0093] In one implementation, the remote control method further includes the following steps:

[0094] In response to the preset function shutdown operation, a second preset command is sent to the controlled device to cause the controlled device to shut down the virtual display in response to the second preset command.

[0095] For example, the remote control software on the control device provides preset function setting controls. A user's action of closing these controls constitutes a preset function shutdown, triggering the sending of a second preset command to the controlled device. The control device can then send this second preset command, triggered by the preset function shutdown action, to the controlled device. Alternatively, the control device can send the relevant data for the preset function shutdown action to a server, which will then generate a second preset command and send it to the controlled device.

[0096] Upon receiving the second preset command, the controlled device shuts down the virtual display. This can be achieved by uninstalling the virtual display driver. Afterwards, the device can revert to using the original display (such as a physical monitor) to provide display functionality.

[0097] In one implementation, reference Figure 5 As shown, the remote control method further includes the following steps S510 and S520:

[0098] Step S510: Receive second display data sent by the controlled device after closing the virtual display; the second display data is display data generated by the controlled device based on third display parameters, which are display parameters used by the controlled device before generating the virtual display.

[0099] Step S520: Display the screen corresponding to the second display data in the remote control display area using the third display parameters.

[0100] In this process, after the controlled device disables the virtual display, it reverts to using the original third display parameters. In the graphics rendering pipeline, the pixel information in the screen space is determined based on these third display parameters, and image or video data is generated as the second display data. This second display data is then matched with the third display parameters. The controlled device sends the second display data to the control device, which can then display the corresponding image in a remotely controlled display area using the third display parameters. This allows for rapid and flexible switching between using and not using a virtual display.

[0101] This disclosure also provides a remote control device. (See reference...) Figure 6 As shown, the remote control device 600 includes the following modules:

[0102] The virtual display generation module 610 is configured to generate a virtual display in response to a first preset instruction by the controlled device; wherein the first preset instruction is an instruction triggered when the first display parameter and the second display parameter of the controlled device do not match, or an instruction triggered based on a preset function activation operation on the control device, and the first display parameter is a display parameter set for the remote control display area on the control device.

[0103] The virtual display setting module 620 is configured to set the display parameters of the virtual display to the first display parameters;

[0104] The display data generation module 630 is configured to generate first display data for the virtual display based on the first display parameters.

[0105] The display data sending module 640 is configured to send the first display data to the control device, so that the control device displays the screen corresponding to the first display data in the remote control display area using the first display parameters;

[0106] The remote controlled module 650 is configured to receive and execute control commands sent by the control device.

[0107] In one embodiment, the remote control device 600 is further configured to:

[0108] After the controlled device closes the virtual display, it generates second display data based on the third display parameters and sends the second display data to the control device, so that the control device displays the image corresponding to the second display data in the remote control display area using the third display parameters; the third display parameters are the display parameters used by the controlled device before generating the virtual display.

[0109] In one implementation, generating the virtual display includes:

[0110] The virtual display is created by installing a virtual display driver on the controlled device.

[0111] In one embodiment, the remote control device 600 is further configured to:

[0112] The controlled device responds to a second preset instruction by unloading the virtual display through the virtual display driver; wherein the second preset instruction is an instruction triggered based on a preset function shutdown operation on the control device.

[0113] In one embodiment, the remote control device 600 is further configured to:

[0114] The controlled device responds by initiating a remote control process and installing the virtual display driver; the remote control process is used to establish a remote control connection between the controlled device and the control device.

[0115] In one embodiment, the remote control device 600 is further configured to:

[0116] When the controlled device generates the virtual display, it sets the virtual display as the only display, so that the virtual display can obtain all or part of the current interface data of the controlled device.

[0117] The step of generating the first display data for the virtual display based on the first display parameters includes:

[0118] The interface data is rendered based on the first display parameters to generate the first display data.

[0119] This disclosure also provides another remote control device. (See reference...) Figure 7 As shown, the remote control device 700 includes the following modules:

[0120] The display setting interaction module 710 is configured to, in response to a preset function activation operation, send a first preset instruction to the controlled device; or, if the first display parameters and the second display parameters of the controlled device do not match, send the first preset instruction to the controlled device; or send the first display parameters to the controlled device so that the controlled device generates the first preset instruction when the first display parameters and the second display parameters do not match; the first display parameters are display parameters set for the remote control display area on the control device; the first preset instruction is used to trigger the controlled device to generate a virtual display, the virtual display being configured to use the first display parameters;

[0121] The display data receiving module 720 is configured to receive first display data sent by the controlled device to the virtual display, and to display the screen corresponding to the first display data in the remote control display area using the first display parameters;

[0122] The remote control module 730 is configured to send corresponding control commands to the controlled device in response to a control operation on the remote control display area, so as to control the controlled device.

[0123] In one embodiment, sending a first preset command to the controlled device when the first display parameter and the second display parameter of the controlled device do not match includes:

[0124] If the first display parameter and the second display parameter do not match, a preset function setting control is provided;

[0125] In response to the activation operation of the preset function setting control, the first preset instruction is sent to the controlled device.

[0126] In one embodiment, the remote control device 700 is further configured to:

[0127] In response to a preset function shutdown operation, a second preset command is sent to the controlled device to cause the controlled device to shut down the virtual display in response to the second preset command.

[0128] In one embodiment, the remote control device 700 is further configured to:

[0129] The controlled device receives second display data after turning off the virtual display; the second display data is display data generated by the controlled device based on third display parameters, which are display parameters used by the controlled device before generating the virtual display.

[0130] The third display parameter is used to display the screen corresponding to the second display data in the remote control display area.

[0131] In one embodiment, the remote control device 700 is further configured to:

[0132] In response to a remote display matching operation, the current display parameters of the control device are used as the first display parameters.

[0133] The specific details of each part of the above-mentioned device have been described in detail in the method section of the implementation plan. For any undisclosed details, please refer to the implementation plan of the method section, and therefore will not be repeated here.

[0134] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to exemplary embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0135] Exemplary embodiments of this disclosure also provide a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the methods described above.

[0136] In one implementation, the computer program product can be a tangible product, such as a computer-readable storage medium storing a computer program. The readable storage medium can be based on electrical, magnetic, optical, electromagnetic, infrared, or other signals, and includes, but is not limited to: random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory, hard disk drive (HDD), solid-state drive (SSD), etc. Exemplarily, the computer program product can be a non-volatile storage medium storing a computer program, such as read-only memory, NAND flash memory, etc.

[0137] In one implementation, the computer program product can be an intangible product. For example, the computer program product can be a virtual digital product, such as an executable file or installation package containing a computer program.

[0138] Computer program code can be written in one or more programming languages. Examples of programming languages ​​include C, Java, and C++. Program code can execute entirely on the user's computing device, partially on the user's computing device, or as a standalone software package. It can also execute partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, such as a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via an internet connection provided by a mobile network operator).

[0139] Computer programs can be carried or transmitted via signals such as electricity, magnetism, light, electromagnetic fields, and infrared radiation. Electronic devices can convert signals carrying computer programs into digital signals, thereby running the computer programs. When a computer program runs on an electronic device, its code is used to cause the electronic device to execute (more specifically, to be executed by the processor of the electronic device) the method steps of various embodiments of this disclosure, for example: Step S210, the controlled device responds to a first preset instruction to generate a virtual display; wherein, the first preset instruction is an instruction triggered when the first display parameters and the second display parameters of the controlled device do not match, or an instruction triggered based on a preset function activation operation on the control device, and the first display parameters are display parameters set for the remote control display area on the control device; Step S220, the display parameters of the virtual display are set to the first display parameters; Step S230, first display data of the virtual display is generated based on the first display parameters; Step S240, the first display data is sent to the control device so that the control device displays the image corresponding to the first display data in the remote control display area using the first display parameters; Step S250, the control instruction sent by the control device is received and executed.

[0140] The above method steps are implemented by a computer program. When the first display parameter and the second display parameter do not match, a first preset command is triggered, or a preset function on the control device is activated, triggering the first preset command. This causes the controlled device to create a virtual display and set its display parameters to the first display parameter. During remote control, first display data matching the first display parameter is generated, enabling the control device to display the corresponding image in the remote control display area using the first display parameter. Even if the controlled device itself does not support the first display parameter, it can still display the image in the remote control display area according to the first display parameter through the virtual display. This overcomes the limitations of the controlled device's display performance, improves the remote control display effect, and enhances the comfort and efficiency of remote operation for the user.

[0141] Exemplary embodiments of this disclosure also provide an electronic device, such as the controlled device or control device described above. The electronic device includes a processor and a memory. The memory stores executable instructions for the processor, such as computer programs. The processor executes these executable instructions to perform the method steps of various exemplary embodiments of this disclosure.

[0142] The following is for reference. Figure 8 The electronic device is illustrated by way of a general-purpose computing device. It should be understood that... Figure 8 The electronic device 800 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0143] like Figure 8As shown, the electronic device 800 may include: a processor 810, a memory 820, a bus 830, an I / O (input / output) interface 840, and a network adapter 850.

[0144] The memory 820 may include volatile memory, such as RAM 821 and cache unit 822, and may also include non-volatile memory, such as ROM 823. The memory 820 may also include one or more program modules 824, including but not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. For example, program module 824 may include the modules described above.

[0145] The processor 810 may include one or more processing units, such as an AP (Application Processor), a modem processor, a GPU, an ISP (Image Signal Processor), a controller, an encoder, a decoder, a DSP (Digital Signal Processor), a baseband processor, and / or an NPU (Neural-Network Processing Unit).

[0146] The processor 810 can be used to execute executable instructions stored in the memory 820 to perform method steps of various embodiments of the present disclosure, such as: step S210, the controlled device generates a virtual display in response to a first preset instruction; wherein, the first preset instruction is an instruction triggered when the first display parameter and the second display parameter of the controlled device do not match, or an instruction triggered based on a preset function activation operation on the control device, and the first display parameter is a display parameter set for the remote control display area on the control device; step S220, the display parameter of the virtual display is set to the first display parameter; step S230, the first display data of the virtual display is generated based on the first display parameter; step S240, the first display data is sent to the control device so that the control device displays the screen corresponding to the first display data in the remote control display area using the first display parameter; step S250, the control instruction sent by the control device is received and executed.

[0147] The processor 810 executes the above method steps. If the first display parameter and the second display parameter do not match, a first preset instruction is triggered; or, based on a preset function activation operation on the control device, the first preset instruction is triggered, causing the controlled device to create a virtual display and set its display parameters to the first display parameter. During remote control, first display data matching the first display parameter is generated, causing the control device to display the image corresponding to the first display data in the remote control display area using the first display parameter. Even if the controlled device itself does not support the first display parameter, it can still display the image in the remote control display area according to the first display parameter through the virtual display, thereby overcoming the limitations of the controlled device's display performance, improving the remote control display effect, and enhancing the comfort and efficiency of remote operation for users.

[0148] Bus 830 is used to connect different components of electronic device 800 and may include a data bus, an address bus and a control bus.

[0149] Electronic device 800 can communicate with one or more external devices 900 (such as keyboard, mouse, external controller, etc.) through I / O interface 840.

[0150] Electronic device 800 can communicate with one or more networks via network adapter 850. For example, network adapter 850 can provide mobile communication solutions such as 3G / 4G / 5G, or wireless communication solutions such as wireless LAN, Bluetooth, and near-field communication. Network adapter 850 can communicate with other modules of electronic device 800 via bus 830.

[0151] although Figure 8 As not shown in the diagram, other hardware and / or software modules may also be configured in the electronic device 800, including but not limited to: a display, microcode, device driver, redundant processor, external disk drive array, RAID (Redundant Arrays of Independent Disks) system, tape drive, and data backup storage system.

[0152] As can be seen from the above, the technical solutions disclosed herein can be implemented as methods, apparatus, systems, computer program products, storage media, electronic devices, etc. Those skilled in the art will understand that various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be referred to as "circuit," "module," or "system," respectively.

[0153] It should be understood that this disclosure is not limited to the specific methods, steps, or structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. Those skilled in the art will readily conceive of other embodiments based on the specific implementations provided in this disclosure. Therefore, the specific implementations provided in this disclosure are merely illustrative, and the scope and spirit of this disclosure are indicated by the claims, and should cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary technical means in the art not disclosed in this disclosure.

Claims

1. A remote control method, characterized in that, The method includes: The controlled device responds to a first preset instruction to generate a virtual display; wherein the first preset instruction is an instruction triggered when the first display parameter and the second display parameter of the controlled device do not match, or an instruction triggered based on a preset function activation operation on the control device, and the first display parameter is a display parameter set for the remote control display area on the control device; Set the display parameters of the virtual display to the first display parameters; Generate first display data for the virtual display based on the first display parameters; The first display data is sent to the control device so that the control device displays the image corresponding to the first display data in the remote control display area using the first display parameters; Receive and execute control commands sent by the control device.

2. The method according to claim 1, characterized in that, The method further includes: After the controlled device closes the virtual display, it generates second display data based on the third display parameters and sends the second display data to the control device, so that the control device displays the image corresponding to the second display data in the remote control display area using the third display parameters; the third display parameters are the display parameters used by the controlled device before generating the virtual display.

3. The method according to claim 1, characterized in that, The generation of the virtual display includes: The virtual display is created by installing a virtual display driver on the controlled device.

4. The method according to claim 3, characterized in that, The method further includes: The controlled device responds to a second preset instruction by unloading the virtual display through the virtual display driver; wherein the second preset instruction is an instruction triggered based on a preset function shutdown operation on the control device.

5. The method according to claim 3, characterized in that, The method further includes: The controlled device responds by initiating a remote control process and installing the virtual display driver; the remote control process is used to establish a remote control connection between the controlled device and the control device.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: When the controlled device generates the virtual display, it sets the virtual display as the only display, so that the virtual display can obtain all or part of the current interface data of the controlled device. The step of generating the first display data for the virtual display based on the first display parameters includes: The interface data is rendered based on the first display parameters to generate the first display data.

7. A remote control method, characterized in that, The method includes: The control device responds to a preset function activation operation by sending a first preset command to the controlled device; or, if the first display parameter and the second display parameter of the controlled device do not match, it sends the first preset command to the controlled device; or it sends the first display parameter to the controlled device so that the controlled device generates the first preset command if the first display parameter and the second display parameter do not match; the first display parameter is a display parameter set for a remote control display area on the control device; the first preset command is used to trigger the controlled device to generate a virtual display, the virtual display being configured to use the first display parameter; Receive the first display data of the virtual display sent by the controlled device, and display the screen corresponding to the first display data in the remote control display area using the first display parameters; In response to a control operation targeting the remote control display area, a corresponding control command is sent to the controlled device to control the controlled device.

8. The method according to claim 7, characterized in that, When the first display parameter and the second display parameter of the controlled device do not match, sending a first preset command to the controlled device includes: If the first display parameter and the second display parameter do not match, a preset function setting control is provided; In response to the activation operation of the preset function setting control, the first preset instruction is sent to the controlled device.

9. The method according to claim 7, characterized in that, The method further includes: In response to a preset function shutdown operation, a second preset command is sent to the controlled device to cause the controlled device to shut down the virtual display in response to the second preset command.

10. The method according to claim 9, characterized in that, The method further includes: The controlled device receives second display data after turning off the virtual display; the second display data is display data generated by the controlled device based on third display parameters, which are display parameters used by the controlled device before generating the virtual display. The third display parameter is used to display the screen corresponding to the second display data in the remote control display area.

11. The method according to any one of claims 7 to 10, characterized in that, The method further includes: In response to a remote display matching operation, the current display parameters of the control device are used as the first display parameters.

12. A remote control device, characterized in that, The device includes: The virtual display generation module is configured to generate a virtual display in response to a first preset instruction from the controlled device; wherein the first preset instruction is an instruction triggered when the first display parameter and the second display parameter of the controlled device do not match, or an instruction triggered based on a preset function activation operation on the control device; and the first display parameter is a display parameter set for the remote control display area on the control device. The virtual display setting module is configured to set the display parameters of the virtual display to the first display parameters; The display data generation module is configured to generate first display data for the virtual display based on the first display parameters; The display data sending module is configured to send the first display data to the control device, so that the control device displays the screen corresponding to the first display data in the remote control display area using the first display parameters; The remote controlled module is configured to receive and execute control commands sent by the control device.

13. A remote control device, characterized in that, The device includes: The display settings interaction module is configured to: 1) send a first preset instruction to the controlled device in response to a preset function activation operation; 2) send a first preset instruction to the controlled device if the first display parameters and the second display parameters of the controlled device do not match; 3) send the first display parameters to the controlled device so that the controlled device generates the first preset instruction if the first display parameters and the second display parameters do not match; 4) the first display parameters are display parameters set for the remote control display area on the control device; 5) the first preset instruction is used to trigger the controlled device to generate a virtual display, the virtual display being configured to use the first display parameters. The display data receiving module is configured to receive first display data sent by the controlled device to the virtual display, and to display the screen corresponding to the first display data in the remote control display area using the first display parameters; The remote control module is configured to send corresponding control commands to the controlled device in response to a control operation on the remote control display area, so as to control the controlled device.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method according to any one of claims 1 to 11.

15. An electronic device, characterized in that, include: processor; Memory for storing the executable instructions of the processor; The processor is configured to execute the method of any one of claims 1 to 11 by executing the executable instructions.

Images