Device display control method and system

Abstract

The embodiment of the present application provides a device display control method and system, wherein the device display control method comprises the following steps: in response to feedback information of at least two devices for a target application, creating a virtual screen for each of the at least two devices, wherein the at least two devices are in communication connection; in the case that the target application in a first device is detected to be started, creating a virtual window of the target application on a first virtual screen of the first device based on the display position of the application window of the target application on the first device, wherein the first device is one of the at least two devices; in response to a cross-device moving instruction submitted for the virtual window, moving the virtual window to a second virtual screen of a second device, wherein the second device is one of the at least two devices and is determined by the cross-device moving instruction; and mapping the display result of the virtual window on the second virtual screen to the device screen of the second device.

Description

Technical Field

[0001] This application relates to the field of Internet technology, and in particular to a device display control method. One or more embodiments of this application also relate to a device display control system, a device display control apparatus, a computing device, and a computer-readable storage medium. Background Technology

[0002] With the rapid development of terminal technology, more and more applications are installed on terminal devices, their functions are becoming increasingly rich, and users are using these applications more frequently. Currently, an application can be used on multiple devices, and users can synchronize application content by logging into the same account on different devices. However, when using the same application on different devices, different user needs may need to be met. In such cases, it may be necessary to switch between applications during use. However, current application switching is still limited to a single device and cannot be achieved across multiple devices or different screens. Therefore, an effective method is urgently needed to solve this problem. Summary of the Invention

[0003] In view of this, embodiments of this application provide a device display control method. One or more embodiments of this application also relate to a device display control system, a device display control apparatus, a computing device, and a computer-readable storage medium, to solve the technical defects in the prior art that make it impossible to realize application window interaction between multiple devices.

[0004] According to a first aspect of the embodiments of this application, a device display control method is provided, comprising:

[0005] In response to feedback from at least two devices regarding a target application, a virtual screen is created for each of the at least two devices, wherein there is a communication connection between the at least two devices;

[0006] If the target application is detected to be launched on the first device, a virtual window of the target application is created on the first virtual screen of the first device based on the display position of the application window of the target application on the first device, wherein the first device is one of the at least two devices;

[0007] In response to a cross-device move command submitted for the virtual window, the virtual window is moved to a second virtual screen of a second device, wherein the second device is one of the at least two devices and is determined by the cross-device move command;

[0008] The display result of the virtual window on the second virtual screen is mapped to the device screen of the second device.

[0009] According to a second aspect of the embodiments of this application, a device display control apparatus is provided, comprising:

[0010] A creation module is configured to create a virtual screen for each of at least two devices in response to feedback information from at least two devices for a target application, wherein there is a communication connection between the at least two devices;

[0011] The detection module is configured to, upon detecting that the target application is launched on the first device, create a virtual window of the target application on a first virtual screen of the first device based on the display position of the application window of the target application on the first device, wherein the first device is one of the at least two devices;

[0012] A mobility module is configured to move the virtual window to a second virtual screen of a second device in response to a cross-device mobility command submitted for the virtual window, wherein the second device is one of the at least two devices and is determined by the cross-device mobility command;

[0013] The mapping module is configured to map the display result of the virtual window on the second virtual screen to the device screen of the second device.

[0014] According to a third aspect of the embodiments of this application, a device display control system is provided, comprising:

[0015] A first device and at least one second device;

[0016] The first device is configured to establish a communication connection with the at least one second device, and in response to feedback information from the at least one second device regarding the target application, to create a virtual screen for each of the first device and the at least one second device;

[0017] Upon detecting that the target application has been launched, a virtual window of the target application is created on the first virtual screen of the first device based on the display position of the application window of the target application on the first device.

[0018] In response to a cross-device move command submitted for the virtual window, the virtual window is moved to a second virtual screen of a target second device, wherein the target second device is one of the at least one second device and is determined by the cross-device move command;

[0019] The display result of the virtual window on the second virtual screen is mapped to the device screen of the target second device.

[0020] According to a fourth aspect of the embodiments of this application, a computing device is provided, comprising:

[0021] Memory and processor;

[0022] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, wherein the processor executes the computer-executable instructions to implement the steps of the device display control method.

[0023] According to a fifth aspect of the present application, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the steps of the device display control method.

[0024] One embodiment of this application implements a device display control method and apparatus. The device display control method includes feedback information from two devices regarding a target application, creating a virtual screen for each of the at least two devices, wherein the at least two devices are connected in communication, and upon detecting that the target application is launched on a first device, a virtual window of the target application is created on a first virtual screen of the first device based on the display position of the application window of the target application on the first device, wherein the first device is one of the at least two devices, and in response to a cross-device movement command submitted for the virtual window, the virtual window is moved to a second virtual screen of a second device, wherein the second device is one of the at least two devices and is determined by the cross-device movement command, and the display result of the virtual window on the second virtual screen is mapped to the device screen of the second device.

[0025] In this embodiment, by creating virtual screens for different devices and creating virtual windows in the virtual screen of the main control device, the application window can be moved across screens or devices by controlling the virtual windows to move between different virtual screens and then mapping the movement results to the real device screen. This improves the user's device experience and helps meet the user's application needs. Attached Figure Description

[0026] Figure 1 This is a flowchart of a device display control method provided in one embodiment of this application;

[0027] Figure 2 This is a schematic diagram illustrating a virtual screen creation result provided in one embodiment of this application;

[0028] Figure 3 This is a schematic diagram of a device display control process provided in one embodiment of this application;

[0029] Figure 4This is a schematic diagram illustrating a specific process of a device display control method in the fitness field according to an embodiment of this application;

[0030] Figure 5 This is a schematic diagram of the structure of a device display control device according to an embodiment of this application;

[0031] Figure 6 This is a schematic diagram of a device display control system provided in one embodiment of this application;

[0032] Figure 7 This is a structural block diagram of a computing device provided in one embodiment of this application. Detailed Implementation

[0033] Many specific details are set forth in the following description to provide a full understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this application; therefore, this application is not limited to the specific embodiments disclosed below.

[0034] The terminology used in one or more embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the scope of one or more embodiments of this application. The singular forms “a,” “the,” and “the” used in one or more embodiments of this application and in the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” used in one or more embodiments of this application refers to and includes any or all possible combinations of one or more associated listed items.

[0035] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this application, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0036] This application provides a device display control method. One or more embodiments of this application also relate to a device display control apparatus, a computing device, and a computer-readable storage medium, which will be described in detail in the following embodiments.

[0037] See Figure 1 , Figure 1 A flowchart of a device display control method according to an embodiment of this application is shown, including the following steps:

[0038] Step 102: In response to feedback information from at least two devices regarding the target application, create a virtual screen for each of the at least two devices, wherein the at least two devices are connected in communication.

[0039] Specifically, the device display control method provided in the embodiments of this specification is applied to a client, which is one of at least two devices, and the client is the master control device. It is used to establish communication connections with other devices among the at least two devices and to create virtual screens for each device. Therefore, among the at least two devices, other than the master control device only need to establish a communication connection with the master control device. Whether or not to establish a communication connection with other devices is not limited in this application embodiment.

[0040] The target application, or APP, is a usable application on the device. When the target application is launched on the device, the application window can be displayed in picture-in-picture mode, that is, the application window is displayed in a certain area of ​​the device screen, and the application screen of the target application can be displayed in this window.

[0041] Since the target application can be downloaded and used on multiple devices, cross-device interaction of the application window can be achieved by moving the application window displayed on one device to another while using the target application on another device.

[0042] Based on this, in this embodiment of the application, the central device can send information acquisition instructions to other devices for the target application. Other devices can return feedback information of the target application to the central device based on the instructions or automatically. The feedback information may include the installation information of the target application. When the central device determines that other devices have installed the target application based on the feedback information, it can create a virtual screen for the device with the target application installed.

[0043] In practical applications, the central device can also query other devices about the installation results of the target application, and the devices that have installed the target application can submit feedback information. In this case, the central device can create a virtual screen for each device that submits feedback information.

[0044] The schematic diagram of the virtual screen creation result provided in this application embodiment is as follows: Figure 2 As shown, Figure 2 The devices in the system include fitness mirrors, televisions, and mobile phones. After creating a virtual screen for each device, they can be combined. The positional relationship of each device in the combination result can be determined according to the positional relationship of each device in the actual scene, or according to actual needs. There are no restrictions here.

[0045] Step 104: Upon detecting that the target application has been launched on the first device, based on the display position of the target application's application window on the first device, a virtual window of the target application is created on the first virtual screen of the first device, wherein the first device is one of the at least two devices.

[0046] Specifically, the first device is one of the aforementioned at least two devices, and the first device has the target application installed and establishes a communication connection with the central device; the first virtual screen is the virtual screen created by the central device for the first device.

[0047] In practical applications, the size of the device can be scaled down proportionally to obtain the size of the device's virtual screen.

[0048] When the target application is detected to be launched on the first device, the first device becomes the master device, that is, the user controls and uses the target application through the first device. In this case, considering that the user may use the target application across devices, a virtual window for the target application on the first device can be created in the first virtual screen of the first device.

[0049] In practical applications, the virtual window can also be created by the first device.

[0050] Furthermore, when the target application's window is displayed in picture-in-picture mode, the position of the virtual window on the first virtual screen can be determined based on the application window's display position on the first device. The relative positions of the application window and the virtual window remain unchanged within both the first device and the first virtual screen.

[0051] Taking a fitness app as an example, devices suitable for fitness include, but are not limited to, exercise bikes, fitness systems combining fitness trackers, fitness boxes, and TV projection, TVs, mobile phones, and fitness mirrors. Since different devices can meet different user needs—for example, rotating the body in front of a fitness mirror to measure body dimensions, sitting on an exercise bike to participate in an exercise class, or standing in front of a TV suitable for regular live classes and post-workout stretching sessions—the same user account can be logged into on different devices, providing a multi-dimensional experience of fitness services. This process may involve cross-device application window interaction. Therefore, upon detecting the launch of the target app on the first device, a virtual window of the target app can be created on the first virtual screen of the first device. This allows for cross-device movement of the application window based on the virtual window, thus achieving cross-device interaction.

[0052] In specific implementation, the first device includes an image acquisition unit;

[0053] Accordingly, before moving the virtual window to the second virtual screen of the second device in response to a cross-device move command submitted for the virtual window, the method further includes:

[0054] The image acquisition unit acquires the target user's gesture control commands, wherein the gesture control commands carry the movement information of the virtual window;

[0055] Generate cross-device movement instructions for the virtual window based on the gesture control instructions.

[0056] Specifically, movement information, including but not limited to the direction of movement and / or the distance of movement.

[0057] The first device includes an image acquisition unit, which can be built into or externally placed in the first device, for acquiring the user's gesture control commands.

[0058] After creating virtual screens for each device, since the target application is launched on the first device, if the user controls the virtual window to move between different virtual screens via gestures, the image acquisition unit of the first device needs to acquire the user's gesture control commands, i.e., the direction and / or distance of movement of the virtual window, in order to generate cross-device movement commands for the virtual window based on the gesture control commands.

[0059] For example, if the first virtual screen of the first device is adjacent to the second virtual screen of the second device, and the first virtual screen is located to the right of the second virtual screen, then if the virtual window moves to the left in the gesture control command, the generated cross-device movement command could include: moving the virtual window from the first virtual screen to the second virtual screen. Furthermore, the movement distance of the virtual window can be determined based on its position on the first virtual screen and the sizes of both the first and second virtual screens.

[0060] In specific implementation, the image acquisition unit acquires the target user's gesture control commands, including:

[0061] The image acquisition unit collects information on changes in the hand bones of the target user.

[0062] The hand bone change information is analyzed, and gesture control commands for the target user are generated based on the analysis results.

[0063] Specifically, hand skeletal change information, which can be information about changes in the bones of the fingers, including but not limited to changes in bone bending or extension. After the image acquisition component acquires hand skeletal change information, it can analyze this information to determine the user's gesture control commands.

[0064] In practical applications, when the first device is the central device, the first device can use a gesture recognition algorithm to process the skeletal change information to identify the user's gestures, such as finger bending, finger extension, or finger movement. Then, a gesture classifier is used to classify the gestures. For example, the gesture corresponding to finger bending can be grasping, the gesture corresponding to finger extension can be releasing, and the gesture corresponding to finger movement can be turning a page.

[0065] When a grab gesture is detected, the gesture control command can be determined to move the virtual window in the virtual screen. Then, when a page-turning gesture is detected, the gesture control command can be determined to determine the movement information of the virtual window based on the page-turning gesture. Finally, when a release gesture is detected, the gesture control command can be determined to stop the movement of the virtual window.

[0066] Furthermore, after determining the gesture control command, a cross-device movement command for the virtual window is generated based on the gesture control command, including:

[0067] Determine the first coordinates of the virtual window in the first virtual screen;

[0068] Based on the movement information carried in the gesture control command, determine the second coordinates of the virtual window in the second virtual screen;

[0069] Based on the first coordinate and the second coordinate, a cross-device movement command for the virtual window is generated.

[0070] Specifically, since it is necessary to determine the exact location of the virtual window on the other virtual screen when moving it from one virtual screen to another, in the case where the first device is the central device, the first device can first determine the first coordinates of the virtual window on the first virtual screen when the gesture control command is detected. Then, based on the movement direction and / or movement position carried in the gesture control command, the first device can determine the second coordinates of the virtual window on the second virtual screen, wherein the first coordinates and the second coordinates are referenced to the same coordinate system. Then, a cross-device movement command for the virtual window can be generated based on the first coordinates and the second coordinates. This cross-device movement command can be: move the virtual window from the location of the first coordinate on the first virtual screen to the location of the second coordinate on the second virtual screen.

[0071] In practical applications, after determining the second coordinate, a gesture smoother can be used to smooth the second coordinate, making the next window movement smoother.

[0072] In this embodiment of the application, in addition to determining the cross-device movement command of the virtual window through the aforementioned gesture control command, the movement of the virtual window can also be controlled by a signal, specifically, the first device includes a signal acquisition unit;

[0073] Accordingly, before moving the virtual window to the second virtual screen of the second device in response to a cross-device move command submitted for the virtual window, the method further includes:

[0074] The signal acquisition unit acquires the target remote control signal emitted towards the virtual window, wherein the target remote control signal carries the movement information of the virtual window;

[0075] The virtual window is generated based on the target remote control signal to perform cross-device movement instructions.

[0076] Furthermore, the signal acquisition unit acquires the target remote control signal emitted towards the virtual window, including:

[0077] The signal acquisition unit acquires the target remote control signal emitted by the remote control device towards the virtual window.

[0078] Accordingly, generating the cross-device movement command for the virtual window based on the target remote control signal includes:

[0079] The target remote control signal is parsed, and the feature code of the remote control device and the mapping relationship between the remote control commands corresponding to each button in the remote control device are determined based on the parsing results.

[0080] Based on the mapping relationship, a target remote control command associated with the target remote control signal is determined;

[0081] Generate cross-device movement instructions for the virtual window based on the target remote control instructions.

[0082] Specifically, in addition to controlling the movement of the virtual window through gestures, users can also control it via signals. This can be achieved through a remote control device, such as a remote controller, which transmits remote control signals to the virtual window. Movement information, including but not limited to movement direction and / or distance, can be determined through the remote control signals. For example, a remote control device may contain four buttons: move up, move down, move left, and move right. However, different remote control devices have different signatures, and different buttons on different devices have different remote control commands. For instance, the first button on remote control A might correspond to the command "move up," while the first button on remote control B might correspond to the command "turn the device on or off." Therefore, upon receiving a target remote control signal, the signature of the remote control device transmitting the signal, as well as the mapping relationship between the buttons and their corresponding remote control commands on that device, can be analyzed. Based on this mapping relationship, the target remote control command associated with the button corresponding to the target remote control signal can be determined, and then a cross-device movement command can be generated based on the target remote control command.

[0083] In practical applications, after creating virtual screens for each device, since the target application starts on the first device, if the user controls the virtual window to move between different virtual screens through signal control, the signal acquisition unit of the first device needs to collect the user's signal control commands, i.e., the direction and / or distance of movement of the virtual window, in order to generate cross-device movement commands for the virtual window based on the signal control commands.

[0084] For example, if the first virtual screen of the first device is adjacent to the second virtual screen of the second device, and the first virtual screen is located to the right of the second virtual screen, then if the virtual window's movement direction in the signal control instruction is to move to the left, the generated cross-device movement instruction could include: moving the virtual window from the first virtual screen to the second virtual screen. Furthermore, the movement distance of the virtual window can be determined based on the virtual window's position on the first virtual screen, the sizes of the first and second virtual screens, and the number of key presses.

[0085] In specific implementation, the first device includes an image acquisition unit;

[0086] Accordingly, before moving the virtual window to the second virtual screen of the second device in response to a cross-device move command submitted for the virtual window, the method further includes:

[0087] The image acquisition unit acquires the target user's location information and the target user's hand bone changes.

[0088] The distance between the target user and each of the at least two devices is determined based on the location information;

[0089] The cross-device movement command for the virtual window is determined based on the distance and the changes in the hand bones.

[0090] Specifically, as mentioned earlier, hand skeletal change information refers to changes in the bones of the fingers, including but not limited to changes in bone bending or extension. After the image acquisition component acquires hand skeletal change information, it can analyze this information to determine the user's gesture control commands.

[0091] However, in practical applications, if there are multiple devices, gestures alone may not be able to accurately determine gesture control commands. For example, if there are two devices to the left of the first device, it is impossible to determine which of the two devices the virtual window should be moved to based solely on the gesture command to move to the left.

[0092] Therefore, in this embodiment of the application, the user's location information is combined, that is, the user's location information is collected by the image acquisition component, and the distance between the user and each device is determined according to the acquisition result. If the distance between the user and the second device is the closest, and the position of the second device relative to the first device is adapted to the movement direction in the gesture control command, then it can be determined that the cross-device movement command can be to move the virtual window to the second device.

[0093] In this embodiment of the application, when the first device is not the central device, before moving the virtual window to the second virtual screen of the second device in response to a cross-device move command submitted for the virtual window, the method further includes:

[0094] Determine whether the first device is the central device;

[0095] If not, the gesture control commands of the target user are acquired through the image acquisition unit of the first device, and the gesture control commands are sent to the central device. The gesture control commands are used by the central device to generate cross-device movement commands for the virtual window.

[0096] Specifically, when the first device is not the central device, after the first device collects the user's gesture control command, it needs to send the gesture control command to the central device, and the central device will parse the gesture control command to determine the cross-device movement command of the virtual window based on the parsing result.

[0097] Step 106: In response to a cross-device move command submitted for the virtual window, move the virtual window to a second virtual screen of a second device, wherein the second device is one of the at least two devices and is determined by the cross-device move command.

[0098] In practice, after receiving a cross-device movement instruction for a virtual window, the central device can move the virtual window from the first virtual screen of the first device to the second virtual screen of the second device according to the movement information of the virtual window in the instruction. The second device is one of the other devices besides the first device among at least two devices, and the second device is determined by the cross-device movement instruction, that is, the cross-device movement instruction contains the device information of the second device.

[0099] In specific implementation, moving the virtual window to the second virtual screen of the second device includes:

[0100] Create a target virtual window for the target application in the second virtual screen of the second device;

[0101] The display position of the target virtual window on the second virtual screen is determined according to the cross-device movement command;

[0102] The target virtual window is displayed in the second virtual screen according to the display position.

[0103] Specifically, since moving a virtual window from the first virtual screen to a virtual window on the second virtual screen actually changes the display of the virtual window from the first virtual screen to the second virtual screen, a target virtual window for the target application can be created on the second virtual screen based on the display of the virtual window. The content of the target virtual window is consistent with the content of the virtual window in the first virtual screen. Then, according to the display position of the target virtual window on the second virtual screen in the cross-device move instruction, the target virtual window is displayed on the second virtual screen, and the display of the virtual window on the first virtual screen is stopped.

[0104] Step 108: Map the display result of the virtual window on the second virtual screen to the device screen of the second device.

[0105] Specifically, after the virtual window is displayed on the second virtual screen, screen inversion mapping can be performed, that is, the virtual window in the second virtual screen is mapped back to the device screen of the second device. During the mapping process, the display size of the virtual window on the device screen of the second device needs to be determined according to the scaling ratio between the second virtual screen and the device screen of the second device, but the relative position of the virtual window in the second virtual screen and the device screen of the second device does not change.

[0106] In specific implementation, mapping the display result of the virtual window on the second virtual screen to the device screen of the second device includes:

[0107] According to the communication protocol between the first device and the second device, the display result of the virtual window on the second virtual screen is encoded, and the encoded result is sent to the second device. The encoded result is used by the second device for decoding, and according to the decoding result, the display result of the virtual window on the second virtual screen is displayed on the device screen of the second device.

[0108] Specifically, since the central device communicates with other devices, and in the case of the first device being the central device, the first device also communicates with the second device and can transmit data with the second device through a preset communication protocol. Therefore, after the first device moves the virtual window to the second virtual screen of the second device, it needs to perform screen inversion mapping to map the virtual window in the second virtual screen back to the device screen of the second device. During this process, the first device can encode the display result of the virtual window on the second virtual screen according to the preset communication protocol and send the encoded result to the second device. After receiving the encoded result, the second device can decode it according to the preset communication protocol to determine the display result of the virtual window on the second virtual screen based on the encoded result, and then display it on the device screen of the second device according to the displayed result.

[0109] A schematic diagram of a device display control process provided in this application embodiment is shown below. Figure 3 As shown. After the device starts up, it first determines whether it is a central device. If so, it can activate central device mode and establish communication connections with other devices. Then, it receives device information from other devices that have the target application installed, and creates virtual screens for these devices. When the target application is launched on the device, a virtual window of the target application can be created in the device's virtual screen. The device's image acquisition component can capture the user's gesture information, and then the virtual window can be manipulated based on the gesture information to change its position in the virtual screen. The position change result is broadcast to other devices, thereby realizing cross-device window interaction.

[0110] Additionally, if the device is not the central device, it will activate sub-device mode, establish a connection with the central device, and, if the target application is installed on the device, send its device information to the central device, which will then create a virtual screen for the device. Next, it can be determined whether the device is the master device. In practical applications, if the target application is running on the device, it can be identified as the master device. If the device is identified as the master device, a virtual window for the target application can be created on its virtual screen. The device's image capture component can also capture the user's gesture information, which is then sent to the central device for gesture analysis and subsequent window movement operations. If the device is not identified as the master device, it can receive window update information and adjust the local window's display position or result based on this information.

[0111] One embodiment of this application implements a device display control method and apparatus. The device display control method includes feedback information from two devices regarding a target application, creating a virtual screen for each of the at least two devices, wherein the at least two devices are connected in communication, and upon detecting that the target application is launched on a first device, a virtual window of the target application is created on a first virtual screen of the first device based on the display position of the application window of the target application on the first device, wherein the first device is one of the at least two devices, and in response to a cross-device movement command submitted for the virtual window, the virtual window is moved to a second virtual screen of a second device, wherein the second device is one of the at least two devices and is determined by the cross-device movement command, and the display result of the virtual window on the second virtual screen is mapped to the device screen of the second device.

[0112] In this embodiment, by creating virtual screens for different devices and creating virtual windows in the virtual screen of the main control device, the application window can be moved across screens or devices by controlling the virtual windows to move between different virtual screens and then mapping the movement results to the real device screen. This improves the user's device experience and helps meet the user's application needs.

[0113] See Figure 4 Taking the application of the device display control method provided in this application embodiment to cross-device window interaction in the fitness field as an example, the device display control method will be further explained. Among them, Figure 4 This application provides a flowchart illustrating the processing steps of a device display control method according to an embodiment, which specifically includes the following steps:

[0114] Step 402, in response to feedback from at least two fitness devices to the target fitness application, create a virtual screen for each of the at least two fitness devices, wherein the at least two fitness devices are communicatively connected.

[0115] Step 404: When the target fitness application is detected to be launched on the first fitness device, a virtual window of the target fitness application is created on the first virtual screen of the first fitness device based on the application window of the target fitness application being displayed on the first fitness device, wherein the first fitness device is one of the at least two fitness devices.

[0116] Step 406: Collect information on changes in the hand bones of the target user through the image acquisition unit included in the first fitness device.

[0117] Step 408: Analyze the hand bone change information and generate the gesture control command for the target user based on the analysis result, wherein the gesture control command carries the movement information of the virtual window.

[0118] Step 410: Determine the first coordinates of the virtual window in the first virtual screen.

[0119] Step 412: Determine the second coordinates of the virtual window in the second virtual screen based on the movement information carried in the gesture control command.

[0120] Step 414: Generate cross-device movement instructions for the virtual window based on the first coordinates and the second coordinates.

[0121] Step 416: In response to a cross-device move command submitted for the virtual window, move the virtual window to a second virtual screen of the second fitness device, wherein the second fitness device is one of the at least two fitness devices and is determined by the cross-device move command.

[0122] Step 418: According to the communication protocol between the first fitness device and the second fitness device, the display result of the virtual window on the second virtual screen is encoded, and the encoded result is sent to the second fitness device. The encoded result is used by the second fitness device for decoding, and according to the decoding result, the display result of the virtual window on the second virtual screen is displayed on the device screen of the second fitness device.

[0123] In this embodiment, by creating virtual screens for different fitness devices and creating virtual windows in the virtual screen of the main fitness device, the application window can be moved across screens or devices by controlling the virtual windows to move between different virtual screens and then mapping the movement results to the real device screen of the fitness device. This improves the user's device experience and meets the user's fitness application needs.

[0124] Corresponding to the above method embodiments, this application also provides an embodiment of a device display control device. Figure 5 A schematic diagram of a device display control apparatus according to an embodiment of this application is shown. Figure 5 As shown, the device includes:

[0125] Creation module 502 is configured to create a virtual screen for each of the at least two devices in response to feedback information from at least two devices for a target application, wherein there is a communication connection between the at least two devices;

[0126] The detection module 504 is configured to, upon detecting that the target application is launched on the first device, create a virtual window of the target application on a first virtual screen of the first device based on the display position of the application window of the target application on the first device, wherein the first device is one of the at least two devices;

[0127] The movement module 506 is configured to move the virtual window to a second virtual screen of a second device in response to a cross-device movement command submitted for the virtual window, wherein the second device is one of the at least two devices and is determined by the cross-device movement command;

[0128] The mapping module 508 is configured to map the display result of the virtual window on the second virtual screen to the device screen of the second device.

[0129] Optionally, the first device includes an image acquisition unit;

[0130] Accordingly, the device display control unit also includes a data acquisition module, configured as follows:

[0131] The image acquisition unit acquires the target user's gesture control commands, wherein the gesture control commands carry the movement information of the virtual window;

[0132] Generate cross-device movement instructions for the virtual window based on the gesture control instructions.

[0133] Optionally, the acquisition module is further configured to:

[0134] The image acquisition unit collects information on changes in the hand bones of the target user.

[0135] The hand bone change information is analyzed, and gesture control commands for the target user are generated based on the analysis results.

[0136] Optionally, the acquisition module is further configured to:

[0137] Determine the first coordinates of the virtual window in the first virtual screen;

[0138] Based on the movement information carried in the gesture control command, determine the second coordinates of the virtual window in the second virtual screen;

[0139] Based on the first coordinate and the second coordinate, a cross-device movement command for the virtual window is generated.

[0140] Optionally, the first device includes a signal acquisition unit;

[0141] Accordingly, the device display control unit further includes a generation module, configured to:

[0142] The signal acquisition unit acquires the target remote control signal emitted towards the virtual window, wherein the target remote control signal carries the movement information of the virtual window;

[0143] The virtual window is generated based on the target remote control signal to perform cross-device movement instructions.

[0144] Optionally, the generation module is further configured to:

[0145] The signal acquisition unit acquires the target remote control signal emitted by the remote control device towards the virtual window.

[0146] The target remote control signal is parsed, and the feature code of the remote control device and the mapping relationship between the remote control commands corresponding to each button in the remote control device are determined based on the parsing results.

[0147] Based on the mapping relationship, a target remote control command associated with the target remote control signal is determined;

[0148] Generate cross-device movement instructions for the virtual window based on the target remote control instructions.

[0149] Optionally, the first device includes an image acquisition unit;

[0150] Accordingly, the device display control unit further includes a determination module, configured to:

[0151] The image acquisition unit acquires the target user's location information and the target user's hand bone changes.

[0152] The distance between the target user and each of the at least two devices is determined based on the location information;

[0153] The cross-device movement command for the virtual window is determined based on the distance and the changes in the hand bones.

[0154] Optionally, the device display control unit further includes a sending module configured to:

[0155] Determine whether the first device is the central device;

[0156] If not, the gesture control commands of the target user are acquired through the image acquisition unit of the first device, and the gesture control commands are sent to the central device. The gesture control commands are used by the central device to generate cross-device movement commands for the virtual window.

[0157] Optionally, the mapping module 508 is further configured to:

[0158] According to the communication protocol between the first device and the second device, the display result of the virtual window on the second virtual screen is encoded, and the encoded result is sent to the second device. The encoded result is used by the second device for decoding, and according to the decoding result, the display result of the virtual window on the second virtual screen is displayed on the device screen of the second device.

[0159] Optionally, the mobile module 506 is further configured to:

[0160] Create a target virtual window for the target application in the second virtual screen of the second device;

[0161] The display position of the target virtual window on the second virtual screen is determined according to the cross-device movement command;

[0162] The target virtual window is displayed in the second virtual screen according to the display position.

[0163] The above is a schematic scheme of a device display control device according to this embodiment. It should be noted that the technical solution of this device display control device and the technical solution of the device display control method described above belong to the same concept. For details not described in detail in the technical solution of the device display control device, please refer to the description of the technical solution of the device display control method described above.

[0164] See Figure 6 , Figure 6 An architectural diagram of a device display control system according to an embodiment of this application is shown, including:

[0165] A first device 602 and at least one second device 604;

[0166] The first device 602 is configured to establish a communication connection with the at least one second device 604, and in response to feedback information from the at least one second device 604 regarding the target application, to create a virtual screen for each of the first device 602 and the at least one second device 604 respectively.

[0167] Upon detecting that the target application has been launched, a virtual window of the target application is created on the first virtual screen of the first device 602 based on the display position of the application window of the target application on the first device 602.

[0168] In response to a cross-device move command submitted for the virtual window, the virtual window is moved to a second virtual screen of a target second device 604, wherein the target second device 604 is one of the at least one second device 604 and is determined by the cross-device move command;

[0169] The display result of the virtual window on the second virtual screen is mapped to the device screen of the target second device 604.

[0170] Optionally, the at least one second device 604 includes a central device;

[0171] The first device 602 is also configured to establish a communication connection with the central device. When the target application is detected to be launched, a virtual window of the target application is created on the first virtual screen of the first device 602 based on the display position of the application window of the target application on the first device 602. The gesture control commands of the target user are collected through the image acquisition unit and the gesture control commands are sent to the central device.

[0172] The central device is configured to generate a cross-device movement instruction for the virtual window according to the gesture control instruction, and move the virtual window to the second virtual screen of the target second device 604 according to the cross-device movement instruction, wherein the target second device is one of the at least one second device 604 and is determined by the cross-device movement instruction;

[0173] The display result of the virtual window on the second virtual screen is mapped to the device screen of the target second device 604.

[0174] In this embodiment, by creating virtual screens for different devices and creating virtual windows in the virtual screen of the main control device, the application window can be moved across screens or devices by controlling the virtual windows to move between different virtual screens and then mapping the movement results to the real device screen. This improves the user's device experience and helps meet the user's application needs.

[0175] The above is a schematic scheme of a device display control system according to this embodiment. It should be noted that the technical solution of this device display control system and the technical solution of the device display control method described above belong to the same concept. For details not described in detail in the technical solution of the device display control system, please refer to the description of the technical solution of the device display control method described above.

[0176] Figure 7A structural block diagram of a computing device 700 according to an embodiment of this application is shown. The components of the computing device 700 include, but are not limited to, a memory 710 and a processor 720. The processor 720 is connected to the memory 710 via a bus 730, and a database 750 is used to store data.

[0177] The computing device 700 also includes an access device 740, which enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 740 may include one or more of any type of wired or wireless network interface (e.g., a Network Interface Card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) interface, a Wi-MAX interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0178] In one embodiment of this application, the aforementioned components of the computing device 700 and Figure 7 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 7 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this application. Those skilled in the art can add or replace other components as needed.

[0179] The computing device 700 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. The computing device 700 can also be a mobile or stationary server.

[0180] The processor 720 is configured to execute the following computer-executable instructions, wherein the processor executes the computer-executable instructions to implement the steps of the device display control method.

[0181] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the device display control method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the device display control method described above.

[0182] One embodiment of this application also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the device display control method.

[0183] The above is an illustrative embodiment of a computer-readable storage medium. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solution of the device display control method described above. Details not described in detail in the technical solution of the storage medium can be found in the description of the technical solution of the device display control method described above.

[0184] The foregoing has described specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0185] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0186] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of this application.

[0187] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0188] The preferred embodiments disclosed above are merely illustrative of this application. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments of this application. These embodiments are selected and specifically described in this application to better explain the principles and practical applications of the embodiments of this application, thereby enabling those skilled in the art to better understand and utilize this application. This application is limited only by the claims and their full scope and equivalents.

Claims

1. A device display control method, comprising: In response to feedback from at least two devices regarding a target application, a virtual screen is created for each of the at least two devices, wherein there is a communication connection between the at least two devices; If the target application is detected to be launched on the first device, a virtual window of the target application is created on the first virtual screen of the first device based on the display position of the application window of the target application on the first device, wherein the first device is one of the at least two devices; In response to a cross-device move command submitted for the virtual window, the virtual window is moved to a second virtual screen of a second device, wherein the second device is one of the at least two devices and is determined by the cross-device move command; The display result of the virtual window on the second virtual screen is mapped to the device screen of the second device.

2. The device display control method according to claim 1, wherein the first device comprises an image acquisition unit; Accordingly, before moving the virtual window to the second virtual screen of the second device in response to a cross-device move command submitted for the virtual window, the method further includes: The image acquisition unit acquires the target user's gesture control commands, wherein the gesture control commands carry the movement information of the virtual window; Generate cross-device movement instructions for the virtual window based on the gesture control instructions.

3. The device display control method according to claim 2, wherein acquiring the gesture control commands of the target user through the image acquisition unit includes: The image acquisition unit collects information on changes in the hand bones of the target user. The hand bone change information is analyzed, and gesture control commands for the target user are generated based on the analysis results.

4. The device display control method according to claim 2 or 3, wherein generating the cross-device movement command of the virtual window based on the gesture control command includes: Determine the first coordinates of the virtual window in the first virtual screen; Based on the movement information carried in the gesture control command, determine the second coordinates of the virtual window in the second virtual screen; Based on the first coordinate and the second coordinate, a cross-device movement command for the virtual window is generated.

5. The device display control method according to claim 1, wherein the first device includes a signal acquisition unit; Accordingly, before moving the virtual window to the second virtual screen of the second device in response to a cross-device move command submitted for the virtual window, the method further includes: The signal acquisition unit acquires the target remote control signal emitted towards the virtual window, wherein the target remote control signal carries the movement information of the virtual window; The virtual window is generated based on the target remote control signal to perform cross-device movement instructions.

6. The device display control method according to claim 5, wherein acquiring the target remote control signal emitted by the virtual window through the signal acquisition unit includes: The signal acquisition unit acquires the target remote control signal emitted by the remote control device towards the virtual window. Accordingly, generating the cross-device movement command for the virtual window based on the target remote control signal includes: The target remote control signal is parsed, and the feature code of the remote control device and the mapping relationship between the remote control commands corresponding to each button in the remote control device are determined based on the parsing results. Based on the mapping relationship, a target remote control command associated with the target remote control signal is determined; Generate cross-device movement instructions for the virtual window based on the target remote control instructions.

7. The device display control method according to claim 1, wherein the first device comprises an image acquisition unit; Accordingly, before moving the virtual window to the second virtual screen of the second device in response to a cross-device move command submitted for the virtual window, the method further includes: The image acquisition unit acquires the target user's location information and the target user's hand bone changes. The distance between the target user and each of the at least two devices is determined based on the location information; The cross-device movement command for the virtual window is determined based on the distance and the changes in the hand bones.

8. The device display control method according to claim 1, further comprising, before moving the virtual window to the second virtual screen of the second device in response to a cross-device movement command submitted for the virtual window: Determine whether the first device is the central device; If not, the gesture control commands of the target user are acquired through the image acquisition unit of the first device, and the gesture control commands are sent to the central device. The gesture control commands are used by the central device to generate cross-device movement commands for the virtual window.

9. The device display control method according to claim 1, wherein mapping the display result of the virtual window on the second virtual screen to the device screen of the second device comprises: According to the communication protocol between the first device and the second device, the display result of the virtual window on the second virtual screen is encoded, and the encoded result is sent to the second device. The encoded result is used by the second device for decoding, and according to the decoding result, the display result of the virtual window on the second virtual screen is displayed on the device screen of the second device.

10. The device display control method according to claim 1, wherein moving the virtual window to the second virtual screen of the second device comprises: Create a target virtual window for the target application in the second virtual screen of the second device; The display position of the target virtual window on the second virtual screen is determined according to the cross-device movement command; The target virtual window is displayed in the second virtual screen according to the display position.

11. A device display control system, comprising: A first device and at least one second device; The first device is configured to establish a communication connection with the at least one second device and, in response to feedback information from the at least one second device regarding the target application, create a virtual screen for each of the first device and the at least one second device. Upon detecting that the target application has been launched, a virtual window of the target application is created on the first virtual screen of the first device based on the display position of the application window of the target application on the first device. In response to a cross-device move command submitted for the virtual window, the virtual window is moved to a second virtual screen of a target second device, wherein the target second device is one of the at least one second device and is determined by the cross-device move command; The display result of the virtual window on the second virtual screen is mapped to the device screen of the target second device.

12. The device display control system according to claim 11, wherein the at least one second device includes a central device; The first device is also configured to establish a communication connection with the central device, and when the target application is detected to be launched, to create a virtual window of the target application on the first virtual screen of the first device based on the display position of the application window of the target application on the first device, to collect the gesture control commands of the target user through the image acquisition unit, and to send the gesture control commands to the central device. The central device is configured to generate cross-device movement instructions for the virtual window based on the gesture control instructions, and to move the virtual window to the second virtual screen of the target second device based on the cross-device movement instructions, wherein... The target second device is one of the at least one second device and is determined by the cross-device movement command; The display result of the virtual window on the second virtual screen is mapped to the device screen of the target second device.

13. A device display control apparatus, comprising: A creation module is configured to create a virtual screen for each of at least two devices in response to feedback information from at least two devices for a target application, wherein there is a communication connection between the at least two devices; The detection module is configured to, upon detecting that the target application is launched on the first device, create a virtual window of the target application on a first virtual screen of the first device based on the display position of the application window of the target application on the first device, wherein the first device is one of the at least two devices; A mobility module is configured to move the virtual window to a second virtual screen of a second device in response to a cross-device mobility command submitted for the virtual window, wherein the second device is one of the at least two devices and is determined by the cross-device mobility command; The mapping module is configured to map the display result of the virtual window on the second virtual screen to the device screen of the second device.

14. A computing device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, wherein when the processor executes the computer-executable instructions, it implements the steps of the device display control method according to any one of claims 1-10.

15. A computer-readable storage medium storing computer instructions that, when executed by a processor, implement the steps of the device display control method according to any one of claims 1-10.

Images