Method for cross-device interaction and electronic device

Abstract

The application provides a cross-device interaction method and an electronic device, and relates to the terminal field. The application provides a cross-device interaction method, which is applied to a first electronic device. The first electronic device and a second electronic device are in a heterogeneous screen projection state. The first electronic device is a source end of screen projection, and the second electronic device is a receiving end of screen projection. After a window manager of the first electronic device receives a first setting instruction, when it is determined that a target screen does not obtain a focus and a separate focus needs to be set, the window manager sets the focus for the target screen and does not set a loss of focus for other screens that obtain the focus. By using the method in the embodiment of the application, in the heterogeneous screen projection scene, during the interaction process between the source end of screen projection and the receiving end of screen projection, the situation that the touch control area of the source end occupies the focus and causes screen projection abnormity can be avoided, and the accuracy of the interaction between the source end and the receiving end is improved.

Description

Technical Field

[0001] This application relates to the field of terminals, and more particularly to a method and electronic device for cross-device interaction. Background Technology

[0002] With the continuous development of technology, wireless screen mirroring technology has emerged. Through screen mirroring, users can project the content displayed on a first electronic device to another device (the receiving end) for display. The first electronic device is one with screen mirroring functionality. After the first electronic device performs the screen mirroring operation, the user can interact with the receiving end through it. For example, using a mobile phone as the source and a TV as the receiver, the mobile phone responds to the user's screen mirroring operation, projecting its displayed content (such as video A) to the TV for display. After video A is displayed on the TV, the user can control the TV to play video A through their mobile phone.

[0003] However, in scenarios involving screen mirroring from different sources, abnormal interactions may occur between the source and receiver. For example, when the receiver displays a game application from the source, the game may pause when the source tries to control the game object displayed on the receiver. Summary of the Invention

[0004] To address the aforementioned technical issues, this application provides a method and electronic device for cross-device interaction, which enables the source and receiver to avoid screen projection errors caused by the touch area of ​​the source device preempting the focus during interaction in heterogeneous screen projection scenarios, thereby improving the accuracy of interaction between the source and receiver.

[0005] In a first aspect, embodiments of this application provide a method for cross-device interaction, applied to a first electronic device, wherein the first electronic device and a second electronic device establish a heterogeneous projection connection, the first electronic device being the projection source and the second electronic device being the projection receiver. The method includes: when the window manager of the first electronic device receives a first setting instruction, determining whether the target screen has acquired focus, the first setting instruction being used to instruct the window manager to set the target screen to acquire focus; when the window manager detects that the target screen has not acquired focus, determining whether the target screen needs to be set to focus separately; when the window manager determines that the target screen needs to be set to focus separately, setting the target screen to acquire focus, and the window manager not performing the operation of setting the first screen to lose focus, wherein the first screen is the screen in the first electronic device that has acquired focus other than the target screen.

[0006] For example, the first electronic device can be a device with a remote projection function, such as a mobile phone, tablet computer, or laptop computer. The second electronic device can be a device with a large screen, such as a television or smart screen.

[0007] For example, the source end of screen mirroring is the device sending the mirrored content, and the receiving end is the device receiving and displaying the mirrored content. In a heterogeneous screen mirroring scenario, the application being mirrored runs on a virtual screen at the source end. The source end transmits the virtual screen to the receiving end for display, but the virtual screen is not displayed at the source end. In other words, the content displayed at the source end and the content displayed at the receiving end can be different.

[0008] For example, the target screen can be the default screen of the first electronic device, which can be understood as the container on which the first electronic device is currently running on the display screen, where views and touchpads can run. The target screen can also be a virtual screen.

[0009] For example, the first screen is a screen in the first electronic device that receives focus other than the target screen. If the target screen is the default screen, the first screen is a virtual screen.

[0010] In this way, when the window manager of the first electronic device determines that the target screen has not acquired focus and needs to be set to focus separately, the window manager sets the target screen to acquire focus and does not perform the operation of the first screen losing focus. This allows the target screen and the first screen to have their own focus at the same time, avoiding the situation where the target screen steals the focus of the first screen. This also avoids the problem of the projected application running abnormally due to the focus being stolen from the first screen (such as pause problems in game applications, login information input errors in learning applications, etc.), improving the accuracy and smoothness of the interaction between the first and second electronic devices during the projection process.

[0011] According to the first aspect, when the window manager detects that the target screen has not acquired focus, determining whether the target screen needs to be set to a separate focus includes: the window manager sending a query command to the screen mirroring module of the first electronic device; the screen mirroring module responding to the query command querying whether the first electronic device is in screen mirroring state and whether there is a running application on the virtual screen of the first electronic device, and obtaining the query result; when the query result indicates that the first electronic device is in screen mirroring state and there is a running application on the virtual screen, determining that the target screen needs to be set to a separate focus; when the query result indicates that the first electronic device is not in screen mirroring state, or indicates that the first electronic device is in screen mirroring state and there is no running application on the virtual screen, determining that the target screen does not need to be set to a separate focus.

[0012] For example, the screen mirroring module of the first electronic device records information about whether the first electronic device is in screen mirroring mode. The screen mirroring module can obtain information about whether the first electronic device is currently in screen mirroring mode in real time.

[0013] For example, wireless screen mirroring includes same-source mirroring and different-source mirroring. The mirroring module only records whether the first electronic device is in mirroring mode, and does not record whether the first electronic device is in a same-source mirroring or different-source mirroring scenario. Because in different-source mirroring, the application being mirrored runs on the virtual screen of the first electronic device; while in the same-source mirroring scenario, the first electronic device does not create a virtual screen for the application being mirrored to run; based on this, the mirroring module can determine whether the first electronic device is in a different-source mirroring state by detecting whether an application is running on the virtual screen when the first electronic device is in mirroring mode. This determination method is simple and accurate.

[0014] In this way, when the screen is projected from a different source, the default screen of the first electronic device is prone to preempting the focus. When the query result returned by the projection module indicates that the first electronic device is in projection mode and there is an application running on the virtual screen, the window manager of the first electronic device can determine that the first electronic device is currently in projection mode. This method of determining whether a separate focus needs to be set is simple and fast; it can avoid the problem of setting a separate focus for the target screen when no separate focus needs to be set.

[0015] According to the first aspect, the screen mirroring module responds to a query command by querying whether the first electronic device is in screen mirroring mode and whether there is a running application on the virtual screen of the first electronic device, and obtains a query result, including: the screen mirroring module responds to the query command by detecting whether the first electronic device is in screen mirroring mode and obtaining a first result; when the first result indicates that the first electronic device is in screen mirroring mode, the screen mirroring module detects whether there is a running application on the virtual screen and obtains a second result; the screen mirroring module uses the first result and the second result as the query result; when the first result indicates that the first electronic device is not in screen mirroring mode, the first result is used as the query result.

[0016] In this way, the screen mirroring module decides whether to further determine whether there is a running application on the virtual screen based on the first result. When the first result indicates that the first electronic device is not in the screen mirroring module, there is no need to perform subsequent judgments, reducing the number of judgment steps and speeding up the acquisition of query results.

[0017] According to the first aspect, when a first application of a first electronic device is projected onto a second electronic device, the method further includes: the first electronic device displaying a first interface of an interactive application, the first interface including a first touch area; the first touch area responding to a first touch operation input by a user, updating the target screen to the default screen of the first electronic device and obtaining a first touch event corresponding to the first touch operation; the first touch area generating a first setting instruction based on information from the target screen; and the first touch area sending the first setting instruction to a window manager.

[0018] For example, the first application is an application in the first electronic device, such as a parkour game application, or an application that requires a login password or username.

[0019] For example, when a first application is projected onto a second electronic device, the first electronic device can interact with the second electronic device through an interactive application. This interactive application could be an air mouse remote control, such as... Figure 2c As shown in (2), the air mouse remote control page 103 is the first interface, which includes a sliding click area (i.e., the first touch area). The air mouse remote control page may also include other spaces, such as sound adjustment controls.

[0020] For example, the first touch operation includes touch operations on the first touch area, such as swiping, clicking, multi-finger (such as two-finger) operations, etc.

[0021] In this way, the first touch area operates on the default screen. When the first touch area receives the first touch operation input by the user, the first touch area can obtain the touch event corresponding to the first touch operation and determine that the target screen is the default screen. The target screen can then be updated to the default screen, and a first setting instruction can be generated based on this. In other words, the first setting instruction contains the information of the target screen so that the window manager can accurately set the focus for the target screen.

[0022] According to the first aspect, after the default screen gains focus, the method further includes: a first touch area sending a first touch event to a second electronic device, and the second electronic device generating a counter-control event based on the first touch event when it detects that the second electronic device is in a projection state; a first application responding to the counter-control event injected by the second electronic device updating the target screen to the virtual screen of the first electronic device; the first application generating a new first setting instruction based on the information of the target screen; the first application sending the first setting instruction to the window manager; after the virtual screen gains focus, the method further includes: the first application performing an operation corresponding to the counter-control event based on the counter-control event.

[0023] For example, the first touch area sends a first touch event to the second electronic device. When the second electronic device detects that it is in projection mode, it generates a reverse control event based on the first touch event and injects the reverse control event into the first application. When the first application receives the reverse control event, since it is running on a virtual screen, it updates the target screen to the virtual screen and generates a new first setting instruction. This allows the window manager to determine whether the virtual screen has gained focus upon receiving the first setting instruction. If the virtual screen does not have focus, it determines whether it needs to be set to focus separately. If it determines that the virtual screen needs to be set to focus separately, it sets the virtual screen to gain focus and does not perform the operation of setting the first screen to lose focus. After the virtual screen gains focus, the first application can execute the operation corresponding to the reverse control event.

[0024] Once the window manager determines that the virtual screen has gained focus, it returns the detection result to the first application. Based on the detection result, the first application determines that the virtual screen has gained focus and then performs the operation corresponding to the control event.

[0025] In this way, when the first application receives a control event, it can determine that the target screen is a virtual screen and update it promptly, avoiding incorrect window manager settings. Furthermore, when the window manager receives the first setting command, it can determine whether the target screen, which has not yet received focus, needs a separate focus setting, ensuring that both the target screen and the first screen have their respective focus, thus preventing focus contention.

[0026] According to the first aspect, after the default screen gains focus, the method further includes: a first touch area sending a first touch event to a first application; the first application responding to the first touch event updating the target screen to a virtual screen of the first electronic device; the first application generating a new first setting instruction based on information from the target screen; the first application sending the first setting instruction to a window manager; and after the virtual screen gains focus, the method further includes: the first application performing an operation corresponding to the first touch event based on the first touch event.

[0027] In this way, after the first touch area gains focus on the default screen, it can send the first touch event to the first application. The first application, responding to the first touch event, can promptly change the target screen to a virtual screen. Furthermore, because the first touch area directly sends the first touch event to the first application, the speed at which the first application responds to the first touch event is improved.

[0028] According to the first aspect, the method further includes: when the window manager determines that the target screen does not need to be set to a separate focus, setting the target screen to gain focus; and setting the first screen to lose focus. In this way, setting the first screen to lose focus when the target screen does not need to be set to a separate focus avoids the problem of multiple screens having their own focus, which could lead to application malfunctions.

[0029] According to the first aspect, when a first application of a first electronic device is projected onto a second electronic device, the method further includes: the first electronic device displaying a first interface of the interactive application, the first interface including a first adjustment control; in response to a user's activation operation of the first adjustment control, displaying a first adjustment box on the first interface, the first adjustment box including at least two different cursor tracking speeds; the interactive application, in response to a user's selection operation on the first adjustment box, obtaining a mapping value corresponding to the cursor tracking speed selected by the user as a target mapping value, the mapping value being a proportional relationship between the distance the first electronic device moves in space and the distance the cursor moves in the display interface of the second electronic device; and the interactive application using the target mapping value.

[0030] For example, after the first electronic device projects the first application onto the second electronic device, the first electronic device can interact with the second electronic device through an interactive application, such as an air mouse or remote control. Figure 10 The first adjustment control is as follows: Figure 10 The control 1002 shown in (1) can be activated by the user through a click operation (such as a single click). In other embodiments, the activation operation can also be other operations, such as a sliding operation.

[0031] For example, the first adjustment box is as follows Figure 10 The first adjustment box shown in (2) is 1002-1; it can also be displayed as a floating window.

[0032] For example, the first adjustment box includes at least two different cursor tracking speeds, each with its own corresponding control for the user to select, as shown in the reference. Figure 10 As shown in the first adjustment box 1002-1 in (2), the first adjustment box 1002-1 includes 10 different cursor tracking speeds. The user can select the desired cursor tracking speed by sliding.

[0033] Thus, the cursor tracking speed is related to the mapping value, which is the ratio between the distance the first electronic device moves in space and the distance the cursor moves in the display interface of the second electronic device. In this example, different cursor tracking speeds correspond to different mapping values. When the user selects a cursor tracking speed in the first adjustment box, the air mouse remote control obtains the mapping value corresponding to the selected cursor tracking speed as the target mapping value and applies it. In this example, the user can adjust the cursor tracking speed as needed, improving the flexibility of user interaction between the first and second electronic devices.

[0034] According to the first aspect, when a first application of a first electronic device is projected onto a second electronic device, the method further includes: the first electronic device displaying a first interface of an interactive application, the first interface including a first touch area; the first touch area responding to a second touch operation input by a user in the first touch area, gaining focus and a second touch event corresponding to the second touch operation, the second touch operation including holding a press operation on the first touch area for a first preset time period and controlling the first electronic device to move in space for the first preset time period; the second touch event including the movement trajectory of the first electronic device in space; when the first application receives the second touch event, controlling the cursor in the display interface of the second electronic device to move according to the movement trajectory.

[0035] Thus, the second touch operation controls the movement of the first electronic device while pressing the first touch area (e.g., Figure 8b As shown in the image, this allows the cursor on the second electronic device to be dragged. This increases the diversity of interaction between the first and second electronic devices, expanding the applicable scenarios and scope of screen mirroring applications.

[0036] According to the first aspect, when a first application of a first electronic device is projected onto a second electronic device, the method further includes: the first electronic device displaying a first interface of an interactive application, the first interface including a first touch area; the first touch area responding to a third touch operation input by a user in the first touch area, gaining focus and a third touch event corresponding to the third touch operation; when the first application receives the third touch event, performing a grabbing operation on a target object in the display interface of the second electronic device.

[0037] In this way, the third touch operation can be a grasping gesture operation, such as... Figure 8a The gesture operation shown in (2) is as follows. In the scenario of heterogeneous screen projection, the grab function is added to the interactive application, which increases the interaction function between the first electronic device and the second electronic device, improves the flexibility of the interaction, and increases the applicable scenarios and scope of the screen projection application.

[0038] According to the first aspect, when a first application of the first electronic device is projected onto the second electronic device, the method further includes: in response to a user tilting the first electronic device, the first application acquires gravity sensing information of the first electronic device; the first application controls a target object in the display interface of the second electronic device to tilt in the tilt direction of the first electronic device according to the gravity sensing information of the first electronic device and a preset gravity sensing mapping, wherein the gravity sensing mapping is a proportional relationship between the tilt angle of the first electronic device in space and the tilt angle of the target object in the display interface of the second electronic device.

[0039] For example, gravity sensing information includes information about the tilt angle of the first electronic device.

[0040] In this way, the first electronic device controls the tilt of the target object on the display interface of the second electronic device through gravity sensing, which increases the fun of interaction and control methods.

[0041] In a second aspect, this application provides an electronic device, including: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory, and when the computer programs are executed by the one or more processors, the electronic device performs the method of cross-device interaction corresponding to the first aspect and any implementation thereof.

[0042] The second aspect and any implementation thereof correspond to the first aspect and any implementation thereof, respectively. The technical effects of the second aspect and any implementation thereof are similar to those of the first aspect and any implementation thereof, and will not be repeated here.

[0043] Thirdly, this application provides a computer-readable medium for storing a computer program that, when run on an electronic device, causes the electronic device to perform the cross-device interaction method corresponding to the first aspect and any implementation thereof. Attached Figure Description

[0044] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1a This is an example of a screen mirroring application scenario;

[0046] Figure 1bThis is an example of a heterogeneous projection application scenario;

[0047] Figures 2a-2b This is an illustrative diagram illustrating the application of heterogeneous projection technology in smart homes.

[0048] Figure 2c This is an illustrative diagram illustrating an anomaly that occurs during the application of heterogeneous projection technology.

[0049] Figures 3a-3d This is an example illustrating the interaction process between a mobile phone and a TV in a heterogeneous projection scenario;

[0050] Figure 4 This is an exemplary hardware structure diagram of an electronic device;

[0051] Figure 5 This is an exemplary software structure block diagram of an electronic device;

[0052] Figures 6a-6c This is an example illustrating the interaction process between a mobile phone and a TV in a heterogeneous screen mirroring scenario;

[0053] Figure 7 This is an illustrative diagram showing the interaction between a mobile phone and a TV in a heterogeneous projection scenario.

[0054] Figure 8a This is an illustrative diagram showing how a mobile phone captures a target object displayed on a TV in a scenario of screen mirroring from a different source.

[0055] Figures 8b-8c This is an illustrative diagram showing the dragging effect of the cursor displayed on the TV when controlled by a mobile phone in a scenario of screen mirroring from a different source.

[0056] Figures 9a-9b This is an exemplary diagram illustrating another scenario where a cursor displayed on a TV is dragged by a mobile phone in a different source projection scenario.

[0057] Figure 10 This is an exemplary diagram illustrating how a user adjusts cursor tracking speed in a heterogeneous projection scenario. Detailed Implementation

[0058] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0059] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0060] The terms "first" and "second," etc., used in the specification and claims of this application are used to distinguish different objects, not to describe a specific order of objects. For example, "first target object" and "second target object," etc., are used to distinguish different target objects, not to describe a specific order of target objects.

[0061] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0062] In the description of the embodiments in this application, unless otherwise stated, "multiple" means two or more. For example, multiple processing units means two or more processing units; multiple systems means two or more systems.

[0063] With the continuous development of screen mirroring technology, more and more devices are equipped with screen mirroring functionality to provide users with screen mirroring services. Screen mirroring technology includes same-source screen mirroring and different-source screen mirroring. In same-source screen mirroring scenarios, the source device directly transmits the currently displayed interface to the receiving device for display. The effect displayed on the receiving device is consistent with the display effect on the source device; that is, the content displayed on the receiving device is the same as the content displayed on the source device.

[0064] The following is combined with Figure 1a This section explains the application scenarios for screen mirroring from the same source. The source end is typically a mobile phone, and the receiving end is typically a TV. For example, refer to... Figure 1a Mobile phones and televisions can establish and maintain a wireless connection through wireless communication technology. When the mobile phone and television are on the same local area network, the mobile phone can initiate screen mirroring to the television. Figure 1aAs shown, in a same-source screen mirroring scenario, the phone's display shows the phone's main page 101, which includes icons for different applications. When the phone transmits the currently displayed main page 101 to the TV, the TV's display shows the mirrored content 201. This mirrored content is exactly the same as the phone's currently displayed main page 101. In other words, in a same-source screen mirroring scenario, when the source device mirrors to the receiving device, the display information on the receiving device is synchronized with the foreground display information on the source device. Same-source screen mirroring uses an extended screen method to mirror the application interface launched on the source device to the receiving device.

[0065] In heterogeneous projection scenarios, the projection source creates a virtual display with the same screen resolution ratio and dots per inch (DPI) as the projection receiver. The projection source then renders the projection image onto the virtual display and transmits it to the projection receiver for display. The content displayed on the projection source can differ from the content displayed on the projection receiver.

[0066] The following is combined with Figure 1b This section describes the application scenarios for heterogeneous screen mirroring. The source end is typically a mobile phone, and the receiving end is typically a television. For example, refer to... Figure 1b A mobile phone and a TV can establish and maintain a wireless connection via wireless communication technology. When the mobile phone and TV are on the same local area network, the mobile phone can initiate screen mirroring to the TV. In this example, the application being mirrored on the mobile phone is a game application. In the screen mirroring scenario, the mobile phone creates a virtual screen. After the game application is launched on the mobile phone, the game application runs on the virtual screen (which is not displayed on the mobile phone) and transmits the virtual screen to the TV for display, such as... Figure 1b As shown, the screen display on the TV shows the interface of a game application cast from the phone (201). After launching the game application, the phone can switch to other interfaces, such as... Figure 1b As shown, when the phone switches to its home screen 101, the content displayed on the TV is different from the content currently displayed on the phone. This means that in a cross-source screen mirroring scenario, the content displayed on the receiving end can be out of sync with the content displayed on the source end.

[0067] Applying heterogeneous projection technology to smart homes enables interconnectivity among smart devices. For example, a "wireless projection" service can project a mobile application onto a TV, displaying the application in a format adapted to the TV's screen. The mobile phone can then interact with the TV using interactive applications (such as an air mouse remote). The following section will illustrate this further. Figure 2a and Figure 2b This section provides a detailed explanation of the application of heterogeneous projection technology in smart homes.

[0068] In this example, smart home devices can include televisions, smart screens, air conditioners, water heaters, speakers, etc. In this example, the source of the screen projection is a mobile phone, and the receiver is a smart home device with a screen, such as a television or smart screen. In this example, the receiver is a television.

[0069] In one example, the mobile phone and the television are on the same local area network (LAN). This LAN can be a wireless communication network, such as a wireless local area network (WLAN), wireless fidelity point-to-point (Wi-Fi P2P), Bluetooth, Zigbee, infrared (IR), or near field communication (NFC) network. For example, the aforementioned LAN can also be a wired communication network, such as a communication network established by a video graphics array (VGA), digital visual interface (DVI), high definition multimedia interface (HDMI), or data transmission cable. As another example, the aforementioned LAN can also be a remote network.

[0070] Once a mobile phone and a television have established mutual trust and recognition, the system formed by the mobile phone and the television is called a trust loop. Furthermore, the process of mutual trust and recognition between devices can refer to relevant technologies. For example, it can be determined based on one or more factors such as whether the devices are logged into the same system account, whether authorization has been granted, and whether near-field communication is used. Specific details will not be elaborated upon here.

[0071] Understandably, within the same trust ring, multiple electronic devices can securely share data. This shared data includes, but is not limited to, message notifications, message logs, and user information from other electronic devices, device types, the running status of various applications, application identifiers, and synchronization of recently used application identifiers. Thus, within the trust ring, cross-platform screen mirroring services can be implemented.

[0072] Reference Figure 2aIn (1), after the mobile phone and the TV are in the same trust ring, the TV can send a request to the mobile phone to enable "wireless screen casting" so that the mobile phone application can be shared with the TV. The mobile phone responds to the request from the TV and displays a prompt box 102 on the current display interface 101. The prompt box 102 includes a prompt message, an enable control 1021, and a cancel control 1022. The prompt message in the prompt box 102 prompts the user to enable the mobile phone's "wireless screen casting" service. The enable control 1021 enables "wireless screen casting," and the cancel control 1022 cancels enabling "wireless screen casting." The mobile phone responds to the user clicking the enable control 1021 and starts the "wireless screen casting" service. The display interface jumps to the air mouse remote control page 103 in the smart space, as shown in the image. Figure 2a (2) The air mouse remote control page 103 includes a button control area and a sliding click area 104. The button control area includes buttons with different functions, such as a composite button, volume up button, volume down button, mute button, input source switch button, menu button, home button, and back button. Among them, the composite button includes a power button and a screenshot button. Users can perform different controls on the TV by clicking different buttons. For example, if a user clicks the mute button on the air mouse remote control page 103, the user can control the TV to perform a mute operation. Users can also control the TV by sliding click area 104, such as clicking and sliding.

[0073] After the "Honor Air Visualization" service is activated, apps on your phone can be shared to your TV. Users can then use the air mouse on their phone to open the shared apps on the TV. For example, the TV screen might display shared apps such as: Game 1, Video 1, and English Learning 1; the user can then use the air mouse on their phone to open Game 1 on the TV.

[0074] Figure 2b The illustration shows different scenarios of using an air mouse remote to control a television.

[0075] When the phone is shaken, the air mouse remote control reads the offset value of the phone's gyroscope to map the position and distance of the cursor on the TV screen, thus achieving the movement and positioning of the cursor on the TV screen. (See reference...) Figure 2b In (1), when the mobile phone is in position 1, the TV cursor 203 in TV 202 is located at point A on the TV screen. When the mobile phone moves from position 1 to position 2, the TV cursor 203 in TV 202 moves from the corresponding position A to point B. When the mobile phone moves from position 2 to position 3, the TV cursor 203 in TV 202 moves from point B to point C.

[0076] When a user clicks or double-clicks on the sliding click area 104 of the air mouse remote control, they can control the TV cursor on the television to perform a selection operation. (See reference...) Figure 2b In (2), when the TV cursor 204 is at point D, the user can control the TV to perform the selected operation by clicking in the sliding click area 104 of the mobile phone.

[0077] When a user performs a swipe operation on the remote control's sliding area 104 on their mobile phone, they can control the scrolling of the page displayed on the TV. (See reference...) Figure 2b In (3), when the user slides in the sliding click area in the direction indicated by the arrow, the display page 205 on the TV also slides synchronously in the direction of the user's operation.

[0078] For example, a user can input a back gesture on the left or right side of the boundary of the swipe-tap area, and the corresponding display page on the TV will perform a back operation (this operation is not shown in the image). Figure 2b (as shown in the image).

[0079] Currently, "wireless screen mirroring" allows users to share apps from their phones to their TVs and then use the air mouse on their phones to control the shared apps on the TV. However, this method can sometimes lead to problems when controlling the shared apps from the phone.

[0080] Reference Figure 2c In (1), after the "wireless screen mirroring" service is enabled on the mobile phone, the user opens the shared parkour game (referred to as Game 1) on the TV using an air mouse remote control. The TV's display interface 210 displays the interface of Game 1. Figure 2c As shown in (2), the air mouse remote control page 103 of the mobile phone includes a sliding click area 104. The user can input interactive operations in the sliding click area 104 to control the game character in game 1 to move left and right, etc. At time T1, the display interface 210 of the TV indicates that the game character moves to the right. At time T1, the user performs a right swipe operation in the sliding click area 104 of the mobile phone; at the next time (e.g., Figure 2c (3) As shown at time T2), the TV screen displays that the game is paused, which means that the game paused during the interaction between the user and the TV via mobile phone.

[0081] The following is combined with Figures 3a-3d This section provides a detailed explanation of the interaction process between a mobile phone and a TV in a heterogeneous screen mirroring scenario.

[0082] Step 301: User inputs touch operation.

[0083] For example, continue to refer to the above. Figure 2cIn the application scenario, let's take game 1 as an example of screen mirroring application A. After the "wireless screen mirroring" service is activated, the user opens the application shared from their mobile phone to the TV using the air mouse remote control of the smart space, and controls game 1 that has been shared to the TV using the air mouse remote control.

[0084] As can be understood, a touchpad refers to the area in interactive applications (such as air mice and remote controls) used to receive touch input. Figure 2c The sliding and clicking area 104 in (2) is the touchpad. Touch operations include clicking / double-clicking, sliding, etc. When "wireless casting" is enabled (i.e., casting from a different source), the TV can interact with the mobile phone through the merged desktop.

[0085] In this example, the user inputs a touch operation (such as...) within the swipe and tap area of ​​the phone. Figure 2c (2) shows the right swipe operation. The touchpad obtains focus and corresponding touch event based on the received touch operation.

[0086] Step 302: The touchpad gains focus and touch events occur.

[0087] In this example, the user is on the phone's touchpad (such as...). Figure 2c The touchpad receives the user's touch input in the sliding click area 104 shown in Figure (2), and obtains focus and the corresponding touch event based on the touch operation. The detailed process of the touchpad obtaining focus and touch event is as follows: Figure 3b As shown, this specifically includes steps 3021 to 3024.

[0088] Step 3021: The touchpad receives a touch event.

[0089] For example, in response to a user's touch input, the touchpad obtains the touch event corresponding to the touch input and determines that the default display has focus. The default display can be understood as the container currently running on the phone's screen, where views and the touchpad can run. Since the touchpad runs on the default display, when the touchpad receives a touch input, it can be determined that the default display where the touchpad resides has focus.

[0090] For example, refer to Figure 2c The sliding click area 104 shown in (2) is a touchpad. The sliding click area 104 runs on the default screen. When the user inputs a touch operation on the sliding click area 104, the touchpad determines that the default screen has focus. The touchpad obtains the touch event corresponding to the received touch operation.

[0091] Step 3022: The touchpad instructs the window manager to give focus to the default screen.

[0092] For example, after receiving a touch event, since the touchpad is running on the default screen, it determines that the target screen to be set is the default screen and generates a first instruction. The touchpad sends the first instruction to the window manager, which instructs the window manager to set the default screen to receive focus.

[0093] Step 3023: Set the default screen to receive focus in the window manager.

[0094] For example, in response to the first instruction, the window manager sets the default screen to gain focus.

[0095] Step 3024: The window manager instructs you to set the virtual screen to lose focus.

[0096] For example, in response to the first instruction, the window manager traverses the screens running on the phone other than the default screen. This traversal includes determining whether a screen has focus. Once the window manager determines that a virtual screen has focus through the traversal, it sets the virtual screen to lose focus.

[0097] After steps 3021 to 3024, the screen gains focus and touch events by default, which means the touchpad gains focus and touch events.

[0098] Step 303: The touchpad sends a touch event to the converged desktop on the TV.

[0099] For example, after the default screen gains focus, the touchpad sends the touch event to the converged desktop on the TV. See [link to detailed procedure] for further information. Figure 3c This will not be elaborated upon here.

[0100] Step 304: The converged desktop on the TV responds to touch operations and injects a reverse control event into the screen mirroring application A on the mobile phone.

[0101] For example, after receiving a touch operation, the converged desktop detects whether the TV is currently in screen mirroring mode. When the converged desktop determines that the TV is currently in screen mirroring mode, it injects a reverse control event into the screen mirroring application A on the mobile phone. For details of this step, please refer to [link to relevant documentation]. Figure 3c This will not be elaborated upon here.

[0102] Step 305: The screen mirroring application A on the phone responds to the event and regains focus.

[0103] For example, the screen mirroring application A on the mobile phone, based on the control event, instructs the window manager to set the virtual screen to regain focus, and performs the corresponding operation based on the feedback event. The specific process of this step can be found in [reference needed]. Figure 3d .

[0104] For detailed procedures of steps 303 to 304 above, please refer to [link / reference]. Figure 3c The specific process includes:

[0105] Step 401: The touchpad gains focus and touch events occur.

[0106] For detailed instructions on this step, please refer to [link / reference]. Figure 3b Steps 3021 to 3024 shown will not be repeated here.

[0107] In this example, in a heterogeneous screen mirroring scenario, the source device is a mobile phone, which includes a touchpad, a screen mirroring application, an input framework, an airsharing transmitter, and a Magic-Link transmitter. The receiving device is a TV, which includes a Magic-Link receiver, an airsharing receiver, a converged desktop tool (i.e., converged desktop), an air gesture application, and a TV system framework. The TV system framework can listen for system events, such as touch operations.

[0108] Step 402: The touchpad transmits the touch event to the TV system framework.

[0109] For example, in a scenario of screen projection from a different source, once the touchpad acquires focus and a touch event, it can send the touch event to the TV system framework of the TV.

[0110] Step 403: The TV system framework sends a touch event to the TV's AirView application.

[0111] For example, the TV system framework can listen for touch events sent by the touchpad. When the TV system framework detects that the TV is in screen casting mode, it sends the received touch events to the TV's AirView application, which can be understood as an application used during screen casting events.

[0112] Step 404: The TV's AirView application sends touch events to the Fusion Desktop tool.

[0113] Step 405: The TV's converged desktop tools encapsulate touch events and put them into a queue.

[0114] For example, converged desktop tools on a TV (i.e. Figure 3aThe Fusion Desktop tool encapsulates touch events into JSON data (hereinafter referred to as encapsulated data). This encapsulated data is then placed into a message queue. In this example, touch events are encapsulated into JSON data for easy data transmission. Furthermore, the encapsulated data is transmitted to the phone via the message queue, eliminating the need for interdependence between the phone and TV systems.

[0115] Step 406: The TV's converged desktop tool sends touch events to the distributed mobilesensing development platform (DMSDP) receiver.

[0116] For example, after receiving a packaged touch event, the TV's converged desktop tool sends the touch event to a DMSDP receiver, where the DMSDP receiver is used to virtualize the device.

[0117] Step 407: The DMSDP receiver sends a touch event to the Airsharing receiver.

[0118] For example, after receiving a touch event, the DMSDP receiver transmits the touch event to the Airsharing receiver.

[0119] Step 408: The Airsharing receiver parses the touch event and encapsulates it into a UIBC control message.

[0120] For example, the Airsharing receiver receives touch events in JSON format and parses the touch events.

[0121] The Airsharing receiver encapsulates the parsed touch events into User Input Back Channel (UIBC) back-control messages. UIBC can transmit user input from the receiver to the sender.

[0122] Step 409: The Airsharing receiver sends the reverse control message as binary data to the Magic-Link receiver.

[0123] For example, the receiving end (i.e., the television) can transmit the control message to the source end through data transmission technology. In this example, the data transmission technology is Magic-Link. The Airsharing receiver can transmit the control message as binary data to the Magic-Link receiver, so that the Magic-Link receiver can send the control message to the Magic-Link transmitter on the mobile phone.

[0124] Step 410: The Magic-Link receiver sends reverse control data to the Magic-Link transmitter.

[0125] For example, the Magic-Link receiver sends binary counter-control messages (i.e. counter-control data) to the Magic-Link transmitter on the mobile phone.

[0126] Step 411: The Magic-Link transmitter transmits control data to the Airsharing transmitter.

[0127] For example, the phone's Magic-Link transmitter transmits reverse control data (i.e., binary reverse control messages) to the phone's Airsharing transmitter.

[0128] Step 412: The Airsharing sender parses the reverse control data and obtains the UIBC reverse control message.

[0129] For example, after receiving the reverse control data, the Airsharing sender parses the reverse control data to obtain the UIBC reverse control message.

[0130] Step 413: The Airsharing sender parses the UIBC reverse control message.

[0131] For example, the Airsharing sender parses the UIBC control message and obtains the corresponding touch event from the control message.

[0132] Step 414: The Airsharing sender injects the touch event into the Input framework.

[0133] Step 415: The phone's Input framework distributes the touch event to the corresponding screen mirroring application A.

[0134] For example, after receiving a touch event, the phone's Input framework distributes the touch event to the corresponding screen mirroring application A (such as...). Figure 2c (Game applications in China).

[0135] For example, after receiving a touch event dispatched by the Input framework, screen mirroring application A responds to the event by instructing the window manager to set the virtual screen to regain focus. After the virtual screen regains focus, application A processes the touch event. For instance, the touch event might be a click on control A, and the corresponding operation is a page jump to page B. After the virtual screen gains focus, screen mirroring application A performs the switch from the current page to page B.

[0136] Figure 3aThe detailed process of step 305, where the screen-casting application A responds to the event and regains focus, can be found in [reference needed]. Figure 3d Specifically, it includes:

[0137] Step 3051: In response to the injected reverse control event, screen mirroring application A instructs the window manager to gain focus from the virtual screen.

[0138] For example, in response to a control event injected by the Input framework, screen mirroring application A determines that the virtual screen running screen mirroring application A has regained focus and generates a second instruction. Screen mirroring application A sends the second instruction to the window manager, which instructs the virtual screen to regain focus.

[0139] Step 3052: The window manager sets the virtual screen to receive focus.

[0140] For example, the window manager responds to the second instruction by setting the virtual screen to focus.

[0141] Step 3053: Window Manager sets the default screen to lose focus.

[0142] For example, in response to the second instruction, the window manager iterates through all screens running on the phone except for the virtual screen. This iteration includes determining whether a screen has focus. Once the window manager determines that the default screen has focus through this iteration, it sets the default screen to lose focus.

[0143] Therefore, in heterogeneous screen mirroring scenarios, the mirroring application runs on a virtual screen. When the application needs to maintain focus on the virtual screen, the default screen may preempt focus, causing the focus to switch back and forth between the default and virtual screens. This can lead to screen mirroring errors (such as...) Figure 2c As shown in the image, when a user inputs commands to control a game character on their phone, the screen mirroring application pauses, affecting the user experience.

[0144] This application provides a method for cross-device interaction. In a heterogeneous projection scenario, when the window manager in the first electronic device (i.e., the projection source) receives an instruction to set focus for a target screen, it determines whether the target screen has acquired focus. If the target screen does not have focus, it determines whether the target screen needs a separate focus. If the window manager determines that the target screen needs a separate focus, it sets that focus for the target screen, ensuring that other screens with focus do not lose focus. This embodiment of the application uses a window manager to set a separate focus, enabling the first electronic device to support multiple screens with their own corresponding focus. This avoids focus contention between currently running screens on the electronic device, thus preventing projection anomalies caused by repeated focus switching in heterogeneous projection scenarios.

[0145] The first electronic device in this application embodiment can be an electronic device with screen projection function, such as a mobile phone, tablet computer, laptop computer, etc.

[0146] Figure 4 The first electronic device 100 shown is merely an example of an electronic device, and the first electronic device 100 may have more or fewer components than shown in the figure, may combine two or more components, or may have different component configurations. Figure 4 The various components shown can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits. In this example, the first electronic device 100 is exemplified by a mobile phone.

[0147] The first electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a Universal Serial Bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, antenna 1, antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a Subscriber Identification Module (SIM) card interface 195, etc. The sensor module 180 may include pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, bone conduction sensors, etc.

[0148] Understandable, Figure 4The components included in the illustrated electronic device do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the first electronic device 100 may include more or fewer components than those illustrated.

[0149] Figure 5 This is a software structure block diagram of the first electronic device according to an embodiment of this application.

[0150] The layered architecture of the first electronic device divides the software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into three layers, from top to bottom: the application layer, the application framework layer, and the kernel layer.

[0151] The application layer can include a series of application packages.

[0152] like Figure 5 As shown, the application package may include applications such as smart space, games, camera, video, and calling.

[0153] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0154] like Figure 5 As shown, the application framework layer may include a window manager, a screen mirroring module, a display manager, a power manager, a view system, power management, a notification manager, and system services, etc.

[0155] The window manager is used to manage windowed applications. It can obtain the screen size, determine if a status bar is present, lock the screen, and capture the screen. In this example, when the window manager receives a command to set focus, it determines whether a separate focus setting is needed. If a separate focus setting is required, it sets the focus for the currently running screen.

[0156] The screen projection module is responsible for managing and supporting screen projection functions between devices. It can record whether the first electronic device is currently projecting, and information about the target device (i.e., the receiving end).

[0157] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0158] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0159] The notification manager allows applications to display notification information in the status bar. It can be used to convey informational messages and can disappear automatically after a short time without user interaction.

[0160] The kernel layer is the layer between hardware and software. The kernel layer includes at least display drivers, camera drivers, Bluetooth drivers, audio drivers, and sensor drivers.

[0161] Understandable, Figure 5 The layers in the illustrated software structure and the components contained in each layer do not constitute a specific limitation on the first electronic device. In other embodiments of this application, the first electronic device may include more or fewer layers than illustrated, and each layer may include more or fewer components; this application does not impose any limitations.

[0162] Figure 6a The flowchart illustrates an exemplary method for cross-device interaction, which includes, but is not limited to, the following steps:

[0163] Step 601: User inputs touch operation.

[0164] Step 601 is similar to step 301. For details of step 601, please refer to the relevant textual description of step 301. It will not be repeated here.

[0165] Step 602: The touchpad gains focus and a touch event occurs.

[0166] Figure 6b This is an illustrative diagram illustrating how a touchpad gains focus and handles touch events. In this example, the source of the screen mirroring is a mobile phone, and the receiving end is a television. The application to be mirrored is one that requires focus to be maintained on its interface, such as a parkour game or an application that requires input of information (such as a login password or username). In this example, a parkour game (referred to as Game 1) is used as the screen mirroring application.

[0167] The mobile phone includes a touchpad, window manager, default screen, virtual screen, and screen mirroring module. Among them, the window manager, default screen, virtual screen, and screen mirroring module are located in the application framework layer.

[0168] Please refer to Figure 6b The process of the touchpad gaining focus and handling touch events includes, but is not limited to, the following steps:

[0169] Step 701: The touchpad receives touch input from the user.

[0170] For example, the touchpad refers to the area on a mobile phone used to receive user input in a screen mirroring scenario. In this example, the mobile phone projects Game 1 onto a TV screen via screen mirroring. The mobile phone can control the game characters in Game 1 using an air mouse remote control. The mobile phone's air mouse remote control is as follows: Figure 7 As shown in (2), it includes a sliding click area 904, which is used to receive touch operations input by the user. The sliding click area 904 can be understood as the touchpad in this example.

[0171] When a user inputs a touch operation in the sliding click area 904, the touchpad can receive the user's touch operation.

[0172] Step 702: Obtain the touch event.

[0173] For example, in response to a user's touch input, the touchpad determines that the default screen should receive focus (i.e., the target screen is updated to the default screen). In this example, the touchpad generates a first setting instruction that includes information about the target screen, such as the screen's identification information. For instance, the first setting instruction instructs the window manager to set the target screen to receive focus.

[0174] Step 703: The touchpad instructs the window manager to give focus to the default screen.

[0175] For example, the touchpad sends a first setting instruction to the window manager to instruct the window manager to give focus to the default screen.

[0176] Step 704: The window manager determines whether the default screen has focus. If the window manager detects that the default screen has no focus, proceed to step 705; if the window manager detects that the default screen has focus, proceed to step 712.

[0177] For example, the window manager records information about whether each running screen on the phone has focus. In response to an instruction (such as a first setting instruction) that includes setting the focus of a target screen, the window manager determines whether the target screen has focus based on the target screen information in the instruction. If the target screen has focus, step 712 is executed; if the target screen does not have focus, step 705 is executed. For example, in this example, the window manager, in response to the first setting instruction, determines whether the default screen has focus based on the target screen information in the first setting instruction. When the window manager detects that the default screen currently has no focus, it sends a query instruction to the phone's screen mirroring module, i.e., executes step 705. When the window manager detects that the default screen has currently acquired focus, it returns the detection result to the touchpad (i.e., executes step 712), which indicates that the default screen has acquired focus. Then, the touchpad sends the acquired touch events to the TV's converged desktop (i.e., executes step 603).

[0178] Step 705: The window manager sends a query command to the screen mirroring module.

[0179] For example, the window manager sends a query command to the screen mirroring module, which instructs the screen mirroring module to check whether the current phone is in screen mirroring mode and whether there are applications running on the virtual screen.

[0180] Step 706: The screen mirroring module obtains the query results based on the screen mirroring status.

[0181] For example, the screen mirroring module can record whether the phone is currently mirroring. When the screen mirroring module receives a query command, it checks whether the phone is currently in a screen mirroring state based on the recorded information. If the screen mirroring module determines that the phone is in a screen mirroring state, it checks whether any application is running on the virtual screen. If the screen mirroring module detects that an application is running on the virtual screen, it determines that the query result is that the phone is in a screen mirroring state and an application is running on the virtual screen. If the screen mirroring module detects that no application is running on the virtual screen, it determines that the query result is that the phone is in a screen mirroring state and no application is running on the virtual screen. If the screen mirroring module determines that the phone is not currently in a screen mirroring state, it determines that the query result is that the phone is not in a screen mirroring state.

[0182] Step 707: The screen mirroring module returns the query results to the window manager.

[0183] Step 708: The window manager determines whether a separate focus needs to be set based on the query results. If the window manager determines that a separate focus needs to be set, proceed to step 709. If the window manager determines that a separate focus does not need to be set, proceed to step 710.

[0184] For example, after receiving the query result, the window manager determines whether a separate focus needs to be set. When the query result indicates that the phone is in screen mirroring mode and an application is running on the virtual screen, the window manager determines that the target screen needs a separate focus and executes step 709. When the query result indicates that the phone is in screen mirroring mode and no application is running on the virtual screen, the window manager determines that the target screen does not need a separate focus and executes step 710; when the query result indicates that the phone is not in screen mirroring mode, the window manager determines that the target screen does not need a separate focus and executes step 710.

[0185] Step 709: Window Manager sets the default screen to receive focus.

[0186] For example, when the window manager determines that the target screen (currently the default screen) needs to have its own focus, it sets the default screen to have focus. Step 711 is not executed after this step; if the current virtual screen has focus, it will not lose focus, thus ensuring that both the default screen and the virtual screen have their respective focus.

[0187] Once this step is completed, step 602 is finished, and the touchpad gains focus and a touch event occurs. Then, step 603 is executed.

[0188] Step 710: Set the default screen to receive focus in the window manager. After this step is completed, proceed to step 711.

[0189] For example, when the window manager determines that the target screen (the current target screen is the default screen) needs to be given separate focus, it sets the default screen to receive focus and performs step 711.

[0190] Step 711: Window Manager sets virtual screen to lose focus.

[0191] For example, the window manager iterates through all the screens running on the phone other than the default screen. This iteration includes determining whether a screen has focus. Once the window manager determines that a virtual screen has focus through this iteration, it sets the virtual screen to lose focus.

[0192] Once this step is completed, step 602 is finished, and the touchpad gains focus and a touch event occurs. Then, step 603 is executed.

[0193] Step 712: The window manager returns the detection results to the touchpad.

[0194] For example, the window manager returns a detection result to the touchpad, indicating that the default screen has gained focus. After receiving this detection result, the touchpad completes step 602, gaining focus and triggering a touch event. Step 603 can then be executed.

[0195] Step 603: The touchpad sends touch events to the TV's converged desktop.

[0196] Step 604: Inject a reverse control event into the screen mirroring application A via the merged desktop.

[0197] For example, the process of the touchpad sending touch events to the converged desktop on the TV and the converged desktop injecting reverse control events into the screen mirroring application A can be referred to Figure 3c This will not be elaborated upon here.

[0198] Step 605: The screen mirroring application A responds to the event and gains focus.

[0199] It should be noted that, in some embodiments, after the touchpad gains focus and a touch event occurs, the touchpad can also directly send a touch event to the projection application A. The projection application A responds to the touch event and gains focus.

[0200] Figure 6c This is an illustrative diagram illustrating how a screen mirroring application A receives focus in response to a touch event, specifically including but not limited to the following steps:

[0201] Step 801: The screen mirroring application A responds to the touch event and instructs the virtual screen to gain focus.

[0202] In one example, after the touchpad gains focus and a touch event occurs, the touchpad sends a touch event to the TV's converged desktop, which then injects a reverse control event into the screen mirroring application (see details of this process). Figure 3c (See the related description). This reverse control event includes touch events. Since the casting application A is running on the virtual screen, casting application A responds to the injected reverse control event, determines that the virtual screen has focus, and updates the target screen to the virtual screen. In this example, casting application A generates a new first setting command based on the information of the new target screen. Casting application A sends the new first setting command to the window manager.

[0203] In another example, after the touchpad gains focus and a touch event occurs, it sends a touch event to the casting application A. Responding to the touch event, casting application A determines that the virtual screen has gained focus and updates the target screen to the virtual screen. Based on the new target screen information, casting application A generates a new first setup command. Casting application A then sends the new first setup command to the window manager.

[0204] Step 802: The window manager determines whether the virtual screen has focus. If the window manager detects that the virtual screen has no focus, proceed to step 803; if the window manager detects that the virtual screen has focus, proceed to step 809.

[0205] For example, the window manager records information about whether each running screen on the phone has focus. In response to the first setting instruction, the window manager determines whether the virtual screen indicated in the target screen's information has gained focus. When it is determined that the virtual screen has gained focus, it returns a second detection result to the screen mirroring application A (i.e., executes step 810); this second detection result indicates that the virtual screen already has focus. Then, the screen mirroring application A processes the reverse control event, i.e., executes step 811.

[0206] When it is determined that the virtual screen has not acquired focus, a query command is sent to the screen mirroring module of the mobile phone, i.e., step 803 is executed.

[0207] Step 803: The window manager sends a query command to the screen mirroring module.

[0208] For example, the window manager sends a query command to the screen mirroring module, which instructs the screen mirroring module to check whether the current phone is in screen mirroring mode and whether there are applications running on the virtual screen.

[0209] Step 804: The screen mirroring module obtains the query results based on the screen mirroring status.

[0210] This step is similar to step 706, and you can refer to the relevant description in step 706. It will not be repeated here.

[0211] Step 805: The screen mirroring module returns the query results to the window manager.

[0212] Step 806: The window manager determines whether the target screen (in this example, the target screen is a virtual screen) needs to be set to a separate focus based on the query results. If the window manager determines that the target screen needs to be set to a separate focus, proceed to step 807. If the window manager determines that the target screen does not need to be set to a separate focus, proceed to step 808.

[0213] For example, after receiving the query result, the window manager determines whether the target screen needs to be set to a separate focus. When the query result indicates that the phone is in screen mirroring mode and an application is running on the virtual screen, the window manager determines that the target screen needs to be set to a separate focus and executes step 807. When the query result indicates that the phone is in screen mirroring mode and no application is running on the virtual screen, the window manager determines that the target screen does not need to be set to a separate focus and executes step 808; when the query result indicates that the phone is not in screen mirroring mode, the window manager determines that the target screen does not need to be set to a separate focus and executes step 808.

[0214] Step 807: Window Manager sets the virtual screen to receive focus.

[0215] For example, when the window manager determines that the target screen needs to be given separate focus, it sets the virtual screen to receive focus. This step is followed by step 811, where the screen mirroring application A handles the control event.

[0216] In this example, it is assumed that the virtual screen has no focus before step 602, and in step 602, the touchpad gains focus (i.e., the default screen gains focus). After step 807, the window manager does not perform the operation of setting the default screen to lose focus, thus allowing the default screen and the virtual screen to each have their corresponding focus.

[0217] Step 808: The window manager sets the virtual screen to receive focus. After this step is completed, proceed to step 809.

[0218] For example, when the window manager determines that the target screen needs to be given separate focus, it sets the virtual screen to receive focus and performs step 809.

[0219] Step 809: Window Manager sets the default screen to lose focus.

[0220] For example, the window manager iterates through all screens running on the phone except for the virtual screen. This iteration includes determining whether a screen has focus. Once the window manager determines that the default screen has focus through this iteration, it sets the default screen to lose focus.

[0221] After this step is completed, proceed to step 811.

[0222] Step 810: The window manager returns the detection results to the screen mirroring application A.

[0223] For example, the current detection result indicates that the virtual screen has gained focus. After receiving this detection result, the screen mirroring application A can process the reverse control event, that is, execute step 811.

[0224] Step 811: Screen mirroring application A processes touch events.

[0225] In one example, after the virtual screen gains focus, the casting application A processes the received touch event and executes the corresponding operation. For instance, in a parkour game, the touch event might be a right swipe, and the casting application would control the game character to move to the right based on this event.

[0226] The following is combined with Figure 7 The example shown illustrates the cross-device interaction process in this example. In this example, the source of the screen mirroring is a mobile phone, and the receiving end is a TV.

[0227] Reference Figure 7In (1), after the "wireless screen mirroring" service is enabled on the mobile phone, the user opens the shared parkour game (referred to as Game 1) on the TV using an air mouse remote control. The TV's display shows the interface 901 of Game 1, which includes the game character 902 and a directional cursor. Figure 7 As shown in (2), the air mouse remote control page 903 of the mobile phone includes a sliding click area 904. The user interacts with the game character in Game 1 by sliding click area 904 to control the character to move left and right, slide up and down, etc. Assume that before time T3, the focus is on the virtual screen. At time T3, the direction indicator cursor in the game interface 901 indicates that the game character moves to the right. At time T3, the user follows the direction indicator cursor and inputs a right swipe touch operation in the sliding click area 904 of the mobile phone, such as... Figure 7 As shown in (2). The sliding click area 904 obtains the corresponding touch event based on the user's input touch operation. Since the sliding click area 904 runs on the default screen, the sliding click area 904 determines that the default screen has focus (i.e., the target screen is the default screen). The sliding click area generates a first setting instruction based on the information of the target screen (such as the identifier of the default screen) and sends the first setting instruction to the window manager. The first setting instruction instructs the window manager to set focus for the target screen. The window manager responds to the first setting instruction and determines whether there is focus on the default screen. When the window manager detects that there is no focus on the default screen, it sends a query instruction to the screen mirroring module of the mobile phone. The query instruction is used to query whether the mobile phone is currently in screen mirroring state and whether there is an application running on the virtual screen. The screen mirroring module responds to the query instruction and queries whether the mobile phone is currently in screen mirroring state. When the screen mirroring module detects that the mobile phone is currently in screen mirroring state, it detects whether there is an application running on the virtual screen, obtains the query result, and returns the query result to the window manager. In this example, the query result indicates that the mobile phone is currently in screen mirroring state and that there is an application running on the virtual screen. The window manager detects that the query results indicate the phone is currently in screen mirroring mode and an application is running on the virtual screen, and determines that a separate focus needs to be set. The window manager sets the default screen to receive focus and will not perform any operations that would cause other screens to lose focus. At this time, the default screen receives focus, which is the currently swiped / clicked area 904.

[0228] The swipe-click area 904 sends the received touch event to the TV's system framework. The TV system framework listens for the touch event and transmits it to the AirSight application on the TV. The AirSight application on the TV transmits the touch event to the TV's converged desktop. The converged desktop on the TV encapsulates the touch event and places it in a message queue. The converged desktop on the TV uses the message queue to transmit the touch event to the TV's DMSDP receiver. The DMSDP receiver transmits the touch event to the TV's Airsharing receiver. The Airsharing receiver parses the touch event and encapsulates it into a UIBC control message. The Airsharing receiver transmits the control message as binary data (denoted as control data) to the TV's Magic-Link receiver. The Magic-Link receiver sends the control data to the phone's Magic-Link sender. The phone's Magic-Link sender transmits the control data to the Airsharing sender. The Airsharing sender parses the control data to obtain the UIBC control message; the Airsharing sender then parses the UIBC control message to obtain the touch event. The Airsharing sender injects the parsed touch event into the phone's Input framework. The phone's Input framework then distributes the touch event to Game 1.

[0229] Since Game 1 runs on a virtual screen, when Game 1 receives a touch event distributed by the Input framework, it determines that the virtual screen has gained focus (i.e., the target screen is updated to the virtual screen). Game 1 generates a new first setting command based on the target screen's information (such as the virtual screen's identifier) ​​and sends this first setting command to the window manager. After receiving the first setting command from Game 1, the window manager determines whether the virtual screen has focus. When the window manager detects focus on the virtual screen, it returns the detection result to Game 1. When Game 1 detects that the detection result indicates focus on the virtual screen, Game 1 processes the touch event distributed by the Input framework. Game 1 controls the game character to move to the right, such as... Figure 7 As shown in (3), the interface 901 displayed in Game 1 changes to interface 905, and the game character 902 is located on the road on the right.

[0230] In this example, the default screen and the virtual screen each have their own focus, thus preventing the default screen from taking over the focus of the virtual screen and causing the application running on the virtual screen to pause.

[0231] In other embodiments, such as when an input box in a screen mirroring application requires at least two characters (e.g., a password), the default screen and the virtual screen can each have their own focus, thus avoiding the situation where the default screen steals the focus of the virtual screen, causing the input of information (e.g., a login password) to fail, thereby improving the user experience of wireless screen mirroring.

[0232] In some embodiments, in a heterogeneous screen mirroring scenario, the screen mirroring application running on the virtual screen can also control the target object in the screen mirroring application via gravity sensing. Specifically, if the screen mirroring application supports gravity sensing, when the phone is tilted, the screen mirroring application can obtain the phone's gravity sensing information. Based on the obtained gravity sensing information, the screen mirroring application controls the target object on the virtual screen to tilt according to the phone's tilt angle, thereby causing the target object on the TV display interface to tilt according to the phone's tilt angle. The gravity sensing information includes information about the tilt angle.

[0233] For example, continue to refer to Figure 7 The example shown uses a parkour game as an example for screen mirroring. The source of the screen mirroring is a mobile phone, and the receiving end is a TV. When Game 1 is mirrored to the TV via "wireless screen mirroring," the user interacts with the TV using an air mouse remote. Game 1 supports gravity sensing; when the user tilts the phone to the right, Game 1 can obtain gravity sensing information. Based on the gravity sensing information and a preset gravity sensing mapping, Game 1 controls the game character to move to the right, such as... Figure 7 As shown in (3) in the figure. Gravity sensing mapping is the proportional relationship between the tilt angle of the mobile phone in space and the tilt angle of the target object (such as a game character) on the display interface of the TV (i.e., on the virtual screen).

[0234] In addition, users can interact with the TV by inputting touch operations on the sliding and clicking area of ​​the air mouse remote control. For example, users can input operations such as swiping up, down, left, and right to control game characters to perform actions such as jumping, crouching, turning left, and turning right. The process of users interacting with the TV through touch operations can be seen in [reference needed]. Figure 7 The relevant descriptions in the original text will not be repeated here. In other words, in addition to controlling the movement of target objects on the TV display screen by sliding and clicking the area on the air mouse remote, users can also control the movement of target objects on the TV display screen by tilting the phone.

[0235] In this example, in a heterogeneous screen mirroring scenario, users can use their phone's gravity sensor to control target objects displayed on the TV's screen in the mirroring application, enhancing the fun of interacting with the TV. Users can also interact with the TV through gravity sensing and touch operations, offering flexible and varied interaction methods and improving the user experience.

[0236] In this embodiment of the application, in a heterogeneous projection scenario, the air mouse remote control also has interactive functions such as press / hold / grab / drag. For example, the press / hold function of the air mouse remote control means that when the user presses down on the sliding click area of ​​the air mouse remote control, the position indicated by the TV cursor on the TV is selected, and the duration of the user pressing down on the sliding click area is the same as the duration of the selection of the position indicated by the TV cursor.

[0237] The grabbing function, when the screen mirroring application supports gesture grabbing, allows users to grab target objects on the TV display screen by swiping and clicking within a designated area. For example, Game 2 on a mobile phone is mirrored to the TV via "wireless screen mirroring." The TV display shows Game 2's interface 1001. Game 2's interface 1001 includes a pineapple 1002. Interface 1001 indicates that the pineapple needs to be moved upwards (e.g., ...). Figure 8a (1) Move in the direction indicated by the arrow in the middle. Figure 8a As shown in (2), the user inputs a "grab" gesture on the sliding click area 1004 of the air mouse remote control page 1003 on the mobile phone to grab the pineapple from position 1 to position 2. The "grab" gesture can be as follows: Figure 8a As shown in (2), pinch inward with two fingers. The sliding click area acquires focus and the touch event corresponding to the "grab" operation (hereinafter referred to as the "grab" event). The process of acquiring focus and the touch event can be referred to in step 602, and will not be repeated here. The sliding click area transmits the "grab" event to the TV's integrated desktop. Based on the "grab" event, the TV's integrated desktop generates a counter-control event and injects it into Game 2. The process of transmitting the "grab" event and injecting the counter-control event can be referred to in... Figure 6a Steps 603 and 604 in the previous steps will not be repeated here. Game 2 responds to the counter-control event (which includes the "grab" event) and performs the operation of grabbing the pineapple, as follows... Figure 8a As shown in (3), the TV display interface 1001 displays an image of the game hand 1005 grabbing a pineapple 1002. Afterwards, the user can input an upward swipe operation in the sliding click area 1004 to move the pineapple in game 2 according to... Figure 8a The black color shown in (3) moves upward.

[0238] The drag-and-drop function refers to the ability to drag the TV cursor by pressing and holding the sliding click area of ​​the air mouse remote control while moving the mobile phone, thus supporting image effects such as drawing and cutting in the "wireless screen casting" scenario.

[0239] For example, Game 3 on the phone can be wirelessly projected onto the TV. The user can then interact with the TV using the phone's air mouse remote. The TV's display shows the Game 3 interface, where the gameplay involves slicing falling fruit, such as... Figure 8c As shown in (1), at time T5, the interface in Game 3 displays the falling fruit.

[0240] Please refer to Figure 8b In (1), at time T5, the user presses and holds the sliding click area of ​​the air mouse remote. At time T5, when the phone is at position A, the cursor is at point A on the game interface, as shown in the reference. Figure 8c (2) In this context, the user presses and holds the sliding click area of ​​the air mouse remote control while simultaneously pressing... Figure 8b The mobile phone's trajectory is shown in the diagram. The air mouse remote responds to the user's "press and hold" input, acquiring the phone's direction, speed, and position in space to record its movement trajectory. The phone sends this trajectory as a touch event to the TV's integrated desktop. The TV's integrated desktop generates a counter-control event based on the touch event and injects it into Game 3. The process of transmitting touch events and injecting counter-control events can be found in [reference needed]. Figure 6a Steps 603 and 604 are not detailed here. Game 3 responds to this reverse control event by moving the cursor along the phone's trajectory within the game interface. Since the game interface is displayed on the TV screen, when the phone moves to position B, the cursor on the corresponding game interface displayed on the TV moves to point B on the game interface. (Refer to...) Figure 8c (2) The phone continues to follow Figure 8b The phone follows the indicated trajectory; at time T7, the phone moves to position C, corresponding to the cursor on the game interface displayed on the TV moving to point C on the game interface. The phone continues to follow... Figure 8b The motion trajectory shown follows; at time T8, the phone moves to position D, corresponding to the cursor on the game interface displayed on the TV moving to point D on the game interface. (Refer to...) Figure 8c As shown in (2), the cursor's movement trajectory on the game interface corresponds to the phone's movement trajectory in space, thus making the cursor appear as a drag effect on the game interface to cut the falling fruit. In this example, the user uses an air mouse remote control to make the interface displayed on the TV display appear as a drag effect, allowing more applications to be used in the heterogeneous projection scenario, improving the fun and flexibility of the interaction between the phone and the TV in the heterogeneous projection scenario, and improving the user experience of using the projection application.

[0241] The drag-and-drop function can also be applied to drawing, handwriting, and other applications. For example, a handwriting app on a mobile phone can be wirelessly projected onto a TV. Users can then interact with the TV using the air mouse remote control on their phone. The TV's display shows the handwriting app's interface. (See reference) Figure 9bAs shown in (1) therein, at time T9, the display screen of the TV shows the interface of the writing application, and the stroke trajectory of the Chinese character to be written is shown in this interface so that the user can write the corresponding strokes according to the stroke trajectory. Among them, the cursor is located at the starting position of the first stroke of the Chinese character to be written, such as Figure 9b the starting position of the "left-falling stroke" shown in (1) therein.

[0242] At time T9, the user presses and holds the sliding click area of the air mouse remote control. At this time, the position E where the mobile phone is located corresponds to the starting position of the TV cursor. While the user presses and holds the sliding click area of the air mouse remote control, the user moves the mobile phone according to the stroke trajectory of the Chinese character in the TV. The air mouse remote control responds to the user's operation, obtains the moving direction, speed and position of the mobile phone to obtain the moving trajectory of the mobile phone. The air mouse remote control transmits the obtained data as a touch event to the fusion desktop of the TV. The fusion desktop of the TV generates a reverse control event based on the touch event and injects the reverse control event into the writing application. Among them, the process of transmitting the touch event and injecting the reverse control event can refer to Figure 6a Steps 603 and 604 therein, which will not be elaborated here. The writing application responds to the reverse control operation, controls the cursor to move on the interface of the writing application according to the moving trajectory of the mobile phone, so that at time T10, when the mobile phone moves to position F; corresponding to the first stroke (i.e., the "left-falling stroke") of the character "person" drawn by the cursor on the interface of the writing application displayed on the TV, such as Figure 9b shown in (2) therein, the black is the actual moving trajectory of the cursor (i.e., the dragging trajectory presented by the cursor on the interface of the writing application).

[0243] In this example, the dragging function of the air mouse remote control improves the flexibility of the interaction between the mobile phone and the TV, enabling more applications to be screen-cast.

[0244] In some embodiments, the user can also adjust the cursor tracking speed of the air mouse remote control according to needs to improve the user experience of using the air mouse remote control.

[0245] Exemplarily, in the heterogeneous screen-casting scenario, the cursor tracking speed of the TV cursor is related to the first mapping value. The first mapping value is the ratio between the moving distance of the mobile phone and the moving distance of the cursor on the display interface of the TV. Multiple first mapping values can be stored in the mobile phone, and different first mapping values correspond to different cursor tracking speeds. The mobile phone responds to the user's operation of adjusting the cursor tracking speed, obtains the first mapping value corresponding to the cursor tracking speed selected by the user, and uses the obtained first mapping value as the target mapping value. The air mouse remote control uses the target mapping value to control the TV cursor tracking speed on the TV side.

[0246] For example, in this example, the application that interacts with the TV in the mobile phone takes the air mouse remote control as an example. Such as Figure 10As shown in (1), the air mouse remote control page 1001 includes a cursor tracking speed adjustment control 1002. In response to the user clicking the cursor tracking speed adjustment control 1002, the air mouse remote control displays an adjustment box 1002-1 on the air mouse remote control page 1001, as shown in (1). Figure 10 (2) of b. Adjustment box 1002-1 includes a cursor speed scale and a slider on the cursor tracking speed scale. The left end of the cursor tracking speed scale corresponds to the slowest cursor tracking speed, and the right end corresponds to the fastest cursor tracking speed. The user can adjust the cursor tracking speed by sliding the slider on the cursor tracking speed scale.

[0247] In this example, the cursor tracking speed scale includes 10 different cursor tracking speeds, and the phone is set with 10 different first mapping values. If the first mapping value (b) = phone movement distance (X) / cursor movement distance (x), then for the same phone movement distance, a smaller first mapping value results in a larger cursor movement distance, thus improving the cursor tracking speed. Alternatively, if the first mapping value (b) = cursor movement distance (x) / phone movement distance (X), then for the same phone movement distance, a larger first mapping value results in a larger cursor movement distance, thus improving the cursor tracking speed.

[0248] It is understood that, in order to achieve the above-mentioned functions, electronic devices include hardware and / or software modules that perform the respective functions. Based on the algorithmic steps of the examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application in conjunction with the embodiments, but such implementation should not be considered beyond the scope of this application.

[0249] This application also provides a chip system including at least one processor and at least one interface circuit. The processor and the interface circuit are interconnected via lines. For example, the interface circuit can be used to receive signals from other devices (e.g., the memory of an electronic device). As another example, the interface circuit can be used to send signals to other devices (e.g., the processor). Exemplarily, the interface circuit can read instructions stored in the memory and send the instructions to the processor. When the instructions are executed by the processor, the electronic device can perform the steps in the above embodiments. Of course, the chip system may also include other discrete devices, and this application does not specifically limit this.

[0250] This embodiment also provides a computer storage medium storing computer instructions. When these computer instructions are executed on an electronic device, the electronic device performs the aforementioned method steps to achieve the cross-device interaction method described in the above embodiment. The storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0251] This embodiment also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned related steps to realize the cross-device interaction method in the above embodiment.

[0252] In this embodiment, the electronic device, computer storage medium, computer program product or chip are all used to execute the corresponding cross-device interaction method provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects of the corresponding method provided above, and will not be repeated here.

[0253] Any content in the various embodiments of this application, as well as any content in the same embodiment, can be freely combined. Any combination of the above content is within the scope of this application.

[0254] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method for cross-device interaction, characterized in that, The method is applied to a first electronic device, which establishes a heterogeneous screen projection connection with a second electronic device. The first electronic device is the source of the projection, and the second electronic device is the receiver. The first electronic device has a default screen and a virtual screen. When the window manager of the first electronic device receives the first setting instruction, it determines whether the target screen has obtained focus. The first setting instruction is used to instruct the window manager to set the target screen to obtain focus. When the window manager detects that the target screen has not acquired focus, it determines whether the target screen needs to be set to a separate focus. The window manager queries whether the first electronic device is in screen mirroring mode and whether there is an application running on the virtual screen of the first electronic device; When the query results indicate that the first electronic device is in screen mirroring mode and there is an application running on the virtual screen, it is determined that the target screen needs to be set with a separate focus. When the window manager determines that the target screen needs to be set to a separate focus, it sets the target screen to gain focus, and the window manager does not perform the operation of setting the first screen to lose focus. The target screen is one of the default screen or the virtual screen, and the first screen is the screen in the first electronic device that has currently gained focus, other than the target screen.

2. The method according to claim 1, characterized in that, When the window manager detects that the target screen has not acquired focus, it determines whether the target screen needs to be given a separate focus, including: The window manager sends a query command to the screen projection module of the first electronic device; The screen mirroring module responds to the query command by querying whether the first electronic device is in screen mirroring mode and whether there is an application running on the virtual screen of the first electronic device, and obtains the query result; When the query result indicates that the first electronic device is in screen mirroring mode and there is an application running on the virtual screen, it is determined that the target screen needs to be set with a separate focus. When the query result indicates that the first electronic device is not in screen mirroring mode, or indicates that the first electronic device is in screen mirroring mode and there is no running application on the virtual screen, it is determined that the target screen does not need to be set with a separate focus.

3. The method according to claim 2, characterized in that, The screen mirroring module responds to the query command by querying whether the first electronic device is in screen mirroring mode and whether an application is running on the virtual screen of the first electronic device, and obtains the query results, including: The screen mirroring module responds to the query command, detects whether the first electronic device is in screen mirroring mode, and obtains a first result; When the first result indicates that the first electronic device is in screen mirroring mode, the screen mirroring module detects whether there is an application running on the virtual screen and obtains a second result; the screen mirroring module uses the first result and the second result as the query result; When the first result indicates that the first electronic device is not in screen mirroring mode, the first result is used as the query result.

4. The method according to claim 1, characterized in that, When a first application of the first electronic device is projected onto the second electronic device, the method further includes: The first electronic device displays a first interface of an interactive application, the first interface including a first touch area; The first touch area responds to a first touch operation input by the user, updates the target screen to the default screen of the first electronic device, and obtains a first touch event corresponding to the first touch operation; The first touch area generates the first setting instruction based on the information of the target screen; The first touch area sends a first setting command to the window manager.

5. The method according to claim 4, characterized in that, After the default screen gains focus, the method further includes: The first touch area sends the first touch event to the second electronic device, and the second electronic device generates a counter-control event based on the first touch event when it detects that the second electronic device is in the screen projection state; In response to a counter-control event injected by the second electronic device, the first application updates the target screen to the virtual screen of the first electronic device; The first application generates a new first setting instruction based on the information of the target screen; The first application sends the first setting instruction to the window manager; After the virtual screen gains focus, the method further includes: The first application performs the operation corresponding to the anti-control event based on the anti-control event.

6. The method according to claim 4, characterized in that, After the default screen gains focus, the method further includes: The first touch area sends the first touch event to the first application; In response to the first touch event, the first application updates the target screen to the virtual screen of the first electronic device; The first application generates a new first setting instruction based on the information of the target screen; The first application sends the first setting instruction to the window manager; After the virtual screen gains focus, the method further includes: The first application performs an operation corresponding to the first touch event.

7. The method according to claim 1, characterized in that, The method further includes: When the window manager determines that the target screen does not need to be set to a separate focus, it sets the target screen to receive focus; The window manager sets the first screen to lose focus.

8. The method according to claim 1, characterized in that, When a first application of the first electronic device is projected onto the second electronic device, the method further includes: The first electronic device displays a first interface of an interactive application, the first interface including a first adjustment control; In response to the user's activation of the first adjustment control, a first adjustment box is displayed on the first interface, the first adjustment box including at least two different cursor tracking speeds; The interactive application responds to the user's selection operation on the first adjustment box, and obtains the mapping value corresponding to the cursor tracking speed selected by the user as the target mapping value. The mapping value is the proportional relationship between the distance the first electronic device moves in space and the distance the cursor moves in the display interface of the second electronic device. The interactive application uses the target mapping value.

9. The method according to any one of claims 1-7, characterized in that, When a first application of the first electronic device is projected onto the second electronic device, the method further includes: The first electronic device displays a first interface of an interactive application, the first interface including a first touch area; The first touch area responds to a second touch operation input by the user in the first touch area, gains focus and a second touch event corresponding to the second touch operation, the second touch operation including a pressing operation on the first touch area for a first preset time period and an operation controlling the first electronic device to move in space during the first preset time period; the second touch event includes the movement trajectory of the first electronic device in space; When the first application receives the second touch event, it controls the cursor in the display interface of the second electronic device to move according to the movement trajectory.

10. The method according to any one of claims 1-7, characterized in that, When a first application of the first electronic device is projected onto the second electronic device, the method further includes: The first electronic device displays a first interface of an interactive application, the first interface including a first touch area; The first touch area responds to a third touch operation input by the user in the first touch area, and gains focus and a third touch event corresponding to the third touch operation; When the first application receives the third touch event, it performs a grabbing operation on the target object in the display interface of the second electronic device.

11. The method according to any one of claims 1-7, characterized in that, When a first application of the first electronic device is projected onto the second electronic device, the method further includes: In response to the user tilting the first electronic device, the first application acquires the gravity sensing information of the first electronic device; The first application controls the target object in the display interface of the second electronic device to tilt in the tilt direction of the first electronic device according to the gravity sensing information of the first electronic device and the preset gravity sensing mapping. The gravity sensing mapping is the proportional relationship between the tilt angle of the first electronic device in space and the tilt angle of the target object in the display interface of the second electronic device.

12. An electronic device, characterized in that, include: A memory and a processor, wherein the memory is coupled to the processor; The memory stores program instructions that, when executed by the processor, cause the electronic device to perform the cross-device interaction method according to any one of claims 1 to 11.

13. A computer-readable storage medium comprising a computer program, characterized in that, When the computer program is run on an electronic device, it causes the electronic device to perform the cross-device interaction method according to any one of claims 1 to 11.

Images