Electronic device and human-computer interaction method

Abstract

Embodiments of the present application relate to an electronic device and a human-computer interaction method. The method comprises: a second electronic device receiving a first picture and identification information of at least one application sent by a first electronic device according to a first operation; the second electronic device displaying the first picture and icons of the at least one application; the second electronic device receiving a second operation acting on a first icon, the first icon being one of the icons of the at least one application; in response to the second operation, the second electronic device displays an application window of the first application, which has the same content as an application window of the first application opened in a first interface. Through the method of the embodiments of the present application, in the scenario of cross-screen display of an application, the user can avoid performing a gesture operation with high accuracy, but can use the execution result of the first operation to achieve cross-screen display of the application by performing the second operation on the execution result, thereby improving the operation experience of the user.

Description

Technical Field

[0001] This application relates to the field of smart terminals. In particular, it relates to an electronic device and a human-computer interaction method. Background Technology

[0002] In the field of smart terminals, users are increasingly using gestures to control electronic devices. To reduce the variety of gestures for easier management, when setting gestures for electronic devices, efforts are made to use a single gesture to accomplish multiple functions. For example, performing the same gesture at different locations within the application window displayed on the electronic device will produce different functions. However, this requires a high degree of precision in the gestures. For instance, when performing a certain function, the gesture is confined to a small operating area on the screen of the electronic device. If the gesture's position is off, it can easily lead to misoperation. Summary of the Invention

[0003] The purpose of this application is to provide an electronic device and a human-computer interaction method to improve the user's operating experience during cross-screen display.

[0004] The first aspect of this application provides a human-computer interaction method, including: a second electronic device receiving a first image and identification information of at least one application sent by a first electronic device according to a first operation, wherein the first image is a screenshot of a first interface currently displayed by the first electronic device, the at least one application is an application corresponding to an application window opened in the first interface, and the identification information is an icon of the application;

[0005] The second electronic device displays the first image and the icon of at least one application, the first image covering a portion of the display area of ​​the second electronic device;

[0006] The second electronic device receives a second operation applied to a first icon, the first icon being one of at least one application icon;

[0007] In response to the second operation, the second electronic device displays the application window of the first application corresponding to the first icon, and the content of the application window of the first application displayed by the second electronic device is the same as that of the application window of the first application opened in the first interface.

[0008] In this embodiment of the application, the first electronic device and the second electronic device can be smart terminals with touch screens, for example, the first electronic device and the second electronic device can be tablet computers. The first operation and the second operation can be gesture operations performed on the touch screens of the first electronic device and the second electronic device, such as one of the following: click operation, swipe operation, and drag operation.

[0009] The first electronic device can simultaneously display the application windows of the open document application, SMS application, and browser application on the screen of the first electronic device. The first image is a screenshot containing the application windows of the document application, SMS application, and browser application, and the corresponding identification information of the application is the icon of the document application, SMS application, and browser application.

[0010] Taking the first operation as a three-finger swipe as an example, since this first operation can be applied to any area of ​​the screen of the first electronic device, it avoids the user having to perform operations with high precision, such as pressing and holding the application window of an application on the screen of the first electronic device, thus improving the user's operating experience.

[0011] The second electronic device can display the first image and the application icon in a partial area of ​​the screen, such as the lower right corner of the screen. The application icon can be located above, below, to the left, or to the right of the first image.

[0012] The second operation can be an operation on the first image, the application icon, or other display areas of the screen of the second electronic device; the second operation can be a click operation on the first image, a click on the application icon, or a drag operation to drag the application icon to a certain area of ​​the screen of the second electronic device, such as dragging it to the top area of ​​the screen, or a click operation on a certain area of ​​the screen of the second electronic device other than the first image and the application icon.

[0013] By performing a second operation on the application icon, the application corresponding to the application icon is moved from the first electronic device to the second electronic device for display. The size of the application window on the screen of the second electronic device may be different from that on the first electronic device. For example, the application window on the screen of the first electronic device is displayed in a thumbnail mode, while the application window on the screen of the second electronic device is displayed in full-screen mode after the application is moved to the second electronic device.

[0014] In one possible implementation of the first aspect described above, in response to the second operation, the second electronic device displays an application window of the first application corresponding to the first icon, including:

[0015] In response to the second operation, the second electronic device sends a first instruction to the first electronic device;

[0016] The second electronic device receives first information sent by the first electronic device in response to the first instruction, the first information being used to represent the application window of the first application corresponding to the first icon;

[0017] The second electronic device displays the application window of the first application corresponding to the first icon based on the first information.

[0018] In this embodiment, the first instruction can be a click or drag operation on the application icon. The first information includes the application name corresponding to the application icon.

[0019] In one possible implementation of the first aspect described above, before the second electronic device receives the first image and at least one application identification information sent by the first electronic device, the method further includes:

[0020] The first electronic device displays a first interface in full screen, and the first interface includes at least one application window corresponding to at least one application.

[0021] The first electronic device receives a first operation applied to the first interface;

[0022] In response to the first operation, the first electronic device sends a first image and identification information of at least one application to the second electronic device.

[0023] In this embodiment of the application, the first electronic device can simultaneously display the application windows of the opened document application, SMS application, and browser application on the screen of the first electronic device. The first operation is a three-finger swipe operation on the screen of the first electronic device. The first image is a screenshot containing the application windows of the document application, SMS application, and browser application. The identification information corresponding to the application is the icon of the document application, SMS application, and browser application.

[0024] In one possible implementation of the first aspect described above, the second electronic device displays an application window for the first application corresponding to the first icon based on the first information, including:

[0025] The second electronic device displays the application window of the first application corresponding to the first icon in full screen according to the first information and the second operation.

[0026] In this embodiment of the application, in response to a click operation performed on the application icon, the second electronic device displays the application window of the application corresponding to the application icon in full screen on the screen of the second electronic device.

[0027] In one possible implementation of the first aspect described above, the second electronic device displays an application window for the first application corresponding to the first icon based on the first information, including:

[0028] Based on the first information and the second operation, the second electronic device displays the application window of the first application corresponding to the first icon on the left side of the screen.

[0029] In one possible implementation of the first aspect described above, the second operation includes dragging the first icon to a first area of ​​the screen of the second electronic device, the first area being located on the left side of the screen of the second electronic device.

[0030] In one possible implementation of the first aspect described above, the second electronic device displays an application window for the first application corresponding to the first icon based on the first information, including:

[0031] Based on the first information and the second operation, the second electronic device displays the application window of the first application corresponding to the first icon on the right side of the screen.

[0032] In one possible implementation of the first aspect described above, the second operation includes dragging the first icon to a first area of ​​the screen of the second electronic device, the first area being located on the right side of the screen of the second electronic device.

[0033] In one possible implementation of the first aspect described above, the second electronic device displays an application window for the first application corresponding to the first icon based on the first information, including:

[0034] The second electronic device displays the application window of the first application corresponding to the first icon at the top of the screen based on the first information and the second operation.

[0035] In one possible implementation of the first aspect described above, the second operation includes dragging the first icon to a first area of ​​the screen of the second electronic device, the first area being located above the screen of the second electronic device.

[0036] In one possible implementation of the first aspect described above, the second electronic device displays an application window for the first application corresponding to the first icon based on the first information, including:

[0037] The second electronic device displays the application window of the first application corresponding to the first icon at the bottom of the screen based on the first information and the second operation.

[0038] In one possible implementation of the first aspect described above, the second operation includes dragging the first icon to a first area of ​​the screen of the second electronic device, the first area being located below the screen of the second electronic device.

[0039] In this embodiment, the first area can be a partial area of ​​the screen of the second electronic device, such as the left half, right half, top half, or bottom half. In response to a drag operation on the application icon, the second electronic device releases the first area of ​​the application icon based on the drag operation on its screen, and displays the application window corresponding to the application icon at the position on the screen of the second electronic device corresponding to the first area. For example, dragging the application icon to the left half of the screen of the second electronic device displays the application window on the left half; similarly, dragging the application icon to the top half of the screen displays the application window on the top half.

[0040] In one possible implementation of the first aspect described above, the second electronic device is wirelessly or wiredly connected to the first electronic device; or the second electronic device and the first electronic device are on the same wireless local area network; or the second electronic device and the first electronic device are logged into the same user account; or the second electronic device is an extension screen of the first electronic device. Alternatively, the second electronic device and the first electronic device may each include two screens within the same electronic device. That is, the same electronic device may include two screens, and the interaction methods between two electronic devices described in the embodiments of this application can be applied to the two screens of the same electronic device. For example, the first electronic device includes a first screen, and the second electronic device includes a second screen; these two screens are located within the same electronic device; when these two screens are on the same electronic device, they share a processor or other control device.

[0041] In one possible implementation of the first aspect above, the first operation is a three-finger swipe operation; or at least one application icon is displayed above, below, to the left or to the right of the first image.

[0042] A second aspect of this application provides a human-computer interaction method, including:

[0043] The first electronic device displays a first interface in full screen, and the first interface includes at least one application window corresponding to at least one application.

[0044] The first electronic device receives a first operation applied to the first interface;

[0045] In response to the first operation, the first electronic device sends a first image and identification information of at least one application to the second electronic device. The first image is a screenshot of the first interface, and the identification information is the icon of the application.

[0046] The second electronic device displays the first image and the icon of at least one application, the first image covering a portion of the display area of ​​the second electronic device;

[0047] The second electronic device receives a second operation applied to a first icon, the first icon being one of at least one application icon;

[0048] In response to the second operation, the second electronic device displays the application window of the first application corresponding to the first icon, and the content of the application window of the first application displayed by the second electronic device is the same as that of the application window of the first application opened in the first interface.

[0049] In this embodiment of the application, the first electronic device and the second electronic device can be smart terminals with touch screens, for example, the first electronic device and the second electronic device can be tablet computers.

[0050] The first electronic device can simultaneously display the application windows of the open document application, SMS application, and browser application on the screen of the first electronic device. The first image is a screenshot containing the application windows of the document application, SMS application, and browser application, and the corresponding identification information of the application is the icon of the document application, SMS application, and browser application.

[0051] The first operation and the second operation can be gesture operations performed on the touch screens of the first electronic device and the second electronic device, such as one of the following: click operation, swipe operation, drag operation.

[0052] Taking the first operation as a three-finger swipe as an example, since this first operation can be applied to any area of ​​the screen of the first electronic device, it avoids the user having to perform operations with high precision, such as pressing and holding the application window of an application on the screen of the first electronic device, thus improving the user's operating experience.

[0053] The second electronic device can display the first image and the application icon in a partial area of ​​the screen, such as the lower right corner of the screen. The application icon can be located above, below, to the left, or to the right of the first image.

[0054] The second operation can be an operation on the first image, the application icon, or other display areas of the screen of the second electronic device; the second operation can be a click operation on the first image, a click on the application icon, or a drag operation to drag the application icon to a certain area of ​​the screen of the second electronic device, such as dragging it to the top area of ​​the screen, or a click operation on a certain area of ​​the screen of the second electronic device other than the first image and the application icon.

[0055] A second operation performed on the application's icon moves the application corresponding to that icon from the first electronic device to the second electronic device. The size of the application window on the second electronic device's screen can differ from that on the first electronic device. For example, the application window on the first electronic device's screen is displayed in a thumbnail format, while on the second electronic device, it is displayed in full-screen mode.

[0056] In one possible implementation of the second aspect described above, the second electronic device is one of at least one third electronic device, each of which is located in a first direction of the first electronic device.

[0057] In one possible implementation of the second aspect above, in response to the first operation, the first electronic device sends a first image and identification information of at least one application to the second electronic device, including:

[0058] The first electronic device selects a third electronic device in the first direction corresponding to the direction of the first operation as the second electronic device;

[0059] The first electronic device sends a first image and identification information of at least one application to the selected second electronic device.

[0060] In one possible implementation of the second aspect above, the first operation includes at least one of sliding up, sliding down, sliding left, sliding right, sliding down to the left, sliding down to the right, sliding up to the left, and sliding up to the right on the first interface.

[0061] In this embodiment, multiple third electronic devices can exist, each located in a different direction from the first electronic device. Based on the direction of the first operation, the third electronic device in the corresponding direction is selected as the second electronic device, and a first image and identification information of at least one application are sent to it. For example, there may be a third electronic device on the left and right sides of the first electronic device. When a leftward swipe gesture is performed on the screen of the first electronic device, the first electronic device identifies the left-side third electronic device as the second electronic device and sends the first image and identification information of at least one application to it. The third electronic devices can also be located below, above, to the upper left, to the upper right, to the lower left, or to the lower right of the first electronic device.

[0062] In one possible implementation of the second aspect above, in response to the second operation, the second electronic device displays an application window of the first application corresponding to the first icon, including:

[0063] In response to the second operation, the second electronic device sends a first instruction to the first electronic device;

[0064] The second electronic device receives first information sent by the first electronic device in response to the first instruction, the first information being used to represent the application window of the first application corresponding to the first icon;

[0065] The second electronic device displays the application window of the first application corresponding to the first icon based on the first information.

[0066] In this embodiment, the first instruction can be a click or drag operation on the application icon. The first information includes the application name corresponding to the application icon.

[0067] In one possible implementation of the second aspect described above, the second electronic device displays an application window for the first application corresponding to the first icon based on the first information, including:

[0068] The second electronic device displays the application window of the first application corresponding to the first icon in full screen according to the first information and the second operation.

[0069] In this embodiment of the application, in response to a click operation performed on the application icon, the second electronic device displays the application window of the application corresponding to the application icon in full screen on the screen of the second electronic device.

[0070] In one possible implementation of the second aspect described above, the second electronic device displays an application window for the first application corresponding to the first icon based on the first information, including:

[0071] Based on the first information and the second operation, the second electronic device displays the application window of the first application corresponding to the first icon on the left side of the screen.

[0072] In one possible implementation of the second aspect described above, the second operation includes dragging the first icon to a first area of ​​the screen of the second electronic device, the first area being located on the left side of the screen of the second electronic device.

[0073] In one possible implementation of the second aspect described above, the second electronic device displays an application window for the first application corresponding to the first icon based on the first information, including:

[0074] Based on the first information and the second operation, the second electronic device displays the application window of the first application corresponding to the first icon on the right side of the screen.

[0075] In one possible implementation of the second aspect described above, the second operation includes dragging the first icon to a first area of ​​the screen of the second electronic device, the first area being located on the right side of the screen of the second electronic device.

[0076] In one possible implementation of the second aspect described above, the second electronic device displays an application window for the first application corresponding to the first icon based on the first information, including:

[0077] The second electronic device displays the application window of the first application corresponding to the first icon at the top of the screen based on the first information and the second operation.

[0078] In one possible implementation of the second aspect described above, the second operation includes dragging the first icon to a first area of ​​the screen of the second electronic device, the first area being located above the screen of the second electronic device.

[0079] In one possible implementation of the second aspect described above, the second electronic device displays an application window for the first application corresponding to the first icon based on the first information, including:

[0080] The second electronic device displays the application window of the first application corresponding to the first icon at the bottom of the screen based on the first information and the second operation.

[0081] In one possible implementation of the second aspect described above, the second operation includes dragging the first icon to a first area of ​​the screen of the second electronic device, the first area being located below the screen of the second electronic device.

[0082] In this embodiment, the first region can be a partial area of ​​the screen of the second electronic device, such as the left half, right half, top half, or bottom half. In response to a drag operation on the application icon, the second electronic device releases the first region of the application icon based on the drag operation on its screen, and displays the application window corresponding to the application icon at the position corresponding to the first region on the screen of the second electronic device. For example, dragging the application icon to the left half of the screen of the second electronic device will display the application window on the left half. The first region can also be the right half, top half, or bottom half of the screen of the second electronic device. For another example, dragging the application icon to the top half of the screen of the second electronic device will display the application window on the top half.

[0083] In one possible implementation of the second aspect described above, the first electronic device, in response to the first operation, also takes a screenshot of the first interface and saves the screenshot of the first interface on the first electronic device.

[0084] In this embodiment of the application, in addition to sending the first image and the application's identification information to the second electronic device, the first electronic device can also perform a screenshot function on the screen of the first electronic device.

[0085] In one possible implementation of the second aspect above, the second electronic device is wirelessly or wiredly connected to the first electronic device; or the second electronic device and the first electronic device are on the same wireless local area network; or the second electronic device and the first electronic device are logged into the same user account; or the second electronic device is an extension screen of the first electronic device.

[0086] In this embodiment of the application, the second electronic device and the first electronic device can establish an extended screen connection, with the screen of the second electronic device serving as an extended screen for the first electronic device.

[0087] In one possible implementation of the second aspect above, the first operation is a three-finger swipe operation; or at least one application icon is displayed above, below, to the left or to the right of the first image.

[0088] A third aspect of this application provides a human-computer interaction method applied to an electronic device including a first screen and a second screen. The method includes:

[0089] The first screen displays a first interface in full screen, and the first interface includes at least one application window corresponding to at least one application.

[0090] The first screen receives a first operation applied to the first interface;

[0091] In response to the first operation, the electronic device controls the second screen to display a first image and the icon of the at least one application, wherein the first image is a screenshot of the first interface and the first image covers a portion of the display area of ​​the second screen;

[0092] The second screen receives a second operation applied to a first icon, which is one of the icons of the at least one application;

[0093] In response to the second operation, the second screen displays the application window of the first application corresponding to the first icon, and the content of the application window of the first application displayed on the second screen is the same as that of the application window of the first application opened in the first interface.

[0094] For example, the electronic device could be a dual-screen computer with two screens.

[0095] In one possible implementation of the third aspect above, the first operation includes at least one of sliding up, sliding down, sliding left, sliding right, sliding down to the left, sliding down to the right, sliding up to the left, and sliding up to the right on the first interface.

[0096] In one possible implementation of the third aspect above, specifically, in response to the second operation, the electronic device controls the second screen to display the application window of the first application corresponding to the first icon.

[0097] In one possible implementation of the third aspect described above, according to the second operation, the electronic device controls the second screen to display the application window of the first application corresponding to the first icon in different forms. For example, when the second operation is clicking the first icon, in response to the second operation, the second screen can display the application window of the first application corresponding to the first icon in full screen. Alternatively, the second operation includes dragging the first icon to a first area of ​​the second screen; when the first area is located on the left side of the second screen, in response to the second operation, the second screen displays the application window of the first application corresponding to the first icon on the left side of the screen; when the first area is located on the right side of the second screen, in response to the second operation, the second screen displays the application window of the first application corresponding to the first icon on the right side of the screen.

[0098] In one possible implementation of the third aspect described above, in response to the first operation, the electronic device may also display a screenshot of the first interface on the first screen; furthermore, based on the user's operation on the screenshot, the electronic device may edit, save, or delete the screenshot.

[0099] In one possible implementation of the third aspect described above, the first operation is a three-finger swipe operation.

[0100] In one possible implementation of the third aspect described above, the icon of the at least one application is displayed above, below, to the left, or to the right of the first image.

[0101] A fourth aspect of this application provides an electronic device, including: a memory storing instructions, and a processor for reading and executing the instructions in the memory to cause the electronic device to perform the human-computer interaction method provided in the first or third aspect.

[0102] A fifth aspect of this application provides an electronic device, including: a memory storing instructions, and a processor for reading and executing the instructions in the memory to cause the electronic device to perform the method executed by the first device in the human-computer interaction method provided in the second aspect above.

[0103] A sixth aspect of this application provides a computer-readable storage medium containing instructions that, when executed by an electronic device, cause the electronic device to implement the human-computer interaction method provided in the first or third aspect described above.

[0104] A seventh aspect of this application provides a computer-readable storage medium containing instructions that, when executed by an electronic device, cause the electronic device to perform the method executed by the first device in the human-computer interaction method provided in the second aspect above.

[0105] An eighth aspect of this application provides a computer program product, including: a non-volatile computer-readable storage medium containing computer program code for performing the human-computer interaction method provided in the first or third aspect.

[0106] A ninth aspect of this application provides a computer program product, including: a non-volatile computer-readable storage medium containing computer program code for performing the method executed by the first electronic device as described in the second aspect above. Attached Figure Description

[0107] Figure 1An embodiment of this application illustrates a scenario where applications can be displayed across multiple electronic devices on different screens.

[0108] Figure 2 An embodiment of this application illustrates a scenario where applications can be displayed across multiple electronic devices on different screens.

[0109] Figure 3 An embodiment of this application illustrates a scenario where applications can be displayed across multiple electronic devices on different screens.

[0110] Figure 4 A hardware structure block diagram of an electronic device is shown according to an embodiment of this application;

[0111] Figure 5 A software structure block diagram of an electronic device is shown according to an embodiment of this application;

[0112] Figure 6 A flowchart of a human-computer interaction method for an electronic device is shown according to an embodiment of this application;

[0113] Figure 7 An embodiment of this application illustrates a scenario where an application is opened and displayed on an electronic device;

[0114] Figure 8 An embodiment of this application illustrates a scenario in which a screenshot gesture is performed on the screen of an electronic device;

[0115] Figure 9 According to an embodiment of this application, a scene diagram is shown in which a screenshot of the screen of a first electronic device and the icon of the application contained in the screenshot are displayed on a second electronic device.

[0116] Figure 10A and Figure 10B An embodiment of this application illustrates a scenario of performing a gesture operation on a screenshot displayed on a second electronic device;

[0117] Figure 11A and Figure 11B An embodiment of this application illustrates a scenario in which a click gesture is performed on an application icon displayed on a second electronic device;

[0118] Figure 12A and Figure 12B An embodiment of this application illustrates a scenario in which a drag-and-drop gesture is performed on an application icon displayed on a second electronic device;

[0119] Figure 13 According to an embodiment of this application, a schematic diagram is shown of dividing the screen of a second electronic device into three display areas;

[0120] Figure 14A and Figure 14B According to an embodiment of this application, a scenario diagram is shown where a click gesture is performed on an area other than the icon and screenshot of an application displayed on a second electronic device;

[0121] Figure 15A and Figure 15B According to an embodiment of this application, a scene diagram is shown where the positions of the first electronic device and the second electronic device are determined after a communication connection is established between the first electronic device and the second electronic device.

[0122] Figure 16 According to an embodiment of this application, a flowchart of another human-computer interaction method for an electronic device is shown;

[0123] Figure 17 According to an embodiment of this application, a scenario diagram is shown in which a screenshot of the screen of an electronic device and the icon of the application included in the screenshot are displayed after a gesture operation is performed on the screen of the electronic device.

[0124] Figure 18 An embodiment of this application illustrates a scenario in which a click gesture is performed on an application icon displayed on an electronic device;

[0125] Figure 19 An embodiment of this application illustrates a scenario in which a click gesture is performed on a list of sending objects displayed on an electronic device;

[0126] Figure 20 An embodiment of this application illustrates a scenario where an electronic device establishes a communication connection with multiple electronic devices.

[0127] Figure 21 According to an embodiment of this application, a scene diagram is shown of an electronic device displaying a screenshot of the screen and the icon of the application contained in the screenshot on a second electronic device corresponding to the direction of the gesture operation.

[0128] Figure 22 According to an embodiment of this application, a scene diagram is shown in which an electronic device displays a screenshot of the screen and the icon of the application contained in the screenshot in a partial area of ​​the screen of a second electronic device corresponding to the direction of the gesture operation.

[0129] Figure 23 This is a flowchart illustrating another human-computer interaction method for an electronic device according to an embodiment of this application. Detailed Implementation

[0130] This application includes, but is not limited to, an electronic device and a human-computer interaction method thereof. To make the objectives, technical solutions, and advantages of this application's embodiments clearer, the implementation methods of this application's embodiments will be further described in detail below with reference to the accompanying drawings.

[0131] As mentioned earlier, for the same gesture operation that corresponds to multiple functions, there is a problem of easy misoperation due to the small operating area. For example, Figure 1 This paper illustrates a scenario in which a user performs a gesture operation on the screen of an electronic device 100 to move the window of a document application 101 on the screen of the electronic device 100 to the screen of an electronic device 200 that is communicatively connected to the electronic device 100, thereby enabling the document application 101 to be displayed across screens.

[0132] like Figure 1 As shown, the screen of electronic device 200 can be used as an extension of the screen of electronic device 100. The window of document application 101 is displayed as a floating window on the screen of electronic device 100. After the user performs a single-finger downward drag gesture on the title bar 1011 of the document application 101 window, as shown... Figure 2 As shown, electronic device 100 moves the window of document application 101 onto the screen of electronic device 200. However, because the area of ​​contact of the user's finger on the screen of electronic device 100 is wider than the title bar 1011 of the document application 101 window, the user's finger will touch the edge of the title bar 1011 of the document application 101 window; at the same time, the edge of the title bar 1011 of the document application 101 window supports single-finger drag gestures to change the window size, such as... Figure 3 As shown, a single-finger drag gesture performed by the user on the title bar 1011 of the document application 101 window will result in an erroneous operation that changes the size of the document application 101 window, such as the window height of the document application 101 becoming smaller.

[0133] To address the aforementioned issues, this application proposes a human-computer interaction method for cross-screen display of applications on electronic devices. In this method, in response to a user's first gesture operation on a first screen, the execution result of the first gesture operation is displayed on a second screen. This execution result can be a screenshot of an currently open application window on the first screen. In response to a user's second gesture operation on the execution result displayed on the second screen, the second gesture operation is used to select at least one application within the execution result, moving the selected application window from the first screen to the second screen for display. The first and second screens can belong to the same electronic device or different electronic devices. For example, after a user performs a three-finger swipe gesture on the screen of the first electronic device, a screenshot of the application window of the first electronic device's screen and the icon of the application included in the screenshot are displayed on the screen of the second electronic device. When the user performs a click or drag gesture on the application icon displayed on the screen of the second electronic device, the application corresponding to that icon is moved from the first electronic device to the second electronic device for display, thereby achieving cross-screen display of the application.

[0134] By employing the method of this application embodiment, in scenarios where applications are displayed across screens, users can avoid performing highly precise gesture operations, such as dragging the application's title bar. Instead, they can utilize the execution result of a first gesture operation and then perform a second gesture operation on that result to achieve cross-screen display of the application. For example, performing a click operation on the icon corresponding to the application window shown in the screenshot, corresponding to the execution result of a three-finger swipe screenshot gesture, enables the application to be displayed across screens. This reduces the operational difficulty of the second gesture operation performed by the user, achieves cross-screen display of the application, and ensures that the execution result of the first gesture operation is not affected.

[0135] It is understood that the technical solutions of this application embodiment can be applied to the field of smart terminals with touch screens or touchpads. The electronic device in this application embodiment is a terminal device with an electronic display screen. Common terminal devices include, for example, in-vehicle devices, mobile phones, tablet computers, laptops, PDAs, mobile internet devices (MIDs), wearable devices (e.g., smartwatches, smart bracelets, pedometers, etc.), personal digital assistants, portable media players, navigation devices, video game devices, set-top boxes, virtual reality and / or augmented reality devices, Internet of Things devices, industrial control devices, streaming media client devices, e-books, reading devices, POS machines, and other devices.

[0136] Figure 4A schematic diagram of the structure of an electronic device 100 according to an embodiment of this application is shown. The electronic device 100 may include a processor 110, an internal memory 120, an interface module 130, a power module 140, a wireless communication module 150, and a screen 160.

[0137] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0138] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0139] The processor 110 may also include a memory for storing instructions and data. In this embodiment, the processor 110 can execute the results of gesture operations performed by the user on the electronic device 100.

[0140] The internal memory 120 can be used to store computer executable program code, which includes instructions. The internal memory 120 may include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as document applications, SMS applications, and browsers), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, phonebook entries, etc.). In this embodiment, the internal memory 120 may store application icons.

[0141] Interface module 130 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of electronic device 100. The external memory card communicates with processor 110 through interface module 130 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0142] The power module 140 receives input from the battery and supplies power to the processor 110, internal memory 120, display screen 111, etc.

[0143] The wireless communication module 150 can provide wireless communication solutions for electronic devices 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, etc.

[0144] Screen 160, which may be a touchscreen composed of a touch sensor and a display screen, also known as a "touchscreen," is used to detect gestures performed on or near it. Screen 160 can transmit the detected gestures to processor 110 to determine the type of gesture. In this embodiment, screen 160 is used to receive gestures performed by the user. For example, screen 160 can determine the type of gesture performed by the user through a touch sensor.

[0145] After introducing the hardware structure of the electronic device 100 in the embodiments of this application, the software structure block diagram of the electronic device 100 is described below.

[0146] Figure 5 This is a software structure block diagram of the electronic device 100 according to an embodiment of the present invention.

[0147] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0148] The application layer can include a series of application packages.

[0149] like Figure 5 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS. In this embodiment, the application package may include a launcher document application, an SMS application 102, and a browser 103.

[0150] The application framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer.

[0151] like Figure 5 As shown, the application framework layer may include: extended screen manager, window manager, content provider, view system, resource manager, notification manager, gesture manager, application manager, etc.

[0152] The gesture manager is used to recognize the gestures performed by the user on the screen of the electronic device 100.

[0153] The extended screen manager is used to identify electronic devices connected to electronic device 100 via wired or wireless means, and configure the screen of the electronic device as an extended screen of electronic device 100. In this embodiment, the extended screen manager can also determine the positional relationship between the extended screen and electronic device 100 based on the positional relationship between the electronic device and electronic device 100.

[0154] A window manager is used to manage window programs. The window manager can obtain the screen size, determine if a status bar is present, lock the screen, and capture the screen, etc. In this embodiment, the window manager can display the applications already launched on the electronic device 100 using thumbnails or split-screen mode.

[0155] Content providers store and retrieve data, making that data accessible to applications. This data can include videos, images, audio, phone calls made and received, browsing history and bookmarks, phone books, and more.

[0156] The view system includes visual controls, such as controls for displaying text and controls for displaying images. The view system can be used to build applications. The display interface can consist of one or more views, which are used to display visual controls within their respective areas and to handle events occurring in those areas. In this embodiment, the view system can display the execution results of gesture operations performed on the electronic device 100 on the screen of the electronic device 200 connected to the electronic device 100. Simultaneously, a display area can be set to one side of the execution results displayed on the screen of the electronic device 200, where the icon of the application displayed on the screen of the electronic device 100 is displayed.

[0157] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0158] The notification manager allows applications to display notification information in the status bar. It can be used to convey informational messages and can disappear automatically after a short time without user interaction.

[0159] An application manager is used to obtain information about running applications, including their names and package names. In this embodiment, the application manager can determine the applications displayed on the screen of the electronic device 100 and their names.

[0160] The Android Runtime consists of core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system.

[0161] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0162] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0163] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0164] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0165] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0166] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0167] A 2D graphics engine is a graphics engine for 2D drawing.

[0168] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0169] The following is through Figure 6 This application provides a detailed description of the cross-screen display method provided in its embodiments. Figure 6The cross-screen display solution in this application is used on electronic devices. Figure 6 The described electronic device could be a tablet computer.

[0170] Specifically, such as Figure 6 As shown, the cross-screen display methods for the above applications include:

[0171] S601: Electronic device 100 displays a first interface, the first interface including one or more application windows, each application window corresponding to an application.

[0172] After the user turns on electronic device 100, such as Figure 7 As shown, a user can click the "Document" icon 101 in the user interface (UI) of the electronic device 100. After receiving the user's click instruction, the electronic device 100 launches the document application 101. Similarly, the electronic device 100 can also launch the SMS application 102 and the browser 103 in sequence.

[0173] Electronic device 100 can display multiple application windows simultaneously. For example, a user can perform a simultaneous display operation on electronic device 100, causing the window manager of electronic device 100 to simultaneously display the application windows of document application 101, SMS application 102, and browser application 103 on the screen of electronic device 100, forming the first interface of electronic device 100. For example, the simultaneous display operation could be a user performing a long press for 3 seconds and then swiping upwards at the bottom edge of the screen of electronic device 100. In one embodiment of this application, after the user performs the simultaneous display operation, such as... Figure 7 As shown, the application windows of document application 101, SMS application 102 and browser 103 are displayed in a thumbnail on the screen of electronic device 100.

[0174] It is understood that, in another embodiment of this application, the window manager of the electronic device 100 can also divide the screen of the electronic device 100 into multiple windows, displaying the document application 101, the SMS application 102, and the browser 103 in a split-screen manner, combining a main window with multiple auxiliary windows to display the document application 101, the SMS application 102, and the browser 103 respectively, allowing the user to focus on the application in each window. Alternatively, when opening various applications, the electronic device 100 can use a non-full-screen window by default, ensuring that the application windows of each opened application are displayed on the screen.

[0175] S602: Electronic device 100 receives the first gesture operation.

[0176] The electronic device 100 can recognize the first gesture operation performed by the user through a gesture manager. For example, in this embodiment, the first gesture operation may be a three-finger swipe gesture performed by the user on the screen of the electronic device 100. Figure 8 As shown, the three-finger swipe gesture, which is the first gesture operation, can specifically be a vertical swipe down with three fingers. This gesture operation can be applied to any position on the screen.

[0177] In another embodiment of this application, the first gesture operation may also be other gesture operations or button functions, such as performing a gesture operation of pressing and holding the bottom of the screen of the electronic device 100 and swiping up, or a two-finger swipe down operation, or a tapping operation of one or more knuckles, etc.

[0178] S603: Electronic device 100 determines whether there is an external electronic device 200. If there is, S605 is executed; if not, S604 is executed.

[0179] Electronic device 100 can determine the presence of an external electronic device 200, for example, by determining that there is an electronic device 200 connected to it as an extended screen; or by determining that there is an electronic device 200 connected to it wirelessly or wiredly. Here, an extended screen refers to another screen that shares a host with electronic device 100. When electronic device 200 and electronic device 100 are on the same local area network, or when electronic device 200 and electronic device 100 are logged into the same user account, it can be assumed that electronic device 200 and electronic device 100 are wirelessly connected, i.e., that an external electronic device 200 exists. Multiple extended screens may be connected to electronic device 100 simultaneously, and multiple external electronic devices 200 may exist simultaneously; in this case, when identifying an external extended screen or electronic device 200, electronic device 100 can further identify the orientation of the external device relative to electronic device 100.

[0180] In another embodiment of this application, step S603 is optional. The electronic device 100 may also respond directly to the first gesture operation without determining whether an external electronic device 200 is connected. For example, when the electronic device 100 is not connected to an external electronic device 200, the electronic device 100 takes a screenshot of the first interface of the screen and saves it. If there is an external electronic device 200, the electronic device 200 displays the execution result of the first gesture operation, that is, the electronic device 200 displays a screenshot of the first interface of the screen of the electronic device 100 and the application identifier corresponding to the application window included in the first interface. The electronic device 100 can also display the screenshot of the first interface, and further, the electronic device 100 can save the screenshot of the first interface. Here, the screenshot of the first interface refers to a snapshot of the first interface; the screenshot is an image, and the content displayed is the first interface. Typically, the electronic device 100 displays the first interface in full screen, so the screenshot of the first interface can be a screen snapshot of the electronic device 100 when displaying the first interface, that is, the screenshot is the content currently displayed on the screen of the electronic device 100.

[0181] S604: In response to the first gesture operation, the electronic device 100 displays the first execution result of the first gesture operation.

[0182] For example, if the first gesture is a three-finger vertical swipe down as described in step S602, the execution result of the electronic device 100 responding to the three-finger vertical swipe down gesture can be a screenshot of the first interface of the screen of the electronic device 100. The screenshot includes the application windows of the document application 101, the SMS application 102, and the browser 103 displayed in thumbnail on the screen of the electronic device 100.

[0183] When electronic device 100 is not connected to electronic device 200 or other electronic devices, such as Figure 8 As shown, the electronic device 100 can display the execution result of the first gesture operation on its screen, which is a thumbnail of the first interface of the screen 100. The user can click the thumbnail to display the screenshot in full screen on the electronic device 100. The user can further edit the screenshot, which will not be elaborated here.

[0184] It should be noted that, in another implementation, if it is determined in S603 that an external electronic device 200 exists, then S604 and S605 can be executed simultaneously. That is, in response to the first gesture operation, the electronic device 100 displays a screenshot of the first interface on its own screen, and the electronic device 100 sends the screenshot of the first interface and the application icon corresponding to the application window included in the first interface to the electronic device 200.

[0185] S605: In response to the first gesture operation, electronic device 100 sends the second execution result of the first gesture operation to electronic device 200.

[0186] It is understood that, when electronic device 100 determines that there is an external electronic device 200 connected, electronic device 100 then sends the second execution result of the first gesture operation to electronic device 200. The second execution result may include: a screenshot of the first interface of the screen of electronic device 100 and the application identifier corresponding to the application window included in the first interface. That is, when an external electronic device 200 is present, electronic device 100 can send a screenshot of the first interface and the application identifier corresponding to the application window included in the first interface to electronic device 200. The application identifier may be an application icon or other information used to identify an application.

[0187] S606: Electronic device 200 displays a screenshot of the first interface and the application icons corresponding to the application windows included in the first interface.

[0188] like Figure 9 As shown, the electronic device 200 displays a screenshot 2001 of the first interface, which includes three windows: a document application 101, a text messaging application 102, and a browser 103. Furthermore, the electronic device 200 also displays application icons corresponding to each application window included in the first interface on the screen. The electronic device 200 can obtain the application icon corresponding to the application identifier sent by the electronic device 100. For example, the electronic device 200 sequentially displays the icons 2002 corresponding to the document application 101, text messaging application 102, and browser 103 above the screenshot 2001 of the first interface. Moreover, this embodiment does not limit the specific location of these application icons. For example, as... Figure 9 As shown, the icons 2002 corresponding to the document application 101, the SMS application 102, and the browser 103 can be aligned with the document application 101, the SMS application 102, and the browser 103 in the screenshot 2001. Alternatively, in another embodiment, these icons may not be perfectly aligned with the windows in the screenshot.

[0189] Specifically, the aforementioned application identifier can be actively sent from electronic device 100 to electronic device 200, or it can be obtained from electronic device 100 by electronic device 200. For example, electronic device 100 can know the application identifier corresponding to the currently open application window. When sending the second execution result to electronic device 200 in response to the first gesture operation, electronic device 100 can actively send the application identifier or application icon to electronic device 200. The application icon can also be considered a type of application identifier. Alternatively, electronic device 200 can obtain the aforementioned application identifier from electronic device 100 through the following process: electronic device 200 obtains the currently running application (e.g., getRecentTasks) from electronic device 100's application manager (e.g., ActivityManager in Android); electronic device 200 then obtains the name of the currently running application and retrieves the application icon based on the name.

[0190] Next, continue to refer to Figure 9 The electronic device 200 displays the icons of the acquired applications and a screenshot of the first interface on its screen. Taking document application 101, SMS application 102, and browser 103 as examples, the electronic device 200 can display screenshot 2001 in a certain area of ​​the screen, such as the lower left, lower right, upper left, upper right, left side, right side, top, and bottom. Simultaneously, the electronic device 2000 can display the icons of the applications included in the screenshot near screenshot 2001, such as displaying icons above the screenshot, or displaying icons 2002 below, to the left, or to the right of the screenshot. In icon 2002, the three windows—document application 101, SMS application 102, and browser 103—can be displayed sequentially according to their order in the screenshot.

[0191] S607: Electronic device 200 receives a second gesture operation.

[0192] Here, the second gesture operation applied to the screen of the electronic device 200 may include the following: the user performs a click gesture operation on the screenshot of the first interface displayed on the electronic device 200, or the user performs a click gesture operation on the application icon displayed above the screenshot, or the user performs a drag gesture operation on the application icon displayed above the screenshot, or the user performs a gesture operation on an area other than the application icon and the screenshot.

[0193] It is understood that in another embodiment of this application, when a user performs a second gesture operation on the screen of an electronic device 200, the electronic device 200 can send the instruction corresponding to the gesture operation to the electronic device 100, and the electronic device 100 can display the execution result corresponding to the instruction on the screen of the electronic device 200 according to the received instruction.

[0194] S608: Electronic device 200 displays the execution result corresponding to the second gesture operation.

[0195] Different second gesture operations correspond to different execution results. For example, a second gesture operation could be a click gesture on a screenshot displayed on the electronic device 200. The corresponding execution result would be that the electronic device 100 can execute a method similar to that in step S604. For example, as... Figure 10A As shown, the user clicks on screenshot 2001 on electronic device 200, and in response to the click, as follows: Figure 10B As shown, the screenshot is displayed in full screen on the screen of electronic device 100, allowing the user to further edit the screenshot; simultaneously, electronic device 200 no longer displays screenshot 2001 and icon 2002. Alternatively, if the user clicks on screenshot 2001 on electronic device 200, the screenshot will be displayed in full screen on the screen of electronic device 200 in response to the click, allowing the user to edit or save the screenshot on electronic device 200.

[0196] For example, the second gesture operation could be a click gesture on the icon corresponding to the application identifier sent by electronic device 100. The corresponding execution result is: electronic device 200 opens the application corresponding to the icon and displays the application window; simultaneously, electronic device 100 removes the application window from the screen. In other words, the user moves the application from the screen of electronic device 100 to the screen of electronic device 200 by clicking the application icon. When the application is moved to electronic device 200 for display, it can be displayed in full-screen mode or not. For example, ... Figure 11A As shown, the icons 2002 corresponding to the document application 101, SMS application 102, and browser 103 can be displayed on one side above the screenshot 2001. When a user clicks the icon of the document application 101 among the icons 2002 displayed on the screen of the electronic device 200, the electronic device 200 sends the click operation instruction to the electronic device 100. This click operation instruction may include: the application name corresponding to the clicked icon and the type of click operation. In response to this click operation instruction, such as... Figure 11B As shown, the electronic device 100 moves the application window of the document application 101 to the screen of the electronic device 200 for display.

[0197] For example, the second gesture operation could also be a drag-and-drop gesture on the icon corresponding to the application identifier sent by electronic device 100. The corresponding execution result is: based on the final position of the application icon after the drag-and-drop operation, electronic device 200 can open the application corresponding to the icon on its screen using left split-screen, right split-screen, or full-screen mode, and display the application window. Simultaneously, electronic device 100 removes the application window from the screen. In other words, the user can move the application from the screen of electronic device 100 to the screen of electronic device 200 by dragging the application icon, and the display method can be determined based on the final position of the application icon. For example, ... Figure 12A As shown, the user presses and holds the icon of document application 101 in icon 2002 displayed on the screen of electronic device 200, drags the icon to the left side of the screen of electronic device 200, and then releases the icon; at this time, as shown Figure 12B As shown, electronic device 200 sends the drag-and-drop instruction to electronic device 100. The drag-and-drop instruction may include: the location where the user drags and releases the icon, the application name corresponding to the icon, and the type of drag-and-drop operation. The instruction responding to the click operation is as follows: Figure 12B As shown, the electronic device 100 moves the application window of the document application 101 to the left split screen of the screen of the electronic device 200. At this time, the screen of the electronic device 100 displays a second interface, which includes the application windows of the SMS application 102 and the browser 103.

[0198] In this embodiment, determining the display method of an application on the screen of an electronic device 200 based on the final position of the application icon dragged and released by the user can be achieved through the following methods. For example, as... Figure 13 As shown, the electronic device 200 can divide its screen into three response areas: a left split-screen response area, a right split-screen response area, and a full-screen response area. These three areas correspond to left split-screen, right split-screen, or full-screen display modes, respectively. The electronic device 100 can, through a communication connection with the electronic device 200, detect in real-time which area on the screen of the electronic device 200 the final position of the application icon dragged and released by the user falls into, thereby determining the display mode of the application on the screen of the electronic device 200. Specifically, the method for dividing the left and right split-screen response areas can be that the left split-screen response area corresponds to the left half of the screen of the electronic device 200, and the right split-screen response area corresponds to the right half of the screen. For the full-screen response area, the electronic device 200 can divide the area formed by 100 pixels vertically downwards from the top edge of the screen as the full-screen response area. In this embodiment, from... Figure 13As can be seen, the areas where the left and right split-screen response areas overlap with the full-screen response area are considered full-screen response area overlaps. It's important to understand that the 100 pixels here are exemplary; any other value can be used.

[0199] In another embodiment of this application, the electronic device 200 can also detect in real time which area of ​​the screen of the electronic device 200 the final position of the application icon dragged and released by the user falls in, thereby determining the display mode of the application window on the screen of the electronic device 200; after receiving the application window sent by the electronic device 100, the electronic device 200 displays the application window according to the determined display mode.

[0200] It is understandable that if a user performs a gesture operation on an area outside the application icon and screenshot, electronic device 200 can delete the current application screenshot and application icon from its screen. Simultaneously, it can display a message on electronic device 100 indicating that the screenshot has been saved and show the save location. For example, as... Figure 14A As shown, the user performs a click operation on an area outside of screenshot 2001 and icon 2002 on the screen of electronic device 200. At this time, as... Figure 14B As shown, electronic device 100 can delete screenshot 2001 and icon 2002 on the screen of electronic device 200, and display a message on the screen of electronic device 100 indicating that the current screen screenshot of electronic device 100 has been saved. In other words, electronic device 100 completes the operation of taking a screenshot of the current screen of electronic device 100 in response to the instruction of the three-finger downward swipe gesture.

[0201] exist Figure 6 In the described steps S601 to S608, the electronic device 100 can communicate with one or more other electronic devices through wired connections, such as USB (Universal Serial Bus) or wireless connections, such as WIFI (Wireless Fidelity) or Bluetooth.

[0202] For example, such as Figure 15A As shown, the method for establishing a communication connection between electronic device 100 and electronic device 200 includes: electronic device 100 searching for electronic device 200 connected to electronic device 100 via wired or wireless means, such as tablet computer 200; electronic device 100 sending a communication connection request to electronic device 200; electronic device 200 displaying a prompt asking whether the user agrees to establish a communication connection; and after the user clicks to confirm, as shown... Figure 15BAs shown, the screen of electronic device 100 can indicate the position of the screen of electronic device 200 relative to the screen of electronic device 100, and the user can choose to position the screen of electronic device 200 below the screen of electronic device 200. It can be understood that electronic device 100 can also establish an extended display connection with electronic device 200, and the screen of electronic device 200 can be configured as an extension screen of the screen of electronic device 100.

[0203] It is understood that, in another embodiment of this application, electronic device 100 can automatically determine the position of the screen of electronic device 200 relative to the screen of electronic device 100 based on the spatial relationship between electronic device 100 and electronic device 200, without requiring user selection. For example, electronic device 100 can determine the spatial relationship between electronic device 100 and electronic device 200 based on the direction of the WIFI signal between electronic device 100 and electronic device 200 or based on UWB (Ultra Wideband) positioning technology, and thus determine the position of the screen of electronic device 200 relative to the screen of electronic device 100.

[0204] In another embodiment of this application, the electronic device 100 may also determine the position of the screen of the electronic device 200, which is communicatively connected to the electronic device 100, relative to the screen of the electronic device 100, according to the default configuration of the operating system of the electronic device 100. For example, the operating system of the electronic device 100 defaults to setting the screen of the electronic device 200, which is communicatively connected to the electronic device 100, on the right side of the screen of the electronic device 100.

[0205] The above Figure 6 The described steps S601 to S608 illustrate a method for cross-screen display where a first gesture operation is performed on electronic device 100, and a second gesture operation is performed on the result of the first gesture operation, thereby moving the application window from electronic device 100 to the screen of electronic device 200. Furthermore, the following describes... Figure 16 A display method for another application of an electronic device 100 according to an embodiment of this application will be described in detail. Embodiment of this application Figure 16 The display scheme of the application in the electronic device 100 can be implemented by executing relevant programs through the processor 110. Figure 16 In this process, based on the execution result of supported gesture operations, such as three-finger swipe operations, electronic device 100 adds judgment conditions or subsequent gesture operations to share the application window with other electronic devices connected to electronic device 100. Figure 16 The electronic devices described herein can be tablets, in-vehicle systems, and mobile phones.

[0206] Specifically, such as Figure 16As shown, the display methods for the above applications include:

[0207] S1601: Electronic device 100 displays a first interface, the first interface including one or more application windows, each application window corresponding to an application.

[0208] Here, step S1601 can be the same as S601, and the user can launch the document application 101, the SMS application 102, and the browser 103 on the electronic device 100. The electronic device 100 displays a first interface, which includes application windows of the document application 101, the SMS application 102, and the browser 103 displayed in a thumbnail format.

[0209] S1602: Electronic device 100 receives the first gesture operation.

[0210] Here, step S1602 can be the same as S602. For example, in this embodiment, the first gesture operation can be a three-finger swipe gesture operation. The execution result corresponding to the three-finger swipe gesture operation can be to take a screenshot of the first interface of the screen of the electronic device 100 and display the screenshot on the screen of the electronic device 100.

[0211] S1603: In response to the first gesture operation, the electronic device 100 displays the first execution result of the first gesture operation.

[0212] Step S1603 here is similar to step S604. For example, in the case where the first gesture operation is a three-finger vertical slide as described in step S1602, such as... Figure 17 As shown, the execution result of the electronic device 100 responding to the three-finger vertical swipe gesture operation can be: the electronic device 100 acquires a screenshot of the first interface of the electronic device 100 containing the current document application 101, SMS application 102, and browser 103, and simultaneously acquires the corresponding icons of document application 101, SMS application 102, and browser 103. The difference from step S607 is that the screenshot 1001 of document application 101, SMS application 102, and browser 103 displayed on the screen of the electronic device 100, and the corresponding icons 1002 of document application 101, SMS application 102, and browser 103 displayed in the display area above the screenshot 1001, as shown... Figure 17 As shown, the icons for document application 101, SMS application 102, and browser 103 can each have a share icon attached.

[0213] S1604: Electronic device 100 receives the second gesture operation and displays the execution result of the second gesture operation.

[0214] The second gesture operation here is a click gesture performed on the application icon displayed on the electronic device 100. That is, a click on the application icon displayed on the electronic device 100. Figure 17 The second gesture operation involves clicking the icons of document application 101, SMS application 102, and browser 103 in icon 1002. The execution result of this second gesture operation could be: the electronic device 100 displays a list of recipients on the screen; this list could include other electronic devices connected to the electronic device 100. For example... Figure 18 As shown, electronic device 100 establishes a Wi-Fi connection with electronic devices 300 and 400. Then, when the user clicks the browser icon 103, a window 1003 appears on the screen of electronic device 100, displaying options for both electronic devices 300 and 400.

[0215] S1605: Electronic device 100 acquires a third gesture operation and sends the execution result of the third gesture operation.

[0216] The third gesture operation here can be: [e.g., ...] Figure 18 The electronic device 100 shown illustrates a tap gesture operation performed on a sending object in a sending object list displayed on the screen. For example, on a... Figure 18 The screen of the electronic device 100 displays a window containing two options: electronic device 300 and electronic device 400. Clicking one of these options is designated as the recipient. The execution result of the third gesture operation can be: the electronic device 100 can send the display content of the application corresponding to the icon of the second gesture operation in step S1604 (i.e., the click operation) to the recipient for display. For example, if the user performs a click gesture on the browser 103 icon on the screen of the electronic device 100 in step S1604, and then clicks the option for electronic device 400 in window 1003 (the recipient list) containing the two options for electronic device 300 and electronic device 400, as shown... Figure 19 As shown, electronic device 400 launches a browser application and displays the content of browser 103 of electronic device 100.

[0217] It is understood that, in the embodiments of this application Figure 15A and Figure 15B In the illustrated scenario, electronic device 100 is only connected to electronic device 200, such as tablet computer 200, with the screen of electronic device 200 positioned directly below the screen of electronic device 100. However, in other embodiments of this application, electronic device 100 can be connected to any number of electronic devices; that is, electronic device 100 can connect to the screens of any number of electronic devices, for example, 2, 4, or 8, without limitation.

[0218] Figure 20The illustration shows a scenario where an electronic device 100 is connected to the screens of two other electronic devices, such as... Figure 20 As shown, the user positions the screens of electronic device 1101 and electronic device 1102 at the lower left and lower right corners of the screen of electronic device 100, respectively. Figure 20 In the scenario where the electronic device 100 already supports gesture operations, the method for moving the application window from the electronic device 100 to the electronic device 1101 or 1102 can be the same as... Figure 6 Steps S601 to S609 are described similarly. The difference lies in that, taking a three-finger swipe as an example, the direction of the three-finger swipe gesture can correspond to the direction between the screens of electronic devices 1101 and 1102 and electronic device 100, such as a three-finger swipe to the lower right and a three-finger swipe to the lower left. Simultaneously, electronic device 100 can display the result of the three-finger swipe—that is, a screenshot of its screen and the application icon included in the screenshot—within the display area on the side of the screen closest to electronic device 100 in the direction of the three-finger swipe gesture. For example... Figure 21 As shown, after the user performs a three-finger swipe gesture on the screen of electronic device 100, the electronic device 100 displays screenshot 11011 on the upper right side of the screen of electronic device 1101, that is, close to the screen of electronic device 100. At the same time, the electronic device 100 displays icon 11012 above the screenshot 11011, and the icons are displayed in the order of document application 101, SMS application 102 and browser 103.

[0219] It is understood that in another embodiment of this application, taking a three-finger swipe gesture supported by the electronic device 100 as an example, if the electronic device 100 detects that there is no connected electronic device in the direction of the three-finger swipe gesture, the electronic device 100 can choose to display the execution result of the gesture operation and the application flag contained in the execution result on the screen of an electronic device close to the gesture direction, while ensuring that the position of the execution result and the application flag contained in the execution result matches the gesture direction. For example, as Figure 22 As shown, electronic device 100 is extended to electronic device 200, with electronic device 200 located directly below electronic device 100. When a user performs a three-finger swipe on electronic device 100, it is a three-finger swipe to the lower left. If electronic device 100 determines that there is no other electronic device in the lower left corner, it can display a screenshot 2001 of its own screen and an icon 2002 of the application contained in the screenshot 2001 on the screen of electronic device 2000. Simultaneously, screenshot 2001 and icon 2002 can be located on the lower left side of the screen of electronic device 100.

[0220] Figure 23 This is a flowchart illustrating another human-computer interaction method for an electronic device according to an embodiment of this application. The method is applied to an electronic device with dual screens. The electronic device includes a first screen and a second screen.

[0221] S701: The first screen of the electronic device displays a first interface, which includes one or more application windows, each application window corresponding to an application.

[0222] S702: The first screen receives the first gesture operation, that is, the electronic device receives the first gesture operation applied to the first screen.

[0223] S703: In response to the first gesture operation, the electronic device controls the second screen to display a screenshot of the first interface and the application icon corresponding to the application window included in the first interface.

[0224] S704: The second screen receives the second gesture operation, that is, the electronic device receives the second gesture operation applied to the second screen.

[0225] S705: In response to the second gesture operation, the electronic device controls the display of the second screen. Specifically, based on different second gesture operations, the electronic device controls the second screen to display different information.

[0226] For example, if the second gesture operation is a click gesture on a screenshot displayed on the second screen, the electronic device controls the second screen or the first screen to display the screenshot in full screen, and then the user can operate on the screenshot, or the electronic device automatically saves the screenshot after a preset time.

[0227] If the second gesture is a click on the first icon displayed on the second screen, the electronic device opens the application corresponding to the first icon on the second screen and displays the application window. For example, the application window can be displayed in full screen or not. The content of the application window displayed on the second screen is the same as that displayed on the first screen. In other words, the second gesture achieves the migration of the application window from the first screen to the second screen.

[0228] If the second gesture is a drag-and-drop gesture on the first icon displayed on the second screen, the electronic device opens the application corresponding to the first icon on the second screen and displays the application window in different positions based on the direction of the drag. For example, if the user drags the first icon to the left of the screen, the electronic device can display the application window corresponding to the first icon in a split-screen format on the left side of the second screen; if the user drags the first icon to the right of the screen, the electronic device can display the application window corresponding to the first icon in a split-screen format on the right side of the second screen. The drag-and-drop operation can be performed by pressing and holding the icon and dragging it on the screen, or by selecting the icon and then sliding it on the screen, in which case the icon may not be dragged.

[0229] Furthermore, the relevant content in the embodiments of this application can be referred to the descriptions in the foregoing embodiments.

[0230] It should be understood that although the terms "first," "second," etc., may be used herein to describe various features, these features should not be limited by these terms. The use of these terms is merely for distinction and should not be construed as indicating or implying relative importance. For example, without departing from the scope of the exemplary embodiments, a first feature may be referred to as a second feature, and similarly, a second feature may be referred to as a first feature.

[0231] Furthermore, the various operations will be described as multiple separate operations in a manner most conducive to understanding the illustrative embodiments; however, the order of description should not be construed as implying that these operations must depend on the order of description, and many of these operations may be performed in parallel, concurrently, or simultaneously. Moreover, the order of the operations may also be rearranged. When the described operations are completed, the process may be terminated, but additional operations not included in the figures may also be possible. A process may correspond to a method, function, procedure, subroutine, subroutine, etc.

[0232] References to "an embodiment," "an embodiment," "an illustrative embodiment," etc., in this specification indicate that the described embodiment may include specific features, structures, or properties; however, each embodiment may or may not necessarily include specific features, structures, or properties. Furthermore, these phrases are not necessarily directed to the same embodiment. Additionally, when specific features are described in conjunction with specific embodiments, the knowledge of those skilled in the art can influence the combination of these features with other embodiments, whether or not those embodiments are explicitly described.

[0233] Unless the context otherwise specifies, the terms “comprising,” “having,” and “including” are synonyms. The phrase “A / B” means “A or B.” The phrase “A and / or B” means “(A), (B), or (A and B).”

[0234] As used herein, the term “module” may refer to, as part of, or include: a memory (shared, dedicated, or grouped) for running one or more software or firmware programs, an application-specific integrated circuit (ASIC), electronic circuitry and / or a processor (shared, dedicated, or grouped), combinational logic circuitry, and / or other suitable components that provide the said functionality.

[0235] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order is not necessary. Rather, in some embodiments, these features may be illustrated in a manner and / or order different from that shown in the illustrative drawings. Furthermore, the inclusion of structural or methodological features in a particular drawing does not mean that all embodiments need to include such features; in some embodiments, these features may be omitted, or they may be combined with other features.

[0236] The embodiments of this application have been described in detail above with reference to the accompanying drawings. However, the use of the technical solutions of the embodiments of this application is not limited to the various applications mentioned in the embodiments of this patent. Various structures and modifications can be easily implemented with reference to the technical solutions of the embodiments of this application to achieve the various beneficial effects mentioned herein. Within the scope of knowledge possessed by those skilled in the art, various changes made without departing from the spirit of the embodiments of this application should be attributed to the patent coverage of the embodiments of this application.

Claims

1. A human-computer interaction method, characterized in that, include: The second electronic device receives a first image and identification information of at least one application sent by the first electronic device according to the first operation. The first image is a screenshot of the first interface currently displayed by the first electronic device, the at least one application is the application corresponding to the application window opened in the first interface, and the identification information is the icon of the application. The second electronic device displays the first image and the icon of the at least one application, wherein the first image covers a portion of the display area of ​​the second electronic device; The second electronic device receives a second operation applied to a first icon, which is one of the icons of the at least one application; In response to the second operation, the second electronic device displays the application window of the first application corresponding to the first icon, and the content of the application window of the first application displayed by the second electronic device is the same as that of the application window of the first application opened in the first interface.

2. The method according to claim 1, characterized in that, In response to the second operation, the second electronic device displays the application window of the first application corresponding to the first icon, including: In response to the second operation, the second electronic device sends a first instruction to the first electronic device; The second electronic device receives first information sent by the first electronic device in response to the first instruction, wherein the first information is used to represent the application window of the first application corresponding to the first icon; The second electronic device displays the application window of the first application corresponding to the first icon based on the first information.

3. The method according to claim 2, characterized in that, Before the second electronic device receives the first image and the identification information of the at least one application sent by the first electronic device, the method further includes: The first electronic device displays the first interface in full screen, and the first interface includes at least one application window corresponding to the at least one application. The first electronic device receives the first operation applied to the first interface; In response to the first operation, the first electronic device sends the first image and the identification information of the at least one application to the second electronic device.

4. The method according to claim 2, characterized in that, The second electronic device displays the application window of the first application corresponding to the first icon based on the first information, including: The second electronic device displays the application window of the first application corresponding to the first icon in full screen according to the first information and the second operation.

5. The method according to claim 2, characterized in that, The second electronic device displays the application window of the first application corresponding to the first icon based on the first information, including: Based on the first information and the second operation, the second electronic device displays the application window of the first application corresponding to the first icon on the left side of the screen.

6. The method according to claim 5, characterized in that, The second operation includes dragging the first icon to a first area of ​​the screen of the second electronic device, the first area being located on the left side of the screen of the second electronic device.

7. The method according to claim 2, characterized in that, The second electronic device displays the application window of the first application corresponding to the first icon based on the first information, including: Based on the first information and the second operation, the second electronic device displays the application window of the first application corresponding to the first icon on the right side of the screen.

8. The method according to claim 7, characterized in that, The second operation includes dragging the first icon to a first area of ​​the screen of the second electronic device, the first area being located on the right side of the screen of the second electronic device.

9. The method according to claim 2, characterized in that, The second electronic device displays the application window of the first application corresponding to the first icon based on the first information, including: The second electronic device displays the application window of the first application corresponding to the first icon at the top of the screen based on the first information and the second operation.

10. The method according to claim 9, characterized in that, The second operation includes dragging the first icon to a first area of ​​the screen of the second electronic device, the first area being located above the screen of the second electronic device.

11. The method according to claim 2, characterized in that, The second electronic device displays the application window of the first application corresponding to the first icon based on the first information, including: Based on the first information and the second operation, the second electronic device displays the application window of the first application corresponding to the first icon at the bottom of the screen.

12. The method according to claim 1, characterized in that, The second operation includes dragging the first icon to a first area of ​​the screen of the second electronic device, the first area being located below the screen of the second electronic device.

13. The method according to any one of claims 1-12, characterized in that: The second electronic device is wirelessly or wiredly connected to the first electronic device; or The second electronic device is on the same wireless local area network as the first electronic device; or The second electronic device is logged into the same user account as the first electronic device; or The second electronic device is an extension screen of the first electronic device; or The second electronic device and the first electronic device each include two screens in the same electronic device.

14. The method according to claim 13, characterized in that, The first operation is a three-finger swipe operation.

15. The method according to claim 13, characterized in that, The icon of at least one application is displayed above, below, to the left, or to the right of the first image.

16. A human-computer interaction method, characterized in that, Applied to electronic devices including a first screen and a second screen, including: The first screen displays a first interface in full screen, and the first interface includes at least one application window corresponding to at least one application. The first screen receives a first operation applied to the first interface; In response to the first operation, the electronic device controls the second screen to display a first image and the icon of the at least one application, wherein the first image is a screenshot of the first interface and the first image covers a portion of the display area of ​​the second screen; The second screen receives a second operation applied to a first icon, which is one of the icons of the at least one application; In response to the second operation, the second screen displays the application window of the first application corresponding to the first icon, and the content of the application window of the first application displayed on the second screen is the same as that of the application window of the first application opened in the first interface.

17. The method according to claim 16, characterized in that, The first operation includes at least one of the following on the first interface: swiping up, swiping down, swiping left, swiping right, swiping down to the left, swiping down to the right, swiping up to the left, and swiping up to the right.

18. The method according to claim 16 or 17, characterized in that, In response to the second operation, the second screen displays the application window of the first application corresponding to the first icon, including: In response to the second operation, the electronic device controls the second screen to display the application window of the first application corresponding to the first icon.

19. The method according to any one of claims 16-17, characterized in that, In response to the second operation, the second screen displays the application window of the first application corresponding to the first icon, including: In response to the second operation, the second screen displays the application window of the first application corresponding to the first icon in full screen.

20. The method according to any one of claims 16-17, characterized in that, In response to the second operation, the second screen displays the application window of the first application corresponding to the first icon, including: In response to the second operation, the second screen displays the application window of the first application corresponding to the first icon on the left side of the screen.

21. The method according to claim 20, characterized in that, The second operation includes dragging the first icon to a first area of ​​the second screen, the first area being located on the left side of the second screen.

22. The method according to any one of claims 16-17, characterized in that, In response to the second operation, the second screen displays the application window of the first application corresponding to the first icon, including: In response to the second operation, the second screen displays the application window of the first application corresponding to the first icon on the right side of the screen.

23. The method according to claim 22, characterized in that, The second operation includes dragging the first icon to a first area of ​​the second screen, the first area being located on the right side of the second screen.

24. The method according to claim 23, characterized in that, In response to the first operation, the electronic device displays a screenshot of the first interface on the first screen.

25. The method according to claim 24, characterized in that, The first operation is a three-finger swipe operation.

26. The method according to claim 24, characterized in that, The icon of at least one application is displayed above, below, to the left, or to the right of the first image.

27. An electronic device, characterized in that, include: Memory, wherein instructions are stored, and A processor for reading and executing instructions in the memory to cause the electronic device to perform the method as described in any one of claims 1-15.

28. An electronic device, characterized in that, include: Memory, wherein instructions are stored, and A processor for reading and executing instructions in the memory to cause the electronic device to perform the method as described in any one of claims 16-25.

Images