Screen projection method of application window and electronic equipment
By implementing multiple screen projections of application windows between mobile phones and PCs, the problems of low usage rate and poor display effect in the existing technology are solved, and the screen projection experience and efficiency are improved.
Patent Information
- Application Number
- CN202410964298.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2020-10-31
- Publication Date
- 2024-12-06
AI Technical Summary
When an existing screen projection technology is projecting a screen to a PC by a mobile phone, the user needs to operate on one device, and the interface usage rate on the other device is low. Due to the small display screen, the screen projection process is not convenient enough. Efficient.
Provide a screen projection method of application window, allowing the first electronic device to establish a connection with the second electronic device, and project the screens separately from the application. Users can view the address book window on the mobile phone on the PC and on the mobile phone. View the chat window, or project images or videos directly to the PC to display, achieving finer screen projection granularity and convenient operation.
Improve the screen projection experience, improve the distribution and viewing efficiency of multiple windows, and avoid the poor interface caused by the small screen of the device. Users can more conveniently operate and view different windows of multiple applications.
Smart Images

Figure CN119088328A_ABST
Abstract
Description
[0001] This application is a divisional application. The application number of the original application is 202011195469.7, and the original application date is October 31, 2020. The entire contents of the original application are incorporated into this application by reference. Technical Field
[0002] The present application relates to the field of screen projection technology, and in particular to a screen projection method and electronic device for an application window. Background Art
[0003] Screen projection is a technology that is currently on the rise, which means that an electronic device can project the display interface on its display screen to another electronic device for display. For example, a movie on a mobile phone can be played on a TV, and pictures on a tablet can be displayed on a TV, etc.
[0004] Taking the mobile phone screen projection to PC as an example, generally speaking, the process of mobile phone screen projection to PC includes: the user triggers the screen projection operation on the mobile phone, the mobile phone and PC establish a screen projection connection, the mobile phone sends the current interface to the PC, and then both the PC and the mobile phone display the current interface. Figure 1 As shown in FIG. 1 , the mobile phone and the PC both display the mobile phone main interface. However, when the two devices display the same interface, in fact, the user only needs to operate on one device, so the usage rate of the interface on the other device is low. Summary of the invention
[0005] The purpose of this application is to provide a screen projection method and electronic device for application windows to enhance the screen projection experience.
[0006] In a first aspect, a method for projecting an application window is provided, which is applicable to a system including a first electronic device and a second electronic device, and the method includes: the first electronic device establishes a connection with the second electronic device; the first electronic device projects a first window of an application to the second electronic device; the second electronic device displays the first window; and the first electronic device displays a second window of the application.
[0007] That is to say, the screen projection granularity can be each window of the application. For example, the Changlian application includes multiple windows, and the first electronic device (such as a mobile phone) can project some or all windows of the Changlian application to the second electronic device (such as a PC). Take the mobile phone to PC screen projection as an example, and take the first window as the address book window of the Changlian application, and the second window as the chat window with contact A; the user can view the address book window on the PC and the chat window with contact A on the mobile phone, without having to switch back and forth between different windows of the application on the mobile phone.
[0008] It should be noted that in the embodiment of the present application, the first electronic device can display the second window while projecting the first window to the second electronic device. That is, the steps: the first electronic device projects the first window of the application to the second electronic device; and the step: the first electronic device displays the second window of the application can be performed at the same time, or of course, can be performed at different times.
[0009] In a possible design, the second window is the previous window of the first window; or, the default window of the application; or, the designated window of the application.
[0010] That is, when the first electronic device projects the first window to the second electronic device, the first electronic device displays the previous window, i.e., the second window, or displays the default window or the specified window. The default window refers to the window set by the electronic device system by default; the specified window refers to the window set by the user; and the first window refers to the window before the current window is opened.
[0011] In one possible design, the first electronic device projects the first window of the application onto the second electronic device for display, including: in response to an operation for opening the first window, the first electronic device projects the first window onto the second electronic device for display.
[0012] That is to say, when the first electronic device detects the operation for opening the first window, the first window is automatically projected to the second electronic device for display, and the first window may not be displayed on the first electronic device. In this way, screen projection is faster and more efficient.
[0013] In one possible design, the operation for opening the first window includes: an operation for opening an image; and / or an operation for playing a video file.
[0014] That is, the first electronic device detects the operation of opening an image and projects the image to the second electronic device for display; or the first electronic device detects the operation of playing a video and projects the video to the second electronic device for display. Taking the projection of a mobile phone to a PC as an example, generally speaking, because the mobile phone display is small, the effect of viewing images or watching videos is not good. Through the method provided by this application, when an image or video is opened on a mobile phone, it can be directly projected to a PC for display, which is more convenient and efficient.
[0015] In one possible design, the first electronic device projects the first window of the application onto the second electronic device for display, including: in response to an operation for opening the first window, displaying the first window on the display screen of the first electronic device; in response to an operation for projecting the second window, the first electronic device projects the first window onto the second electronic device for display.
[0016] That is, under the operation for opening the first window, the first electronic device first displays the first window, and under the operation for projecting the first window, the first electronic device projects the first window to the second electronic device, and the first electronic device closes the first window and displays the second window. Taking the projection of a mobile phone to a PC as an example, the first window is first displayed on the mobile phone. If the user wants to project the first window, the first window is projected, which helps to avoid the automatic projection of windows including user privacy and leakage to others.
[0017] In one possible design, before the first electronic device projects the first window of an application to the second electronic device, it also includes: the first electronic device displays the first window and the second window; the first electronic device displays the second window of the application, including: the first electronic device displays the second window in full screen.
[0018] That is, when the first electronic device displays the first window and the second window in split screen, if the first electronic device projects the first window onto the second electronic device, the second window can be displayed in full screen on the first electronic device. In this way, multiple windows of the application can be displayed on different devices, avoiding the problem of being unable to view multiple windows of the application well due to the small display screen of the first electronic device.
[0019] In a possible design, the first electronic device projects the first window of the application onto the second electronic device for display, including: in response to a first operation, displaying multiple identifiers of multiple devices to be projected; in response to an operation of selecting the identifier of the second electronic device from the multiple identifiers, the first electronic device projects the first window onto the second electronic device for display. That is, the user can select the second electronic device to which the user wants to project from the multiple devices to be projected, and then project the first window onto the second electronic device, which provides a better user experience.
[0020] In a possible design, the first electronic device projects the first window of the application onto the second electronic device for display, including: in response to a second operation, displaying multiple candidate windows of the application; in response to an operation of selecting the first window from the multiple candidate windows, the first electronic device projects the first window onto the second electronic device for display. That is, the user can select the first window that the user wants to project from the multiple candidate windows of the application, which provides a better experience.
[0021] In a possible design, the candidate window includes:
[0022] All windows of the application that have been opened by the first electronic device; or,
[0023] a preset window of the application; or,
[0024] A window specified by a user in the application; or
[0025] A window in the application that has been opened more times or more frequently than a preset value; or
[0026] A window in the application that runs for a longer period than a preset period of time is running in the foreground.
[0027] It should be noted that the above-mentioned candidate windows are only examples and are not limitations. Other types of windows are also feasible as candidate windows, and the embodiments of the present application do not limit this.
[0028] In one possible design, the method also includes: the first electronic device opens a third window of the application; the first electronic device projects the third window onto the second electronic device; the second electronic device displays the third window; and the third window on the second electronic device completely or partially covers the first window.
[0029] That is to say, when the first electronic device opens a new window of the application (i.e., the third window), it can also project the new window to the second electronic device, i.e., the projection granularity is the individual windows of the application. Therefore, two windows are displayed on the second electronic device, i.e., the third window and the second window, and the third window can completely cover the first window (i.e., the first window is hidden), or the third window may not completely cover the first window, such as covering part of the first window, or not covering the first window at all (i.e., split-screen display or multi-window simultaneous display).
[0030] In one possible design, the method further includes: in response to a first instruction for restoring the first window to be displayed on the first electronic device, the first electronic device displays the first window, and the second electronic device closes the first window. In other words, the first window that has been projected to the second electronic device can be restored to be displayed on the first electronic device, which is flexible in operation and provides a high user experience.
[0031] In a possible design, the first instruction for restoring the first window to be displayed on the first electronic device includes: an operation instruction for closing the first window on the second electronic device detected on the second electronic device; or an operation instruction for opening the first window on the first electronic device detected on the first electronic device. That is, when the user can close the first window on the second electronic device, the first window can be restored to be displayed on the first electronic device, or when the user can open the first window on the first electronic device, the first window can be restored to be displayed on the first electronic device.
[0032] In a possible design, the application also includes a fourth window, and the method also includes: an operation for opening the second window is detected on the second electronic device; in response to the operation for opening the second window, the second electronic device sends a second instruction to the first electronic device; in response to the received second instruction, the first electronic device projects the second window to the second electronic device; the second electronic device displays the second window; the second window on the second electronic device completely or partially covers the first window; the first electronic device displays the fourth window. That is, the first electronic device can project two windows of the application to the second electronic device one after another, and the two windows can be displayed overlapping or non-overlapping on the second electronic device.
[0033] In one possible design, the application also includes a fourth window, the system also includes a third electronic device, and the method further includes: the first electronic device is connected to the third electronic device; the first electronic device projects the second window to the third electronic device; the third electronic device displays the second window; the first electronic device displays the fourth window of the application. In other words, the first electronic device can project the first window of the application to the second electronic device, and project the second window of the application to the third electronic device. In this way, multiple windows of the application can be displayed on multiple electronic devices for easy viewing by users.
[0034] In one possible design, the first electronic device projects the first window of an application to the second electronic device, including: the first electronic device creates the first window in a first task stack; the window in the first task stack is configured to be projected to the second electronic device for display; wherein the first task stack is different from the second task stack where the second window is located, and the window in the second task stack is configured to be displayed on the display screen of the first electronic device.
[0035] It should be noted that, in the embodiment of the present application, the first window and the second window of the application are respectively in different task stacks, the window in the second task stack is displayed on the first electronic device, and the window in the first task stack is projected and displayed on the second electronic device. In other words, by stacking different windows of the application, the effect of displaying different windows on different devices is achieved.
[0036] In one possible design, the first electronic device projects the first window of an application to the second electronic device, including: in response to an operation for opening the first window, the first electronic device creates the first window in a second task stack; the window of the second task stack is configured to be displayed on the display screen of the first electronic device; in response to an operation for projecting the first window, the first electronic device creates the first window in the first task stack and closes the first window in the second task stack; the window in the first task stack is configured to be projected to the second electronic device for display.
[0037] It should be noted that in an embodiment of the present application, the same task stack (such as the second task stack) of the application may include two windows, such as a first window and a second window. If the first electronic device wants to project the first window onto the second electronic device for display, the first window can be stacked so that the first window is in the first task stack, thereby projecting the first window onto the second electronic device. In other words, different windows originally in the same task stack are split into different task stacks to achieve the effect of displaying on different devices, such as the first window split into the first task stack is projected onto the second electronic device for display, and the second window still in the second task stack is displayed on the first electronic device.
[0038] In a possible design, the method further includes: in response to an operation for restoring the first window to be displayed on the first electronic device, the first electronic device determines that the original task stack of the first window is the second task stack according to the original stack mark of the first task stack; the first electronic device creates the first window in the second task stack; the first electronic device closes the first window in the first task stack. That is, if the first electronic device wants to restore the projected first window to be displayed on the first electronic device, the first window in the first task stack can be restored to the second task stack where it originally was, so that the first window is displayed on the first electronic device.
[0039] In one possible design, the first electronic device projects the first window of the application to the second electronic device, including: in response to a preset operation, the first electronic device displays multiple task stacks of the application; in response to an operation for selecting a third task stack from the multiple task stacks, the first electronic device projects all windows of the third task stack to the second electronic device. In other words, the first electronic device can project all windows of a task stack to the second electronic device, i.e., the task stack of the application is used as the projection granularity.
[0040] In a second aspect, a method for projecting an application window is provided, which is applicable to a first electronic device, wherein the first electronic device includes an application, and the application includes a first window and a second window; the method includes:
[0041] The first electronic device is connected to the second electronic device;
[0042] The first electronic device projects the first window onto the second electronic device for display;
[0043] The first electronic device displays the second window.
[0044] In a possible design, the second window is the previous window of the first window; or, the default window of the application; or, the designated window of the application.
[0045] In one possible design, the first electronic device projects the first window of the application onto the second electronic device for display, including: in response to an operation for opening the first window, the first electronic device projects the first window onto the second electronic device for display.
[0046] In one possible design, the operation for opening the first window includes: an operation for opening an image; and / or an operation for playing a video file.
[0047] In one possible design, the first electronic device projects the first window of the application onto the second electronic device for display, including: in response to an operation for opening the first window, the first electronic device displays the first window; in response to an operation for projecting the second window, the first electronic device projects the first window onto the second electronic device for display.
[0048] In a possible design, before the first electronic device projects the first window to the second electronic device, it also includes: the first electronic device displays the first window and the second window; the first electronic device displays the second window, including: the first electronic device displays the second window in full screen.
[0049] In one possible design, the first electronic device projects the first window of the application onto the second electronic device for display, including: in response to a first operation, displaying multiple identifiers of multiple devices to be projected; in response to an operation of selecting the identifier of the second electronic device from the multiple identifiers, the first electronic device projects the first window onto the second electronic device for display.
[0050] In one possible design, the first electronic device projects the first window of the application onto the second electronic device for display, including: in response to a second operation, displaying multiple candidate windows of the application; in response to an operation of selecting the first window from the multiple candidate windows, the first electronic device projects the first window onto the second electronic device for display.
[0051] In a possible design, the candidate window includes:
[0052] All windows of the application that have been opened by the first electronic device; or,
[0053] a preset window of the application; or,
[0054] A window specified by a user in the application; or
[0055] A window in the application that has been opened more times or more frequently than a preset value; or
[0056] A window in the application that runs for a longer period than a preset period of time is running in the foreground.
[0057] In one possible design, in response to a first instruction for restoring the first window to be displayed on the first electronic device, the first electronic device displays the first window and controls the second electronic device to close the first window.
[0058] In one possible design, the first instruction includes: a first instruction received from the second electronic device, the first instruction being an operation instruction detected on the second electronic device for closing the first window on the second electronic device; or, an operation instruction detected on the first electronic device for opening the first window on the first electronic device.
[0059] In one possible design, the application also includes a fourth window, the system also includes a third electronic device, and the method also includes: the first electronic device is connected to the third electronic device; the first electronic device projects the second window to the third electronic device; and the first electronic device displays the fourth window.
[0060] In one possible design, the first electronic device projects the first window onto the second electronic device for display, including: creating a first window of an application in a first task stack; the window in the first task stack is configured to be projected onto the display screen of the second electronic device for display; wherein the first task stack is different from the second task stack where the second window is located, and the window in the second task stack is configured to be displayed on the display screen of the first electronic device.
[0061] In one possible design, the first electronic device projects the first window onto the second electronic device for display, including: in response to an operation for opening the first window, the first electronic device creates the first window in a second task stack; the window in the second task stack is configured to be displayed on the display screen of the first electronic device; in response to an operation for projecting the first window, the first electronic device creates the first window in the first task stack and closes the first window in the second task stack; the window in the first task stack is configured to be projected onto the display screen of the second electronic device for display.
[0062] In one possible design, the method also includes: in response to an operation for restoring the first window to be displayed on the first electronic device, the first electronic device determines that the original task stack of the first window is a second task stack based on the original stack mark of the first task stack; the first electronic device creates the first window in the second task stack; and the first electronic device closes the first window in the first task stack.
[0063] In one possible design, the first electronic device projects the first window of an application to the second electronic device, including: in response to a preset operation, the first electronic device displays multiple task stacks of the application; in response to an operation for selecting a third task stack from the multiple task stacks, the first electronic device projects all windows of the third task stack to the second electronic device.
[0064] In a third aspect, a method for projecting an application window is also provided, which is applicable to a second electronic device, and the method includes:
[0065] The second electronic device is connected to the first electronic device;
[0066] The second electronic device receives the first display information sent by the first electronic device, and displays a first window according to the first display information;
[0067] The second electronic device receives second display information sent by the first electronic device, and displays a second window according to the second display information, wherein the second window is a window displayed on the first electronic device when the second electronic device displays the first window, and the second window and the first window are from the same application on the first electronic device;
[0068] The second window completely or partially covers the first window.
[0069] In one possible design, the method also includes: the second electronic device receives a first instruction for restoring the first window to the first electronic device, and closes the first window; and sends a second instruction to the first electronic device, where the second instruction is used to instruct the first electronic device to display the first window.
[0070] In one possible design, the first instruction for restoring the first window to the first electronic device includes: an operation instruction detected on the second electronic device for closing the first window on the second electronic device; or, receiving a first instruction from the first electronic device, the first instruction being used to instruct opening the first window on the first electronic device.
[0071] In a fourth aspect, an electronic device is provided, comprising: a display screen, one or more processors, a memory, and one or more programs; wherein the one or more programs are stored in the memory, and the one or more programs include instructions, and when the instructions are executed by the electronic device, the electronic device executes the method provided in any one of the first to third aspects above.
[0072] In the fifth aspect, an embodiment of the present application further provides an electronic device, which includes a module / unit for executing the method provided in any one of the first to third aspects above; these modules / units can be implemented through hardware, or the corresponding software can be implemented through hardware.
[0073] In the sixth aspect, an embodiment of the present application also provides a chip, which is coupled to a memory in an electronic device, and is used to call a computer program stored in the memory and execute the technical solution provided by any one of the first to third aspects of the embodiments of the present application; in the embodiments of the present application, "coupling" refers to the direct or indirect combination of two components with each other.
[0074] In the seventh aspect, a computer-readable storage medium is also provided, wherein the computer-readable storage medium includes a computer program. When the computer program runs on an electronic device, the electronic device executes the technical solution provided in any one of the first to third aspects above.
[0075] In an eighth aspect, a program product is also provided, comprising instructions, which, when executed on a computer, enable the computer to execute the technical solution provided in any one of the first to third aspects above.
[0076] In a ninth aspect, a system is also provided, comprising a first electronic device and a second electronic device, wherein the first electronic device is used to execute the method steps provided in the second aspect, and the second electronic device executes the method steps provided in the third aspect.
[0077] In the tenth aspect, a graphical user interface on an electronic device is also provided, wherein the electronic device has a display screen, one or more memories, and one or more processors, wherein the one or more processors are used to execute one or more computer programs stored in the one or more memories, and the graphical user interface includes a graphical user interface displayed when the electronic device executes the technical solution provided in any one of the first to third aspects above.
[0078] For the beneficial effects of the second to tenth aspects mentioned above, please refer to the beneficial effects of the first aspect and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0079] Figure 1 A schematic diagram of projecting the screen of a mobile phone to a PC provided in an embodiment of the present application;
[0080] Figure 2A A schematic diagram of same-source screen projection provided in one embodiment of the present application;
[0081] Figure 2B A schematic diagram of heterogeneous screen projection provided in one embodiment of the present application;
[0082] Figure 3 A schematic diagram of an application scenario provided by an embodiment of the present application;
[0083] Figure 4A A schematic diagram of the hardware structure of a master device or a target device provided in an embodiment of the present application;
[0084] Figure 4B A schematic diagram of a software structure of a master device or a target device provided in an embodiment of the present application;
[0085] Figure 5 A schematic diagram of a first exemplary scenario provided for an embodiment of the present application;
[0086] Figure 6 A schematic diagram of a flow chart of a method for projecting an application window provided in one embodiment of the present application;
[0087] FIG. 7A to FIG. 7D A schematic diagram of the implementation principle of the first exemplary scenario provided in an embodiment of the present application;
[0088] Fig. 8A A schematic diagram of a second exemplary scenario provided for an embodiment of the present application;
[0089] FIG. 8B to FIG. 8E A schematic diagram of a second exemplary scenario provided for an embodiment of the present application;
[0090] FIG. 9A to FIG. 9B A schematic diagram of the implementation principle of a third exemplary scenario provided in an embodiment of the present application;
[0091] Fig. 9C A schematic diagram of a third exemplary scenario provided for an embodiment of the present application;
[0092] Fig. 10A A schematic diagram of a fourth exemplary scenario provided for an embodiment of the present application;
[0093] FIG. 10B to FIG. 10G A schematic diagram of the implementation principle of a fourth exemplary scenario provided in an embodiment of the present application;
[0094] FIG. 11A to FIG. 11B A schematic diagram of an implementation principle of opening a new window on a target device provided by an embodiment of the present application;
[0095] Fig.12 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION
[0096] The following is an explanation of some of the terms used in the embodiments of the present application.
[0097] (1) Application (app for short), referred to as application, is a software program that can realize one or more specific functions. Usually, multiple applications can be installed in an electronic device, for example, instant messaging applications, video applications, audio applications, image capture applications, etc. Among them, instant messaging applications, for example, may include text messaging applications, Changlian, WeChat, WhatsApp Messenger, Line, photo sharing (Instagram), Kakao Talk, DingTalk, etc. Image capture applications, for example, may include camera applications (system camera or third-party camera applications). Video applications, for example, may include Youtube, Twitter, Douyin, iQiyi, Tencent Video, etc. Audio applications, for example, may include Google Music, Kugou Music, Xiami, QQ Music, etc. The applications mentioned in the following embodiments may be applications that are installed on the electronic device when it leaves the factory, or they may be applications downloaded by the user from the Internet or obtained from other electronic devices during the use of the electronic device.
[0098] (2) The application window refers to the display area of the application. Each application window corresponds to an activity object, and all windows are presented by a view. An application can correspond to multiple windows, and different windows can display different content of the application.
[0099] (3) Tasks and stacks, where a task is a collection of multiple activities, and can also be understood as a container for placing activities. A stack is a collection of multiple tasks. Generally speaking, a stack and a task are one-to-one, but this application does not limit this, and the relationship can be one-to-many. For the sake of convenience, this article takes the example of a one-to-one relationship between a stack and a task, that is, a stack includes one task, and the tasks in different stacks are different. In this case, the task and / or stack can be referred to as a task stack.
[0100] The task stack places activities in a "first in, last out" format. The so-called "first in, last out" can be understood as: create a task stack and place the first activity in the task stack. The first activity is at the top and the first activity is displayed in the foreground. If the second activity is created in the task stack later, since the first activity has been pushed into the stack before the second activity, the second activity will be pushed into the stack later and the second activity will be pushed into the first activity, that is, the second activity is located on the upper layer of the first activity. If the display positions of the first and second activities are the same, the second activity will be displayed in the foreground and cover the first activity. When the second activity is moved out of the first task stack (referred to as popping the stack), since the first activity is at the top of the task stack, the first activity is displayed in the foreground and the second activity is not displayed.
[0101] (4) At least one of the embodiments of the present application includes one or more; wherein, more than one means greater than or equal to two. In addition, it should be understood that in the description of the present application, the words "first", "second", etc. are only used for the purpose of distinguishing the description, and cannot be understood as expressing or implying relative importance, nor can they be understood as expressing or implying an order.
[0102] The screen projection method of the application window provided in this application is applicable to the screen projection scenario. Generally speaking, the screen projection scenario includes a transmitter and a receiver. The transmitter sends the display information to the receiver for display. For the convenience of description, the transmitter is referred to as the main device (or source device) and the receiver is referred to as the target device. Among them, the main device can be a mobile phone, a tablet computer, a PC, a mobile phone, a watch, etc. The target device can be a large-screen device such as a mobile phone, a tablet computer, a PC, a TV, etc. It can be understood that in addition to the main device and the target device, the transmitter and the receiver can also have other names, for example, the transmitter is an active screen projection device and the receiver is a passive screen projection device; or, the transmitter is a first electronic device, the receiver is a second electronic device, etc., and this application does not limit this. Among them, the roles of the transmitter and the receiver can be interchanged, that is, the transmitter can project the screen to the receiver, and correspondingly, the receiver can also project the screen to the transmitter.
[0103] Currently, commonly used screen projection technologies include same-source screen projection and different-source screen projection.
[0104] See also Figure 2AThe figure shows a schematic diagram of the same-source screen projection process. The main device projects the foreground display information (such as window A) to the target device, and the target device displays the display information (such as window A). When the foreground display information of the main device changes, such as switching from window A to window B, the display information on the target device also changes accordingly, that is, switching from window A to window B. In other words, in the same-source screen projection scenario, when the main device projects the screen to the target device, the display information on the target device is synchronized with the foreground display information of the main device.
[0105] Different-source screen projection is different from same-source screen projection. Simply put, in different-source screen projection scenarios, when the main device projects the screen to the target device, the display information projected from the main device on the target device can be out of sync with the display information on the main device front end.
[0106] Specifically, see Figure 2B The figure shows a schematic diagram of the heterogeneous screen projection process. The window of application B is displayed on the main device. When the main device projects the window of application B to the target device, the window of application B is displayed in the created virtual display area (the virtual display area is invisible to the user). The display information of the virtual display area is projected to the target device for display by default. In addition to the virtual display area, the main device also includes a user-visible display area. When the main device projects the window of application B to the target device, the user-visible display area displays the window of application A, that is, the main device displays the window of application A in the foreground. Assume that the user-visible area on the main device switches from the window of application A to the window of application C, that is, the main device displays the window of application C in the foreground. At this time, the window of application B is still displayed on the target device.
[0107] Continue with Figure 2B For example, in the current heterogeneous screen projection technology, all windows opened by application B on the main device are displayed in the virtual display area by default, that is, all windows opened by application B will be projected to the target device for display. In other words, the main device projects the entire application B to the target device. In simple terms, the projection granularity is the application.
[0108] In the application window projection method provided in this application, the screen can be projected at a finer granularity, for example, using individual application windows as the projection granularity. For example, taking a mobile phone as the main device, an application on the mobile phone may include multiple windows, and the mobile phone may project all or part of the multiple windows to the PC.
[0109] The following introduces the screen projection scenario provided by the embodiments of the present application.
[0110] like Figure 3 The figure is an exemplary schematic diagram of a screen projection scenario provided by the present application. The screen projection scenario includes a mobile phone, a PC and a TV.
[0111] Take the example of projecting the screen from a mobile phone to a PC. Figure 3 (a), the user opens the phone application on the phone, and the phone application's "dial pad" window is displayed on the phone's display. It should be understood that for ease of understanding, Figure 3 The interface in the middle circle is an enlarged view of the display interface on the phone. In addition to the "Dial Keypad" window, the Phone app also includes other windows, such as the "Contacts" window, the "Recent Calls" window, and the "Connect" window. Users can choose to project the "Contacts" window of the Phone app to the PC. Of course, they can also project the "Recent Calls" window to the PC. See Figure 3 As shown in (b), optionally, the PC can display the two windows projected from the mobile phone separately. That is, the mobile phone can project different windows of an application (such as a phone application) to the PC, and the corresponding PC displays different windows of the application.
[0112] In the above example, the screen projection from a mobile phone to a PC is used as an example. In simple terms, it is a main device that projects the screen to a target device. This application is also applicable to scenarios where a main device projects the screen to multiple target devices. For example, a mobile phone projects the screen to both a PC and a TV. For example, continue to refer to Figure 3 (a) and Figure 3 (b) The phone has already projected the "Contacts" window and "Recent Calls" window of the phone application to the PC. The phone can also project the "Changlian" window of the phone application to the TV. See Figure 3 (c) shows that the mobile phone can project different windows of an application to different target devices. In this way, users can view different windows of the same application through multiple devices, which has a better visual effect.
[0113] It should be noted that Figure 3 The introduction is made using the phone application on a mobile phone as an example. The principles of screen projection for other applications on the mobile phone are the same and will not be listed one by one.
[0114] The following introduces electronic equipment. Figure 4A 1 shows a schematic diagram of the structure of an electronic device 100. The electronic device 100 can be a master device or a target device. Figure 2AAs shown, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.
[0115] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Among them, different processing units may be independent devices or integrated into one or more processors. Among them, the controller may be the nerve center and command center of the electronic device 100. The controller may generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions. A memory may also be set in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a high-speed cache memory. The memory may store instructions or data that the processor 110 has just used or circulated. If the processor 110 needs to use the instruction or data again, it can be directly called from the memory. Repeated access is avoided, the waiting time of the processor 110 is reduced, and the efficiency of the system is improved.
[0116] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transmit data between the electronic device 100 and peripheral devices. The charging management module 140 is used to receive charging input from the charger. The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and provides power to the processor 110, the internal memory 121, the external memory, the display screen 194, the camera 193, and the wireless communication module 160.
[0117] The wireless communication function of the electronic device 100 can be implemented by antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modulation and demodulation processor and baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antenna. For example, antenna 1 can be reused as a diversity antenna of a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.
[0118] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.
[0119] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.
[0120] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).
[0121] The display screen 194 is used to display the display interface of the application, such as the viewfinder interface of the camera application. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.
[0122] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.
[0123] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.
[0124] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.
[0125] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.
[0126] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.
[0127] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.
[0128] The internal memory 121 can be used to store computer executable program codes, which include instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system, and software code of at least one application (such as iQiyi application, WeChat application, etc.). The data storage area can store data generated during the use of the electronic device 100 (such as captured images, recorded videos, etc.). In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.
[0129] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as pictures and videos are stored in the external memory card.
[0130] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.
[0131] Among them, the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.
[0132] The pressure sensor 180A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194. The gyroscope sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B.
[0133] The gyroscope sensor 180B can be used for anti-shake shooting. The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist in positioning and navigation. The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip according to the magnetic sensor 180D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip, the characteristics such as automatic unlocking of the flip are set. The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in all directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device 100, and is applied to applications such as horizontal and vertical screen switching and pedometers.
[0134] Distance sensor 180F, used to measure distance. The electronic device 100 can measure distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focus. The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode, and the pocket mode automatically unlocks and locks the screen.
[0135] The ambient light sensor 180L is used to sense the brightness of the ambient light. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived brightness of the ambient light. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches. The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to achieve fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.
[0136] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 reduces the performance of a processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 due to low temperature. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 performs a boost on the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.
[0137] The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.
[0138] The bone conduction sensor 180M can obtain vibration signals. In some embodiments, the bone conduction sensor 180M can obtain vibration signals of vibrating bones of the human body. The bone conduction sensor 180M can also contact the human body's pulse to receive blood pressure beating signals.
[0139] The button 190 includes a power button, a volume button, etc. The button 190 can be a mechanical button. It can also be a touch button. The electronic device 100 can receive the button input and generate a key signal input related to the user settings and function control of the electronic device 100. The motor 191 can generate a vibration prompt. The motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization. The indicator 192 can be an indicator light, which can be used to indicate the charging status, power changes, and can also be used to indicate messages, missed calls, notifications, etc. The SIM card interface 195 is used to connect the SIM card. The SIM card can be inserted into the SIM card interface 195, or pulled out from the SIM card interface 195 to achieve contact and separation with the electronic device 100.
[0140] Understandably, Figure 4A The components shown do not constitute a specific limitation on the electronic device 100. The electronic device may also include more or fewer components than shown, or combine some components, or separate some components, or arrange the components differently. Figure 4A The combination / connection relationship between the components can also be adjusted and modified.
[0141] Figure 4B This is a software architecture diagram of a screen projection system provided in an embodiment of the present application.
[0142] Figure 4B The left half is the software architecture diagram of the main device, which is the device that actively projects the screen, such as a mobile phone. The software architecture of the main device may include an application layer, a framework layer, and a driver layer; it should be noted that it may also include more or fewer layers, which is not limited in the embodiments of the present application.
[0143] The application layer includes an application module and a screen projection management module. The application module includes various types of applications, such as video playback applications, image capture applications, etc. The screen projection management module is used to manage screen projection. For example, the screen projection management module includes a device connection module, which can send instructions for controlling the connection between the main device and the target device to the cross-device management module in the framework layer.
[0144] The framework layer includes the basic framework module, the cross-device management module, and the data management module.
[0145] The basic framework module includes an event management module, a window display module, a life cycle module, and a task stack management module. The event management module is responsible for sensing events, for example, receiving input events reported by the driver layer, such as input events for a window of the projected application.
[0146] The window display module is responsible for managing the display mode of the application window, including the coordinate size of the window display, the window display level, etc.
[0147] The life cycle module is used to manage the life cycle of the window. For example, the life cycle module can activate the window through the callback (onResume) interface, and the activated window is displayed in the foreground; or, the life cycle module can pause the window through the pause (onPause) interface to switch the window from the foreground to the background. Taking the topmost window (top Activity) in the task stack as an example, the top Activity calls onResume to activate it, so the top Activity is displayed in the foreground. When a new window is added to the task stack and the new window is located on the upper layer of the original top Activity, the original top Activity calls onPause to pause the window so that the original top Activity is switched from the foreground to the background. Similar principles can be used to manage the life cycle of new windows.
[0148] The task stack management module is used to create and close task stacks, etc.
[0149] The cross-device management module is responsible for controlling the driver layer to implement proximity discovery, authentication, connection and other functions between the main device and the target device.
[0150] The data management module is responsible for the data streams such as audio, video, and layers transmitted between the master device and the target device. It can also be responsible for reverse event control (i.e., events triggered by the target device to control the master device) and other functions.
[0151] The driver layer includes the underlying driver, which is responsible for discovery, authentication, connection, etc. For example, the driver layer receives commands issued by the cross-device management module in the framework layer and performs actions such as connection and disconnection. Specifically, the driver layer includes a device discovery module, a device authentication module, and a device connection module. The device discovery module is responsible for device discovery, the device authentication module is responsible for device authentication, and the device connection module is responsible for device connection. Of course, the driver layer can also be a hardware driver such as a display driver.
[0152] Figure 4B The right half of the diagram is the software architecture diagram of the target device. The target device is the device to be projected, such as a PC or a TV. The software architecture of the target device may include an application layer, a framework layer, and a driver layer. The functional modules of each layer and the functions of the functional modules can refer to the above-mentioned main device. In other words, the roles of the target device and the main device can be interchanged, and the target device can also project the screen to the main device.
[0153] Combine the following Figure 4A and Figure 4B , introduces the implementation process of the application window projection method provided in the embodiment of the present application.
[0154] The display screen of the master device displays the first window of the application. When the driver layer of the master device receives a hardware interrupt (e.g., a user opens the second window of the application on the touch screen), the input event corresponding to the hardware interrupt is reported to the application in the application layer. When the application determines to open the second window of the application based on the input event, the second window of the application is displayed on the display screen.
[0155] When the driver layer of the main device receives a hardware interrupt (such as the user clicking the projection icon on the touch screen), the driver layer reports the input event corresponding to the hardware interrupt to the application. The application triggers the cross-device management module through the projection management module in the application layer to send instructions to the driver layer through the cross-device management module to control the driver layer to realize authentication and connection between the main device and the target device.
[0156] After the driver layer of the main device determines that the main device is successfully connected to the target device, it sends an instruction to the event management module in the basic framework module in the framework layer, and the instruction is used to instruct the second window of the application to be projected. The event management module sends an instruction to the task stack management module to implement the stacking processing of the second window and the first window through the task stack management module (the specific content is introduced later). The stacked second window is sent to the target device for display through the data management module. After the task stack management module completes the stacking processing of the second window and the first window, the first window is located at the highest layer, and the first window can be activated through the life cycle module, and the first window can be displayed through the window display module, that is, the first window is switched from the background to the foreground display.
[0157] The following takes the example that the main device is a mobile phone and the target device is a PC or a TV, and describes in detail the screen projection method of the application window provided in the embodiment of the present application in combination with the accompanying drawings.
[0158] In the screen projection method of the application window provided in the embodiment of the present application, the screen projection granularity can be each window of the application. For example, an application on a mobile phone includes multiple windows, and the mobile phone can project at least one of the multiple windows to the PC, and display the other windows of the multiple windows on the mobile phone. Specifically, the screen projection method of the application window provided in the present application includes the following embodiments 1 to 4.
[0159] Embodiment 1
[0160] The first embodiment of the present invention provides a method for projecting an application window. Specifically, the mobile phone displays window A of the application in the foreground. In response to the user operation, the mobile phone switches the foreground from window A of the application to window B of the application. At this time, if the user wants to project window B to the PC, the user can trigger the projection operation to establish a projection connection between the mobile phone and the PC. The mobile phone projects window B to the PC. Since window B is projected, the mobile phone may not display window B in the foreground, but display other windows of the application, such as window A, that is, when the mobile phone projects window B, the foreground of the mobile phone switches from window B to window A.
[0161] Take the browser application on the mobile phone as an example. Figure 5 An exemplary scenario of the first embodiment is given.
[0162] like Figure 5 As shown in (a), the user opens the browser application on the mobile phone, and the mobile phone displays the homepage window of the browser application, which includes the introduction of news A and the introduction of news B. When the mobile phone receives an operation for opening news A (e.g., clicking the introduction of news A in the homepage window), the following is displayed Figure 5 (b) shows a window showing news A (for the sake of convenience, this window is called the window of news A). At this point, if the user wants to project news A to the PC, the user can trigger the screen projection operation to establish a screen projection connection between the mobile phone and the PC. The mobile phone projects the window of news A to the PC, and then the window of news A is displayed on the PC, as shown in FIG. Figure 5 (c). After projecting the window of news A, the mobile phone foreground may not display the window of news A, but display other windows of the browser application, for example, the window that was last opened before the window of news A was opened, for example, Figure 5 (d) shows the home page window of the browser application.
[0163] The following is Figure 6 Taking the flowchart shown as an example, the implementation principle of the screen projection method of the application window provided in the first embodiment is introduced.
[0164] like Figure 6 As shown, the process of this embodiment 1 may include S1 to S15. Among them, S1 to S3 introduce the process of opening the first window of the application on the mobile phone. Figure 5 Taking the scenario as an example, the application may be a browser application, and the first window may be a homepage window of the browser application. S4 to S6 describe the process of opening the second window of the application by the mobile phone. Figure 5 For example, the second window may be a window of news A. S7 to S15 introduce a process in which the mobile phone projects the second window of an application (eg, a window of news A) to the PC.
[0165] S1, a mobile phone receives a first operation for opening a first window of an application.
[0166] For example, see Fig. 7A (a), the mobile phone displays the main interface, and the first operation can be an operation of clicking an icon of an application on the main interface. If the application is a browser application, the first operation is an operation of clicking an icon of the browser application in the main interface. Of course, the first operation can also be other types of operations, such as a voice command for instructing to open the first window of the application, etc., which is not limited in the embodiment of the present application.
[0167] S2: The mobile phone creates a first task stack, and creates a first window in the first task stack.
[0168] See also Figure 7B (a), under the action of the first operation, the mobile phone creates a first task stack and creates a first window in the first task stack. The windows in the first task stack are displayed on the default display screen, which refers to the display area visible to the user. Since the first window is the topmost window in the first task stack, the first window is displayed on the mobile phone display screen, that is, S3.
[0169] S3, the mobile phone displays the first window of the application on the display screen.
[0170] For example, see Fig. 7A (b) The mobile phone displays a first window of the application in response to the first operation.
[0171] S4, the mobile phone receives a second operation for opening a second window of the application.
[0172] by Fig. 7A (b) as an example, the second operation may be an operation for opening a second window of the application in the first window. Figure 5 In the case of the home page window of the browser application shown in (a), the second operation may be, for example, a click operation on the introduction of news A.
[0173] S5: The mobile phone creates a second window in the first task stack.
[0174] Continue to see Figure 7B As shown, under the effect of the second operation, the mobile phone creates a second window in the first task stack. Because the first window is stacked first and the second window is stacked later, the second window is located on the upper layer of the first window. Generally speaking, the second window located on the upper layer and the first window located on the lower layer have the same display position, so the second window will completely cover the first window, so the mobile phone foreground switches from the first window to the second window, that is, S6.
[0175] S6, the phone foreground switches from the first window to the second window.
[0176] See also Fig. 7A (c), in response to the second operation, the second window is displayed on the mobile phone, and the first window is no longer displayed. Figure 5 For example, combined with Figure 5 (b) and Figure 5 (a), the window of news A (i.e., the second window) completely covers the homepage window of the browser application (i.e., the first window).
[0177] At this point, if you want to project the second window onto the target device, the mobile phone can continue to execute S7 to S15 as follows.
[0178] S7, the mobile phone receives a third operation for projecting a second window.
[0179] Among them, the third operation can be an upward swipe operation at the lower edge of the display screen, or a right swipe operation at the left edge of the display screen, or a left swipe operation at the right edge of the display screen, or a downward swipe operation at the upper edge of the display screen, etc.
[0180] Alternatively, the third operation includes a swiping up operation from the bottom edge of the display screen and clicking on the projection icon in the swiping up menu (the swiping up menu displayed in response to the swiping up operation); alternatively, the third operation includes a swiping down operation from the top status bar of the phone screen and clicking on the “wireless projection” icon in the notification panel (the notification panel displayed in response to the swiping down operation).
[0181] Alternatively, the third operation can also be used to open the multi-device control center, such as sliding vertically upward from the lower left edge or lower right edge of the mobile phone to open the multi-device control center, and displaying the icon of at least one device that supports screen projection in the multi-device control center, and the user can select the target device from it.
[0182] The above is an example of the third operation, and is not intended to limit the third operation. The third operation may also be other types of operations, such as a voice command for projecting the second window, etc., which are not listed one by one in the embodiments of the present application.
[0183] If the mobile phone is not currently connected to the target device, the mobile phone can first connect to the target device and then project the second window to the target device. The process of connecting the mobile phone to the target device includes the following steps S8-S10.
[0184] S8, the phone displays a list of devices.
[0185] Continue with Fig. 7A (c) as an example, when the second window is displayed on the mobile phone, assuming that the mobile phone detects the third operation, a device list may be displayed, such as Figure 7C (a). The device list may include a device identifier, which may be an icon of the device, such as a TV icon, a PC icon, etc., and may also include a device model, etc., to facilitate user identification. If the third operation is an operation for opening a multi-device control center, then the device list here is the multi-device control center.
[0186] S9, phone select the target device.
[0187] For example, the mobile phone receives a click operation on an icon of a device in the device list, and in response to the click operation, determines that the device corresponding to the clicked icon is the target device.
[0188] S10. The mobile phone establishes a screen projection connection with the target device.
[0189] In the above example, when the mobile phone detects the third operation, it opens the device list, selects the target device in the device list and connects with the target device. Optionally, in addition to the above methods, there are other ways to establish a connection with the target device. For example, you can also scan the code to connect, such as opening the multi-screen collaboration interface on the mobile phone, clicking the scan code connection in the multi-screen collaboration interface, and scanning the QR code on the PC to complete the connection. For another example, you can also connect by "touch and pay", such as turning on the NFC function on the mobile phone, touching the NFC area on the mobile phone to the NFC area on the PC keyboard, and establishing a near-field communication connection between the mobile phone and the PC. For another example, you can also connect by proximity, such as turning on the proximity connection function on the PC; after turning on the Bluetooth function on the mobile phone, when the mobile phone is close to the PC, the PC discovers the mobile phone and displays a prompt message to confirm whether to connect the mobile phone. When the PC receives the confirmation instruction, it completes the connection with the mobile phone.
[0190] The mobile phone can stack the first window and the second window of the application, and project the second window after the stacking process onto the target device. Specifically, it can be achieved through the following steps S11 to S15.
[0191] S11, the phone creates a virtual display.
[0192] Among them, the virtual display screen is relative to the real display screen of the mobile phone (visible to the user), and the virtual display screen is invisible to the user. The virtual display screen may include various display parameters, such as resolution, display screen size, and so on. The display parameters of the virtual display screen may be consistent with the display parameters of the real display screen of the mobile phone. Alternatively, after the main device and the target device establish a screen projection connection, the purpose of creating the virtual display screen is to display the window in the virtual display screen on the display screen of the target device, so the display parameters of the virtual display screen may be consistent with the display parameters of the display screen of the target device, for example, the size of the virtual display screen is consistent with the size of the display screen of the target device, so that the display content on the virtual display screen is more adapted to the display screen of the target device.
[0193] S12, the mobile phone creates a second task stack, the second task stack is displayed on the virtual display screen, and the second window in the first task stack is moved to the second task stack.
[0194] The execution order of S11 and S12 is not limited. For example, S11 may be executed first and then S12, that is, a virtual display screen is created after a connection is established with the target device. Alternatively, S12 may be executed first and then S11.
[0195] The second window is moved from the original first task stack to the second task stack, specifically including: creating the second task stack, creating the second window in the second task stack, and then closing the second window in the first task stack. Figure 7B As shown, under the action of the third operation, the second window is moved out of the original first task stack to the second task stack. Since the second window is moved out of the first task stack and the first window is at the top, the mobile phone foreground switches from the second window to the first window, that is, S13.
[0196] S13, the mobile phone foreground switches from the second window to the first window.
[0197] S14, the mobile phone sends the display information of the second window in the second task stack to the target device.
[0198] S15: Displaying a second window of the application on the target device.
[0199] It should be noted that the execution order of S13 to S15 is not limited, for example, S13 and S14 can be executed at the same time.
[0200] For example, see Figure 7C (a), assuming that the mobile phone determines that the target device is a PC, the mobile phone establishes a connection with the PC, sends the display information of the second window to the PC, and the second window is displayed on the PC, such as Figure 7C (b), and the mobile phone foreground switches from the second window to the first window, such as Figure 7C (c).
[0201] It should be noted that the above embodiment is described by taking the example that when the mobile phone opens the second window, the second window completely covers the first window. It is understandable that the display position of the second window can be different from the display position of the first window. For example, the second window and the first window are displayed in split screens. Fig.7D (a). In this case, the user can choose to project the first window or the second window to the PC. Taking projecting the second window as an example, after the mobile phone detects the third operation for projecting the second window, S8 to S15 are executed to complete the projection process of the second window.
[0202] It should be noted that in the case of split screen display, in order to facilitate the mobile phone to distinguish which window the user wants to cast, when the mobile phone detects that the first window is selected, it is determined that the user wants to cast the first window, and when the mobile phone detects that the second window is selected, it is determined that the user wants to cast the second window. The operation for selecting a window can be double-clicking the window, or clicking the window three times in a row, or long pressing the window, etc. For example, continue to refer to Fig.7D As shown in (a), when the mobile phone detects a long press operation on the second window, a device list is displayed, such as Fig.7D (b) When the mobile phone detects that the user selects the PC, the mobile phone establishes a connection with the PC and projects the second window onto the PC. Fig.7D (c) At this time, the first window is displayed in full screen on the front of the phone. Fig.7D (d).
[0203] In the above embodiment, when the second window is projected, the foreground of the mobile phone switches from the second window to the first window, and the first window is the last window opened before the second window is opened. It can be understood that when the second window is projected, the foreground of the mobile phone can also display other windows of the application, such as a default window, which can be, for example, the home window of the application, or a window specified by the user.
[0204] In summary, in Example 1, when the mobile phone displays a window of an application, under the action of the user's screen projection operation, when the mobile phone completes the screen projection connection with the PC, the window displayed in the foreground of the mobile phone is projected to the PC, and other windows of the application are displayed in the foreground of the mobile phone, that is, the projection granularity is the window of the application.
[0205] Embodiment 2
[0206] This second embodiment provides a method for projecting an application window. Specifically, when a mobile phone has established a projection connection with a target device, when the mobile phone detects an operation for opening a window C of an application, the window C is displayed on the mobile phone, and when an operation for projecting the window C is detected, the window C is projected onto the target device. For the convenience of description, this second embodiment is introduced by taking the example that the mobile phone currently establishes a projection connection with only one target device (such as a PC).
[0207] An exemplary scenario of the second embodiment is described below.
[0208] Assume that the mobile phone has established a screen projection connection with the PC, and the second window of the application (such as the Changlian application) displayed on the mobile phone has been projected to the PC, and the first window of the application (such as the Changlian application) is displayed on the mobile phone. For example, the first window is a chat window with contact A in the Changlian application. Fig. 8A (a), the first window includes an image 801 sent by contact A. At this time, if the mobile phone detects that the user clicks on the image 801, the third window is displayed on the mobile phone, such as Fig. 8A (b), that is, image 801 is enlarged and displayed. When the mobile phone detects the operation of projecting the third window, image 801 is projected to the PC for display, such as Fig. 8A (c), then you can switch back to the first window on your phone, such as Fig. 8A (d).
[0209] Below Figure 6 The implementation principle of the second embodiment is introduced as an example. Specifically, the process of the second embodiment may include: Figure 6 The steps S1 to S23 in the embodiment include only S10, S16 to S23, and the following description takes the steps S1 to S23 as an example. For the description of S1 to S15, please refer to the first embodiment.
[0210] S16. The mobile phone detects a fourth operation for opening a third window of the application.
[0211] The implementation method of the fourth operation can refer to the introduction of the implementation method of the second operation in Example 1, and will not be repeated.
[0212] S17. The mobile phone creates a third window in the first task stack.
[0213] See also Figure 8B When the first task stack already includes the first window, under the effect of the fourth operation, the mobile phone creates a third window in the first task stack, and the third window covers the first window, so the foreground of the mobile phone switches from the first window to the third window, that is, S18.
[0214] S18, the mobile phone foreground switches from the first window to the third window.
[0215] For example, see Figure 8C (a), under the effect of the fourth operation, the mobile phone switches from the first window to the third window, such as Figure 8C (b).
[0216] S19. The mobile phone detects a fifth operation for projecting a third window.
[0217] For example, the fifth operation may be a long press and drag operation on the third window, or a long press and drag operation on the third window, and the drag operation is moved to the upper / lower / left / right edge of the display screen. Alternatively, the fifth operation is an operation of clicking an icon of a target device in the multi-device center.
[0218] S20: The mobile phone creates a third task stack, displays the third task stack on the virtual display screen, and moves the third window from the first task stack to the third task stack.
[0219] See also Figure 8B As shown, under the fifth operation, the third window is moved from the first task stack to the third task stack, which specifically includes: the mobile phone creates a third task stack, creates a third window in the third task stack, and closes the third window in the first task stack, so the mobile phone foreground switches from the third window to the first window, that is, S21.
[0220] S21, the mobile phone foreground switches from the third window to the first window.
[0221] S22, the mobile phone sends the display information of the third window to the target device.
[0222] S24, the target device displays a third window.
[0223] See also Figure 8C (b), under the fifth operation, the mobile phone projects the third window to the PC, and the third window is displayed on the PC, such as Figure 8C (c), at this time, the mobile phone foreground switches from the third window to the first window, such as Figure 8C (d).
[0224] Considering that the mobile phone has projected the second window before the third window is projected, the PC includes two windows projected from the mobile phone. Figure 8C In (c), the second window and the third window are displayed vertically on the PC as an example. It is understandable that the second window and the third window may also be displayed in other ways, such as Fig.8D (a) The third window completely covers the second window, or, Fig.8D (b) The third window covers part of the second window, or, Fig.8D As shown in (c), the second window and the third window are displayed horizontally, and so on.
[0225] In the above example, the mobile phone is used to establish a screen projection connection with only one target device (i.e. PC). Figure 8CIn the example, when the mobile phone detects the fifth operation for projecting the third window (e.g., long pressing and dragging the third window), the third window is projected to the PC by default. Alternatively, each time the mobile phone detects an operation for projecting a window (e.g., the fifth operation), a device list may be displayed for the user to select a target device. Figure 8C For example, when the mobile phone detects the fifth operation for projecting the third window, it does not directly project the third window to the PC, but displays a list of devices and selects the target device in the device list. If the selected target device is a PC, since the mobile phone has established a projection connection with the PC, the mobile phone can directly project the third window to the PC without executing the connection process with the PC. If the target device selected by the user is a TV, and the mobile phone has not yet established a projection connection with the TV, the mobile phone will establish a projection connection with the TV and then project the third window to the TV.
[0226] Considering that the number of target devices currently connected to the mobile phone may be more than one. Therefore, when the mobile phone detects an operation for projecting a window, it needs to determine which target device the window is projected to. For example, see Fig. 8E (a), the phone displays the third window of the application and displays the icons of all currently connected target devices. Fig. 8E (b) When the mobile phone detects that the third window is long pressed and dragged to the icon corresponding to the PC, the third window is projected to the PC; or, when the mobile phone detects that the third window is long pressed and dragged to the icon corresponding to the TV, the third window is projected to the TV. Optionally, the mobile phone can also display the windows that have been projected to each target device. Assuming that the second and third windows of the application have been projected to the PC, continue to refer to Fig. 8E (b), when the mobile phone detects the operation of clicking button 810, it can display the following Fig. 8E (c) Displaying the mark of the second window and the mark of the third window at the icon corresponding to the PC, indicating that the second window and the third window are projected onto the PC. To facilitate user identification, the mark of the window may be a reduced version of the window.
[0227] When the mobile phone has established a screen projection connection with the first target device, the window can also be projected to the unconnected second target device. For example, when the mobile phone detects an operation for calling up a device list, the device list is displayed, and the device list includes the currently unconnected target device for the user to select. Among them, the operation for calling up the device list can be, for example, the third operation in Example 1, which is not repeated here.
[0228] Embodiment 3
[0229] In the second embodiment, when the mobile phone has established a screen projection connection with the target device, when the mobile phone detects an operation for opening window C of the application, the window C is displayed on the mobile phone, and when the operation for projecting the window C is detected, the window C is projected to the target device. Different from the second embodiment, in the third embodiment, when the mobile phone has established a screen projection connection with the target device, when the mobile phone detects an operation for opening window C of the application, the mobile phone may not open window C, but directly project the window C to the target device for display.
[0230] Below Figure 6 The implementation principle of the third embodiment is introduced as an example. Specifically, the process of the third embodiment may include Figure 6 In other words, after the mobile phone has established a screen projection connection with the target device, under the effect of the fourth operation for opening the third window, the mobile phone does not create the third window in the first task stack, but directly creates the third window in a third task stack different from the first task stack. Fig. 9A , when the first task stack already includes the first window, and the second task stack already includes the second window (that is, the second window has been projected to the PC), under the effect of the fourth operation, the mobile phone creates a third task stack, the third task stack is displayed on the virtual display screen, and a third window is created in the third task stack. Since the third window is not created in the first task stack, the third window is not opened in the foreground of the mobile phone. For example, see Fig. 9B (a), the first window is displayed on the foreground of the mobile phone. When the mobile phone detects the fourth operation for opening the third window, the mobile phone projects the third window to the PC, and the first window is still displayed on the foreground of the mobile phone.
[0231] An exemplary scenario of the third embodiment is described below.
[0232] Assume that the mobile phone establishes a screen projection connection with the PC, and projects the second window of the application (such as the Changlian application) displayed on the mobile phone to the PC, and the first window of the application (such as the Changlian application) is displayed on the mobile phone. For example, the first window is a chat window with contact A, see Fig. 9C As shown in (a), the first window includes an image 901 sent by contact A. At this time, if the mobile phone detects that the user clicks on the image 901 (i.e., the fourth operation), the image can be projected onto the PC for display. Fig. 9C (b) shows the third window, but the image 901 is not enlarged on the mobile phone.
[0233] It should be noted that. Fig. 9CFor example, when the mobile phone detects that the user wants to open the image, the image is projected to the target device for display. It is understandable that other scenarios are also applicable, for example, the mobile phone displays the home window of the browser application, and the home window includes a video. When the mobile phone detects an operation for playing the video, the video is projected to the target device for display, and the video may not be opened on the mobile phone.
[0234] Optionally, if the screen projection method of Example 2 is used as the first screen projection method and the screen projection method of Example 3 is used as the second screen projection method, the mobile phone can use the first screen projection method or the second screen projection method by default. Of course, the user can also choose a certain screen projection method, for example, setting a screen projection method switching button on the mobile phone, and using this button to switch between the first screen projection method and the second screen projection method.
[0235] Embodiment 4
[0236] In addition to the first and second screen projection methods described above, this fourth embodiment provides a third screen projection method. Different from the first and second screen projection methods, in the third screen projection method, the mobile phone can display candidate windows of the application, and the number of the candidate windows can be multiple, so that the user can select the window to be projected.
[0237] The candidate windows may be all windows opened after the application is started. For example, taking a browser application as an example, after the browser application is started, the home window, the window of news A, and the window of news B are opened, and the candidate windows include these three windows.
[0238] Alternatively, the candidate window may be a preset fixed window. Taking the phone application as an example, the candidate windows may include a contact book window, a recent call window, a dial pad window, and a smooth connection window.
[0239] Alternatively, the candidate window may be a window specified by the user. Taking the Changlian application as an example, the user may set a chat window with a certain contact as a candidate window, and may also specify a chat window with a certain group as a candidate window.
[0240] It should be noted that the above is an example of a candidate window, and there may be other ways to set the candidate window, such as a window that has been opened more than a preset number of times or opened more than a preset frequency in the application, a window that has been displayed in the foreground of the application for a longer time than a preset time, etc. The embodiments of the present application will not give examples one by one.
[0241] Optionally, the candidate window of each application may be different. The candidate window of each application may be the window listed above.
[0242] The mobile phone may display the candidate window of the application in such a manner that, when a preset operation for calling out the candidate window of the application is detected, the candidate window of the application is displayed.
[0243] Among them, the operation type of the preset operation is not limited in this application, for example, it can be a double-click or a long press of the home button, or, it can be an operation of swiping up from the bottom of the screen and pausing to enter the multi-tasking management interface, which includes the candidate window; or, double-clicking or long pressing the screen projection icon (such as located in the pull-up menu); or, after the mobile phone establishes a connection with the target device, a candidate window button is displayed on the foreground of the mobile phone, and the candidate window is called out when a click operation on the button is detected.
[0244] It is understandable that when the window of application A is displayed in the foreground of the mobile phone, the candidate window of application A can be called out through the above preset operation, or the candidate window of each application in all currently running applications (such as application A and application B) can be called out.
[0245] For example, see Fig. 10A As shown in (a), the mobile phone displays the application window D in the foreground. When the mobile phone detects a preset operation, the preset operation is displayed in response to the preset operation. Fig. 10A (b) shows an interface including candidate windows of the application, including windows A to D of the application. Assuming that the mobile phone detects that the user selects windows C and D, the mobile phone projects windows C and D to the PC, see Fig. 10A (c) At this time, the mobile phone displays window A or window B in the foreground. For example, if window B is on the upper layer of window A, window B is displayed. Fig. 10A (d).
[0246] Optional, Fig. 10A In (b), windows A to D can be window snapshots. For example, for each window opened on the mobile phone, a snapshot of the window is stored, such as the window snapshot is stored in SurfaceFlinger. Therefore, SurfaceFlinger stores a large number of window snapshots of multiple applications. In order to facilitate SurfaceFlinger to distinguish snapshots of different windows, each window snapshot can be set with a unique identifier, and there is a corresponding relationship between the unique identifier and the activity handle of the window. Therefore, when the mobile phone displays window D of the application in the foreground, when the mobile phone detects the above-mentioned preset operation, the window snapshot of each window of the application is obtained from SurfaceFlinger according to the activity handle of each window of the application and the corresponding relationship and displayed.
[0247] Fig. 10A In the example shown, the implementation principle of projecting the application window C and window D by the mobile phone is similar to the implementation principle of projecting the second window in the first embodiment, that is, stacking the window C and window D. For details, please refer to Fig. 10BAs shown, the first task stack includes windows A to D. Since window D is at the top, the mobile phone displays window D in the foreground. Fig. 10A (a). See also Fig. 10B , when the mobile phone detects an operation for projecting window C and window D to the PC, window C is transferred to the second task stack and window D is moved to the third task stack. The second task stack and the third task stack are displayed on the virtual display screen, and the windows in the second task stack and the third task stack are projected to the PC. Here, taking the case where window C and window D are in different task stacks after being moved out of the first task stack as an example, it can be understood that after being moved out of the first task stack, window C and window D can also be in the same task stack, for example, both are in the second task stack, so there is no need to establish a third task stack.
[0248] It should be noted that when window C is moved from the first task stack to the second task stack, the second task stack can set the original task stack mark, which is used to represent the original task stack to which the window of the second task stack belongs, that is, the original task stack mark set on the second task stack is the mark of the first task stack, such as the ID of the first task stack. By setting the task stack mark, it is convenient to restore the window in the second task stack to the original first task stack when exiting the screen projection. Similarly, when window D is moved from the first task stack to the third task stack, the original task stack mark set by the third task stack is also the ID of the first task stack.
[0249] above Fig. 10A and Fig. 10B Taking the case where windows A to D are in the same task stack, i.e., the first task stack, as an example, the user can select the window to be projected in the task stack. In simple terms, the selection is made based on the dimension of the window. Optionally, the mobile phone can also make a selection based on the dimension of the task stack. For example, see Fig. 10C (a), the mobile phone displays the application window D in the foreground. When the mobile phone detects a preset operation, in response to the preset operation, the following is displayed Fig. 10C (b) shows an interface, which includes all task stacks of the foreground application, or all task stacks of each application in all currently running applications. Take the interface showing all task stacks of the foreground application as an example, such as task stack A and task stack B, task stack A includes two windows, and task stack B includes two windows; suppose that when the mobile phone detects the operation of selecting task stack B, all windows in task stack B are projected to the PC, such as Fig. 10C (c) At this time, window B is displayed on the mobile phone (eg, window B is at the top of task stack A).
[0250] Optional, Fig. 10CIn (b), windows A to D may also be window snapshots. For example, when the mobile phone detects the above preset operation, it obtains the window snapshot of each window from SurfaceFlinger and displays the window snapshot in the form of a task stack.
[0251] Optionally, the window projected to the PC can be closed, for example, see Fig. 10A Window C includes a close control 1001. When the PC detects that an operation of clicking the "close" control 1001 in window C is performed, window C is closed, that is, window C is no longer displayed on the PC.
[0252] Optionally, when window C is closed on the PC, window C can also be returned to the mobile phone for display. Fig. 10A When the PC detects the operation of clicking the "Close" control 1001 in window C, the PC sends an instruction to the mobile phone, which is used to instruct the mobile phone to display window C. Therefore, window C is closed on the PC and window C is displayed on the mobile phone. Fig. 10D .
[0253] The return of window C to the mobile phone can be achieved through the stack restoration principle. For details, see Fig. 10E , window C is located in the second task stack. Since the original task stack tag set by the second task stack is the ID of the first task stack, the mobile phone can determine that the original task stack of window C is the first task stack based on the original task stack tag. Therefore, the mobile phone restores window C from the second task stack to the first task stack, and then window C in the first task stack covers window B and window A. Therefore, window C is displayed on the mobile phone and closed on the PC. Optionally, the mobile phone can also close the second task stack, such as Fig. 10E .
[0254] If when window C is closed on the PC, window C is returned to the mobile phone for display as the first method, and when window C is closed on the PC, window C is not returned to the mobile phone for display as the second method. The mobile phone may use the first method or the second method by default. Alternatively, the user may select the first method or the second method, for example, the mobile phone or the PC includes a switch button, and the first method or the second method is selected through the switch button.
[0255] Optionally, after window C is projected to the PC, when the mobile phone detects an operation for opening window C, window C on the PC can also be pulled back. For example, when the mobile phone detects an operation for opening window C, the mobile phone sends a command to the PC, which is used to instruct the PC to close window C. Therefore, window C is closed on the PC and window C is displayed on the mobile phone. Fig. 10D .
[0256] Optionally, when the mobile phone detects an operation for opening window C, window C on the PC may not be pulled back. Fig. 10A (d) as an example, the mobile phone displays window B, and the PC displays windows C and D. At this time, when the mobile phone detects an operation for opening window C, the mobile phone displays window C in the foreground, and the PC still displays window C. Fig.10F If the mobile phone does not pull back the window C on the PC, the implementation principle can be seen in Figure 10G When the mobile phone detects an operation for opening window C, window C is created in the first task stack, and the second task stack still includes window C, so window C is displayed on both the mobile phone and the PC.
[0257] Optional, Figure 10G In the example, the window C created in the first task stack may be the reused window C in the second task stack, or may be a newly created window C. Alternatively, the mobile phone may determine whether the window C supports multiple instances, if so, create a new window C, if not, reuse the window C in the second task stack.
[0258] If the first method is to pull the window C on the PC back to the mobile phone when the window C is opened on the mobile phone, and the second method is to not pull the window C back to the mobile phone when the mobile phone opens the window C. The mobile phone can use the first method or the second method by default. Alternatively, the user can select the first method or the second method, for example, the mobile phone or the PC includes a switch button, and the first method or the second method is selected through the switch button.
[0259] Optionally, the user can open a new window of the application on the PC. For example, see Fig.11A , window B is displayed on the mobile phone, and windows C and D are displayed on the PC. When a click operation on the "+" button in window D is detected on the PC, window E of the application is displayed on the PC. Among them, window E can be a fixed window by default, such as the home window of the application or a window specified by the user. Alternatively, window E can also be an upper window or a lower window located in window D among all windows that have been opened by the application. Taking the lower window as an example, when the PC detects a click operation on the "+" button in window D, it sends an instruction to the mobile phone. The mobile phone determines that the next layer window of window D is window C, and window C has been displayed on the PC. Then the mobile phone can continue to determine whether the next layer window of window C, that is, window B, is projected to the PC. If not, window B is projected to the PC. Then the mobile phone can no longer display window B in the foreground, but display the next layer window of window B, that is, window A.
[0260] Specifically, the implementation principle of opening the application window E on the PC can be seen in Fig. 11BAs shown, when the PC detects an operation for opening window E, it sends a command to the mobile phone, which is used to indicate that window E needs to be opened on the PC. After receiving the command, the mobile phone creates a fourth task stack, creates window E in the fourth task stack, and displays the fourth task stack on the virtual display screen, so it is projected to the PC, so window E is displayed on the PC. If window E is window B, that is, Fig. 11B Window B in the first task stack is moved to the fourth task stack.
[0261] Based on the above embodiments, the present application also provides an electronic device, which may be the master device or the target device mentioned above. Fig.12 As shown, the electronic device 1200 may include: a display screen 1201 , one or more processors 1202 , a memory 1203 , and one or more computer programs (not shown in the figure). The above components may be coupled via one or more communication buses 2704 .
[0262] The display screen 1201 is used to display the interface, image, video and other related user interfaces of the application. The memory 1203 stores one or more computer programs, and the one or more computer programs include instructions; the processor 1202 calls the instructions stored in the memory 1203, so that the electronic device 1200 executes the screen projection method of the application window provided in the embodiment of the present application.
[0263] In the embodiments provided by the present application, the method provided by the embodiments of the present application is introduced from the perspective of the electronic device as the execution subject. In order to implement the functions in the methods provided by the embodiments of the present application, the terminal device may include a hardware structure and / or a software module to implement the above functions in the form of a hardware structure, a software module, or a hardware structure plus a software module. Whether a function of the above functions is executed in the form of a hardware structure, a software module, or a hardware structure plus a software module depends on the specific application and design constraints of the technical solution.
[0264] As used in the above embodiments, the term "when..." or "after..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrase "when determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "when (the stated condition or event) is detected" or "in response to detecting (the stated condition or event)", depending on the context. In addition, in the above embodiments, relational terms such as first and second are used to distinguish one entity from another, without limiting any actual relationship and order between the entities.
[0265] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.
[0266] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented with hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein the communication media include any media that facilitates the transmission of a computer program from one place to another. The storage medium can be any available medium that a general or special-purpose computer can access.
[0267] The specific implementation methods described above further describe the purpose, technical solutions and beneficial effects of the embodiments of the present application in detail. It should be understood that the above is only the specific implementation methods of the embodiments of the present application, and is not intended to limit the protection scope of the embodiments of the present application. Any modifications, equivalent replacements, improvements, etc. made on the basis of the technical solutions of the embodiments of the present application should be included in the protection scope of the embodiments of the present application. The above description of the specification of the present application can make any technical content of the present application embodiment that can be utilized or implemented in the art, and any modification based on the disclosed content should be considered obvious in the art. The basic principles described in the embodiments of the present application can be applied to other variations without departing from the inventive essence and scope of the present application. Therefore, the content disclosed in the embodiments of the present application is not only limited to the described embodiments and designs, but can also be extended to the maximum scope consistent with the principles of the present application and the disclosed new features.
[0268] Although the present application has been described in conjunction with specific features and embodiments thereof, it is obvious that various modifications and combinations may be made thereto without departing from the spirit and scope of the embodiments of the present application. Accordingly, this specification and the drawings are merely exemplary illustrations of the present application as defined by the appended claims, and are deemed to have covered any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art may make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalent technologies, the embodiments of the present application are also intended to include these changes and variations.
Claims
1. A screen projection method for an application window, characterized in that: The method comprises: The first electronic device establishes a connection with the second electronic device; The first electronic device projects the first window of the application to the second electronic device; The second electronic device displays the first window; The first electronic device displays a second window of the application.
2. The method according to claim 1, characterized in that The second window is a previous window of the first window; or, is a default window of the application; or, is a designated window of the application.
3. The method according to claim 1 or 2, characterized in that The first electronic device projects a first window of an application to the second electronic device, including: In response to an operation for opening the first window, the first electronic device projects the first window to the second electronic device.
4. The method according to claim 3, characterized in that The operation for opening the first window includes: an operation for opening an image; and / or an operation for playing a video file.
5. The method according to claim 1 or 2, characterized in that: The first electronic device projects a first window of an application to the second electronic device, including: In response to an operation for opening the first window, the first electronic device displays the first window; In response to an operation for projecting the second window, the first electronic device projects the first window to the second electronic device.
6. The method according to any one of claims 1 to 5, characterized in that: Before the first electronic device projects the first window of the application to the second electronic device, the method further includes: The first electronic device displays the first window and the second window; The first electronic device displays the second window of the application, including: The first electronic device displays the second window in full screen.
7. The method according to any one of claims 1 to 6, characterized in that: The first electronic device projects a first window of an application to the second electronic device, including: In response to the first operation, multiple identifiers of multiple devices to be projected are displayed; In response to an operation of selecting the identifier of the second electronic device from the multiple identifiers, the first electronic device projects the first window to the second electronic device.
8. The method according to any one of claims 1 to 7, characterized in that: The first electronic device projects a first window of an application to the second electronic device, including: In response to a second operation, displaying a plurality of candidate windows of the application; In response to an operation of selecting the first window from among the multiple candidate windows, the first electronic device projects the first window to the second electronic device.
9. The method according to claim 8, characterized in that The candidate window includes: All windows of the application that have been opened by the first electronic device; or, a preset window of the application; or, A window specified by a user in the application; or A window in the application that has been opened more times or more frequently than a preset value; or A window in the application that runs for a longer period than a preset period of time is running in the foreground.
10. The method according to any one of claims 1 to 9, characterized in that: The method further comprises: The first electronic device opens a third window of the application; The first electronic device projects the third window onto the second electronic device; The second electronic device displays the third window; the third window on the second electronic device completely or partially covers the first window.
11. The method according to any one of claims 1 to 10, characterized in that: The method further comprises: In response to a first instruction for restoring the first window to be displayed on the first electronic device, the first electronic device displays the first window, and the second electronic device closes the first window.
12. The method according to claim 11, characterized in that The first instruction for restoring the first window to be displayed on the first electronic device includes: an operation detected on the second electronic device for closing the first window on the second electronic device; or, An operation detected on the first electronic device for opening the first window on the first electronic device.
13. The method according to any one of claims 1 to 12, characterized in that: The application further includes a fourth window, and the method further includes: An operation for opening the second window is detected on the second electronic device; In response to the operation for opening the second window, the second electronic device sends a second instruction to the first electronic device; In response to the received second instruction, the first electronic device projects the second window to the second electronic device; The second electronic device displays the second window; the second window completely or partially covers the first window; The first electronic device displays the fourth window.
14. The method according to any one of claims 1 to 12, characterized in that: The application further includes a fourth window, the system further includes a third electronic device, and the method further includes: The first electronic device is connected to the third electronic device; The first electronic device projects the second window to the third electronic device; The third electronic device displays the second window; The first electronic device displays the fourth window.
15. The method according to any one of claims 1 to 14, characterized in that: The first electronic device projects a first window of an application to the second electronic device, including: The first electronic device creates the first window in a first task stack; the window in the first task stack is configured to be projected onto the second electronic device for display; The first task stack is different from a second task stack where the second window is located, and the window in the second task stack is configured to be displayed on a display screen of the first electronic device.
16. The method according to any one of claims 1 to 14, characterized in that: The first electronic device projects a first window of an application to the second electronic device, including: In response to an operation for opening the first window, the first electronic device creates the first window in a second task stack; the window of the second task stack is configured to be displayed on a display screen of the first electronic device; In response to the operation for projecting the first window, the first electronic device creates the first window in a first task stack and closes the first window in the second task stack; the window in the first task stack is configured to be projected onto the second electronic device for display.
17. The method according to claim 16, characterized in that The method further comprises: In response to an operation for restoring the first window to be displayed on the first electronic device, the first electronic device determines, according to an original stack mark of the first task stack, that the original task stack of the first window is a second task stack; The first electronic device creates the first window in the second task stack; The first electronic device closes the first window in the first task stack.
18. The method according to any one of claims 1 to 17, characterized in that: The first electronic device projects a first window of an application to the second electronic device, including: In response to a preset operation, the first electronic device displays a plurality of task stacks of the application; In response to an operation for selecting a third task stack from the plurality of task stacks, the first electronic device projects all windows of the third task stack onto the second electronic device.
19. A screen projection method for an application window, characterized in that: Applied to a first electronic device, the first electronic device includes an application, the application includes a first window and a second window; the method includes: The first electronic device is connected to the second electronic device; The first electronic device projects the first window onto the second electronic device for display; The first electronic device displays the second window.
20. The method of claim 19, wherein: The second window is a previous window of the first window; or, a default window of the application; or, a designated window of the application.
21. The method according to claim 19 or 20, characterized in that The first electronic device projects the first window onto the second electronic device for display, including: In response to the operation for opening the first window, the first electronic device projects the first window onto the second electronic device for display.
22. The method according to claim 21, characterized in that The operation for opening the first window includes: an operation for opening an image; and / or an operation for playing a video file.
23. The method according to claim 19 or 20, characterized in that The first electronic device projects the first window onto the second electronic device for display, including: In response to an operation for opening the first window, the first electronic device displays the first window; In response to the operation for projecting the second window, the first electronic device projects the first window onto the second electronic device for display.
24. The method according to any one of claims 19 to 23, characterized in that: Before the first electronic device projects the first window onto the second electronic device, the method further includes: The first electronic device displays the first window and the second window; The first electronic device displays the second window, including: The first electronic device displays the second window in full screen.
25. The method according to any one of claims 19 to 23, characterized in that: The first electronic device projects the first window onto the second electronic device for display, including: In response to a second operation, displaying a plurality of candidate windows of the application; In response to an operation of selecting the first window from among the multiple candidate windows, the first electronic device projects the first window onto the second electronic device for display.
26. The method according to any one of claims 19 to 25, characterized in that: The application further includes a third window, the system further includes a third electronic device, and the method further includes: The first electronic device is connected to the third electronic device; The first electronic device projects the second window to the third electronic device; The third electronic device displays the second window; The first electronic device displays the third window.
27. The method according to any one of claims 19 to 26, characterized in that: The first electronic device projects the first window onto the second electronic device, including: The first electronic device creates the first window in a first task stack; the window in the first task stack is configured to be projected onto the second electronic device for display; wherein the first task stack is different from the second task stack where the second window is located, and the window in the second task stack is configured to be displayed on the display screen of the first electronic device.
28. The method according to any one of claims 19 to 26, characterized in that: The first electronic device projects the first window onto the second electronic device, including: In response to an operation for opening the first window, the first electronic device creates the first window in a second task stack; the window of the second task stack is configured to be displayed on a display screen of the first electronic device; In response to the operation for projecting the first window, the first electronic device creates the first window in a first task stack and closes the first window in the second task stack; the window in the first task stack is configured to be projected onto the second electronic device for display.
29. An electronic device, characterized in that: include: A display screen, one or more processors, a memory, and one or more programs; wherein the one or more programs are stored in the memory, and the one or more programs include instructions, which, when executed by the electronic device, enable the electronic device to perform the method as described in any one of claims 1 to 28.
30. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and when the computer program is run on a computer, the computer is caused to execute the method according to any one of claims 1 to 28.
