Multi-screen collaboration method and system, and electronic device

Abstract

The present application relates to the field of terminals and provides a multi-screen collaboration method and system, and an electronic device, for enabling application functions of various applications provided by a source device to be integrated with application functions on a destination device to implement multi-screen collaboration. The method includes a source device transmitting application information of at least one source device application installed on the source device to a destination device. The destination device displays an icon of the at least one source device application on a first interface and an icon of at least one destination device application installed on the destination device on the first interface. In response to an operation performed by a user on the icon of a first application of the source device application, the destination device acquires first display data generated during execution of the first application on the source device. The destination device displays an application window of the first application based on the first display data. In response to an operation performed by a user on an icon of a second application of the destination device application, the destination device launches the second application and displays the application window of the second application.

Description

[Technical field]

[0001] This application claims priority to Chinese Patent Application No. 201911303934.1, entitled "Multi-screen collaboration method and system, and electronic device," filed with the State Intellectual Property Office of the People's Republic of China on December 17, 2019, the entirety of which is incorporated herein by reference.

[0002] The present application relates to the field of terminals, and in particular to a multi-screen collaboration method and system, and an electronic device. [Background technology]

[0003] With the development of intelligent terminal technology, a user or a family usually has multiple electronic devices that can communicate with each other. Various electronic devices generally have their own device characteristics. For example, a mobile phone is more portable, a television screen has a better display effect, and a sound box has a better sound quality. In order to fully utilize the device characteristics of different electronic devices, the electronic device can switch the display of display data between multiple devices by screen projection, etc.

[0004] For example, a mobile phone is used as a source device. The mobile phone may project an application interface or a task being performed (e.g., a video playback task) to a destination device (e.g., a TV or a PC) for display. For example, when the mobile phone runs a video app, if a user enables the screen projection function of the mobile phone and selects a PC as a destination device, the mobile phone may be used as a source device to project the application interface of the video app to the PC for display.

[0005] In other words, when a mobile phone performs screen projection to a PC, the PC can be used as another display of the mobile phone to display the contents in the mobile phone. In this case, a user can operate the PC to control related applications of the mobile phone, but the user cannot operate the PC to perform related functions originally provided by the PC. In short, after a source device performs screen projection to a destination device, the functions provided by the source device and the destination device cannot be effectively integrated on the destination device to perform a multi-screen cooperation function. As a result, a user cannot use the functions provided by the source device and the functions provided by the destination device by screen projection at the same time on the destination device. Summary of the Invention [Means for solving the problem]

[0006] The present application provides a multi-screen collaboration method and system, as well as an electronic device, so as to facilitate a user to operate different applications installed on a source device and a destination device on the destination device, and so that application functions of various applications provided by the source device can be integrated with application functions on the destination device to implement the multi-screen collaboration function.

[0007] In order to achieve the aforementioned objectives, the following technical solutions are used in this application:

[0008] According to a first aspect, the present application provides a multi-screen collaboration method, including: a source device transmitting application information of at least one application (which may be referred to as a source device application) installed on the source device to a destination device; the destination device displaying an icon of the at least one application (which may be referred to as a destination device application) installed on the destination device in a first interface based on the application information; the destination device displaying an icon of the at least one application (which may be referred to as a destination device application) installed on the destination device in the first interface; when detecting an operation performed by a user on an icon of the first application (which may be referred to as an application of a source device application) on the destination device, the destination device may obtain first display data generated during execution of the first application on the source device; the destination device may display an application window of the first application on the first interface based on the first display data; and when detecting an operation performed by a user on an icon of a second application (which may be referred to as an application of a destination device application) on the destination device, the destination device may launch the second application and display an application window of the second application in the first interface.

[0009] In short, all the device functions of the source device can be presented on the destination device in the form of application icons of the source device, and all the device functions of the destination device can also be presented on the destination device in the form of application icons of the destination device. In this way, a user can conveniently use the applications provided by the destination device and the applications provided by the source device on the destination device, and the application functions provided by the source device can be effectively integrated with the application functions on the destination device without affecting the normal operation of the destination device. This improves the user experience in multi-screen interaction and collaboration.

[0010] For example, the source device may run a first operating system and the destination device may run a second operating system. The first operating system may be the same as or different from the second operating system. For example, the first operating system may be an Android operating system and the second operating system may be a Windows operating system.

[0011] In a possible implementation, the application information may include an identifier of at least one source device application, in which the destination device obtaining first display data generated during execution of the first application on the source device in response to an operation performed by a user on an icon of the first application on the destination device may specifically include the destination device sending a first launch message to the source device in response to an operation performed by a user on an icon of the first application on the destination device, the first launch message including an identifier of the first application, and in response to the first launch message, the source device launching the first application based on the identifier of the first application and sending the first display data generated during execution of the first application to the destination device.

[0012] In addition to the identifier of the application, the application information may further include an icon of at least one source device application. In this way, after receiving the application information, the destination device may display the icon in the application information on the first interface to present the device capabilities of the source device to the user.

[0013] In possible implementations, the first display data may be drawing instructions generated for the first application interface in a corresponding buffer queue during execution of the first application, the first display data may be frame data (which may also be referred to as graphics data) generated by SurfaceFlinger of the source device for the first application interface based on the drawing instructions during execution of the first application, or the first display data may be pixel data (e.g., RGB data) generated by HWcomposer of the source device for the first application interface based on the frame data during execution of the first application.

[0014] For example, before transmitting the first display data to the destination device, the source device may first encode the first display data using a video encoder to reduce the amount of data by compressing the display data to be transmitted to the destination device.

[0015] In a possible implementation, the application window of the first application may include a first application interface and a title bar of the first application, where the title bar includes a minimize button, a maximize button, and a close button of the first application, or the title bar may further include information such as the application name. Of course, the application window of the first application may alternatively not include the aforementioned title bar.

[0016] Similarly, the application window of the second application may include a second application interface and a title bar of the second application. In this manner, a user may conveniently open the source device application or the destination device application on the destination device.

[0017] In possible implementations, when the destination device displays an application window of the first application, the method further includes the source device displaying a first application interface of the first application, or the source device displaying an application interface of a third application, the third application being an application of the source device different from the first application. In short, the source device may launch the first application in the foreground or background and display the first application on the destination device in cross-screen mode without affecting applications running on the source device.

[0018] In a possible implementation, the source device includes a first display module (e.g., display module 1) and a second display module (e.g., display module 2). In other words, the source device supports a multi-display module mode. The first display module is configured to provide display data to the source device, and the second display module is configured to provide display data to the destination device, and the first display data displayed by the destination device may be stored in the second display module. In this case, the destination device obtaining the first display data generated during execution of the first application on the source device includes the destination device obtaining the first display data generated during execution of the first application from the second display module of the source device.

[0019] In a possible implementation, after the destination device displays the application window of the first application on the first interface based on the first display data as the application window of the first application displayed in the cross-screen mode, the method further includes the source device receiving a second launch message sent by the destination device, the second launch message including an identifier of a fourth application, the fourth application being an application of the source device different from the first application, the source device launching the fourth application in response to the second launch message, further storing in the second display module second display data generated during the execution of the fourth application, and the destination device retrieving the second display data from the second display module and displaying in the first interface the application window of the fourth application based on the second display data, thereby implementing the display of multiple applications of the source device in the cross-screen mode.

[0020] For example, each display module of the source device corresponds to a physical screen having a specific size. After the source device receives the second launch message sent by the destination device, the method further includes the source device sending position information of the second display data to the destination device, and the position information is used to indicate a specific position of the application window of the fourth application on the screen corresponding to the second display module. In this case, the destination device obtaining the second display data from the second display module includes the destination device obtaining the second display data from the second display module based on the position information. In this way, when the second display module stores the display data of the multiple applications, the destination device may obtain the display data of the application currently operated by the user based on the corresponding position information. This implements the display of the multiple application windows in the cross-screen mode at application granularity.

[0021] Alternatively, the second display module includes a first display space and a second display space. In other words, the second display module is divided into multiple real spaces in advance during the execution of multiple applications. For example, the first display space is used to store the first display data, and the second display space is used to store the second display data. In this case, the destination device obtaining the second display data from the second display module includes the destination device obtaining the second display data from the second display space. In this case, the destination device does not need to obtain position information of the second display data relative to the second display module.

[0022] In a possible embodiment, the source device may further include a third display module, and the third display module may also be configured to provide display data to the destination device. After the destination device displays the application window of the first application on the first interface based on the first display data, the method further includes: the source device receives a second launch message sent by the destination device, the second launch message includes an identifier of a fourth application, the fourth application being an application of the source device different from the first application; the source device launches the fourth application in response to the second launch message, and stores the second display data generated during the execution of the fourth application in the third display module; and the destination device obtains the second display data from the third display module and displays the application window of the fourth application on the first interface based on the third display data. In short, each application on the source device that needs to be executed on the destination device may correspond to one display module. In this way, the display modules corresponding to the applications do not overlap, and the destination device may obtain the display data of the related application from the corresponding display module for display.

[0023] In a possible implementation, after the destination device displays the application window of the first application in the first interface based on the first display data, the method further includes: in response to an operation of opening the first application by the user on the source device, the source device stores the first display data generated during the execution of the first application in a first display module and continues to provide the display data to the source device using the first display module; or in response to an operation of opening the first application by the user on the source device, the source device reads and displays the first display data stored in the second display module. Specifically, after the first application of the source device is displayed on the destination device in the cross-screen mode, the user may further open and display the first application on the source device. For example, the user may open the first application that is running using a multitasking window on the source device. In this case, the source device may store the display data of the first application in a corresponding first display module and provide the display data for the source device's display using the first display module, or the source device may provide the display data for the source device's display using the second display module.

[0024] In a possible implementation, after the destination device displays the application window of the first application in the first interface based on the first display data, the method further includes: in response to a first operation input by a user in the application window of the first application, the destination device sends a first control message corresponding to the first operation to the source device, the first control message including an identifier of the first application; and in response to the first control message, the source device executes an operation instruction corresponding to the first control message for the first application. In this way, the user may manipulate the content projected to the destination device for display, thereby implementing various functions of the associated application of the source device on the destination device.

[0025] In a possible implementation, after the source device transmits application information of at least one source device application installed on the source device to the destination device, the method further includes the destination device displaying a first button in the first interface, where the destination device displaying an icon of the at least one source device application in the first interface based on the application information includes the destination device displaying an icon of the at least one source device application in the first interface based on the application information when detecting that a user has selected the first button.

[0026] For example, the first button may be located on a preset presentation bar, and the application information may further include a usage frequency or a running status of the application, and the method further includes the destination device displaying, on the presentation bar, icons of one or more applications whose usage frequency is greater than a preset value based on the application information, or the destination device displaying, on the presentation bar, icons of one or more applications running on the source device based on the application information.

[0027] The preset presentation bar may be a dock bar or a floating window, etc., and the position of the preset presentation bar may be set or dragged by the user.

[0028] In a possible implementation, the first button is located on a pre-configured presentation bar, and the method further includes the destination device adding an icon of the new application to the presentation bar in response to a user operation of adding a new application.

[0029] In a possible implementation, the destination device may display an icon of at least one source device application on the desktop, start menu, taskbar, or tile of the destination device, which is not limited to the embodiments of the present application.

[0030] In a possible embodiment, the destination device may include an electronic label or an NFC chip, and the source device may also include an electronic label or an NFC chip. Before the source device transmits application information of at least one source device application installed on the source device to the destination device, the method further includes the source device establishing a communication connection to the destination device when the source device approaches or touches the electronic label or the NFC chip of the destination device. In this way, the source device may be triggered to display one or more applications of the source device on the destination device in a cross-screen mode in a "touch" manner between the source device and the destination device.

[0031] According to a second aspect, the present application provides a multi-screen collaboration method, the multi-screen collaboration method including: a destination device receiving application information of at least one source device application transmitted by a source device; the source device running a first operating system, the destination device running a second operating system; the destination device displaying an icon of the at least one source device application in a first interface based on the application information; the destination device displaying an icon of at least one destination device application installed on the destination device in the first interface; in response to an operation performed by a user on an icon of the first application (the first application is an application of a source device application) on the destination device, the destination device acquiring first display data generated during the execution of the first application on the source device; the destination device displaying an application window of the first application in the first interface based on the first display data; in response to an operation performed by a user on an icon of a second application (the second application is an application of a destination device application) on the destination device, the destination device launching a second application and displaying an application window of the second application in the first interface.

[0032] In short, all the device functions of the source device can be presented on the destination device in the form of application icons of the source device, and all the device functions of the destination device can also be presented on the destination device in the form of application icons of the destination device. In this way, a user can conveniently use the applications provided by the destination device and the applications provided by the source device on the destination device, and the application functions provided by the source device can be effectively integrated with the application functions on the destination device without affecting the normal operation of the destination device. This improves the user experience in multi-screen interaction and collaboration.

[0033] In a possible implementation, the application information includes an identifier of at least one source device application, and the destination device obtaining, in response to an operation performed by a user on an icon of the first application on the destination device, first display data generated during execution of the first application on the source device includes the destination device sending a first launch message to the source device in response to an operation performed by a user on an icon of the first application on the destination device, the first launch message including an identifier of the first application, and the destination device obtaining, from the source device, the first display data generated during execution of the first application.

[0034] In a possible implementation, the application window of the first application may include a first application interface and a title bar of the first application, the title bar including a minimize button, a maximize button, and a close button of the first application.

[0035] Similarly, the application window of the second application includes a second application interface of the second application and a title bar, which may of course include further information such as the application name.

[0036] In a possible implementation, the source device includes a first display module and a second display module, the first display module configured to provide display data to the source device and the second display module configured to provide display data to the destination device, and the first display data is stored in the second display module. The destination device obtaining the first display data generated during execution of the first application on the source device includes the destination device obtaining, from the second display module, the first display data generated during execution of the first application.

[0037] In a possible implementation, after the destination device displays an application window of the first application in the first interface based on the first display data, the method further includes: in response to an operation performed by a user on an icon of a third application on the destination device, the destination device sending a second launch message to the source device, the second launch message including an identifier of the third application, the third application being an application of the source device different from the first application; and the destination device obtaining, from the second display module, second display data generated during execution of the third application, and displaying the application window of the third application in the first interface based on the second display data.

[0038] In a possible implementation, after the destination device sends the second wake-up message to the source device, the method further includes the destination device receiving location information of the second display data sent by the source device. The destination device obtaining, from the second display module, the second display data generated during execution of the third application includes the destination device obtaining, from the second display module based on the location information, the second display data generated during execution of the third application.

[0039] In a possible implementation, the second display module includes a first display space and a second display space, the first display space being used to store the first display data and the second display space being used to store the second display data. The destination device obtaining from the second display module the second display data generated during execution of the third application includes the destination device obtaining from the second display space the second display data generated during execution of the third application.

[0040] In a possible implementation, the source device further includes a third display module, the third display module also configured to provide display data to the destination device. After the destination device displays the application window of the first application in the first interface based on the first display data, the method further includes: in response to an operation performed by a user on an icon of the third application on the destination device, the destination device sends a second launch message to the source device, the second launch message including an identifier of the third application, the third application being an application of the source device different from the first application; and the destination device obtains from the third display module second display data generated during the execution of the third application, and displays the application window of the third application in the first interface based on the second display data.

[0041] In a possible implementation, after the destination device displays the application window of the first application in the first interface based on the first display data, the method further includes: in response to a first operation input by a user in the application window of the first application, the destination device sends a first control message corresponding to the first operation to the source device, such that the source device executes an operation instruction corresponding to the first control message for the first application, the first control message including an identifier of the first application. In this way, the user may manipulate the content projected to the destination device for display. This allows various functions of the associated application of the source device to be implemented on the destination device.

[0042] In possible implementations, the destination device may further display a first button in the first interface, where the destination device displaying an icon of the at least one source device application in the first interface based on the application information includes the destination device displaying an icon of the at least one source device application in the first interface based on the application information when detecting that the user has selected the first button.

[0043] For example, the first button may be located on a preset presentation bar, and the application information may further include a usage frequency or a running status of the application, and the method further includes the destination device displaying, on the presentation bar, icons of one or more applications whose usage frequency is greater than a preset value based on the application information, or the destination device displaying, on the presentation bar, icons of one or more applications running on the source device based on the application information.

[0044] The preset presentation bar may be a dock bar or a floating window, etc., and the position of the preset presentation bar may be set or dragged by the user.

[0045] In a possible implementation, the first button is located on a pre-configured presentation bar, and the method further includes the destination device adding an icon of the new application to the presentation bar in response to a user operation of adding a new application.

[0046] In a possible embodiment, the application icons of the source device may be displayed on the desktop, start menu, taskbar, or tiles of the destination device. In short, the application list projected by the source device to the destination device for display does not affect the application list originally displayed by the destination device. Thus, on the destination device, the user may not only manipulate the icons provided by the destination device to use related functions, but also manipulate the application icons provided by the source device to use related functions of the source device. This implements the multi-screen cooperation function.

[0047] According to a third aspect, the present application provides a multi-screen collaboration method, the multi-screen collaboration method including: a source device transmitting application information of at least one source device application installed on the source device to a destination device, such that the destination device displays an icon of the at least one source device application based on the application information, the source device running a first operating system and the destination device running a second operating system; the source device receiving a first launch message transmitted by the destination device, the first launch message including an identifier of a first application (the first application being an application of a source device application); and in response to the first launch message, the source device launching the first application based on the identifier of the first application and transmitting first display data generated during the execution of the first application to the destination device.

[0048] The source device may send application information of one or more applications installed on the source device to the destination device, so that the destination device may know and present an application list of applications installed on the source device to the user. In this way, all device capabilities of the source device may be presented on the destination device in the form of an application list, and the user may open and display one or more applications installed on the source device on the destination device according to the user's request. This implements the display of one or more applications on different devices in cross-screen mode, improving the user experience.

[0049] In possible implementations, the first display data may be drawing instructions generated for the first application interface in a corresponding buffer queue during execution of the first application, the first display data may be frame data (which may also be referred to as graphics data) generated by SurfaceFlinger of the source device for the first application interface based on the drawing instructions during execution of the first application, or the first display data may be pixel data (e.g., RGB data) generated by HWcomposer of the source device for the first application interface based on the frame data during execution of the first application.

[0050] In a possible implementation, before the source device transmits to the destination device the first display data generated during execution of the first application, the method further includes the source device encoding the first display data using a video encoder to reduce the amount of data by compressing the display data to be transmitted to the destination device.

[0051] In a possible implementation, the source device launching the first application based on the identifier of the first application includes the source device launching the first application in the foreground or background based on the identifier of the first application.

[0052] In a possible implementation, the source device includes a first display module and a second display module, the first display module configured to provide display data to the source device, the second display module configured to provide display data to the destination device, and the first display data is stored in the second display module.

[0053] In a possible implementation, after the source device transmits to the destination device the first display data generated during execution of the first application, the method further includes the source device receiving a second launch message transmitted by the destination device, the second launch message including an identifier of the second application, the second application being one of the at least one source device application, the second application being different from the first application, and the source device launching the second application in response to the second launch message and transmitting the second display data generated during execution of the second application to the second display module.

[0054] In a possible implementation, in response to the second wake-up message, the method further includes the source device transmitting location information of the second display data to the destination device.

[0055] In a possible implementation, the second display module includes a first display space and a second display space, the first display space being used to store the first display data and the second display space being used to store the second display data. The source device transmitting the second display data generated during execution of the second application to the second display module includes the source device transmitting the second display data generated during execution of the second application to the second display space.

[0056] In a possible implementation, the source device further includes a third display module, the third display module also configured to provide display data to the destination device, and after the source device transmits the first display data generated during execution of the first application to the destination device, the method further includes the source device receiving a second launch message sent by the destination device, the second launch message including an identifier of the second application, the second application being one of the at least one source device application, the second application being different from the first application, and the source device launching the second application in response to the second launch message and transmitting the second display data generated during execution of the second application to the third display module.

[0057] In a possible implementation, after the source device transmits to the destination device first display data generated during execution of the first application, the method further includes the source device receiving a first control message transmitted by the destination device, the first control message including an identifier of the first application, and in response to the first control message, the source device executing, for the first application, an operation instruction corresponding to the first control message.

[0058] According to a fourth aspect, the present application provides a destination device including a display, a communication module, one or more processors, one or more memories, and one or more computer programs. The processor is coupled to the communication module, the communication module, and the memory. The one or more computer programs are stored in the memory. When the destination device operates, the processor executes the one or more computer programs stored in the memory to enable the destination device to perform the multi-screen cooperation method according to the second aspect or any one of the possible implementations of the second aspect.

[0059] According to a fifth aspect, the present application provides a source device including a display, a communication module, one or more processors, one or more memories, and one or more computer programs. The processor is coupled to the communication module, the communication module, and the memory. The one or more computer programs are stored in the memory. When the source device operates, the processor executes the one or more computer programs stored in the memory to enable the source device to perform the multi-screen collaboration method according to the third aspect or any one of the possible implementations of the third aspect.

[0060] According to a sixth aspect, the present application provides a multi-screen collaboration system including a destination device and a source device as mentioned above, wherein the destination device and the source device are capable of performing the multi-screen collaboration method according to the first aspect or any one of the possible implementations of the first aspect by interaction.

[0061] According to a seventh aspect, the present application provides a computer-readable storage medium comprising computer instructions, which, when executed on a destination device or a source device as described above, enable the destination device or the source device to perform a multi-screen cooperation method according to any one of the aspects described above.

[0062] According to an eighth aspect, the present application provides a computer program product, which, when running on a destination device or a source device as described above, enables the destination device or the source device to execute a multi-screen collaboration method according to any one of the aspects described above.

[0063] It should be understood that the destination device, source device, multi-screen collaborative system, computer-readable storage medium, and computer program product provided in the above aspects are all applicable to the above corresponding methods. For the beneficial effects that can be achieved by the destination device, source device, multi-screen collaborative system, computer-readable storage medium, and computer program product, please refer to the beneficial effects achieved by using the above corresponding system or method. Details will not be described again here. [Brief description of the drawings]

[0064] [Figure 1] FIG. 1 is a schematic diagram of the architecture of a multi-screen collaborative system according to an embodiment of the present application. [Diagram 2] FIG. 1 is a schematic diagram of an application scenario of a multi-screen collaboration method according to an embodiment of the present application. [Figure 3A(1)] FIG. 2 is a schematic diagram of an application scenario of the multi-screen collaboration method according to an embodiment of the present application. [Figure 3A(2)] FIG. 2 is a schematic diagram of an application scenario of the multi-screen collaboration method according to an embodiment of the present application. [Figure 3B] FIG. 3 is a schematic diagram of an application scenario of the multi-screen collaboration method according to an embodiment of the present application. [Figure 3C] FIG. 4 is a schematic diagram of an application scenario of the multi-screen collaboration method according to an embodiment of the present application. [Figure 4(a)] FIG. 5 is a schematic diagram of an application scenario of the multi-screen collaboration method according to an embodiment of the present application. [Figure 4(b)] FIG. 5 is a schematic diagram of an application scenario of the multi-screen collaboration method according to an embodiment of the present application. [Diagram 5] FIG. 1 is a schematic diagram of a structure of an electronic device according to an embodiment of the present application. [Figure 6] FIG. 2 is a schematic diagram of an architecture of an operating system in an electronic device according to an embodiment of the present application. [Figure 7]4 is a schematic flowchart of processing display data in a multi-screen cooperation method according to an embodiment of the present application. [Figure 8] 1 is a schematic flowchart of a multi-screen collaboration method according to an embodiment of the present application; [Figure 9(a)] FIG. 6 is a schematic diagram of an application scenario of the multi-screen collaboration method according to an embodiment of the present application. [Figure 9(b)] FIG. 6 is a schematic diagram of an application scenario of the multi-screen collaboration method according to an embodiment of the present application. [Figure 10] FIG. 7 is a schematic diagram of an application scenario of the multi-screen collaboration method according to an embodiment of the present application. [Figure 11(a)] FIG. 8 is a schematic diagram of an application scenario of the multi-screen collaboration method according to an embodiment of the present application. [Figure 11(b)] FIG. 8 is a schematic diagram of an application scenario of the multi-screen collaboration method according to an embodiment of the present application. [Figure 12A] FIG. 9 is a schematic diagram of an application scenario of the multi-screen collaboration method according to an embodiment of the present application. [Figure 12B] FIG. 9 is a schematic diagram of an application scenario of the multi-screen collaboration method according to an embodiment of the present application. [Figure 13] 1 is a schematic diagram 10 of an application scenario of a multi-screen collaboration method according to an embodiment of the present application. [Figure 14] FIG. 11 is a schematic diagram of an application scenario of the multi-screen collaboration method according to an embodiment of the present application. [Figure 15A] FIG. 12 is a schematic diagram of an application scenario of the multi-screen collaboration method according to an embodiment of the present application. [Figure 15B] FIG. 12 is a schematic diagram of an application scenario of the multi-screen collaboration method according to an embodiment of the present application. [Figure 16] FIG. 13 is a schematic diagram of an application scenario of the multi-screen collaboration method according to an embodiment of the present application. [Figure 17A] FIG. 14 is a schematic diagram of an application scenario of the multi-screen collaboration method according to an embodiment of the present application. [Figure 17B] FIG. 14 is a schematic diagram of an application scenario of the multi-screen collaboration method according to an embodiment of the present application. [Figure 18] FIG. 15 is a schematic diagram of an application scenario of the multi-screen collaboration method according to an embodiment of the present application. [Figure 19] FIG. 16 is a schematic diagram of an application scenario of the multi-screen collaboration method according to an embodiment of the present application. [Figure 20] FIG. 17 is a schematic diagram of an application scenario of the multi-screen collaboration method according to an embodiment of the present application. [Figure 21] 1 is a schematic diagram of the structure of a notebook computer according to an embodiment of the present application; [Figure 22] FIG. 2 is a schematic diagram of a structure of an electronic device according to an embodiment of the present application. [Figure 23] 3 is a schematic diagram of a structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0065] Hereinafter, implementations of the embodiments will be described in detail with reference to the accompanying drawings.

[0066] The multi-screen collaboration method provided in the embodiment of the present application may be applied to the multi-screen collaboration system 200 shown in Fig. 1. As shown in Fig. 1, the multi-screen collaboration system 200 may include a first electronic device 101 and a second electronic device 102.

[0067] Specifically, the first electronic device 101 and the second electronic device 102 may each be a device having a display function, such as a mobile phone, a tablet computer, a smart TV, a notebook computer, an ultra-mobile personal computer (UMPC), a handheld computer, a netbook, a personal digital assistant (PDA), a wearable electronic device, an in-vehicle device, or a virtual reality device, which is not limited in the embodiment of the present application.

[0068] For example, both the first electronic device 101 and the second electronic device 102 may be mobile phones, or both the first electronic device 101 and the second electronic device 102 may be notebook computers, or the first electronic device 101 may be a mobile phone or a tablet computer and the second electronic device 102 may be a notebook computer or a smart TV.

[0069] In some embodiments, the first electronic device 101 may be used as a source device and the second electronic device 102 may be used as a destination device for the first electronic device 101. The first electronic device 101 may project content (e.g., applications, pictures, videos, audio, or documents) of the first electronic device 101 onto a display of the electronic device 102 for display to implement a cross-screen display function.

[0070] For example, the first electronic device 101 is a mobile phone and the second electronic device 102 is a notebook computer. In some scenarios, a user may use the mobile phone as a source device and project one or more applications of the mobile phone onto the notebook computer (i.e., the destination device) for display.

[0071] For example, as shown in Fig. 2, an electronic label 201 may be disposed on a notebook computer (i.e., the second electronic device 102), and the electronic label may also be called an NFC (near field communication) label. A coil is usually disposed on the electronic label 201, and device information of the notebook computer may be written in advance on the coil of the electronic label 201 before the notebook computer is shipped. For example, the device information may include the name of the notebook computer and a Bluetooth MAC (media access control) address, etc. Alternatively, an NFC chip may be disposed on the notebook computer, and the NFC chip stores the device information of the notebook computer.

[0072] In this case, when a user needs to project an application of the mobile phone onto the notebook computer for display, the user can enable the NFC function of the mobile phone and bring the mobile phone close to or into contact with the electronic label 201 (or NFC chip) of the notebook computer. In this way, when approaching the electronic label 201 (or NFC chip), the mobile phone can read the device information of the notebook computer from the electronic label 201 (or NFC chip) by transmitting a short-range wireless communication signal. Then, the mobile phone can establish a wireless communication connection to the notebook computer based on the device information of the notebook computer. For example, the wireless communication connection can specifically be a Bluetooth connection, a Wi-Fi connection, or a Wi-Fi P2P (peer to peer) connection. This is not limited in the embodiment of the present application.

[0073] Of course, in addition to the method in which the mobile phone establishes a wireless communication connection to the notebook computer by contacting the electronic label 201 of the notebook computer, a person skilled in the art may design another method for establishing a communication connection between the mobile phone and the notebook computer, which is not limited in the embodiment of the present application. For example, a user may connect the mobile phone to the notebook computer using a data cable to establish a communication connection between the mobile phone and the notebook computer. In another example, the mobile phone may obtain the device information of the notebook computer by reading a QR code or a barcode displayed by the notebook computer, and establish a wireless communication connection to the notebook computer.

[0074] As shown in FIG. 3A(1) and FIG. 3A(2), after the mobile phone establishes a communication connection to the notebook computer, the mobile phone may send application information of N (N is an integer greater than 0) applications installed on the mobile phone to the notebook computer. The application information may include an icon and an identifier of the application. The identifier of the application may be an application package name or ID, etc. The icon of the application may also be called an application shortcut. Alternatively, the application information may further include parameters such as the running status of the application (e.g., running in the foreground or background) and the frequency of use. Then, as further shown in FIG. 3A(1) and FIG. 3A(2), the notebook computer may display icons 302 of the N applications on the desktop 301 of the notebook computer based on the received application information. In this way, the application list of the mobile phone is displayed on the notebook computer.

[0075] Alternatively, as shown in FIG. 3B, when displaying icons 302 of N applications of the mobile phone on the desktop 301, the notebook computer may further display icons 303 of one or more applications or files originally installed on the notebook computer on the desktop 301. In short, the application list projected by the mobile phone to the notebook computer for display does not affect the application list originally displayed by the notebook computer. Therefore, on the notebook computer, the user may not only operate the icon 303 to use the related function of the notebook computer, but also operate the icon 302 to use the related function of the mobile phone. This implements the multi-screen cooperation function. When the mobile phone functions as a source device, the application installed on the mobile phone may be called a source device application. When the notebook computer functions as a destination device, the application installed on the notebook computer may be called a destination application.

[0076] Alternatively, as shown in Fig. 3C, the notebook computer may display the icon 302 in the start menu 304 of the notebook computer. After the user opens the start menu 304 of the notebook computer, the notebook computer may simultaneously display the icons 303 of one or more applications or files installed on the notebook computer and the icons 302 of the N applications of the mobile phone in an "All Apps" option 305. In this way, the user may conveniently use the applications provided by the notebook computer and the applications provided by the mobile phone on the notebook computer. This implements the non-discriminatory integration of the functions of multiple devices on the destination device.

[0077] Of course, the notebook computer may alternatively display the icon 302 in a location such as the notebook computer's taskbar, tile, toolbar, or status bar, which is not limited to the embodiment of the present application.

[0078] Alternatively, as shown in FIG. 4(a), after receiving the application information sent by the mobile phone, the notebook computer may display the button 401 on the desktop 301. When the notebook computer detects that the user selects the button 401, as shown in FIG. 4(b), the notebook computer may display a corresponding application list 402 based on the application information of the N applications, and the application list 402 may include icons of the N applications. The specific display form and position of the application list 402 and the button 401 are not limited in the embodiment of the present application. For example, a presentation bar (e.g., a dock bar) may be added to the desktop 301 of the notebook computer to display the button 401, or the button 401 may be displayed on the original task bar, toolbar, or start menu on the desktop 301 of the notebook computer.

[0079] In addition, when the mobile phone sends application information to the notebook computer and the notebook computer displays the icon 302, the mobile phone may continue to run the application originally running on the mobile phone normally, or the mobile phone may be in a screen-off state or a screen-lock state.

[0080] Then, when the user needs to use a related function of application 1 of the mobile phone on the notebook computer, the user may select an icon of application 1 from the N application icons 302 displayed on the notebook computer. Furthermore, the notebook computer may send an identifier of application 1 corresponding to the icon selected by the user to the mobile phone to trigger the mobile phone to execute application 1, and send display data of the application interface of application 1 generated during the execution of application 1 to the notebook computer for display, so that application 1 of the mobile phone (i.e., source device) is displayed on the notebook computer (i.e., destination device) with application granularity in cross-screen mode. This facilitates the user to use the application of the source device on the destination device and implements the multi-screen cooperation function.

[0081] Of course, if the user needs to further open application 2 of the mobile phone on the notebook computer, the user may continue to select the icon of application 2 on the desktop 301 to trigger the mobile phone to continue to display application 2 on the notebook computer in cross-screen mode using the method described above. In this way, the user may open multiple applications of the mobile phone (i.e., the source device) on the notebook computer (i.e., the destination device), thereby implementing the display of multiple applications in cross-screen mode.

[0082] According to the multi-screen cooperation method provided in the present application, the application interface displayed by the source device (e.g., mobile phone) is not projected to the destination device (e.g., notebook computer) for display, but the application information of N applications installed on the source device is sent to the destination device, so that the destination device can know and present the application list of the applications installed on the source device to the user. In this way, all the device functions of the source device can be presented on the destination device in the form of an application list, and the user can open and display one or more applications installed on the source device on the destination device according to the user's request. This realizes the display of one or more applications on different devices in cross-screen mode, improving the user experience.

[0083] In addition, when displaying the application list of the source device, the destination device may still display the applications and files of the destination device. In short, the device functions of both the source device and the destination device may be presented on the destination device, and the user may conveniently use the applications provided by the destination device and the applications provided by the source device on the destination device, and the application functions provided by the source device may be effectively integrated with the application functions on the destination device without affecting the normal operation of the destination device. This improves the user experience in multi-screen interaction and collaboration.

[0084] In the following embodiments, a specific method for displaying one or more applications of a source device on a destination device in a cross-screen mode will be described with reference to a specific example, and therefore will not be described in detail here.

[0085] For example, a mobile phone serves as the first electronic device 101 of the multi-screen collaborative system 200. Figure 5 is a schematic diagram of the structure of a mobile phone.

[0086] The mobile phone may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) port 130, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a handset 170B, a microphone 170C, a headset jack 170D, and a sensor module 180, among others.

[0087] It will be understood that the structure shown in this embodiment of the invention does not constitute a specific limitation on the mobile phone. In some other embodiments of the present application, the mobile phone may include more or fewer components than those shown in the figures, some components may be combined, some components may be divided, or there may be a different arrangement of components. The components shown in the figures may be implemented using hardware, software, or a combination of software and hardware.

[0088] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be separate components or may be integrated into one or more processors.

[0089] A memory may further be disposed in the processor 110 and configured to store instructions and data. In some embodiments, the memory in the processor 110 is a cache. This memory may store instructions or data that have just been used or are used periodically by the processor 110. When the processor 110 needs to use the instructions or data again, the processor 110 may retrieve the instructions or data directly from the memory. This avoids repeated accesses, reduces the latency of the processor 110, and improves system efficiency.

[0090] In some embodiments, the processor 110 may include one or more interfaces, which may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0091] The wireless communication functions of the mobile phone may be implemented using antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, a modem processor, a baseband processor, and the like.

[0092] Antenna 1 and Antenna 2 are configured to transmit and receive electromagnetic wave signals. Each antenna of the mobile phone may be configured to cover one or more communication bands. Different antennas may be further multiplexed to improve antenna utilization. For example, Antenna 1 may be multiplexed as a diversity antenna in a wireless local area network. In some other embodiments, the antennas may be used in combination with tuning switches.

[0093] The mobile communication module 150 may provide a wireless communication solution applied to mobile phones, including 2G / 3G / 4G / 5G, etc. The mobile communication module 150 may include at least one filter, switch, power amplifier, and low noise amplifier (LNA), etc. The mobile communication module 150 may receive electromagnetic waves via the antenna 1, perform processing such as filtering and amplification on the received electromagnetic waves, and send the processed electromagnetic waves to the modem processor for demodulation. The mobile communication module 150 may further amplify the signal modulated by the modem processor and convert the signal to electromagnetic waves via the antenna 1 for radiation. In some embodiments, at least some functional modules of the mobile communication module 150 may be disposed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be disposed in the same device.

[0094] The wireless communication module 160 may provide wireless communication solutions applied to mobile phones, including wireless local area networks (WLAN) (e.g., wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC) technology, or infrared (IR) technology, etc. The wireless communication module 160 may be one or more components incorporating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signal, and transmits the processed signal to the processor 110. The wireless communication module 160 may further receive a transmission signal from the processor 110, perform frequency modulation and amplification on the signal, and convert the signal into an electromagnetic wave for radiation via the antenna 2.

[0095] In some embodiments, antenna 1 is coupled to mobile communication module 150 and antenna 2 is coupled to wireless communication module 160 in a mobile phone, such that the mobile phone can communicate with a network and other devices using wireless communication technologies that can include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. GNSS may include the global positioning system (GPS), the global navigation satellite system (GLONASS), the Beidou navigation satellite system (BDS), the quasi-zenith satellite system (QZSS), and / or satellite based augmentation systems (SBAS).

[0096] The mobile phone performs display functions using a GPU, a display 194, an application processor, and the like. The GPU is a microprocessor for image processing and is connected to the display 194 and the application processor. The GPU is configured to perform mathematical and geometric calculations and render images. The processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0097] The display 194 is configured to display images, videos, and the like. The display 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled, MicroLed, Micro-oLed, or a quantum dot light emitting diode (QLED), and the like. In some embodiments, the mobile phone can include one or N displays 194, where N is a positive integer greater than 1.

[0098] The mobile phone may implement the photography function using an ISP, a camera 193, a video codec, a GPU, a display 194, an application processor, and the like.

[0099] The ISP is configured to process data fed back by the camera 193. For example, during shooting, the shutter is pressed and light is sent to the camera's light receiving element through the lens. The light signal is converted into an electrical signal, which the camera's light receiving element sends to the ISP for processing to convert the electrical signal into a visible image. The ISP may further perform algorithmic optimization on noise, brightness, and color of the image. The ISP may further optimize parameters such as exposure and color temperature for the shooting scenario. In some embodiments, the ISP may be located in the camera 193.

[0100] The camera 193 is configured to capture still images or videos. An optical image of an object is generated by a lens and projected onto a photoreceptor. The photoreceptor may be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) photoelectric transistor. The photoreceptor converts the optical signal into an electrical signal, which is then sent to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard format, such as RGB or YUV. In some embodiments, the mobile phone may include one or N cameras 193, where N is a positive integer greater than 1.

[0101] The digital signal processor is configured to process digital signals, and may process other digital signals in addition to the digital image signal, for example when the mobile phone selects a frequency, the digital signal processor is configured to perform a Fourier transform or the like on the frequency energy.

[0102] A video codec is configured to compress or decompress digital video. A mobile phone may support one or more video codecs. In this manner, a mobile phone may play or record video in multiple encoding formats, e.g., moving picture experts group (MPEG) 1, MPEG 2, MPEG 3, and MPEG 4.

[0103] The external memory interface 120 may be configured to connect to an external storage card, such as a Micro SD card, to expand the storage capacity of the mobile phone. The external storage card communicates with the processor 110 via the external memory interface 120 to perform data storage functions. The external storage card stores files, such as music and videos.

[0104] The internal memory 121 may be configured to store computer executable program code. The executable program code includes instructions. The processor 110 executes the instructions stored in the internal memory 121 to perform various functional applications and data processing of the mobile phone. The internal memory 121 may include a program storage area and a data storage area. The program storage area may store an operating system and applications required by at least one function (e.g., a sound playback function or an image playback function), etc. The data storage area may store data created during use of the mobile phone (e.g., audio data or an address book), etc. In addition, the internal memory 121 may include a high-speed random access memory and may include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory, or a universal flash storage (UFS).

[0105] The mobile phone may implement audio functions such as music playback and recording functions through an audio module 170, a speaker 170A, a handset 170B, a microphone 170C, a headset jack 170D, and an application processor.

[0106] Audio module 170 is configured to convert digital audio information to an analog audio signal output, and is also configured to convert an analog audio input to a digital audio signal. Audio module 170 may be further configured to encode and decode audio signals. In some embodiments, audio module 170 may be located in processor 110, or some functional modules within audio module 170 are located in processor 110.

[0107] The speaker 170A, also called a "loudspeaker," is configured to convert audio electrical signals into voice signals. A mobile phone may use the speaker 170A to listen to music or answer hands-free calls.

[0108] The handset 170B, also referred to as the "earpiece," is configured to convert electrical audio signals into speech signals. When using a mobile phone to answer a call or receive voice information, the handset 170B may be held close to a person's ear to hear the sound.

[0109] The microphone 170C, also referred to as a "microphone" or "mic", is configured to convert audio signals into electrical signals. When making a call or sending a voice message, a user may speak near the microphone 170C to input an audio signal to the microphone 170C. At least one microphone 170C may be disposed on the mobile phone. In some other embodiments, two microphones 170C may be disposed on the mobile phone to collect audio signals and further perform noise reduction functions. In some other embodiments, three, four, or more microphones 170C may alternatively be disposed on the mobile phone to collect audio signals, reduce noise, identify sound sources, perform directional recording functions, etc.

[0110] Headset jack 170D is configured to connect to a wired headset and may be a USB port 130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0111] The sensor module 180 may include a pressure sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, an optical proximity sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

[0112] Of course, the mobile phone may further include a charging management module, a power management unit, a battery, a button, an indicator, and one or more SIM card interfaces, etc. This is not limited to the embodiments of the present application.

[0113] The software system of the mobile phone may use a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. In the embodiment of this application, the Android system with a layered architecture is used as an example to describe the software structure of the mobile phone.

[0114] FIG. 6 is a block diagram of the software structure of a mobile phone according to one embodiment of the present application.

[0115] In a layered architecture, software is divided into several layers, each of which has a clear role and task. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: application layer, application framework layer, Android runtime and system libraries, and kernel layer.

[0116] The application layer may include a set of application packages.

[0117] As shown in FIG. 6, APPs (applications), such as phone, memo, browser, contacts, camera, gallery, calendar, map, Bluetooth, music, video, and message, may be installed in the application layer.

[0118] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predetermined functions.

[0119] As shown in FIG. 6, the application framework layer may include an NFC service, a window manager, a content provider, a view system, a telephony manager, a resource manager, and a notification manager, etc.

[0120] After enabling the NFC function, the mobile phone may start to execute the NFC service. When the mobile phone approaches or touches an electronic label or an NFC chip of another electronic device (e.g., a notebook computer), the NFC service may call the kernel layer NFC driver to read the information in the electronic label, and the mobile phone establishes a wireless communication connection to the other electronic device based on the information.

[0121] A window manager is configured to manage window programs and may be configured to get the size of the display, determine if there is a status bar, perform screen locking, take screenshots, and the like.

[0122] Content providers are configured to store and retrieve data and enable the data to be accessed by applications. Data may include video, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0123] The view system can be configured to build display interfaces for an application. Each display interface can include one or more controls. In general, controls can include interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.

[0124] The telephone manager is configured to provide communication functions of the mobile phone, such as management of call status (including answering or rejecting, etc.).

[0125] The resource manager provides various resources for an application, such as localized strings, icons, pictures, layout files, and video files.

[0126] The notification manager allows applications to display notification information in the status bar and may be configured to convey notification type messages, and the displayed notification information may disappear automatically after a short period without user interaction. For example, the notification manager is configured to notify download completion and message reminders, etc. The notification manager may alternatively be a notification displayed in the status bar at the top of the system in the form of a graph or scrollbar text, e.g., a notification of an application running in the background, or a notification displayed on the screen in the form of a dialog window. For example, text information may be displayed in the status bar, an alert sound may be played, a vibration may be generated, or an indicator light may flash.

[0127] As shown in FIG. 6, the system library may include multiple functional modules, such as a SurfaceFlinger, Media Libraries, a 3D graphics processing library (e.g., OpenGL ES), and a 2D graphics engine (e.g., SGL).

[0128] SurfaceFlinger is configured to manage the display subsystem and provide integration of 2D and 3D layers for multiple applications. The media library supports playback and recording of multiple commonly used audio and video formats as well as still image files, etc. The media library may support multiple audio and video encoding formats such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG. The 3D graphics processing library is configured to perform 3D graphics drawing, image rendering, composition, layer processing, etc. The 2D graphics engine is a drawing engine for 2D drawing.

[0129] The Android Runtime includes the core libraries and the virtual machine. The Android runtime is responsible for scheduling and managing the Android system.

[0130] The core library contains two parts: the functions that need to be called in java language and the android core library.

[0131] The application layer and the application framework layer run on a virtual machine. The virtual machine executes the java files of the application layer and the application framework layer as binary files. The virtual machine is configured to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0132] The kernel layer is a layer between hardware and software. The kernel layer includes at least a HWcomposer, a camera driver, an audio driver, and a sensor driver, etc. This is not limited to the embodiment of the present application.

[0133] In an embodiment of the present application, as shown in FIG. 7, after the application layer starts executing an application (e.g., music APP 701), the mobile phone may allocate a corresponding buffer queue, such as buffer queue 1, to the music APP 701. The buffer queue 1 is used to buffer drawing instructions, such as openGL instructions, generated during the execution of the music APP 701. For example, during the execution of the music APP 701, the window manager and view system of the application framework layer may be called to generate drawing instructions for all controls and layers of the application interface of the music APP 701, and the application interface of the music APP 701 may be drawn by executing these drawing instructions. In this case, after these drawing instructions are generated in real time for the music APP 701, these drawing instructions may be pushed to the corresponding buffer queue 1 for storage. Similarly, the mobile phone may execute multiple applications (e.g., application 2 and application 3 shown in FIG. 7) in the application layer, and the drawing instructions generated for each application during the execution of each application may be stored in the corresponding buffer queue.

[0134] As further shown in FIG. 7, the drawing instructions in the buffer queue may be sent to SurfaceFlinger in the system library, and SurfaceFlinger may draw corresponding layers and controls by executing the drawing instructions in each buffer queue. In addition, SurfaceFlinger may compose these layers and controls to obtain each frame of frame data (which may also be referred to as graphic data) of an application interface. For example, the frame data may be in RGB format (R represents red, G represents green, and B represents blue). For example, SurfaceFlinger may call hardware such as a GPU to perform the composition for obtaining the frame data of the application interface, or may use a software module having graphic composition capabilities to perform the composition for obtaining the frame data of the application interface. SurfaceFlinger may store each frame of the generated frame data in a frame buffer for output or display.

[0135] As further shown in Figure 7, the frame data in the frame buffer may be sent to HWcomposer in the kernel layer. A display controller in HWcomposer may convert the frame data of the application interface into pixel data that may be directly displayed on the display. The pixel data may also be in RGB format. If the application to which the pixel data belongs is an application running in the foreground, HWcomposer may send the generated pixel data to the display of the mobile phone, so that the application interface of the application may be displayed on the display based on the pixel data.

[0136] In some embodiments, as further shown in FIG. 7, the mobile phone may further include a video encoder (Media Encoder). The video encoder may obtain the frame data output by SurfaceFlinger or the pixel data output by HWcomposer. The video encoder may be configured to encode and compress the frame data output by SurfaceFlinger or the pixel data output by HWcomposer. In subsequent embodiments, the data obtained by encoding and compressing by the video encoder may be referred to as encoded data, and the amount of encoded data may be significantly reduced compared to the amount of unencoded and uncompressed frame data and pixel data. For example, the encoded data may be in a YUV format (Y represents luminance, and U and V represent chrominance).

[0137] In the embodiment of the present application, the mobile phone (i.e., source device) is still used as an example to project the N applications installed on the notebook computer (i.e., destination device). When detecting that the user opens one of the N applications (e.g., music APP701) on the notebook computer, the notebook computer can send the identifier of music APP701 to the mobile phone to indicate that the mobile phone should project the application interface of music APP701 onto the notebook computer for display.

[0138] In this case, the mobile phone may start to execute the music APP 701 at the application layer. In the process of executing the music APP 701, as shown in FIG. 7, a set of drawing instructions of the application interface of the music APP 701 generated during the execution of the music APP 701 may be stored in a buffer queue 1 corresponding to the music APP 701. Then, the mobile phone may send the set of drawing instructions stored in the buffer queue 1 in real time to the notebook computer. Then, the notebook computer may draw the application interface of the music APP 701 on the desktop of the notebook computer according to the drawing instructions. In this way, the mobile phone projects the music APP 701 to the notebook computer for display.

[0139] Alternatively, when the mobile phone projects the music APP 701 onto the notebook computer, as further shown in FIG. 7, the mobile phone may send frame data of the application interface of the music APP 701, which is generated by SurfaceFlinger during the execution of the music APP 701, to the notebook computer. Then, the notebook computer may draw the application interface of the music APP 701 on the desktop of the notebook computer based on the frame data. In this way, the mobile phone projects the music APP 701 onto the notebook computer for display.

[0140] Alternatively, when the mobile phone projects the music APP 701 to the notebook computer, as further shown in Figure 7, the mobile phone may send pixel data of the application interface of the music APP 701, which is generated in HWcomposer during the execution of the music APP 701, to the notebook computer. Then, the notebook computer may draw the application interface of the music APP 701 on the desktop of the notebook computer based on the pixel data. In this way, the mobile phone projects the music APP 701 to the notebook computer for display.

[0141] Alternatively, when the mobile phone projects the music APP 701 to the notebook computer, as further shown in Fig. 7, the mobile phone may send the encoded data output by the video encoder during the execution of the music APP 701 to the notebook computer. Then, after decoding the encoded data, the notebook computer may draw the application interface of the music APP 701 on the desktop of the notebook computer according to the decoded data. In this way, the mobile phone projects the music APP 701 to the notebook computer for display.

[0142] Similarly, when a user opens multiple applications installed on the mobile phone on the notebook computer, the mobile phone may execute each application using the above-described method and send display data of the application interface of each application (the display data may be the above-described drawing instructions, frame data, pixel data, or encoded data) generated during the execution of each application to the notebook computer for display, thereby implementing the display of multiple applications in cross-screen mode.

[0143] In addition, when the mobile phone acts as a source device and projects an application to a destination device (e.g., a notebook computer), the mobile phone may execute the application in the foreground of the mobile phone to synchronously display the application interface of the application on the destination device. Alternatively, the mobile phone may execute the application in the background of the mobile phone. In this case, the mobile phone may display the application interface of another application in the foreground or in a screen-off state. In other words, when the mobile phone projects the application interface of the application on the destination device, the mobile phone does not need to synchronously display the application interface of the application on the mobile phone, and the user may use the mobile phone normally to perform the related functions provided by the mobile phone.

[0144] The multi-screen collaboration method provided in the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0145] For example, a user projects an application on a mobile phone (i.e., a source device) to a notebook computer (i.e., a destination device) for display, as shown in Figure 8. A multi-screen collaboration method provided in an embodiment of the present application may include the following steps.

[0146] S801: A mobile phone establishes a communication connection to a notebook computer.

[0147] The communication connection may be a wireless communication connection, such as a Wi-Fi connection, a Bluetooth connection, a P2P connection, an NFC connection, or a cellular network-based data connection. Alternatively, the communication connection may be a wired connection. For example, a mobile phone and a notebook computer may be connected using a data cable or a dock device. This is not limited in this embodiment of the present application.

[0148] For example, an NFC chip or electronic label may be placed in the notebook computer, and the device information of the notebook computer, such as the MAC address of the notebook computer, is stored in the NFC chip or electronic label. Furthermore, after enabling the NFC function, the mobile phone may obtain the device information of the notebook computer by approaching or touching the NFC chip or electronic label of the notebook computer. Then, the mobile phone may establish a communication connection, such as a Bluetooth connection, a Wi-Fi connection, or a P2P connection, to the notebook computer based on the device information.

[0149] In another example, a pre-set QR code may be displayed on the notebook computer, and the QR code includes the device information of the notebook computer. In this case, the mobile phone may alternatively obtain the device information of the notebook computer by reading the QR code of the notebook computer. Then, the mobile phone may establish a communication connection, such as a Bluetooth connection, a Wi-Fi connection, or a P2P connection, to the notebook computer based on the device information.

[0150] Of course, the user may alternatively use a mobile phone to create a Wi-Fi hotspot, and connect the notebook computer to the Wi-Fi hotspot created by the mobile phone to establish a communication connection between the mobile phone and the notebook computer. The specific method of establishing a communication connection between the mobile phone and the notebook computer is not limited in this embodiment of the present application.

[0151] S802: The mobile phone sends application information of the N applications installed on the mobile phone to the notebook computer.

[0152] The N applications may be all or some of the applications installed on the mobile phone. For example, the N applications may be all applications installed on the mobile phone, the N applications may be all third-party applications installed on the mobile phone by the user, or the N applications may be applications that are used by the user more frequently than a preset value in the past week. This is not limited in this embodiment of the present application.

[0153] For example, after the mobile phone detects that a communication connection to another device (e.g., a notebook computer) has been established, the mobile phone may be triggered to obtain identifiers (e.g., package names of the applications) and icons of each of the N currently installed applications, and the mobile phone may then send these identifiers and icons of the applications as application information to the notebook computer via the communication connection.

[0154] Alternatively, the mobile phone may periodically maintain an application list of installed applications. For example, when a user installs a new application on the mobile phone or uninstalls an installed application from the mobile phone, the mobile phone may update the identifiers and icons of the applications in the application list. Then, after the mobile phone establishes a communication connection to the notebook computer, the mobile phone may send the identifiers and icons of each application in the updated application list to the notebook computer as application information via the communication connection.

[0155] In addition to the identifiers and icons of each of the N applications, the application information may further include parameters related to the application, such as the running status or usage frequency of the application. For example, when the WeChat APP of the mobile phone is running on the mobile phone, the running status of the WeChat APP may be 01, and when the music APP of the mobile phone is not running on the mobile phone, the running status of the music APP may be 00. In another example, when the launch frequency or running time of the WeChat APP of the mobile phone in the last week is greater than a preset value, the usage frequency of the WeChat APP may be set high, and when the launch frequency or running time of the fitness APP of the mobile phone in the last week is less than a preset value, the usage frequency of the fitness APP may be set low.

[0156] It should be noted that the mobile phone may send application information of the N applications to the notebook computer after first detecting that the mobile phone has established a communication connection to the notebook computer, and the notebook computer may store the application information. Thereafter, after the mobile phone re-establishes a communication connection to the notebook computer, the mobile phone may no longer send application information to the notebook computer. Alternatively, after the mobile phone re-establishes a communication connection to the notebook computer, in addition to the application information of the N applications, the mobile phone may send application information of one or more applications that need to be displayed on the notebook computer in the cross-screen mode to the notebook computer. For example, in addition to the application information of the N applications, the mobile phone may send application information of a newly installed application to the notebook computer.

[0157] S803: The notebook computer displays an application list of the N applications based on the application information.

[0158] In step S803, after receiving the application information of the N applications sent by the mobile phone, the notebook computer may display an application list including the N applications installed on the mobile phone on the display of the notebook computer based on the application information, thereby implementing projection of the application list installed on the mobile phone for display on the notebook computer.

[0159] For example, as shown in FIG. 9(a), after receiving the application information sent by the mobile phone, the notebook computer may display a preset presentation bar, for example, a dock bar 902, on the desktop 901 of the notebook computer. A button 903 used to open an application list of the mobile phone is arranged on the dock bar 902. When detecting that the user has selected the button 903, for example, when detecting that the user has moved the cursor by operating the mouse of the notebook computer to click the button 903, the notebook computer may display icons of each application carried in the application information of the N applications on the desktop 901 to form an application list 904 of the N applications, as shown in FIG. 9(b). In this way, the notebook computer may present various device functions of the mobile phone (i.e., the source device) to the user using the application list 904, and the user may select a specific application that needs to be projected to the notebook computer from the application list 904. When displaying the application list 904, the notebook computer may further display files or applications of the notebook computer as usual. Specifically, the user may also open files on the notebook computer or applications of the notebook computer. In other words, when the mobile phone projects the application list 904 onto the notebook computer, the normal operation of the notebook computer is not affected.

[0160] In some embodiments, the application information sent by the mobile phone further includes the running status of the application. For example, the applications running on the mobile phone include WeChat APP and music APP. In this case, as shown in Fig. 10, the notebook computer may further display the icon 1001 of the WeChat APP and the icon 1002 of the music APP running on the mobile phone in the dock bar 902. In other words, the notebook computer (i.e., the destination device) may implement the projection of the icon of the application running on the mobile phone (i.e., the source device) to the interface displayed by the notebook computer, so that the user can quickly open the application running on the mobile phone on the notebook computer.

[0161] In some embodiments, the application information sent by the mobile phone further includes the frequency of use of the application. For example, the applications whose frequency of use is greater than a preset value include a calculator APP and a weather APP. In this case, as shown in FIG. 11(a), the notebook computer may further display icons of these frequently used applications, i.e., an icon 1101 of the calculator APP and an icon 1102 of the weather APP, on the dock bar 902. In this way, the user may conveniently and quickly find and open the frequently used applications on the mobile phone on the notebook computer.

[0162] Alternatively, the user may manually add one or more application icons of the mobile phone that need to be displayed to the dock bar 902. As shown in Fig. 11(b), after the user selects the button 903 to open the application list 904 on the notebook computer, the user may use the cursor to drag one or more application icons in the application list 904 to the dock bar 902, and the mobile phone may add the corresponding application icons to the dock bar 902 for display in response to the user's drag operation. Of course, an add button 1103 may be set on the dock bar 902 of the notebook computer, and the user may manually add application icons to the dock bar 902 using the add button 1103.

[0163] In addition, the dock bar 902 may be displayed in a floating manner on the desktop 901 of the notebook computer, and the user may manually adjust the position or size of the dock bar 902 on the desktop 901. Alternatively, if the notebook computer does not detect an operation input by the user on the dock bar 902 within a certain time, the notebook computer may hide the dock bar 902 on the desktop 901.

[0164] It is understood that those skilled in the art may set other buttons on the dock bar 902 based on practical experience or practical application scenarios. For example, a setting button may be displayed on the dock bar 902 to set parameters of cross-screen display from mobile phone to notebook computer. In another example, an end button may be further displayed on the dock bar 902 to end a current cross-screen display task from mobile phone to notebook computer. This is not limited in this embodiment of the present application.

[0165] Of course, in addition to displaying the N applications projected by the mobile phone on the dock bar 902 on the notebook computer, the notebook computer may alternatively display an application list containing the N applications in another location, such as the desktop or task bar, which is not limited in this embodiment of the present application.

[0166] Based on steps S801 to S803, the mobile phone may project an application list including N installed applications onto the notebook computer for display, so that the user may know that the application in the application list may be operated according to the user's request in the cross-screen display process to implement the corresponding function. In the process of performing steps S801 to S803, the mobile phone may continue to display the application originally running in the foreground, or may be in a screen-off state or a screen-lock state. In other words, the process of the mobile phone projecting the application list onto the notebook computer does not affect the original operating status of the mobile phone, and the user may still operate various applications and functions of the mobile phone normally.

[0167] S804: In response to a user's operation of selecting a first application from the application list, the notebook computer sends a first launch message to the mobile phone, where the first launch message includes an identifier of the first application.

[0168] For example, after the notebook computer displays the application list 904 on the desktop 901, the user may manipulate the cursor on the notebook computer to select a specific application that currently needs to be projected onto the notebook computer for display. For example, as further shown in FIG. 9(b), if the user wants to project the gallery APP of the mobile phone onto the notebook computer for display, the user may manipulate the cursor on the notebook computer to click the icon 905 of the gallery APP. After detecting that the user clicks the icon 905 of the gallery APP in the application list 904, the notebook computer may include an identifier (e.g., package name) of the gallery APP in the first launch message and send the first launch message to the mobile phone to indicate to the mobile phone to project the gallery APP onto the notebook computer for display.

[0169] Of course, if the icon of the application that the user intends to open is located on the dock bar 902, the user may also manipulate the cursor to click on the corresponding application icon on the dock bar 902. Similarly, in response to the user clicking on the application icon, the notebook computer may send a corresponding launch message to the mobile phone, where the launch message includes an identifier of the particular application clicked by the user.

[0170] S805: In response to the first launch message, the mobile phone sends display data of the application interface of the first application, which is generated during the execution of the first application, to the notebook computer.

[0171] After the mobile phone receives the first launch message sent by the notebook computer, the mobile phone may inquire whether applications running on the mobile phone include a gallery APP based on an identifier of the first application (e.g., a gallery APP) carried in the first launch message.

[0172] When the mobile phone is running the gallery APP, as shown in FIG. 12A, the drawing command of the application interface of the gallery APP generated during the execution of the gallery APP is stored in the corresponding buffer queue 1. For example, the drawing command may be specifically a GPU rendering command or an openGL command, etc. The application interface corresponding to the gallery APP may be drawn by executing the drawing command. In this case, in step S805, the mobile phone may send the drawing command stored in the buffer queue 1 in real time to the notebook computer as the display data of the gallery APP, so that the notebook computer may draw the application interface of the gallery APP according to the drawing command.

[0173] Alternatively, as further shown in FIG. 12A, during the execution of the gallery APP, SurfaceFlinger may be invoked to execute drawing instructions in the buffer queue 1 to obtain each frame of frame data of the application interface of the gallery APP by configuration. For example, the frame data may be display data in RGB format in particular. The frame data is typically stored in a frame buffer of SurfaceFlinger. In this case, in step S805, the mobile phone may send the frame data of the gallery APP in the frame buffer to the notebook computer as display data of the gallery APP, so that the notebook computer may draw the application interface of the gallery APP based on the frame data.

[0174] 12A, the frame data output by SurfaceFlinger during the execution of the gallery APP is finally sent to HWcomposer, and HWcomposer converts the frame data into pixel data that can be displayed on a display, for example, pixel data in RGB format. In this case, in step S805, the mobile phone can send the pixel data of the gallery APP output in real time by HWcomposer to the notebook computer as display data of the gallery APP, so that the notebook computer can draw the application interface of the gallery APP based on the pixel data.

[0175] Alternatively, as further shown in FIG. 12A, the frame data output by SurfaceFlinger (or the pixel data output by HWcomposer) during the execution of the gallery APP may be encoded and compressed by a video encoder to generate encoded data corresponding to the application interface of the gallery APP, for example, display data in YUV format. In this case, in step S805, the mobile phone may send the encoded data of the gallery APP output by the video encoder in real time to the notebook computer as the display data of the gallery APP, so that the notebook computer may draw the application interface of the gallery APP after decoding the encoded data, and the amount of data transmission during data transmission may be reduced.

[0176] Correspondingly, if the mobile phone has not started to execute the gallery APP when it receives the first start message, the mobile phone may first create an application process of the gallery APP to start the gallery APP. For example, the mobile phone may start the gallery APP in the foreground, or may start the gallery APP in the background. In addition, as further shown in FIG. 12A, during the execution of the gallery APP after the start, the mobile phone may send the drawing command, frame data, encoding data, or pixel data generated for the gallery APP to the notebook computer in real time as the display data of the gallery APP, so that the notebook computer may draw the application interface of the gallery APP according to the display data.

[0177] S806: The notebook computer displays a first application window of the first application based on the display data.

[0178] It is still used as an example that the mobile phone projects the gallery APP to the notebook computer. If the display data sent by the mobile phone is a drawing command generated during the execution of the gallery APP, after receiving the drawing command, the notebook computer may use a display module such as the GPU or the graphic card of the notebook computer to execute the drawing command. In this case, as shown in FIG. 12B, the notebook computer may draw the application interface 1301 of the gallery APP (which may also be referred to as the application window of the gallery APP) on the desktop 901 by executing the drawing command. Since the application interface of the gallery APP running on the mobile phone may be updated in real time, the mobile phone may send new drawing commands to the notebook computer in real time, and the notebook computer may continuously draw the application interface displayed for the gallery APP on the desktop 901 in real time.

[0179] Alternatively, if the display data sent by the mobile phone is frame data generated during the execution of the gallery APP, after receiving the frame data, the notebook computer may also use the display module of the notebook computer to draw the application interface of the gallery APP based on the frame data to obtain the application interface 1301 shown in FIG. 12B.

[0180] Alternatively, if the display data sent by the mobile phone is pixel data generated during the execution of the gallery APP, after receiving the pixel data, the notebook computer may also use the display module of the notebook computer to draw the application interface of the gallery APP based on the pixel data to obtain the application interface 1301 shown in FIG. 12B.

[0181] Alternatively, if the display data sent by the mobile phone is encoded data generated during the execution of the gallery APP, after receiving the encoded data, the notebook computer may first decode the received encoded data using a video decoder (Media Decoder), and then the notebook computer may use the display module of the notebook computer to render the application interface of the gallery APP based on the decoded data to obtain the application interface 1301 shown in FIG. 12B.

[0182] In some other embodiments, after the notebook computer draws the application interface 1301 of the gallery APP based on the display data sent by the mobile phone, as shown in FIG. 13, the notebook computer may add a title bar 1401 to the application interface 1301 to display the application interface of the gallery APP in a window format. The title bar 1401 and the application interface 1301 together form the application window of the gallery APP. The title bar 1401 may include an application name 1402, a minimize button 1403, a maximize button 1404, and a close button 1405, etc. This is not limited in this embodiment of the present application. Then, the application projected on the notebook computer may be displayed in the display manner of the operating system of the notebook computer. This improves the user experience.

[0183] In short, based on steps S801 to S806, the mobile phone projects the application list of the mobile phone to the notebook computer, so that the user can project the required destination application of the mobile phone to the notebook computer for display, thereby implementing the display of applications on different devices in cross-screen mode, and improving the user experience in the cross-screen display scenario.

[0184] S807: In response to a first operation input by a user in the first application window, the notebook computer sends a corresponding first control message to the mobile phone.

[0185] It is still used as an example that the mobile phone projects the gallery APP to the notebook computer. After the notebook computer displays the application window of the gallery APP on the desktop 901, the user can operate the application window displayed on the notebook computer to control the gallery APP running on the mobile phone. This allows various functions of the related application of the source device to be implemented on the notebook computer (i.e., the destination device).

[0186] For example, when drawing the application interface 1301 of the camera APP based on the display data, the notebook computer may establish a correspondence between the application interface 1301 displayed on the notebook computer and the corresponding application interface of the camera APP running on the mobile phone. For example, if the size of the application interface of the camera APP sent by the mobile phone is 300*400, the notebook computer may draw the application interface 1301 based on the size of 300*400. Alternatively, the notebook computer may draw the application interface 1301 based on a specific scale ratio. For example, the notebook computer may draw the application interface 1301 based on the size of 600*800.

[0187] As shown in Fig. 14, the notebook computer may use the upper left vertex of the application interface 1301 as the origin O to establish a Cartesian coordinate system. Since the application interface 1301 displayed on the notebook computer corresponds to the application interface of the camera APP running on the mobile phone, the notebook computer may convert any coordinate in the application interface 1301 to the corresponding coordinate in the application interface of the camera APP running on the mobile phone based on the correspondence. When the notebook computer detects that the user inputs a click operation, a double click operation, a drag operation, a mouse scroll operation, or a keyboard input operation, etc. in the application interface 1301, the notebook computer may generate a corresponding control message and send the control message to the mobile phone.

[0188] The control message may include an identifier of the application to which the application interface 1301 belongs, such as the package name of the gallery APP, so that the mobile phone knows that the current control message is the control message of the gallery APP. In addition, the control message may further include the operation type of the operation input by the user, such as a click operation, a double click operation, a drag operation, a mouse scroll operation, or a keyboard input operation.

[0189] When the operation type is indicated as a click operation or a double-click operation, the notebook computer may include in the corresponding control message coordinates corresponding to the click operation in the application window on the notebook computer. When the operation type is indicated as a drag operation, the notebook computer may include in the corresponding control message start and end coordinates corresponding to the drag operation in the application window on the notebook computer. When the size of the application interface 1301 of the gallery APP displayed on the notebook computer is the same as the size of the application interface of the gallery APP running on the mobile phone, the aforementioned coordinates may be coordinates corresponding to the current operation in the Cartesian coordinate system. Alternatively, when the size of the application interface 1301 of the gallery APP displayed on the notebook computer is different from the size of the application interface of the gallery APP running on the mobile phone, the notebook computer may convert the aforementioned coordinates corresponding to the current operation in the Cartesian coordinate system into corresponding coordinates in the application interface of the gallery APP running on the mobile phone, and include the converted coordinates in the control message.

[0190] When the operation type is indicated as a mouse scroll operation, the notebook computer may include a specific mouse scroll amount in the corresponding control message, and when the operation type is indicated as a keyboard input operation, the notebook computer may include a specific character string input by the user in the corresponding control message.

[0191] Alternatively, when the operation type is indicated as a drag operation, the notebook computer may include the coordinates of the cursor detected in real time in the application window on the notebook computer with a certain detection frequency in a series of control messages, and send the series of control messages to the mobile phone. In this way, the mobile phone may respond to the user's operation in the application interface 1301 in real time based on the coordinates in the series of control messages to perform corresponding functions by controlling the gallery APP.

[0192] Similarly, when the operation type is indicated as a mouse scroll operation, the notebook computer may send a control message to the mobile phone every time the notebook computer detects that the mouse scroll wheel has scrolled one unit, so as to report the current mouse scroll operation to the mobile phone in real time. Similarly, the mobile phone may respond to the user's mouse scroll operation in real time based on the series of received control messages, so as to perform corresponding operations on the gallery APP. In addition, to avoid the impact on the processing performance of the mobile phone caused by sending a large amount of control messages to the mobile phone in a short time, the notebook computer may set an upper limit on the amount of control messages sent to the mobile phone per unit time (e.g., 10 messages per second). When the number of sent control messages exceeds the upper limit, the notebook computer may stop generating the corresponding control messages or discard redundant control messages.

[0193] For example, as further shown in FIG. 14, when detecting that a user controls the cursor to click a point A of a photo 1501 in the application interface 1301, the notebook computer may convert the coordinate (x, y) of the point A into a corresponding coordinate A' in the application interface of the camera APP running on the mobile phone. Then, the notebook computer may include the operation type "click operation" of the current operation, the coordinate of the point A', and the package name of the gallery APP in a first control message, and send the first control message to the mobile phone. After that, the mobile phone may report a corresponding input event to the gallery APP based on the first control message, so as to trigger the execution of a corresponding action command of the gallery APP.

[0194] The user may not only input corresponding operations in the application interface 1301, but also operate the title bar 1401 in the application interface 1301. For example, when detecting that the user clicks the minimize button 1403 of the title bar 1401, the notebook computer may display the minimized window of the gallery APP in the dock bar 902 or display the task bar on the desktop 901, and hide the application interface 1301. When detecting that the user clicks the maximize button 1404 of the title bar 1401, the notebook computer may display the application interface 1301 in tile mode on the desktop 901. When detecting that the user clicks the close button 1405 of the title bar 1401, the notebook computer may send a close message to the mobile phone to trigger the mobile phone to terminate the running gallery APP. In this case, the mobile phone no longer sends the display data generated for the gallery APP to the notebook computer, and the notebook computer no longer continues to display the title bar 1401 and the application interface 1301.

[0195] Additionally, or alternatively, the user may manually adjust the position and size of the application interface 1301 of the gallery APP on the desktop 901. For example, if the user enlarges the application interface 1301 on the desktop 901 by two times, the notebook computer may update a specific position of the application interface 1301 on the desktop 901. Then, when the notebook computer subsequently detects that the user inputs a corresponding operation in the application interface 1301, the notebook computer may reduce the value of the coordinate corresponding to the operation in the Cartesian coordinate system by half, and restore the operation to the operation in the application interface 1301 before being enlarged. Then, the notebook computer may include the coordinate value obtained by the half reduction in a control message, and send the control message to the mobile phone.

[0196] S808: In response to the first control message, the mobile phone executes, for the first application, an operation instruction corresponding to the first control message.

[0197] A user clicks on point A of the photo 1501 in the application interface 1301 is still used as an example. The first control message sent by the notebook computer may include an identifier of the gallery APP, an operation type of the current operation "click operation", and a click position corresponding to point A' on the display of the mobile phone. Then, the mobile phone may convert the first control message into a touch event of the user at point A' and report the touch event to the gallery APP. For the gallery APP, it can be understood that after the gallery APP receives the touch event, the user clicks on point A' of the running gallery APP, which indicates that the user wants to open the photo 1501 corresponding to point A' (i.e., the corresponding point A on the notebook computer). In this case, for the gallery APP, a corresponding function may be called to execute an operation instruction for opening the photo 1501.

[0198] When the action command to open the photo 1501 is executed for the gallery APP, new display data (e.g., the drawing command, frame data, or pixel data mentioned above) is generated. In this case, the mobile phone may continue to send the new display data to the notebook computer using the method described in steps S805 and S806 to trigger the notebook computer to update the application interface of the gallery APP on the desktop 901. In this way, the user may manipulate the content projected and displayed on the notebook computer (i.e., the destination device). This allows various functions of the associated application of the mobile phone (i.e., the source device) to be implemented on the destination device.

[0199] In some embodiments, after receiving the first control message sent by the notebook computer, the mobile phone may trigger the execution of the corresponding action instruction of the gallery APP in the foreground, or may trigger the execution of the corresponding action instruction of the gallery APP in the background. As shown in FIG. 15A and FIG. 15B, the mobile phone is running the desktop 1601, and when it receives the first control message, the mobile phone is running the gallery APP in the background and displaying the desktop 1601 in the foreground. In this case, after converting the first control message into the touch event of the user at point A, the mobile phone may report the touch event to the gallery APP running in the background. Then, in response to the touch event in the background, an action instruction to open the photo 1501 may be executed for the gallery APP. In this case, as further shown in FIG. 15A and FIG. 15B, while displaying the desktop 1601 in the foreground, the mobile phone may send new display data generated for the gallery APP in the background to the notebook computer, and trigger the notebook computer to display the application interface 1602 of the gallery APP after the photo 1501 is opened. In short, when a user operates the mobile phone APP on the notebook computer to perform related functions, the original operating status of the mobile phone will not be affected, and the user can still operate various applications and functions of the mobile phone normally.

[0200] In some embodiments, after the mobile phone projects the application list of the mobile phone to the notebook computer, the user may open multiple applications in the application list on the notebook computer. This implements displaying multiple applications in cross-screen mode. For example, after the notebook computer displays a first application window of a first application (i.e., step S806), the user may further continue to open a second application in the application list on the notebook computer. In this case, the mobile phone and the notebook computer may further continue to execute the following steps S809 to S813.

[0201] S809: In response to the user's operation of selecting a second application from the application list, the notebook computer sends a second launch message to the mobile phone, where the second launch message includes an identifier of the second application.

[0202] Similar to step S804, after the user opens the gallery APP of the mobile phone from the application list 904 on the notebook computer, the user can further continue to open the application list 904 and select the application that needs to be opened from the application list 904. For example, as shown in Fig. 16, if the user wants to further project the music APP of the mobile phone to the notebook computer for display, the user can operate the cursor on the notebook computer to click the icon 1701 of the music APP. After detecting that the user clicks the icon 1701 of the music APP, the notebook computer can include the identifier of the music APP in the second start-up message and send the second start-up message to the mobile phone to indicate that the mobile phone should project the music APP to the notebook computer for display.

[0203] S810: In response to the second launch message, the mobile phone sends display data of the application interface of the second application, which is generated during the execution of the second application, to the notebook computer.

[0204] S811: The notebook computer displays a second application window of a second application based on the display data.

[0205] Similar to steps S805 and S806, in steps S810 and S811, after the mobile phone receives the second startup message of the music APP sent by the notebook computer, if the mobile phone is not running the music APP, the mobile phone may first startup the music APP.

[0206] After the mobile phone launches the music APP, in the process of executing the music APP (this process is similar to the process of executing the gallery APP by the mobile phone shown in FIG. 12A), as shown in FIG. 17A, the mobile phone may send the drawing command in the corresponding buffer queue 2 to the notebook computer as the display data of the music APP, and the drawing command is generated during the execution of the music APP. Alternatively, the mobile phone may send the frame data of the music APP in the frame buffer of SurfaceFlinger to the notebook computer as the display data of the music APP, and the frame data is obtained during the execution of the music APP. Alternatively, the mobile phone may send the pixel data of the music APP to the notebook computer as the display data of the music APP, and the pixel data is output by HWcomposer in real time during the execution of the music APP. Alternatively, the mobile phone may send the encoded data of the music APP to the notebook computer as the display data of the music APP, and the encoded data is output by a video encoder in real time during the execution of the music APP.

[0207] After the notebook computer receives the display data of the music APP sent by the mobile phone, the notebook computer may use the display module of the notebook computer to draw the application interface 1801 of the music APP based on the display data, as shown in Fig. 17B. Of course, the notebook computer may further add a title bar 1802 to the application interface 1801 to display the application interface of the music APP in the form of a window. In this case, the notebook computer may simultaneously display the application window of the music APP and the application window of the gallery APP on the desktop 901. This implements the display of multiple applications in cross-screen mode.

[0208] S812: In response to a second operation input by the user in the second application window, the notebook computer sends a corresponding second control message to the mobile phone.

[0209] S813: In response to the second control message, the mobile phone executes, for the second application, an operation instruction corresponding to the second control message.

[0210] Similar to step S807 and S808, in step S812 and S813, after the notebook computer displays the application window of the music APP on the desktop 901, the user can operate the application window of the music APP on the notebook computer to control the music APP running on the mobile phone, so that various functions of the related application of the source device are implemented on the notebook computer (i.e., the destination device).

[0211] For example, when detecting that a user inputs a second operation in the application window of the music APP, the notebook computer may include the identifier of the music APP, the operation type of the second operation, and the coordinates corresponding to the second operation in the application window in the second control message, and send the second control message to the mobile phone. Then, the mobile phone may convert the second control message into a corresponding touch event and report the touch event to the music APP, so that the corresponding function of the music APP may be invoked to execute the corresponding operation command. In this way, the user may operate the content projected and displayed on the notebook computer (i.e., the destination device). This allows various functions of the related application of the mobile phone (i.e., the source device) to be implemented on the destination device.

[0212] In some embodiments of the present application, during operation, the mobile phone may support a multi-display mode, in which the mobile phone may output different display data to one or more external display devices for multi-screen display.

[0213] For example, after enabling the multi-display mode, the mobile phone may start running a display manager service (DisplayManagerService). The DisplayManagerService supports mirroring by multiple displays in multiple display types, including the mobile phone's local display type, the HDMI display type, and a display type supporting the WIFI Display protocol, to control the logical display executed by the currently connected external display device. In other words, the DisplayManagerService may create and manage one or more logical displays (which may hereinafter be referred to as display modules). The display module may store display data on the mobile phone or the external display device.

[0214] For example, the DisplayManagerService may create a display module 1 having the same size as the size of the mobile phone display, and send display data that needs to be displayed on the mobile phone display to the display module 1. Correspondingly, the mobile phone display may obtain the corresponding display data from the display module 1 to display the corresponding interface.

[0215] In another example, in addition to the display module 1, the DisplayManagerService may further create a display module 2 having a certain size, and send display data of one or more applications that need to be displayed on the external display device 1 to the display module 2. Correspondingly, the external display device 1 may obtain the corresponding display data from the display module 2 to display the corresponding interface.

[0216] In another example, in addition to the display module 1, the DisplayManagerService may further create a display module 2 and a display module 3. The display module 2 and the display module 3 both correspond to an external display device 1. The display module 2 may be configured to store display data of an application 1, and the display module 3 may be configured to store display data of an application 2. Correspondingly, the external display device 1 may obtain the display data of the application 1 from the display module 2 for display, and the external display device 1 may obtain the display data of the application 2 from the display module 2 for display.

[0217] As shown in FIG. 18, when the mobile phone executes each application, the HWcomposer of the mobile phone may convert the frame data output by SurfaceFlinger into pixel data. Then, the HWcomposer may store the pixel data of each application in the corresponding display module as display data. In addition to the HWcomposer, other hardware (e.g., GPU) or software modules having graphic processing capabilities in the mobile phone may also convert the frame data output by SurfaceFlinger into pixel data and store the pixel data in the corresponding display module. In addition, the display data stored in each display module may be encoded data obtained by encoding by a video encoder (not shown), or may be data that is not encoded by a video encoder. This is not limited in this embodiment of the present application.

[0218] 18, the display module 1 of the mobile phone may be configured to store display data of one or more applications that need to be displayed by the mobile phone. In other words, when a user opens one or more applications (e.g., application 1) on the mobile phone, the mobile phone may store display data (e.g., pixel data) generated during the execution of application 1 in the display module 1. Correspondingly, the display of the mobile phone may retrieve the pixel data of application 1 from the display module 1 to display the corresponding application interface.

[0219] In addition, each display module of the mobile phone other than the display module 1 (e.g., the display module 2 and the display module 3) is configured to store each application that needs to be displayed by the external display device 1. In other words, the mobile phone may separately store the display data of one or more applications that need to be displayed by the external display device 1 in the corresponding display module. For example, when the application 2 needs to be displayed on the external display device 1, the mobile phone may store the display data generated for the application 2 in the display module 2. When the application 3 also needs to be displayed on the external display device 1, the mobile phone may store the display data generated for the application 3 in the display module 3. The external display device 1 may separately obtain the display data of the APPs from the display module 2 and the display module 3 to display the corresponding application interfaces on the external display device 1.

[0220] In some embodiments of the present application, the notebook computer (i.e., destination device) may be used as an external display device (e.g., external display device 1) of the mobile phone. After the mobile phone receives one or more application launch messages sent by the notebook computer, the mobile phone may enter a multi-display mode and execute the DisplayManagerService. In this case, as further shown in FIG. 18 , the mobile phone may store in the display module 1 the relevant display data of the application 1 that needs to be output on the display of the mobile phone. In addition, if the mobile phone receives the launch message of the application 2 sent by the notebook computer, the mobile phone may start executing the application 2 and store in the display module 2 the display data generated during the execution of the application 2.

[0221] In the following, as shown in FIG. 21, the notebook computer may include a communication module 2101, a video decoding module 2102, an acquisition module 2103, and a display module 2104. The communication module 2101 may obtain display data generated in real time for the application 2 from the display module 2 of the mobile phone. If the display data is data encoded by a video encoder, the video decoding module 2102 may decode the display data. When storing the display data of the application 2 in the display module 2, the mobile phone may store the display data of the application 2 in the display module 2 in full screen mode, or may store the display data of the application 2 at a specific position of the display module 2 based on a specific window size. The mobile phone may further send position information of the display data of the application 2 to the communication module 2101 of the notebook computer. For example, the display module 2 corresponds to a display having a size of 400*600, the size of the application window corresponding to the display data generated during the execution of the application 2 by the mobile phone is 100*200, and the coordinates of the top left vertex of the application window for the display having a size of 400*600 are (0,0). In this case, when the mobile phone executes the application 2, the mobile phone may transmit the size (100*200) of the application window corresponding to the application 2 and the coordinates (0,0) of the top left vertex of the application window to the notebook computer as the position information of the display data.

[0222] In this case, the capture module 2103 of the notebook computer captures display data corresponding to the application window of application 2 from the display module 2 based on the position information, and then the display module 2104 draws the application window of application 2 on the notebook computer based on the display data, thereby implementing the display of applications on different devices in cross-screen mode.

[0223] In addition, when the notebook computer displays the application interface of application 2, the user may further continue to open another application (e.g., application 3) of the mobile phone on the notebook computer. If the mobile phone receives the start message of application 3 sent by the notebook computer, the mobile phone may start the execution of application 3, and store the display data generated during the execution of application 3 in the display module 3. Similarly, the communication module 2101 of the notebook computer may obtain the display data generated in real time for application 3 from the display module 3, the video decoding module 2102 and the capture module 2103 respectively decode and capture the display data in the display module 3, and finally, the display module 2104 draws the application interface of application 3 on the notebook computer based on the captured display data. In this case, the notebook computer may simultaneously display the application interface of application 1 and the application interface of application 2. This implements the display of multiple applications in cross-screen mode.

[0224] In some other embodiments, as shown in Fig. 19, each display module of the mobile phone other than display module 1 (e.g., display module 2 and display module 3) may be configured to correspond to one external display device. In this case, display module 2 is used as an example. Display module 2 may store display data of one or more applications that need to be displayed on the external display device (e.g., external display device 1).

[0225] Further, as shown in FIG. 19, it is used as an example that a notebook computer (i.e., a destination device) is used as an external display device 1 of a mobile phone. The mobile phone may store relevant display data of application 1 that needs to be output to the display of the mobile phone in the display module 1. In addition, when the mobile phone receives a start message of application 2 sent by the notebook computer, the mobile phone may start to execute application 2, and store the display data generated during the execution of application 2 in the display module 2. Then, the notebook computer may obtain the display data generated in real time for application 2 from the display module 2, and draw the application interface of application 2 on the notebook computer based on the display data. This implements the display of applications on different devices in cross-screen mode.

[0226] The difference lies in the following: when the notebook computer displays the application interface of application 2, if the mobile phone receives a launch message of application 3 sent by the notebook computer, the mobile phone will start to run application 3, and may also store and display data generated during the running of application 3 in the display module 2. In other words, the display data of all applications that need to be displayed on the notebook computer will be stored in the corresponding display module 2.

[0227] When the display module 2 includes both the display data of the application interface of the application 2 and the display data of the application interface of the application 3, the application interface of the application 2 and the application interface of the application 3 may overlap on the display module 2. Therefore, the mobile phone may further transmit to the notebook computer the position information of the application interface of the application 3 currently operated by the user relative to the display module 2. In this way, after the notebook computer obtains the display data in the display module 2, the capture module 2103 of the notebook computer may capture display data corresponding to the application interface of the application 3 based on the position information. Then, the notebook computer may draw the application interface of the application 3 based on the captured display data. This implements the display of multiple applications in cross-screen mode.

[0228] Alternatively, different display templates may be pre-configured in the display module 2 for different applications that need to be displayed by the notebook computer. For example, when the notebook computer needs to display only one application of a mobile phone, all display spaces in the display module 2 may be configured to store the display data of the application. When the notebook computer needs to simultaneously display application 1 and application 2 of a mobile phone, the display space of the display module 2 may be divided into space 1 and space 2 having equal sizes. Space 1 is used to store the display data of application 1, and space 2 is used to store the display data of application 2.

[0229] As shown in FIG. 20, when the notebook computer displays the application interface of application 2, if the mobile phone receives a launch message of application 3 sent by the notebook computer, the mobile phone may store the display data generated during the execution of application 2 in space 1 of the display module 2, and store the display data generated during the execution of application 3 in space 2 of the display module 2. In this way, no occlusion occurs in the display data of different applications in the display module 2. Then, the notebook computer may obtain the display data generated in real time for application 2 from space 1 of the display module 2, and draw the application interface of application 2 on the notebook computer based on the display data, and the notebook computer may obtain the display data generated in real time for application 3 from space 2 of the display module 2, and draw the application interface of application 3 on the notebook computer based on the display data. This implements the display of multiple applications in cross-screen mode.

[0230] Of course, after the notebook computer displays the application interface of Application 1 or the application interface of Application 2, the user may further input related operations, such as dragging a window, maximizing, minimizing, or clicking a control in the application interface, according to the related descriptions in the preceding embodiments, to implement the functions of controlling the related applications of the mobile phone on the notebook computer (i.e., the destination device), which is not limited in this embodiment of the present application.

[0231] As shown in Fig. 22, an embodiment of the present application discloses a source device, for example the aforementioned mobile phone. The source device may specifically include a touch screen 2201 including a touch sensor 2206 and a display 2207, one or more processors 2202, a memory 2203, a communication module 2208, one or more applications (not shown), and one or more computer programs 2204. These components may be connected using one or more communication buses 2205. The one or more computer programs 2204 are stored in the memory 2203 and configured to be executed by the one or more processors 2202. The one or more computer programs 2204 include instructions. The instructions may be used to execute the relevant steps executed by the source device in the aforementioned embodiment.

[0232] Optionally, the source device may further include another component, such as an NFC chip or an electronic label, which is not limited in this embodiment of the present application.

[0233] As shown in Fig. 23, an embodiment of the present application discloses a destination device, for example the aforementioned notebook computer. The destination device may specifically include a display 2301, one or more processors 2302, a memory 2303, a communication module 2306, one or more applications (not shown), and one or more computer programs 2304. These components may be connected using one or more communication buses 2305. The one or more computer programs 2304 are stored in the memory 2303 and configured to be executed by the one or more processors 2302. The one or more computer programs 2304 include instructions. The instructions may be used to execute the relevant steps executed by the destination device in the aforementioned embodiment.

[0234] Optionally, the destination device may further include another component, such as a keyboard, a touch sensor, an NFC chip, or an electronic label, which is not limited in this embodiment of the present application.

[0235] The above description of the embodiments allows those skilled in the art to understand that the above division into functional modules is used as an example for illustration for the sake of convenience. In practical application, the above functions can be allocated and implemented into different modules as required, that is, the internal structure of the device is divided into different functional modules to implement all or part of the above-described functions. For the detailed operation processes of the above-described system, device and unit, please refer to the corresponding processes in the above-described method embodiments, and the details will not be described again here.

[0236] The functional units in the embodiments of the present application may be integrated into one processing unit, each unit may exist physically alone, or two or more units may be integrated into one unit. The integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0237] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, the integrated unit may be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the embodiments of the present application essentially, or the part contributing to the prior art, or all or part of the technical solution may be implemented in the form of a software product. The computer software product is stored in a storage medium and includes some instructions for instructing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in the embodiments of the present application. The aforementioned storage medium includes any medium that can store program code, such as a flash memory, a removable hard disk, a read-only memory, a random access memory, a magnetic disk, or an optical disk.

[0238] The above description is only a specific implementation of the embodiments of the present application, and is not intended to limit the protection scope of the embodiments of the present application. Any variation or replacement within the technical scope disclosed in the embodiments of the present application shall be within the protection scope of the embodiments of the present application. Therefore, the protection scope of the embodiments of the present application shall be subject to the protection scope of the claims. [Explanation of symbols]

[0239] 101 First Electronic Device 102 Second Electronic Device 110 Processor 120 External Memory Interface 121 Internal Memory 130 USB ports 150 Mobile communication module 160 Wireless Communication Module 170 Audio Module 170A Speaker 170B Handset 170C Microphone 170D Headset Jack 180 Sensor Module 193 Camera 194 Display 200 Multi-screen Collaboration System 201 Electronic Label 301 Desktop 302 Icon 303 Icon 304 Start Menu 305 "All Apps" option 401 Button 402 Application List 701 Music APP 901 Desktop 902 Dock Bar 903 Button 904 Application List 905 Icon 1001 Icon 1002 Icons 1101 Icon 1102 Icon 1103 Add button 1301 Application Interface 1401 Title Bar 1402 Application Name 1403 Minimize button 1404 Maximize button 1405 Close button 1501 Photos 1601 Desktop 1602 Application Interface 1701 Icon 1801 Application Interface 1802 Title Bar 2101 Communication Module 2102 Video Decoding Module 2103 Acquisition Module 2104 Display module 2201 Touch Screen 2202 Processor 2203 Memory 2204 Computer Programs 2205 Communication Bus 2206 Touch Sensor 2207 Display 2208 Communication Module 2301 Display 2302 Processor 2303 Memory 2304 Computer Programs 2305 Communication Bus 2306 Communication Module

Claims

1. A multi-screen collaboration method, the method comprising: transmitting, by a source device, application information of at least one source device application installed on the source device to a destination device, the application information including an identifier of the at least one source device application, the source device running a first operating system, and the destination device running a second operating system; displaying, by the destination device, an icon of the at least one source device application on a first interface based on the application information, and displaying, by the destination device, an icon of at least one destination device application installed on the destination device on the first interface; sending, by the destination device, a first launch message to the source device in response to an operation performed by a user on an icon of a first application on the destination device, the first launch message including an identifier of the first application; launching, by the source device, in response to the first launch message, the first application based on the identifier of the first application and transmitting first display data generated during execution of the first application to the destination device; acquiring, by the destination device, the first display data generated during execution of the first application on the source device, the at least one source device application including the first application, and displaying, by the destination device, an application window of the first application on the first interface based on the first display data; after the step of displaying, by the destination device, an application window of the first application on the first interface based on the first display data; sending, by the destination device, in response to a first operation input by the user in the application window of the first application, a first control message corresponding to the first operation to the source device, the first control message including the identifier of the first application; executing, by the source device, in response to the first control message, an operation instruction corresponding to the first control message for the first application; launching, by the destination device, a second application and displaying an application window of the second application in the first interface in response to an operation performed by the user on an icon of the second application on the destination device, wherein the at least one destination device application includes the second application; A multi-screen collaboration method, including:

2. the first display data being drawing instructions generated for a first application interface during execution of the first application; the first display data is frame data generated by a SurfaceFlinger of the source device for the first application interface based on the drawing instructions during execution of the first application; or The first display data is pixel data generated by a Hwcomposer of the source device for the first application interface based on the frame data during execution of the first application. The method of claim 1.

3. The method comprises: encoding, by the source device, the first display data using a video encoder.

3. The method of claim 2, further comprising:

4. 4. The method of claim 2 or 3, wherein the application window of the first application comprises the first application interface of the first application and a first title bar, the first title bar comprising a minimize button, a maximize button, and a close button of the first application.

5. 5. The method of claim 4, wherein the application window of the second application comprises a second application interface and a second title bar of the second application, the second title bar comprising a minimize button, a maximize button, and a close button of the second application.

6. When the destination device displays the application window of the first application, the method further comprises: displaying, by the source device, the first application interface of the first application; or displaying, by the source device, an application interface of a third application, the third application being an application of the source device different from the first application; 6. The method of claim 2, further comprising:

7. the source device comprises a first display module and a second display module, the first display module configured to provide display data to the source device, the second display module configured to provide display data to the destination device, the first display data being stored in the second display module; The step of acquiring, by the destination device, first display data generated during execution of the first application on the source device, comprises: obtaining, by the destination device, from the second display module, the first display data generated during execution of the first application.

7. The method of any one of claims 1 to 6, comprising:

8. After the step of displaying, by the destination device, an application window of the first application on the first interface based on the first display data, the method further comprises: receiving, by the source device, a second wake-up message sent by the destination device, the second wake-up message including an identifier of a fourth application, the at least one source device application including the fourth application, the fourth application being different from the first application; launching, by the source device, the fourth application in response to the second launch message, and further storing, in the second display module, second display data generated during execution of the fourth application; obtaining, by the destination device, the second display data from the second display module, and displaying an application window of the fourth application on the first interface based on the second display data; 8. The method of claim 7, further comprising:

9. After the step of receiving, by the source device, a second wake-up message sent by the destination device, the method further comprises: transmitting, by the source device, location information of the second display data to the destination device; and the step of obtaining, by the destination device, the second display data from the second display module further comprises: obtaining, by the destination device, the second display data from the second display module based on the location information.

9. The method of claim 8, comprising:

10. the second display module comprises a first display space and a second display space, the first display space is used to store the first display data, and the second display space is used to store the second display data; The step of obtaining, by the destination device, the second display data from the second display module, further comprises: obtaining, by the destination device, the second display data from the second display space; 9. The method of claim 8, comprising:

11. the source device further comprises a third display module, the third display module also configured to provide display data to the destination device; After the step of displaying, by the destination device, an application window of the first application on the first interface based on the first display data, the method further comprises: receiving, by the source device, a second wake-up message sent by the destination device, the second wake-up message including an identifier of a fourth application, the at least one source device application including the fourth application, the fourth application being different from the first application; launching, by the source device, the fourth application in response to the second launch message, and storing, in the third display module, second display data generated during execution of the fourth application; obtaining, by the destination device, the second display data from the third display module, and displaying an application window of the fourth application on the first interface based on the second display data; 8. The method of claim 7, comprising:

12. After the step of displaying, by the destination device, an application window of the first application on the first interface based on the first display data, the method further comprises: storing, by the source device, in response to an operation of opening the first application by the user on the source device, the first display data generated during execution of the first application in the first display module, and subsequently providing display data to the source device using the first display module; or reading and displaying, by the source device, the first display data stored in the second display module in response to an operation of opening the first application by the user on the source device; 12. The method of claim 7, further comprising:

13. After the step of transmitting, by the source device, to a destination device, application information of at least one source device application installed on the source device, the method further comprises: displaying, by the destination device, a first button on the first interface. and wherein the step of displaying, by the destination device, an icon of the at least one source device application on a first interface based on the application information further comprises: displaying, by the destination device, the icon of the at least one source device application on the first interface based on the application information when detecting that the user has selected the first button.

13. The method of any one of claims 1 to 12, comprising:

14. The first button is disposed on a pre-defined presentation bar, and the application information further comprises a usage frequency or a running status of the at least one source device application, and the method further comprises: displaying, by the destination device, icons of one or more of the at least one source device application, the application having a usage frequency greater than a preset value, on the presentation bar based on the application information; or displaying, by the destination device, icons of one or more of the at least one source device applications running on the source device on the presentation bar based on the application information.

14. The method of claim 13, further comprising:

15. The first button is located on a pre-defined presentation bar, and the method further comprises: adding, by the destination device, an icon of the new application to the presentation bar in response to an operation by the user to add the new application.

14. The method of claim 13, further comprising:

16. The method of claim 1 , wherein the icon of the at least one source device application is placed on a desktop, a start menu, a taskbar, or a tile of the destination device.

17. The destination device includes an electronic label or a near field communication (NFC) chip, and before the step of transmitting, by the source device, to the destination device, application information of at least one source device application installed on the source device, the method further includes: Establishing, by the source device, a communication connection to the destination device when the source device approaches or touches the electronic label or the NFC chip of the destination device.

17. The method of any one of claims 1 to 16, further comprising:

18. An electronic device, comprising: A display and one or more processors; Memory, Communication module and 18. An electronic device comprising: a memory for storing one or more computer programs, the one or more computer programs including instructions that, when executed by the electronic device, enable the electronic device to perform the multi-screen collaboration method according to any one of claims 1 to 17.

19. A computer-readable storage medium, the computer-readable storage medium storing a computer program, the computer program including instructions, the instructions being executed on a terminal device to enable the terminal device to perform the multi-screen collaboration method according to any one of claims 1 to 17.

Images