Cross-platform display method for applications, readable medium and electronic device

Abstract

The application relates to the computer technical field, and discloses a cross-platform display method of an application, a readable medium and an electronic device. The method comprises the following steps: a first system platform acquires a first size of a first window of a first application running on a second system platform, and creates a third window corresponding to the first application in the first system platform, wherein the third window is used for displaying a display interface of the first application running on the second system platform; the first system platform acquires a second size of a second window of a second application running on the second system platform, and creates a fourth window corresponding to the second application in the first system platform, wherein the fourth window is used for displaying a display interface of the second application running on the second system platform. Therefore, the display interfaces of multiple second applications running on the second system platform can be displayed on the first system platform.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a cross-platform display method of application, readable medium and electronic device. BACKGROUND

[0002] With the development of communication technology, cross-platform running application becomes more and more common; for example, a user can run an application on a first system platform on a second system platform. The application on the first system platform can be run on the second system platform. The application on the first system platform can be run on the second system platform.

[0003] For example, the application on the first system platform can be run on the second system platform by installing an Android virtual machine or using a screen projection technology. The application on the first system platform can be run on the second system platform. However, it is difficult to display the content of multiple Android applications on the second system platform at the same time by installing an Android virtual machine or using a screen projection technology. And the user cannot control the size or position of the window of the Android application displayed on the second system platform. SUMMARY

[0004] The present application provides a cross-platform display method of application, readable medium and electronic device.

[0005] In a first aspect, the present application provides a cross-platform display method of application, comprising: a first system platform acquiring a first size of a first window of a first application running on a second system platform, and creating a third window corresponding to the first application in the first system platform, wherein the third window is used to display a display interface of the first application running on the second system platform; the first system platform acquiring a second size of a second window of a second application running on the second system platform, and creating a fourth window corresponding to the second application in the first system platform, wherein the fourth window is used to display a display interface of the second application running on the second system platform.

[0006] For example, the first system platform can be a second operating system running on a second electronic device, for example, a Windows system, also known as a desktop system; the second system platform can be a second operating system running on a first electronic device, for example, an Android system, also known as an Android system; the second system platform can also be a second operating system running on a virtual machine of a third electronic device, for example, a Windows system, also known as a desktop system. The second electronic device and the third electronic device can be a computer, and the first electronic device can be a mobile phone.

[0007] ​​​For example, the first window of the first application can be an Android application window of the "calculator" application below. The third window of the first application can be a desktop application window of the "calculator" application below. The first size of the first window of the first application can be parameters of the Android application window of the "calculator" application below, wherein the parameters of the Android application window include but are not limited to an identifier of the Android application window, a size of the Android application window, and a window region position coordinate, etc.

[0008] For example, the second window of the second application can be an Android application window of the "settings" application below. The fourth window of the second application can be a desktop application window of the "settings" application below. The second size of the second window of the second application can be parameters of the Android application window of the "calculator" application below.

[0009] For example, the phone can send the parameters of the Android application window of the "calculator" to the computer, wherein the parameters of the Android application window include a size of the Android application window and a window region position coordinate; the computer creates a desktop application window based on the received parameters of the Android application window of the "calculator" application and sends a window handle of the desktop application window to the phone, wherein the window handle of the desktop application window is used to uniquely identify the desktop application window; the phone generates drawing information using the received window handle and sends the drawing information to the computer; the computer can call a graphics library of the computer to perform image rendering on the desktop application window based on the received drawing information; as shown in the following Figure 1 B As shown, the desktop application window of the "calculator" application displays the content of the Android application "calculator". It can be understood that, according to the above method, as shown in the following Figure 1 C As shown, the computer can start multiple Android applications, create multiple Android application windows and simultaneously display the content of the multiple Android applications.

[0010] In a possible implementation of the first aspect, the method further includes: the first system platform obtaining the first size or the second size from the second system platform in a case where the first system platform detects that a user opens the first application or the second application in the first system platform.

[0011] For example, the computer can synchronously display the Android application of the phone. For example, after the phone establishes a distributed connection with the computer, the computer can synchronously display the Android application by creating a shortcut of the Android application on the desktop of the computer. The computer can obtain parameters of the started Android application by detecting that a user opens an operation of the shortcut of the Android application created on the desktop of the computer.

[0012] For example, the computer can also create a display button of the Android application on the display interface. The computer can also obtain parameters of the Android application by detecting that a user opens an operation of the display button of the Android application created on the display window of the computer.

[0013] For example, the mobile phone can start the Android application based on the parameters of the Android application. In the process of starting the Android application, the mobile phone creates an Android application window of the application, and the mobile phone sends the parameters of the Android application window to the computer.

[0014] In a possible implementation of the first aspect, the method further includes that the second system platform does not display the first window and the second window.

[0015] For example, the Android application can be started on the virtual screen of the mobile phone or on the display screen of the mobile phone. When the Android application is started on the virtual screen of the mobile phone, the computer can start the Android application, and the display content of the started Android application is only displayed on the computer. For example, as shown in Figure 1 B the content of the Android application "Calculator" is only displayed on the computer. Thus, the distributed application display of cross-platform applications is realized without affecting the normal use of the mobile phone.

[0016] It can be understood that, as shown in the following Figure 10 the display screen of the mobile phone can not display the Android application window of the "Video Player" application. The desktop application window executes the drawing information of the "Video Player" application to realize video image rendering and consume the GPU resources of the computer. The Android application window does not execute the instruction and thus does not consume the GPU resources of the mobile phone. Thus, the display interface of the application of the mobile phone can be displayed on the computer, and the GPU resources of the mobile phone are not consumed.

[0017] In a possible implementation of the first aspect, the method further includes that the first system platform creates a third window corresponding to the first application or creates a fourth window corresponding to the second application in the first system platform, and includes that the first system platform sets a first identifier for the first application or sets a second identifier for the second application. The first identifier is used to identify that the third window is used to display the display interface of the first application, and the second identifier is used to identify that the fourth window is used to display the display interface of the second application.

[0018] For example, the first identifier can be the window handle of the desktop application window of the "Calculator" application, and the second identifier can be the window handle of the desktop application window of the "Settings" application. The window handle of the desktop application window of the "Calculator" application is used to uniquely identify the desktop application window of the "Calculator" application, and the window handle of the desktop application window of the "Settings" application is used to uniquely identify the desktop application window of the "Settings" application.

[0019] It can be understood that the computer creates the desktop application window based on the parameters of the Android application window, and assigns a window handle to the created desktop application window. The assigned window handle is used to uniquely identify the created desktop application window. For example, the computer creates 10 desktop application windows, and assigns a window handle to each desktop application window; when the computer operates the desktop application window, the window handle of the desktop application window needs to be obtained before the desktop application window can be operated.

[0020] In a possible implementation of the first aspect, the method further includes: the first system platform sending the first identifier or the second identifier to the second system platform.

[0021] For example, after the mobile phone receives the window handle of the desktop application window sent by the computer, it continues to create an Android application window and associates the Android application window with the window handle of the desktop application window. In the process of the mobile phone continuing to create the Android application window, the Android application window is associated with the window handle of the desktop application window, and the mobile phone can use the drawing information generated by the window handle, so that the generated drawing information can be directly executed on the computer.

[0022] In a possible implementation of the first aspect, the method further includes: the first system platform displaying the first application running on the second system platform in the third window by: the first system platform receiving first drawing information from the second system platform, where the first drawing information includes the first identifier; and the first system platform displaying, in the third window, a first display interface of the first application drawn according to the first drawing information, according to the first identifier included in the first drawing information.

[0023] For example, the first drawing information can be the drawing information of the "calculator" application, and the computer can call the image library of the computer, such as EGL, OpenGL, etc., according to the received drawing information of the "calculator" application. The first display interface can be the display interface obtained by the computer calling the image library for image rendering on the desktop application window of the "calculator" for displaying the display interface of the "calculator" application.

[0024] It can be understood that the mobile phone only needs to send the parameters of the Android application window and the drawing information to the computer, and the computer creates the desktop application window based on the received parameters of the Android application window and the drawing information and calls the graphics library to draw the window display content, thereby realizing the starting and displaying of the content of the Android application on the computer. Compared with the existing cross-platform application distributed display of the screen projection technology, the application distributed display of the present application can realize the distributed display without transmitting a large amount of video image data between devices.

[0025] In a possible implementation of the first aspect, the method further includes: the first system platform displaying the second application running on the second system platform in the fourth window by: the first system platform receiving second drawing information from the second system platform, where the second drawing information includes the second identifier; and the second system platform displaying, in the fourth window, a second display interface of the second application according to the second drawing information, according to the second identifier included in the second drawing information.

[0026] For example, the second drawing information can be drawing information of a "Settings" application, and the computer can call an image library, such as an EGL, an OpenGL, or the like, according to the received drawing information of the "Settings" application. The second display interface can be a display interface of the "Calculator" application obtained by the computer calling the image library to perform image rendering on the desktop application window of the "Settings".

[0027] In a possible implementation of the first aspect, the method further includes: the first system platform detecting a first operation of a user on a third window on the first system platform; the first system platform generating operation information and sending the operation information to the second system platform, where the operation information includes the first identifier; the first system platform receiving third drawing information from the second system platform, where the third drawing information includes the first identifier, and the third drawing information is generated by the second system platform by performing an operation corresponding to the operation information; and the first system platform displaying, in the third window, a third display interface based on the third drawing information, where the third display interface is a display interface of the first display interface after the first operation.

[0028] For example, as shown in the following Figure 1 B , the display content in the desktop application window can be a number or an operator symbol in the desktop application window of the "Calculator" application; the desktop application window size can be a position of the desktop application window of the "Calculator" application in the display interface, a display area size of the desktop application window of the "Calculator" application, or the like; and the desktop application window state of the "Calculator" application can be open, close, minimize, or the like.

[0029] As shown in the following Figure 1 B , the computer creates the desktop application window based on the received parameters of the Android application window and the drawing information, and calls the graphics library to draw the display content of the window, thereby realizing the display of the desktop application window of the "Calculator" application in Figure 1 B . Therefore, the absolute coordinates of the window area of the Android application window and the absolute coordinates of the window area of the desktop application window have a mapping relationship, that is, the mobile phone can redraw the content displayed by the Android application window based on the operation position coordinates in the operation information, to generate operation drawing information.

[0030] For example, the third drawing information can be operation drawing information of a "calculator" application as follows. For example, a user performs a "zoom-in" operation on a desktop application window of the "calculator" application of the mobile phone, the desktop system of the computer detects the "zoom-in" operation on the desktop application window, the desktop system of the computer generates window region position coordinates and operation position coordinates of the "zoom-in" operation, and the desktop system sends control operation information of the "zoom-in" to the mobile phone agent process of the mobile phone; the mobile phone agent process of the mobile phone adjusts the position coordinates of the Android application window by calling the resize Task interface of the ActivityTaskManagerService based on the received window region position coordinates and operation position coordinates. The Android system of the mobile phone adjusts the region coordinates of the Android application window of the "calculator" application and generates operation drawing information. The operation drawing information is used to draw the content displayed by the desktop application window of the "calculator" application after zoom-in.

[0031] For example, the third drawing information can also be operation drawing information of a "settings" application as follows. For example, the relative position coordinates of the region position coordinates of the desktop application window of the "settings" application and the region position coordinates of the Android application window have a mapping relationship. As shown in the following Figure 13A For example, the third drawing information can also be operation drawing information of a "settings" application as follows. For example, the relative position coordinates of the region position coordinates of the desktop application window of the "settings" application and the region position coordinates of the Android application window have a mapping relationship. As shown in the following

[0032] In a possible implementation of the first aspect, the method further includes that the first operation includes at least one of the following operations: an operation of changing the size of the first display interface; and an operation on a control in the first display interface.

[0033] For example, the first operation of the user on the third window of the first system platform can be a control operation of the user on display content (for example, a control of a display interface), a window state, and a window size, and the like in the desktop application window of the "calculator" application through a mouse, a keyboard, and touch, and the like. The operation information can include the size of the desktop application window, window region position coordinates, and operation position coordinates, and the like.

[0034] In a possible implementation of the first aspect, the method further includes that the first system platform and the second system platform have different operating systems.

[0035] In a possible implementation of the first aspect, the method further includes that the first system platform and the second system platform run on different electronic devices.

[0036] In a possible implementation of the first aspect, the method further includes that the first system platform is an operating system running on a virtual machine of the electronic device, and the second system platform is an operating system of the electronic device.

[0037] In a second aspect, the application provides a cross-platform display method of an application, which includes the following steps: a first application runs on a second system platform; a first system platform obtains a first size of a first window of the first application from the second system platform; the first system platform creates a third window corresponding to the first application and a first identifier in the first system platform, where the first identifier is used to identify that the third window is used to display a display interface of the first application; after the second system platform receives the first identifier from the first system platform, the second system platform generates first drawing information, where the first drawing information includes the first identifier; the first system platform receives the first drawing information from the second system platform, and displays a first display interface of the first application drawn according to the first drawing information in the third window according to the first identifier included in the first drawing information.

[0038] In a possible implementation of the second aspect, the method further includes that, when the first system platform detects that a user opens the first application in the first system platform, the first system platform sends first application information to the second system platform; the second system platform runs the first application according to the first application information, and creates a first window of the first application in the second system platform; and the second system platform sends the first size of the first window of the first application to the first system platform.

[0039] In a possible implementation of the second aspect, the method further includes that the second system platform does not display the first window.

[0040] In a possible implementation of the second aspect, the method further includes that the first system platform detects a first operation of a user on the third window on the first system platform; the first system platform generates operation information, and sends the operation information to the second system platform, where the operation information includes the first identifier; the second system platform generates third drawing information by performing an operation corresponding to the operation information, and sends the third drawing information to the first system platform; and the first system platform displays a third display interface in the third window based on the third drawing information, where the third display interface is a display interface of the first display interface changed after the first operation.

[0041] In a possible implementation of the second aspect, the method further includes that the first operation includes at least one of the following operations: an operation of changing a size of the first display interface; an operation on a control in the first display interface.

[0042] In a possible implementation of the second aspect, the method further includes that the first system platform and the second system platform have different operating systems, and the first system platform and the second system include at least one of the following: the first system platform and the second system platform run on different electronic devices; the first system platform is an operating system running on a virtual machine of the electronic device, and the second system platform is an operating system of the electronic device.

[0043] In a third aspect, the embodiments of the present application provide a cross-platform display method of an application, and the method includes: a first application runs on a second system platform; the second system platform sends a first size of a first window of the first application to a first system platform; after the second system platform receives a first identifier from the first system platform, the second system platform generates first drawing information, where the first identifier is used to identify that a third window in the first system platform is used to display a display interface of the first application, the first drawing information includes the first identifier, and the first drawing information can be used by the first system platform to draw the first display interface of the first application displayed in the third window.

[0044] In a fourth aspect, the embodiments of the present application provide a readable medium, and the readable medium stores instructions, where the instructions are executed on an electronic device to make the electronic device perform the cross-platform display method of the application in any one of the first aspect to the third aspect and the various possible implementations of the first aspect to the third aspect.

[0045] In a fifth aspect, the embodiments of the present application provide an electronic device, and the electronic device includes:

[0046] a memory, configured to store instructions executed by one or more processors of the electronic device, and

[0047] a processor, which is one of the processors of the electronic device, and is configured to execute the cross-platform display method of the application in any one of the first aspect to the third aspect and the various possible implementations of the first aspect to the third aspect. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 A According to some embodiments of the present application, a schematic diagram of establishing a distributed display connection between a mobile phone and a computer is shown.

[0049] Figure 1 B According to some embodiments of the present application, a schematic diagram of distributed display of a "Settings" application is shown.

[0050] Figure 1 CAccording to some embodiments of the present application, a distributed display schematic diagram of multiple applications is shown;

[0051] Figure 1 D According to some embodiments of the present application, a display window "zoom-in" schematic diagram of an application is shown;

[0052] Figure 2 According to some embodiments of the present application, a scene diagram of a cross-platform display system of an application is shown;

[0053] Figure 3 According to some embodiments of the present application, a cross-platform application distributed display interaction diagram of a cross-platform display system of an application is shown;

[0054] Figure 4 According to some embodiments of the present application, another cross-platform application distributed display interaction diagram of a cross-platform display system of an application is shown;

[0055] Figure 5 According to some embodiments of the present application, a schematic diagram of starting a listening function of a mobile phone agent process is shown;

[0056] Figure 6 According to some embodiments of the present application, a schematic diagram of starting a listening function of a desktop agent process is shown;

[0057] Figure 7 According to some embodiments of the present application, a schematic diagram of successfully establishing a distributed display connection between a mobile phone and a computer is shown;

[0058] Figure 8 According to some embodiments of the present application, a schematic diagram of displaying an Android application on a computer display interface is shown;

[0059] Figure 9 According to some embodiments of the present application, another schematic diagram of displaying an Android application on a computer display interface is shown;

[0060] Figure 10 According to some embodiments of the present application, a distributed display schematic diagram of a "video player" application is shown;

[0061] Figure 11 According to some embodiments of the present application, an interaction diagram of "zoom-in" operation on a desktop application window of an Android application is shown;

[0062] Figure 12 According to some embodiments of the present application, a schematic diagram of a "calculator" application before and after desktop application window size adjustment is shown;

[0063] Figure 13AAccording to some embodiments of the present application, a display diagram of a desktop application window of a "Settings" application before adjustment is shown;

[0064] Figure 13B According to some embodiments of the present application, a display diagram of a desktop application window of a "Settings" application after adjustment is shown;

[0065] Figure 14 According to some embodiments of the present application, an interaction diagram of a control operation on the display content of a desktop application window of an Android application is shown;

[0066] Figure 15 According to some embodiments of the present application, another scenario diagram of cross-platform application distributed display is shown;

[0067] Figure 16 According to some embodiments of the present application, another interaction diagram of cross-platform distributed display is shown;

[0068] Figure 17 According to some embodiments of the present application, a diagram of another listening function of a desktop agent process is shown;

[0069] Figure 18 According to some embodiments of the present application, a diagram of successful establishment of distributed display connection between a virtual machine and a computer is shown;

[0070] Figure 19 According to some embodiments of the present application, another distributed display diagram of a "Video Player" application is shown;

[0071] Figure 20 According to some embodiments of the present application, a structural diagram of an electronic device is shown;

[0072] Figure 21 According to some embodiments of the present application, a software structure block diagram of an electronic device is shown;

[0073] Figure 22 According to some embodiments of the present application, another structural diagram of an electronic device is shown;

[0074] Figure 23 According to some embodiments of the present application, another software structure block diagram of an electronic device is shown. DETAILED DESCRIPTION

[0075] The illustrative embodiments of the present application include, but are not limited to, a cross-platform display method of an application, a readable medium, and an electronic device.

[0076] To address the aforementioned issues, this application provides a cross-platform display method that can be applied to two electronic devices with different operating systems installed, or to a single device with two operating systems installed.

[0077] For example, such as Figure 1 A As shown, this applies to 100 phones with Android installed and those with... After establishing a connection between the system's computer 200 and mobile phone 100, the computer 200 can launch the Android application "Calculator" by sending parameters of the Android application "Calculator" to the mobile phone 100. The mobile phone 100 can generate the Android application window of "Calculator" based on the received parameters, and can send the parameters of the Android application window of "Calculator" to the computer 200. These parameters include the size of the Android application window and the coordinates of the window area. The computer 200 creates a desktop application window based on the received parameters of the Android application window of "Calculator" and sends the window handle of the desktop application window to the mobile phone 100. The window handle of the desktop application window is used to uniquely identify the desktop application window. The mobile phone 100 uses the received window handle to generate drawing information and sends the drawing information to the computer 200. The computer 200 can use the received drawing information to call its graphics library to render the image of the desktop application window. Figure 1 B As shown, this allows the desktop application window 2003-2 to display the contents of the Android application "Calculator". It can be understood that, according to the above method, as... Figure 1 C As shown, computer 200 can launch multiple Android applications, create multiple Android application windows, and display the content of multiple Android applications simultaneously.

[0078] Furthermore, computer 200 can also control the displayed content or window size of desktop application window 2003-2. For example, computer 200 can "zoom in" on desktop application window 2003-2 and generate operation information; computer 200 can send the operation information to mobile phone 100, whereby the operation information may include the size of the desktop application window, the window area position coordinates, and the operation position coordinates; mobile phone 100 can adjust the Android application window of "Calculator" based on the operation information and generate operation drawing information; mobile phone 100 sends the generated operation drawing information to computer 200; computer 200 calls functions in its graphics library to render the image of the desktop application window based on the operation drawing information; such as Figure 1 D As shown, this achieves the "enlargement" of the desktop application window 2003-2.

[0079] To make the purpose, technical solution, and advantages of this application clearer, the following description is in conjunction with Figures 1 to 12. Figure 23 The technical solution of this application is described in detail.

[0080] Figure 2 According to some embodiments of the present application, a scenario diagram of an application cross-platform display system 20 is shown.

[0081] Specifically, as shown in Figure 2 , the application cross-platform display system 20 includes a first electronic device 100 and a second electronic device 200. The first electronic device 100 can be established with the second electronic device 200. The first electronic device 100 can run a first operating system 11. The second electronic device 200 can run a second operating system 21.

[0082] As shown in Figure 2 , the display interface 110 of the first electronic device 100 can be a user operation interface of the first operating system 11. The display interface 210 of the second electronic device 200 can be a user operation interface of the second operating system 21.

[0083] In some embodiments, the first operating system 11 can be a Harmony operating system (HarmonyOS), an iPhone operation system (iOS), an Android operating system (Android OS), a Windows Phone operating system (Windows Phone OS), a Blackberry operating system (Blackberry OS), a Palm operating system (Palm OS), and the like. The second operating system 21 can be an Android operating system (Android OS), an iOS operating system (iOS OS), a MAC operating system (MAC OS), and the like.

[0084] In some embodiments, the first electronic device 100 can be communicatively connected with the second electronic device 200 through a wireless communication manner such as Bluetooth, WIFI, or NFC. The first electronic device 100 can also be communicatively connected with the second electronic device 200 through a wired communication manner, for example, the first electronic device 100 is communicatively connected with the second electronic device 200 through a data line and a Universal Serial Bus (USB) interface.

[0085] In some embodiments, the first electronic device 100 and the second electronic device 200 can be various computing devices capable of communicating with each other, for example, the first electronic device 100 or the second electronic device 200 can include but is not limited to a laptop computer, a desktop computer, a tablet computer, a mobile phone, a server, a wearable device, a head-mounted display, a mobile email device, a car machine device, a portable game console, a portable music player, a reader device, a television with one or more processors embedded or coupled therein, or other electronic devices capable of accessing a network.

[0086] The following takes the first electronic device 100 as a mobile phone 100, the first operating system as the operating system, the second electronic device 200 as a computer 200, and the second operating system as the operating system as an example to illustrate the scheme of the present application. The following operating system is referred to as an Android system, the operating system is referred to as a desktop system.

[0087] Figure 3 According to some embodiments of the present application, a cross-platform application distribution display interaction diagram of a cross-platform application distribution display system 20 is shown.

[0088] As Figure 3 shown, the process of the cross-platform application distribution display method of the cross-platform application distribution display system 20 includes the following steps:

[0089] S301: The mobile phone 100 establishes a distributed display connection with the computer 200.

[0090] In some embodiments of the present application, after the mobile phone 100 and the computer 200 establish a wired communication connection or a wireless communication connection, the computer 200 can obtain the verification code of the mobile phone 100 and verify the verification code of the mobile phone 100, thereby establishing a distributed display connection with the mobile phone 100. The verification code of the mobile phone 100 can be the IP address of the mobile phone 100, or a random code generated by the mobile phone 100, etc. The process of the computer 200 verifying the verification code of the mobile phone 100 is described in detail below, and is not repeated here.

[0091] S302: The computer 200 detects an operation of starting an Android application and obtains the parameters of the started Android application.

[0092] In some embodiments of the present application, the parameters of the Android application include the package name and class name of the Android application, and the mobile phone 100 can open the Android application according to the parameters of the Android application. For example, the computer 200 can detect an operation of the user opening the Android application on the computer 200, and the computer 200 generates an instruction to open the Android application.

[0093] In some embodiments of the present application, the computer 200 can synchronously display the Android application of the mobile phone 100. For example, after the mobile phone 100 and the computer 200 establish a distributed connection, the computer 200 can create a shortcut of the Android application on the desktop of the computer 200, thereby realizing that the computer 200 synchronously displays the Android application. The computer 200 can obtain the parameters of the started Android application by detecting an operation of the user opening the shortcut of the Android application on the desktop of the computer 200.

[0094] In some embodiments of the present application, the computer 200 can create a display button of the Android application on the display interface. The computer 200 can also obtain the parameters of the Android application by detecting the operation of the user opening the display button of the Android application created on the display window of the computer 200.

[0095] S303: The computer 200 sends the parameters of the Android application to the mobile phone 100.

[0096] S304: The mobile phone 100 starts the application based on the parameters of the Android application and creates an Android application window.

[0097] In some embodiments of the present application, the mobile phone 100 can start the Android application based on the parameters of the Android application. In the process of starting the Android application, the mobile phone 100 creates an Android application window of the application. In the process of creating the Android application window, the mobile phone 100 first sends the parameters of the Android application window to the computer 200. The computer 200 can create a desktop application window based on the parameters of the Android application window. The parameters of the Android application window include but are not limited to the identification of the Android application window, the size of the Android application window, and the window area position coordinates, etc.

[0098] In embodiments of the present application, the Android application can be started on the virtual screen of the mobile phone 100 or on the display screen of the mobile phone 100. When the Android application is started on the virtual screen of the mobile phone 100, the computer 200 can start the Android application, and the display content of the started Android application is only displayed on the computer 200. For example, as shown in FIG. 2B, the content of the Android application "Calculator" is only displayed on the computer 200. Thus, the cross-platform application distributed application display is realized without affecting the normal use of the mobile phone 100. Figure 1 B

[0099] S305: The mobile phone 100 sends the parameters of the Android application window to the computer 200.

[0100] S306: The computer 200 creates a desktop application window based on the parameters of the Android application window.

[0101] In embodiments of the present application, the computer 200 creates a desktop application window based on the parameters of the Android application window, and assigns a window handle to the created desktop application window. The assigned window handle is used to uniquely identify the created desktop application window.

[0102] For example, the computer 200 creates 10 desktop application windows, and the computer 200 assigns a window handle to each desktop application window. When the computer 200 operates the desktop application window, the window handle of the desktop application window needs to be obtained first before the desktop application window can be operated.

[0103] ​S307: The computer 200 sends the window handle of the desktop application window to the mobile phone 100.

[0104] S308: The mobile phone 100 continues to create the Android application window and associates the Android application window with the window handle of the desktop application window.

[0105] In some embodiments of the present application, after the mobile phone 100 receives the window handle of the desktop application window, the mobile phone 100 continues to create the Android application window and associates the Android application window with the window handle of the desktop application window.

[0106] For example, in the process of the mobile phone 100 continuing to create the Android application window, the mobile phone 100 can use the drawing information generated by the window handle, so that the generated drawing information can be directly executed on the computer 200.

[0107] S309: The mobile phone 100 creates drawing information using the window handle.

[0108] In some embodiments of the present application, the drawing information is used to call an embedded graphics library (EGL), an open graphics library (OpenGL), or the like to draw the display content of the Android application.

[0109] In some embodiments of the present application, the mobile phone 100 generates the drawing information to execute image rendering of the Android application window on the mobile phone 100, so as to complete the display of the display content of the Android application on the desktop application window. In some other embodiments of the present application, the mobile phone 100 generates the drawing information without executing it on the mobile phone 100, so as to achieve the display of the display content of the Android application on the desktop application window without affecting the normal use of the mobile phone 100 and consuming the GPU performance of the mobile phone 100. Details are described below, and are not described herein again.

[0110] For example, the drawing information is used to call an Embedded-System Graphics Library (EGL) or an Open Graphics Library (OpenGL) to draw display content of an Android application, where the mobile phone 100 sends the drawing information to the computer 200, and the computer 200 executes the drawing information to render an image on a desktop application window of the computer 200, so as to display the Android application on the desktop application window of the computer 200. Specifically, the mobile phone 100 sends the drawing information to the computer 200, and the computer 200 executes the drawing information to draw an image using the OpenGL. In this case, the mobile phone 100 creates a rendering surface, i.e., an EGLSurface, and when an interface in the NativeWindowType option is used, the mobile phone 100 replaces a window handle (e.g., native_window) of the computer 200 with a window handle of an Android application window, so that the drawing information created by the mobile phone 100 can be executed on the computer 200, and in a case where the computer 200 opens a plurality of desktop application windows of Android applications, according to information of a window handle contained in the drawing information, a desktop application window that needs to be redrawn is determined.

[0111] In some embodiments, when the computer 200 finishes drawing using the OpenGL, a eglSwapBuffers method is used to exchange front and back buffers, so as to display the drawing content on a display screen of the computer 200; when the drawing is finished and the EGL is no longer needed, the computer 200 cancels the binding of the eglMakeCurrent, and destroys the EGLDisplay, the EGLSurface, and the EGLContext, so as to release the EGL environment.

[0112] S310: The mobile phone 100 sends drawing information to the computer 200.

[0113] S311: The computer 200 executes the drawing information to display content of an Android application on a desktop application window.

[0114] In some embodiments of the present application, the computer 200 calls an image library of the computer 200, such as the EGL, the OpenGL, etc., according to the drawing information, so as to render an image on the desktop application window, and complete display of the display content of the Android application on the desktop application window. Details are described below, and are not repeated here.

[0115] As shown in steps S301 to S311, the mobile phone 100 only needs to send the parameters and drawing information of the Android application window to the computer 200. The computer 200 then creates a desktop application window based on the received parameters and drawing information and calls a graphics library to draw the window's display content, thereby enabling the Android application to be launched and displayed on the computer 200. Compared to existing screen mirroring technologies for cross-platform distributed application display, this application's distributed application display does not require the transmission of large amounts of video and image data between devices to achieve distributed display.

[0116] S312: Computer 200 detected a control operation on the desktop application window and generated operation information.

[0117] In the embodiments of this application, the computer 200 detects user control operations on the desktop application window and generates operation information. For example, the user controls the displayed content (e.g., controls of the display interface), window state, and window size in the desktop application window using a mouse, keyboard, or touch. The operation information may include the size of the desktop application window, the window area coordinates, and the operation position coordinates.

[0118] For example, such as Figure 1 B As shown, the content displayed in the desktop application window (e.g., controls of the display interface) can be numbers or operators in the desktop application window 2003-2; the size of the desktop application window can be the position of the desktop application window 2003-2 in the display interface 210, the size of the display area of ​​the desktop application window 2003-2, etc.; the state of the desktop application window 2003-2 can be open, closed, minimized, etc.

[0119] S313: Computer 200 sends operation information to mobile phone 100.

[0120] S314: Mobile phone 100 generates operation drawing information based on operation information.

[0121] In some embodiments of this application, the operation information sent from computer 200 to mobile phone 100 includes a task identifier for the Android application window. This task identifier is used to... Figure 1 C When multiple windows are displayed as shown, mobile phone 100 can determine which Android application windows need to have their images redrawn.

[0122] like Figure 1 B As shown, computer 200 creates a desktop application window based on the parameters and drawing information received from the Android application window, and calls the graphics library to draw the window display content, thereby achieving... Figure 1 Bthe display of the desktop application window 2003-2 in FIG. 13B. Therefore, the absolute coordinates of the window area of the Android application window have a mapping relationship with the absolute coordinates of the window area of the desktop application window, that is, the mobile phone 100 can re-draw the content displayed by the Android application window based on the operation position coordinates in the operation information to generate operation drawing information. The specific content is the same as that in step S310 of generating drawing information, which is not repeated here.

[0123] S315: The mobile phone 100 sends the operation drawing information to the computer 200.

[0124] S316: The computer 200 executes the operation drawing information to re-render the image displayed by the desktop application window. The specific content is the same as that in step S311 of generating drawing information, which is not repeated here.

[0125] It can be understood that according to the above description, the computer 200 creates a desktop application window based on the received parameters of the Android application window and the drawing information, and calls a graphics library to draw the window display content, thereby realizing the display of the desktop application window 2003-2 in FIG. 13B. Figure 1 B It can be understood that according to the above description, the computer 200 creates a desktop application window based on the received parameters of the Android application window and the drawing information, and calls a graphics library to draw the window display content, thereby realizing the display of the desktop application window 2003-2 in FIG. 13B. Figure 1 B As shown in FIG. 13C, the mobile phone 100 can start and run the "Settings" application on the virtual screen of the mobile phone 100, thereby realizing the distributed display of cross-platform applications without affecting the use of the mobile phone 100.

[0126] Further, the computer 200 detects the control operation of the desktop application window to generate operation information, the mobile phone 100 generates operation drawing information based on the operation information generated by the computer 200, and the computer 200 re-draws the desktop application window based on the operation drawing information to display the content of the Android application after the control operation on the desktop application window. Therefore, the user can operate the numbers or operation symbols in the desktop application window 2003-2 shown in FIG. 13B, move the desktop application window 2003-2 shown in FIG. 13B on the display interface 21, and enlarge, reduce, close, minimize, or the like the desktop application window 2003-2 shown in FIG. 13B. Figure 1 B Figure 1 B Figure 1 B

[0127] In other embodiments of the present application, the computer 200 can include a desktop agent process 201, and the mobile phone 100 can include a mobile phone agent process 101. Figure 4 FIG. 13 shows a cross-platform application distributed display interaction diagram of another application cross-platform display system 20. As shown in FIG. 13, the cross-platform application distributed display process is as follows: Figure 4

[0128] S401: The mobile phone 100 detects the start of the listening operation in the mobile phone agent process 101. ​​​​

[0129] In some embodiments of the present application, the phone agent process 101 is used to provide the IP address of the phone 100, and the phone 100 starts the listening function through the phone agent process 101; after the phone 100 starts the listening, the desktop system 21 can establish a connection with the Android system 11 through the IP address of the phone 100.

[0130] In some embodiments of the present application, the phone agent process 101 can be an application installed in the phone 100, and when the phone agent process 101 is opened, the display interface of the phone agent process 101 can display a button for closing the listening, a button for starting the listening, and the IP address of the phone 100, etc.

[0131] For example, as shown in (a), the user can start the phone agent process 101 by clicking the "phone agent" icon on the display interface 101 of the phone 100; after starting the phone agent process 101, as shown in (b), the user can start the listening function of the phone agent process 101 by clicking the "start listening" button on the display interface 102. As shown in (c), the user can also determine whether the listening function of the phone agent process 101 has been started by checking the system notification of the phone 100. For example, as shown in (d), when the listening function of the phone agent process 101 is started, the system notification window 103 will appear. Figure 5 Figure 5 Figure 5 Figure 5

[0132] For example, as shown in (b), the listening address 10.1.1.1 in the display interface 102 is the IP address of the phone 100, and the desktop agent process 201 of the computer 200 can establish a connection with the phone agent process 101 of the phone 100 according to the IP address of the phone 100. In other embodiments of the present application, the IP address of the phone 100 can also be set to 11.1.1.1. It should be noted that in the specific implementation process, the present application does not make specific limitations on the specific content of the IP address of the phone 100. Figure 5 In other embodiments of the present application, the user can also start the phone agent process 101 of the phone 100 and start the listening function of the phone agent process 101 through other ways, which are not limited in the present application according to the actual application situation.

[0133] S402: The computer 200 detects the operation of starting the listening in the desktop agent process 201 of the computer 200.

[0134]

[0135] ​​​​​In some embodiments of the present application, the desktop agent process 201 can be used to input the IP address of the mobile phone 100, the desktop agent process 201 can also be used to provide the communication connection mode of the mobile phone 100 and the computer 200, and the desktop agent process 201 can also be used to start the listening function of the computer 200.

[0136] In some embodiments of the present application, the desktop agent process 201 can be software installed on the computer 200. When the desktop agent process 201 is opened, the display interface 220 of the desktop agent process 201 can display an input box for inputting the IP address of the mobile phone 100, the display interface 220 of the desktop agent process 201 can also be used to display the communication connection mode of the mobile phone 100 and the computer 200, such as USB connection or wifi connection; the display interface 220 of the desktop agent process 201 can also display a button for starting the listening function, and the desktop agent process 201 can also be used to display whether the desktop system 21 and the Android system 11 are connected successfully.

[0137] In embodiments of the present application, as shown in Figure 6 (a), the user can start the desktop agent process 201 by clicking the shortcut icon 211 of the desktop agent on the display interface 210 of the computer 200; after starting the desktop agent process 201, as shown in Figure 6 (b), the user can input the IP address of the mobile phone 100 on the input box 221, and then click the "connect" button 222 to start the listening function of the desktop agent process 201.

[0138] In other embodiments of the present application, the user can also start the desktop agent process 201 of the computer 200 by other ways, and start the listening function of the desktop agent process 201; the present application does not make specific limitation according to the actual application.

[0139] S403: Send / request between the mobile phone agent process 101 of the mobile phone 100 and the desktop agent process 201 of the computer 200, and establish distributed display connection.

[0140] In some embodiments of the present application, the mobile phone 100 and the computer 200 can be connected through wired or wireless network. Based on the network connection established between the mobile phone 100 and the computer 200, the user can start the listening function of the mobile phone agent process 101 of the mobile phone 100 and obtain the IP address of the mobile phone 100; then, the user inputs the IP address of the mobile phone 100 on the desktop system 21 and clicks the "connect" button, after the desktop agent process 201 learns the IP address of the mobile phone 100, the desktop agent process 201 of the computer 200 and the mobile phone agent process 101 of the mobile phone 100 send / receive requests, wherein the sent / received requests can be socket requests. The socket request is used for bidirectional communication between different electronic devices or between different application programs of different electronic devices in wireless network or wired network.

[0141] The mobile phone agent process 101 of the mobile phone 100 and the desktop agent process 201 of the computer 200 send / receive requests to establish distributed display connection. As shown in Figure 7 After the mobile phone agent process 101 of the mobile phone 100 and the desktop agent process 201 of the computer 200 successfully establish connection, the display interface 210 displays the "connection success" icon 224, that is, the desktop agent process 201 successfully opens the listening function; when the mobile phone agent process 101 of the mobile phone 100 and the desktop agent process 201 of the computer 200 fail to establish distributed display connection, the display interface 210 displays the "connection failure" icon, that is, the desktop agent process 201 fails to successfully open the listening function.

[0142] In the above S401 to S403, the user inputs the IP address of the mobile phone 100 on the display interface of the computer 200. The desktop agent process 201 of the computer 200 learns the IP address of the mobile phone 100 and sends a socket request to the mobile phone agent process 101 of the mobile phone 100 based on the IP address of the mobile phone 100. In other embodiments of the present application, the user can also input the verification code of the mobile phone 100 on the display interface of the computer 200. The desktop agent process 201 of the computer 200 learns the verification code of the mobile phone 100 and sends a socket request to the mobile phone agent process 101 of the mobile phone 100 based on the verification code of the mobile phone 100. It can be understood that the verification code of the mobile phone 100 can be the IP address of the mobile phone 100 or a random code generated by the mobile phone 100. According to actual application, the type of the verification code of the mobile phone 100 learned by the computer 200 is not limited in the present application.

[0143] S404: The computer 200 detects the operation of starting the "settings" application of the mobile phone 100 on the desktop agent process 201.

[0144] For ease of description, the following explanation uses the example of a user launching the "Settings" app on phone 100 from desktop proxy process 201. It can be understood that other apps on phone 100 can also be launched from desktop proxy process 201. The specific launch process can be found in the process of launching the "Settings" app on phone 100 from desktop proxy process 201, and will not be elaborated upon here.

[0145] In some embodiments of this application, an application button for launching the mobile phone 100 can be created on the display interface 220 of the desktop agent process 201 of the computer 200. The user can launch the application of the mobile phone 100 by clicking the application button on the display interface 220 of the desktop agent process 201 of the computer 200. Furthermore, if there is no application button for the mobile phone 100 on the display interface 220 of the computer 200, the user can also launch the application of the mobile phone 100 by entering the application package name and / or class name of the mobile phone 100 in the input box of the display interface 220 of the computer 200.

[0146] For example, such as Figure 8 As shown, the user launches the "Settings" application on the mobile phone 100 by clicking the "Settings" button 225 on the display interface 220 of the desktop proxy process 201 on the computer 200. The user can also open the application on the mobile phone 100 by entering the package name and class name in the "Input Box" 226 and / or "Input Box" 227 on the computer 200 respectively and then clicking the "Launch" button 227. For example, the user can also launch the "Settings" application on the mobile phone 100 by entering the package name com.android.settings in the "Input Box" 22 of the display interface 220 and then clicking the "Launch" button 228.

[0147] In other embodiments of this application, desktop shortcuts for applications of the mobile phone 100 can also be created on the display interface 210 of the computer 200, allowing users to open the applications of the mobile phone 100 by clicking the desktop shortcut icon. For example, as Figure 9 As shown, the user launches the "Settings" application on the phone 100 by clicking the desktop shortcut icon 212 of the "Settings" application on the display interface 210.

[0148] For example, in some embodiments, the process of creating a desktop shortcut of the application of the mobile phone 100 on the display interface 210 by the computer 200 can include: the desktop agent process 201 of the computer 200 sends an application query instruction to the mobile agent process 101 of the mobile phone 100; after receiving the application query instruction, the mobile agent process 101 of the mobile phone 100 obtains all the installed application package names on the mobile phone 100 by calling the getAllPackages interface of the PackageManagerService service of the Android system 11, and then obtains the applicationPackageManager by getPackageManager(), and queries the application package name by the getApplicationIcon interface of the applicationPackageManager to obtain the application icon according to the application package name; the mobile agent process 101 of the mobile phone 100 traverses all the application package names to obtain the icon of each application on the mobile phone 100; the mobile agent process 101 of the mobile phone 100 sends the parameters of the icon of each application on the mobile phone 100 to the desktop agent process 201 of the computer 200; the desktop agent process 201 of the computer 200 calls the Application Programming Interface (API) of the 32-bit platform of Microsoft to create a desktop shortcut for each application of the mobile phone 100.

[0149] S405: The desktop agent process 201 of the computer 200 sends the parameters of the started application of the mobile phone 100 to the mobile agent process 101.

[0150] In some embodiments, the desktop agent process 201 of the computer 200 sends the parameters of the Android application to be started to the mobile agent process 101 of the mobile phone 100, and the mobile agent process 101 of the mobile phone 100 can open the application of the mobile phone 100 through the parameters of the application. Thus, the user can start the application of the mobile phone 100 on the computer 200, thereby making the user experience more convenient.

[0151] In some embodiments of the present application, the parameters of the Android application include the package name of the application and the class name of the application. For example, the user starts the "Settings" application of the mobile phone 100 on the desktop agent process 201, and the desktop agent process 201 can send the package name com.android.settings of the "Settings" application to the mobile agent process 101.

[0152] S406: The mobile agent process 101 of the mobile phone 100 calls the interface of the Android system 11 to start the "Settings" application of the mobile phone 100.

[0153] In some embodiments of the present application, for example, the phone agent process 101 starts the "Settings" application of the phone 100 according to the received parameters of the "Settings" application, by calling the startActivity interface of the ActivityTaskManagerService of the Android system 11.

[0154] S407: The Android system 11 of the phone 100 creates an Android application window of the "Settings" application.

[0155] For example, in some embodiments of the present application, the phone agent process 1002 of the phone 100 calls the interface of the Android system 11 to start the "Settings" application of the phone 100; during the starting process of the "Settings" application, the Android system 11 creates an Android application window of the "Settings" application and sends the parameters of the created Android application window of the "Settings" application to the phone agent process 101 of the phone 100. In embodiments of the present application, the "Settings" application of the phone 100 can be started on the virtual screen of the phone 100 or on the display screen of the phone 100.

[0156] In some embodiments of the present application, the Android system 11 of the phone 100 specifies the starting parameter android.activity.windowingMode as the freeform mode, and the Android system 11 of the phone 100 specifies to start in the FLAG_ACTIVITY_NEW_TASK mode, i.e., the Android system 11 of the phone 100 creates a task associated with the Android application window of the "Settings" application.

[0157] S408: The Android system 11 of the phone 100 sends the parameters of the Android application window of the "Settings" application to the phone agent process 101 of the phone 100.

[0158] In some embodiments, the Android system 11 of the phone 100 can first send the parameters of the Android application window of the "Settings" application to the phone agent process 101 of the phone 100, and the phone agent process 101 of the phone 100 sends the parameters of the Android application window of the "Settings" application. The desktop agent process 201 of the computer 200 can create a desktop application window of the "Settings" application according to the parameters of the Android application window of the "Settings" application.

[0159] In some embodiments of the present application, the parameters of the Android application window can include the task identifier of the Android application window, the width and height of the Android application window, the area position coordinates of the Android application window, etc. The area position coordinates of the Android application window can be represented as {A l , A t , A r , A b}, wherein A lA is a position coordinate of a left border of an Android application window region of the Android application, A t A is a position coordinate of an upper border of the Android application window region of the Android application, A r A is a position coordinate of a right border of the Android application window region of the Android application, A l A is a position coordinate of a lower border of the Android application window region of the Android application.

[0160] S409: The phone agent process 101 of the phone 100 sends the parameters of the Android application window of the application to the desktop agent process 201 of the computer 200.

[0161] S410: The desktop agent process 201 of the computer 200 creates the desktop application window of the "Settings" application on the desktop system 21.

[0162] In some embodiments of the present application, for example, the desktop agent process 201 of the computer 200 creates the desktop application window of the "Settings" application on the desktop system 21 based on the parameters of the Android application window of the "Settings" application sent by the phone agent process 101 of the phone 100.

[0163] Specifically, the desktop agent process 201 of the computer 200 checks whether there is a window associated with the task identifier of the Android application window of the "Settings" application in the currently created desktop application window 2003-1 of the "Settings" application; if there is no window associated with the task identifier, the desktop agent process 201 newly creates a parent window associated with the task identifier, and the desktop agent process 201 creates the desktop application window 2003-1 of the "Settings" application under the newly created parent window; if there is a window associated with the task identifier, the desktop agent process 201 of the computer 200 creates the desktop application window of the "Settings" application under the parent window associated with the task identifier.

[0164] It can be understood that the desktop agent process 201 of the computer 200 determines the region position coordinate of the desktop application window of the "Settings" application in the display interface 210 of the computer 200, the window size, etc. based on the parameters of the Android application window of the "Settings" application.

[0165] In some embodiments, the task identifier of the Android application window is used to uniquely identify the Android application window. The desktop application window created by the computer 200 based on the parameters of the Android application window is associated with the task identifier of the Android application window. The parameters of the Android application window include the task identifier of the Android application window. Therefore, before creating the desktop application window, the desktop agent process 201 checks whether there is a desktop application window associated with the task identifier of the Android application window of the "Settings" application in the currently created desktop application windows, thereby preventing the application of one mobile phone 100 from displaying two desktop application windows of the "Settings" application on the computer 200. In addition, for the created desktop application window, when the image displayed by the Android application window or the size of the window is adjusted, the desktop agent process 201 of the computer 200 synchronously adjusts the created desktop application window by checking the task identifier of the Android application window. This will be described in detail below, and will not be described here.

[0166] S411: The desktop system 21 of the computer 200 sends the window handle of the desktop application window of the "Settings" application to the mobile agent process 101 of the mobile phone 100.

[0167] In some embodiments, the window handle of the desktop application window is used to uniquely identify the desktop application window. For example, the window handle of the desktop application window of the "Settings" application can be used to identify the desktop application window, and the desktop system 21 operates the desktop application window of the "Settings" application by obtaining the window handle of the desktop application window of the "Settings" application.

[0168] S412: The mobile agent process 101 of the mobile phone 100 sends the window handle of the desktop application window of the "Settings" application of the computer 200 to the Android system 11 of the mobile phone 100.

[0169] S413: The Android system 11 of the computer 200 associates the window handle of the desktop application window 2003-1 of the "Settings" application with the Android application window of the mobile phone 100.

[0170] In some embodiments, the Android system 11 of the computer 200 associates the window handle of the desktop application window of the "Settings" application with the Android application window of the mobile phone 100, so that the operation of the Android application window of the "Settings" application by the mobile phone 100 can be synchronized to the desktop application window 2003-1 of the computer 200.

[0171] S414: The Android system 11 of the mobile phone 100 creates drawing information using the window handle.

[0172] In some embodiments of the present application, the drawing information can be drawing information created by the Android system 11 using the window handle of the desktop application window of the "Settings" application. The drawing information can be drawing information created by the Android system 11 using the window handle of the desktop application window of the "Settings" application and the window handle of the Android application window. The drawing information is used to execute on the desktop system 21 of the computer 200 and render the image on the desktop application window identified by the window handle.

[0173] For example, in some embodiments of the present application, after the Android system 11 of the mobile phone 100 receives the window handle of the desktop application window of the "Settings" application of the mobile phone 100, the Android system 11 associates the window handle of the desktop application window 2003-1 with the Android application window of the "Settings" application, and continues to create the Android application window of the "Settings" application. When the Android system 11 continues to create the Android application window of the "Settings" application, it needs to generate drawing information to render the display content of the Android application window as an image. The drawing information is used to call the Embedded-System Graphics Library (EGL), Open Graphics Library (OpenGL), etc. to draw the display content of the Android application.

[0174] As can be seen from the above description, since the Android system 11 of the mobile phone 100 uses the associated window handle as a parameter of the Android application window to create drawing information, when the drawing information is executed on the desktop application window, there is no problem of not finding the desktop application window to execute the drawing information, and compared with the drawing information being executed on the Android system 11 and rendering the image, the drawing information being executed on the desktop application window and rendering the image belongs to on-screen rendering, which can reduce one buffer operation and improve the GPU performance of the computer 200 during image rendering operation. Among them, the drawing information being executed on the desktop application window and rendering the image belongs to on-screen rendering, and the drawing information being executed on the Android system 11 and rendering the image belongs to off-screen rendering.

[0175] S415: The Android system 11 of the mobile phone 100 sends the drawing information to the desktop agent process 201 of the computer 200.

[0176] S416: The desktop agent process 201 of the computer 200 executes the drawing information.

[0177] In some embodiments of this application, the desktop agent process 201 of the computer 200 executes drawing information and calls image libraries in the graphics service, such as EGL and OpenGL, through the graphics device interface, thereby realizing the display of image rendering on the desktop application window of the "Settings" application created in step S410, and completing the display of the Android application's content on the desktop application window. For example, as Figure 1 B The desktop application window of the "Settings" app shown in Figure 2003-2 displays the following content.

[0178] Based on the process described in steps S404 to S416, where the desktop agent process 201 of computer 200 launches an application, creates a desktop application window 2003-2, and renders the image of the content displayed on the desktop application window 2003-2, computer 200 can also launch other Android applications and display the content of other Android applications. For example, such as Figure 1 C As shown, the desktop application window 2003-1 of the "Settings" application, the desktop application window 2003-2 of the "Calculator" application, and the desktop application window 2003-3 of the "Mail" application can be displayed simultaneously on the display interface 210 of the computer 200.

[0179] The Android system 11 of mobile phone 100 receives the window handle of desktop application window 2003-1 and associates the window handle of desktop application window 2003-1 with the Android application window of the "Settings" application of mobile phone 100. It can be understood that for multiple applications on mobile phone 100, Android system 11 can associate the window handles of the desktop application windows 2003 of the applications on mobile phone 100 with the Android application windows one by one, thereby enabling the simultaneous opening and display of multiple desktop application windows 2003 of mobile phone 100 on the desktop system 21. For example, as... Figure 1 C As shown, the computer 200's display interface 210 simultaneously displays the desktop application window 2003-1 of the "Settings" application, the desktop application window 2003-2 of the "Calculator" application, and the desktop application window 2003-3 of the "Mail" application of the mobile phone 100.

[0180] In other embodiments of this application, the desktop agent process 201 of the computer 200 can also receive drawing information in real time and render the display content of the desktop application window, thereby enabling dynamic image content to be displayed on the desktop application window of the computer 200. For example... Figure 10 As shown, the desktop application window 2003-4 displayed on the display interface 210 of computer 200 can be the desktop application window 2003-4 of the "Video Player" application on mobile phone 100; the Android application window of the "Video Player" application on mobile phone 100 can be displayed on the virtual screen of mobile phone 100, while the display screen of mobile phone 100 displays real-time captured mobile phone performance data. Figure 10The content displayed on the display screen of the mobile phone 100 in real time shows that the desktop application window 2003-4 of the drawing application executes the drawing information to realize video image rendering and consumes the GPU resources of the computer 200, while the Android application window does not execute the instruction and thus does not consume the GPU resources of the mobile phone 100.

[0181] In some embodiments of the present application, as known from steps S404 to S413, the desktop agent process 201 of the computer 200 can create a desktop application window of the "video player" application on the desktop system 21, and the Android system 11 of the mobile phone 100 can associate the window handle of the desktop application window of the "video player" application with the Android application window of the "video player" application on the mobile phone 100. The Android system 11 of the mobile phone 100 can create drawing information using the window handle of the desktop application window of the "video player" application and send the drawing information to the desktop system 21 of the computer 200, and the desktop system 21 of the computer 200 executes the drawing information and renders the image on the desktop application window of the "video player" application.

[0182] For example, the application window displayed on the display interface 210 of the computer 200 is the desktop application window 2003-4 of the "video player" application of the mobile phone 100 and the window displays that a video is being played; as known from the above steps, the window handle of the desktop application window 2003-4 of the "video player" application of the mobile phone 100 can be associated with the Android application window of the "video player" application, and the Android system 11 uses the associated window handle as the parameter of the Android application window of the "video player" application to create the drawing information.

[0183] It can be understood that, for example, for an Android application "Settings", the window handle of the desktop application window of the application is associated with the Android application window of the application, and the desktop application window of the application is created based on the parameter of the Android application window, wherein the parameter of the Android application window contains the task identifier of the Android application window. Therefore, the drawing information created by the Android system 11 using the window handle of the desktop application window of the "Settings" application can not only be run on the desktop system 21, but also the desktop system 21 can determine the desktop application window 2003-3 that needs to display the image rendered using the drawing information according to the drawing information. Moreover, when the desktop agent process 201 of the computer 200 detects the control operation of the desktop application window 2003-3, operation information is generated. The control of the Android application on the computer 200 is described in detail below, and thus is not described herein.

[0184] As known from the above steps, the window handle of the desktop application window of the "video player" application of the mobile phone 100 is associated with the Android application window of the "video player" application of the mobile phone 100, and the Android system 11 of the mobile phone 100 uses the associated window handle as the parameter of the Android application window of the "video player" application to create the drawing information. Figure 4The cross-platform application distributed display interaction flow of the cross-platform display system 20 of the application can know that the Android system 11 of the mobile phone 100 generates drawing information using the received window handle and sends the drawing information to the desktop agent process 201 of the computer 200, and the desktop agent process 201 of the computer 200 can call the graphics library of the computer 200 to perform image rendering on the desktop application window based on the received drawing information, so as to realize that the created desktop application window 2003-2 displays the content of the Android application “calculator”. Moreover, the Android system 11 of the mobile phone 100 generates drawing information using the received window handle, so that multiple desktop application windows of Android applications can be displayed on the display interface of the computer 200, and multiple Android applications can be opened and displayed at the same time. For example, as shown in Figure 1 B so as to realize that the desktop application window 2003-2 displays the content of the Android application “calculator”. It can be understood that, according to the above method, as shown in Figure 1 C the computer 200 can start multiple Android applications, create multiple Android application windows, and display the content of multiple Android applications at the same time.

[0185] In some other embodiments of the present application, the user can control the display content (for example, the control of the display interface) in the desktop application window, the window state, and the window size through a mouse, a keyboard, and touch, and the like, so as to realize that the Android application can not only be displayed on the computer 200, but also be controlled on the computer 200.

[0186] The following will be described in detail several types of control of the Android application on the computer 200.

[0187] For example, as shown in Figure 1 B the user can perform a “zoom-in” operation on the window in the desktop application window through a mouse, touch, and the like, and the computer 200 can detect the “zoom-in” operation of the user on the desktop application window. Figure 11 shows an interaction diagram of a “zoom-in” operation on the desktop application window of the Android application.

[0188] S1101: The desktop system 21 of the computer 200 detects the control operation of the desktop application window, and generates window region position coordinates and operation position coordinates.

[0189] For example, as shown in Figure 1 BAs shown, the display interface 210 of the computer 200 displays a desktop application window 2003-2 of a "calculator" application of the mobile phone 100, and the user can click the zoom-in icon by using a mouse or touch, etc. to perform a zoom-in operation on the desktop application window 2003-2. After the desktop system 21 of the computer 200 detects the control operation of the desktop application window, the desktop system 21 of the computer 200 adjusts the area position coordinates of the desktop application window. The desktop system 21 of the computer 200 sends the area position coordinates of the desktop application window before and after the adjustment and the position coordinates at which the desktop system 21 of the computer 200 detects the control operation of the desktop application window to the mobile phone agent process 101 of the mobile phone 100.

[0190] In some embodiments of the present application, after the desktop system 21 of the computer 200 detects the control operation of the desktop application window, the desktop system 21 of the computer 200 adjusts the window size of the desktop application window 2003-2 of the "calculator" application. The desktop system 21 of the computer 200 generates the window area position coordinates and the operation position coordinates.

[0191] For example, after the desktop system 21 of the computer 200 detects the control operation of the desktop application window, the area coordinates of the desktop application window 2003-2 are adjusted from [W l , W t , W r , W b ] to [W′ l , W′ t , W′ r , W′ b ], where W l is the position coordinate of the left boundary of the desktop application window area of the "calculator" application before the adjustment, W t is the position coordinate of the top boundary of the desktop application window area of the "calculator" application before the adjustment, W r is the position coordinate of the right boundary of the desktop application window area of the "calculator" application before the adjustment, and W b is the position coordinate of the bottom boundary of the desktop application window area of the "calculator" application before the adjustment; W′ l is the position coordinate of the left boundary of the desktop application window area of the "calculator" application after the adjustment, W′ t is the position coordinate of the top boundary of the desktop application window area of the "calculator" application after the adjustment, W′ r is the position coordinate of the right boundary of the desktop application window area of the "calculator" application after the adjustment, and W′ b is the position coordinate of the bottom boundary of the desktop application window area of the "calculator" application after the adjustment.

[0192] Figure 12 FIG. 2B is a schematic diagram of the desktop application window 2003-2 of the "calculator" application before and after the size adjustment. As shown, the desktop application window 2003-2 of the "calculator" application before the size adjustment is a rectangle with a left boundary at W l , a top boundary at W t , a right boundary at W r , and a bottom boundary at W b . The desktop application window 2003-2 of the "calculator" application after the size adjustment is a rectangle with a left boundary at W′ l , a top boundary at W′ t , a right boundary at W′ r , and a bottom boundary at W′ b .Figure 12 As shown in (a), the original area coordinates of the desktop application window 2003-2 are [W l W t W r W b ].like Figure 12 As shown in (b), after the user clicks the "zoom in" icon in the desktop application window 2003-2, the coordinates of the desktop application window 2003-2 area are adjusted to [W′]. l , W′ t , W′ r , W′ b It is understandable that the desktop system 21 of computer 200 only adjusts the size of the window. The desktop system 21 of computer 200 can render the display content image of the enlarged desktop application window 2003-2 based on the operation drawing information sent by mobile phone 100.

[0193] The following explanation uses the example of how the desktop system 21 of computer 200 detects the "zoom in" operation of the desktop application window 2003-2 of the "Calculator" application on mobile phone 100 and generates the position coordinates of the "zoom in" window area and the operation position coordinates. It can be understood that the desktop system 21 of computer 200 can also detect the size of other application windows on mobile phone 100 and perform operations accordingly. For details, please refer to the example of the desktop system 21 of computer 200 detecting the "zoom in" operation of the desktop application window 2003-2 of the "Calculator" application on mobile phone 100; these will not be elaborated upon here.

[0194] S1102: The desktop system 21 of the computer 200 sends the window area position coordinates and operation position coordinates to the mobile agent process 101 of the mobile phone 100.

[0195] S1103: Mobile phone agent process 101 of mobile phone 100 calls the Android interface to readjust the size and position coordinates of the Android application window of the "Calculator" application.

[0196] In some embodiments of this application, the mobile agent process 101 of the mobile phone 100, based on the received window area position coordinates and operation position coordinates, calls the `resize Task` interface of `ActivityTaskManagerService` to reset the size and position coordinates of the task corresponding to the task identifier associated with the Android application window of the "Calculator" application, thereby adjusting the Android application window of the "Calculator" application. The window area position coordinates and operation position coordinates include the task identifier of the Android application window of the "Calculator" application on the mobile phone 100.

[0197] S1104: The Android system 11 of mobile phone 100 adjusts the window area position coordinates of the Android application window based on the window area position coordinates and operation position coordinates, and generates operation drawing information.

[0198] For example, in some embodiments of this application, when a user zooms in on the desktop application window 2003-2 of the "Calculator" application on mobile phone 100, the desktop system 21 of computer 200 detects the zoom-in operation and generates the window area position coordinates and operation position coordinates for the zoom-in operation. The desktop system 21 then sends the zoom-in control operation information to the mobile agent process 101 of mobile phone 100. Based on the received window area position coordinates and operation position coordinates, the mobile agent process 101 of mobile phone 100 adjusts the position coordinates of the Android application window by calling the resize Task interface of ActivityTaskManagerService. The window area position coordinates and operation position coordinates for the zoom-in operation include the task identifier of the Android application window of the "Calculator" application on mobile phone 100.

[0199] In some embodiments of this application, the Android system 11 of the mobile phone 100, based on the "zoomed" window area position coordinates and operation position coordinates, performs control operations on the task corresponding to the task identifier associated with the Android application window of the "calculator" application by calling the resize Task interface of ActivityTaskManagerService, thereby adjusting the area position coordinates of the Android application window of the "calculator" application; the area position coordinates of the Android application window of the "calculator" application are determined by [A l A t A r A b ] changed to [A' l A' t A' r A' b ], where A l For the position coordinates of the left edge of the Android application window area of ​​the "Calculator" app before adjustment, A t For the position coordinates of the upper boundary of the Android application window area of ​​the "Calculator" app before adjustment, A r For the position coordinates of the right edge of the Android application window area of ​​the "Calculator" app before adjustment, A b The coordinates of the bottom edge of the Android application window area of ​​the "Calculator" app before adjustment; A' l For the adjusted left edge coordinates of the Android application window area of ​​the "Calculator" app, A t A represents the coordinates of the upper boundary of the Android application window area for the "Calculator" app.r A represents the coordinates of the right edge of the Android application window area for the "Calculator" app. b These are the coordinates of the bottom edge of the Android application window area for the "Calculator" app.

[0200] In some embodiments of this application, the Android system 11 of the mobile phone 100 calls the system service interface to adjust the area coordinates of the Android application window based on the window area position coordinates and the operation position coordinates. The adjusted area coordinates of the Android application window are [A' l A' t A' r A' b The calculation relationship of formulas (1) and (2) must be satisfied as follows:

[0201]

[0202]

[0203] It's understandable that Android 11 on phone 100 adjusts the area coordinates of the "Calculator" app's Android application window and generates operation drawing information. This operation drawing information is used to draw the content displayed in the enlarged desktop application window of the "Calculator" app.

[0204] S1105: The Android system 11 of mobile phone 100 sends the operation drawing information to the desktop agent process 201 of computer 200.

[0205] S1106: Desktop agent process 201 of computer 200 performs drawing operations.

[0206] For example, the desktop agent process 201 of computer 200 draws a magnified desktop application window 2003-2 of the "Calculator" application on desktop system 21 based on the operation drawing information. (See details for further information.) Figure 4 The content of step S415, which generates drawing information, will not be elaborated here.

[0207] It's understandable that PC200 can also detect the user's "minimize" operation on the desktop application window; please refer to the above for details. Figure 11 The process described will not be elaborated here.

[0208] The following describes how to control the display content (e.g., controls on the display interface) of Android applications on mobile phone 100 from computer 200. For example, as shown in Figure 13, ... Figure 13A As shown, computer 200 can detect the user's click on button icon 218 in desktop application window 2003-1, and adjust the display content of desktop application window 2003-1 to... Figure 13BThe content shown, and the option 4 of the phone 100 is changed from the closed state to the open state. The following describes the control of the operation of the desktop application window of the Android application in combination with Figure 14 The interactive diagram of the control of the operation of the desktop application window of the Android application is shown. As shown in

[0209] Figure 14 The interactive diagram of the control of the operation of the display content of the desktop application window of the Android application is shown. As shown in Figure 14 The specific process of the control of the operation of the display content of the desktop application window of the Android application is as follows:

[0210] S1401: The desktop system 21 of the computer 200 detects the control operation of the desktop application window, and generates the operation position coordinates.

[0211] For example, according to the process described in Figure 4 The computer 200 can create the desktop application window 2003-1 of the "Settings" application, and display the content of the Android application "Settings" on the desktop application window 2003-1, which is displayed on the display interface 210 of the computer 200. As shown in Figure 13A The desktop system 21 of the computer 200 can detect that the user clicks the button icon 218 of the "Settings" application by the mouse, and generate the operation position icon. The generated operation position icon can be the position coordinates of the area where the button icon 218 is located.

[0212] S1402: The desktop system 21 of the computer 200 sends the operation position coordinates to the mobile agent process 101 of the phone 100.

[0213] S1403: The mobile agent process 101 of the phone 100 sends the operation position coordinates to the Android system 11 of the phone 100.

[0214] S1404: The Android system 11 of the phone 100 generates the operation drawing information based on the operation position coordinates.

[0215] In some embodiments of the present application, according to the distributed display process of Figure 4 The relative position coordinates of the area position coordinates of the desktop application window of the "Settings" application and the area position coordinates of the Android application window have a mapping relationship. For example, as shown in Figure 13AAs shown, the Android system 11 of mobile phone 100 detects the operation on setting option 4 based on the regional position coordinates of the button icon 218 of "Settings Option 4". The Android system 11 of mobile phone 100 executes control operations on the task corresponding to the task identifier associated with the Android application window of the "Settings" application by calling the InputManagerService service interface, and generates operation drawing information. For example, setting option 4 is airplane mode, and button icon 218 is the airplane mode status switch button. The Android system 11 of mobile phone 100 turns on airplane mode by calling the InputManagerService service interface and generates operation drawing information, which is used to draw the display window after airplane mode is turned on.

[0216] S1405: The Android system 11 of mobile phone 100 sends the operation drawing information to the desktop agent process 201 of computer 200.

[0217] S1406: Desktop agent process 201 of computer 200 performs drawing operations.

[0218] For example, such as Figure 13B As shown, the desktop agent process 201 of computer 200 draws the desktop application window 2003-1 of the "Settings" application after closing setting option 4 on desktop system 21 based on operation drawing information. See details for further information. Figure 4 The content of step S415, which generates drawing information, will not be elaborated here.

[0219] Understandable. Figure 3 The cross-platform display method of the application shown can be applied to electronic devices 100 and 200 that have different operating systems installed. In other embodiments of this application, the cross-platform display method of the application can also be applied to scenarios where two operating systems run on one electronic device.

[0220] Figure 15 According to some embodiments of this application, another scenario for distributed display in cross-platform applications is illustrated. Figure 15 The scenario shown includes a third electronic device 300; wherein the third electronic device 300 runs a second operating system 21, and the third electronic device 300 also has a virtual machine 400 running a first operating system 11 installed. For a description of the first operating system 11 and the second operating system 21, please refer to the description above, which will not be repeated here.

[0221] like Figure 15 As shown in (a), the user can open the "Mobile Application Engine" application shortcut icon 321 on the display interface 320 of the third electronic device 300 by double-clicking it with a mouse or touch. Figure 15(b) the user interface 410 of the virtual machine 400 running the first operating system 11.

[0222] In some embodiments of the present application, the third electronic device 300 can be an electronic device capable of running the first operating system 11 and the second operating system 21 simultaneously, for example, the third electronic device 300 can include but is not limited to a mobile phone, a notebook computer, a tablet computer, a desktop computer, a laptop computer, an Ultra-mobile Personal Computer (UMPC), a handheld computer, a netbook, a Personal Digital Assistant (PDA), and the like.

[0223] Hereinafter, another cross-platform application distributed display process is described with the third electronic device 300 as a computer 300, the first operating system 11 as an Android system 11, and the second operating system 21 as a desktop system 21. Figure 16 To illustrate another cross-platform distributed display according to an embodiment of the present application, an interaction diagram is shown as follows: Figure 16 As shown, the specific process of cross-platform application distributed display is as follows:

[0224] S1601: The virtual machine 400 detects the operation of starting the listening of the mobile agent process 101.

[0225] In some embodiments of the present application, the virtual machine of the Android system 11 is installed on the desktop system 21 of the computer 300, and the user can click the "mobile application engine" application shortcut icon 321 of the display interface 320 of Figure 15 (a) to open the display interface 410 of the virtual machine 400 of the Android system 11. As shown in Figure 15 (b), the user can click the "mobile agent" icon on the display interface 401 of the virtual machine 400 to start the mobile agent process 101 by using a mouse or touch, etc. The specific content of starting the listening function of the mobile agent process 101 is described with reference to step S401, which is not repeated here.

[0226] S1602: The computer 300 detects the operation of starting the listening of the desktop agent process 201 of the computer 300.

[0227] In some embodiments of the present application, as shown in Figure 17 (a), the user can start the desktop agent process 201 by clicking the shortcut icon 331 of the desktop agent on the display interface 330 of the computer 300, and the specific process is described with reference to step S402, which is not repeated here.

[0228] In some embodiments of the present application, as shown in Figure 17(a) as shown, the user can start the desktop agent process 331 by clicking the shortcut icon 331 of the desktop agent on the display interface 320 of the computer 300; after starting the desktop agent process, as shown in Figure 17 (b) as shown, the user can click the "USB" button 341 on the display interface 340, the "USB" button 341 option indicates that the desktop agent process 201 can be connected with the mobile agent process 101 through the USB connection mode, then the user clicks the "connect" button 342, and the desktop agent process 201 of the computer 300 starts the listening operation.

[0229] In other embodiments of the present application, the user can also start the desktop agent process 201 of the computer 300 in other ways and start the listening function of the desktop agent process 201; the present application does not make specific limitations according to the actual application.

[0230] S1603: The mobile agent process 101 of the virtual machine 400 and the desktop agent process 201 of the computer 300 send / receive requests to establish a distributed display connection.

[0231] In some embodiments of the present application, since the virtual machine 400 is started on the desktop system 21, the mobile agent process 101 and the desktop agent process 201 can be connected through the network card address, as shown in Figure 17 (b) as shown, when the user clicks the "USB" button 341 on the display interface 340, the desktop agent process 201 is configured as an Android debug bridge (adb) port mapping, then the user clicks the "connect" button 342, and the desktop agent process 201 of the computer 200 sends / receives requests to the mobile agent process 101 of the mobile phone 100, wherein the sending / receiving request can be a socket request. Wherein the local loopback address can be 127.0.0.1.

[0232] As shown in Figure 18 When the mobile agent process 101 of the virtual machine 400 and the desktop agent process 201 of the computer 300 successfully establish a connection, the display interface 340 will display a "connection successful" icon 343.

[0233] S1604: The computer 300 detects the operation of starting the application of the virtual machine 400 on the desktop agent process 201, for details, please refer to step S404, which is not repeated here.

[0234] S1605: The desktop agent process 201 of the computer 300 sends the parameters of the started Android application of the virtual machine 400 to the mobile agent process 101. For details, please refer to step S405, which is not repeated here.

[0235] S1606: The phone agent process 101 of the virtual machine 400 calls an interface of the Android system 11 to start the application of the virtual machine 400. For details, refer to step S406, which will not be repeated here.

[0236] S1607: The Android system 11 of the virtual machine 400 creates an Android application window of the application. For details, refer to step S407, which will not be repeated here.

[0237] S1608: The Android system 11 of the virtual machine 400 sends parameters of the Android application window to the phone agent process 101 of the virtual machine 400.

[0238] S1609: The phone agent process 101 of the virtual machine 400 sends the parameters of the Android application window of the application to the desktop agent process 201 of the computer 300.

[0239] S1610: The desktop agent process 201 of the computer 300 creates a desktop application window of the application on the desktop system 21 based on the parameters of the Android application window. For details, refer to step S410, which will not be repeated here.

[0240] S1611: The desktop system 21 of the computer 300 sends a window handle of the desktop application window of the application to the phone agent process 101 of the virtual machine 400.

[0241] S1612: The phone agent process 101 of the virtual machine 400 sends the window handle of the desktop application window of the application of the computer 300 to the Android system 11 of the virtual machine 400.

[0242] S1613: The Android system 11 of the computer 300 associates the window handle of the desktop application window of the application with the Android application window of the virtual machine 400.

[0243] In some embodiments of the present application, the Android system 11 associates the window handle of the desktop application window with the Android application window of the application of the virtual machine 400 by receiving the window handle of the desktop application window. Multiple applications of the virtual machine 400 can be displayed on multiple desktop application windows, where each Android application window of the application of the virtual machine 400 is associated with each desktop application window one by one.

[0244] S1614: The Android system 11 of the virtual machine 400 creates drawing information using the window handle. For details, refer to step S414, which will not be repeated here.

[0245] S1615: The Android system 11 of the virtual machine 400 sends the drawing information to the desktop agent process 201 of the computer 300.

[0246] S1616: The desktop agent process 201 of the computer 300 executes the drawing information created using the window handle on the desktop system 21. For details, refer to step S416, which will not be repeated here.

[0247] For example, as shown in FIG. 13, the display interface 320 of the computer 300 displays a desktop application window 3003 of the "video player" application of the virtual machine 400. Figure 19

[0248] It can be understood that the desktop application window displayed on the display interface 320 of the computer 300 can be the desktop application window 3003 of the "video player" application of the opened virtual machine 400. Compared with executing the drawing information on the Android system 11 and rendering the image, executing the drawing information on the desktop application window and rendering the image belongs to on-screen rendering, which can reduce one buffer operation and improve the GPU performance during the image rendering operation of the computer 300. The way of executing the drawing information on the desktop application window and rendering the image belongs to on-screen rendering, and the way of executing the drawing information on the Android system 11 and rendering the image belongs to off-screen rendering.

[0249] In some embodiments of the present application, the desktop of the computer 300 can also simultaneously display display windows of multiple Android applications of the virtual machine 400. The creation and image rendering process of each window will be described below with reference to the content of Figure 16 , which will not be repeated here.

[0250] In some other embodiments of the present application, the user can also control the display content (for example, the control of the display interface), the window state, and the window size of the desktop application window of the computer 300 by using the mouse, the keyboard, and the touch, and so on, to realize that the Android application of the virtual machine 400 can not only be displayed on the computer 300, but also be controlled on the computer 300. For details, refer to the process described in Figure 11 or FIG. 13, which will not be repeated here.

[0251] ​It can be understood that the application of the virtual machine 400 is displayed on the desktop application window of the desktop system 21, and the desktop application window of the multiple applications of the virtual machine 400 can be opened and displayed at the same time; and when the desktop application window plays a video and needs to render an image in real time, the drawing information generated by the Android system 11 of the virtual machine 400 can be executed on the desktop system 21, so as to realize the off-screen rendering mode to render the image displayed by the desktop application window, so that one buffer operation can be reduced, and the GPU performance during the image rendering operation of the computer 300 can be improved; and the user can also perform operations such as magnification, reduction and movement on the desktop application window displaying the application of the virtual machine 400, and the window operation is more convenient, and thus the user experience is improved.

[0252] Figure 20 A structural schematic diagram of the electronic device 100 is shown.

[0253] As shown in Figure 20 The electronic device 100 can include a processor 110, a memory 121, a universal serial bus (USB) interface 130, a power management module 141, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a display screen 194, and a camera 193, and the like.

[0254] It can be understood that the structure shown in the embodiment of the application does not constitute a specific limitation on the electronic device 100. In other embodiments of the application, the electronic device 100 can include more or fewer components than the diagram, or combine certain components, or decompose certain components, or different component arrangements. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0255] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units can be independent devices, or can be integrated in one or more processors.

[0256] The USB interface 130 is an interface conforming to the USB standard specification, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. In some embodiments of the present application, the electronic device 100 can be connected in communication with the electronic device 200 through the USB interface 130.

[0257] The power management module 141 is used to connect the battery and the charging management module with the processor 110. The power management module 141 receives the input of the battery and / or the charging management module, and supplies power to the processor 110, the memory 121, the display screen 194, the camera 193, and the wireless communication module 160, etc.

[0258] The wireless communication function of the electronic device 100 can be realized through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor, etc.

[0259] The antenna 1 and the antenna 2 are used to emit and receive electromagnetic wave signals. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example, the antenna 1 can be multiplexed as a diversity antenna of a wireless local area network. In some other embodiments, the antennas can be used in combination with a tuning switch.

[0260] The wireless communication module 160 can provide a wireless communication solution applied on the electronic device 100, including a wireless local area network (WLAN) (such as a wireless fidelity (Wi-Fi) network), Bluetooth (BT), etc. The wireless communication module 160 can be one or more devices integrating at least one communication processing module.

[0261] The wireless communication module 160 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive signals to be sent from the processor 110, perform frequency modulation and amplification on the signals, and convert the signals into electromagnetic wave radiation via the antenna 2. The performance parameters in the above can include the available network bandwidth of the Wi-Fi network, wherein the available network bandwidth of the Wi-Fi network is the network bandwidth available for transmission.

[0262] It can be understood that in some embodiments of the present application, the electronic device 100 and the electronic device 200 can be connected in communication through the wireless communication module 160; the electronic device 100 and the electronic device 200 perform cross-platform distributed display, the electronic device 200 can send the parameters of the application to the electronic device 100, and the electronic device 100 can also send the parameters of the Android application window to the electronic device 200, based on the parameters, the electronic device 200 can create a window to display the content of the application of the electronic device 100.

[0263] The electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs that execute program instructions to generate or change display information.

[0264] The display screen 194 is used to display images, videos, etc. In some embodiments, the launched Android application can be launched on the display screen 194, and the Android application window and the rendered image created by the electronic device 100 can be displayed on the display screen 194.

[0265] The memory 121 can be used to store computer executable program codes, which include instructions. The memory 121 can include a program storage area and a data storage area.

[0266] It can be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation of the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than illustrated, or combine certain components, or split certain components, or different arrangement of components. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0267] Figure 21 is a software structure block diagram of the electronic device 100 of the embodiments of the present application.

[0268] The layered architecture divides the software into several layers, each layer has a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom, the application layer, the application framework layer, the Android runtime and the system library, and the kernel layer.

[0269] The application layer can include a series of application packages.

[0270] Figure 21 As shown, the application package can include camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc.

[0271] The application framework layer provides an application programming interface (API) and programming framework for the applications of the application layer. The application framework layer includes some pre-defined functions.

[0272] Figure 21 As shown, the application framework layer can include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc.

[0273] For example, the window manager can include the phone agent service 101 of the electronic device 100, for managing window programs. The window manager can create a desktop application window according to the parameter calling interface of the Android application. The window manager can also send the parameters and drawing information of the Android application window to the internal phone agent service 101, and the phone agent service 101 sends the parameters and drawing information of the Android application window to other electronic devices.

[0274] The content provider is used to store and obtain data, and make the data accessible to the applications. The data can include videos, images, audios, dialed and received calls, browsing history and bookmarks, phone books, etc.

[0275] The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a short message notification icon can include a view for displaying text and a view for displaying pictures.

[0276] The resource manager provides various resources for the applications, such as localized strings, icons, pictures, layout files, video files, etc.

[0277] In embodiments of the present application, the resource manager can also be used to store Overlay configuration files.

[0278] The notification manager enables the applications to display notification information in the status bar, which can be used to convey notification type messages that can automatically disappear after a short stay without user interaction. For example, the notification manager is used to notify the completion of downloading, message reminders, etc. The notification manager can also be a notification in the form of a chart or a scroll bar text appearing in the top status bar of the system, such as a notification of an application running in the background, and can also be a notification in the form of a dialog window appearing on the screen. For example, prompting text information in the status bar, issuing a prompt sound, the electronic device vibrating, the indicator light flashing, etc.

[0279] The Android Runtime includes a core library and a virtual machine. The Android runtime is responsible for scheduling and managing the Android system.

[0280] The core library contains two parts: one part is the function function that the java language needs to call, and the other part is the core library of Android.

[0281] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the java files of the application layer and the application framework layer into binary files. The virtual machine is used to perform the functions of object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0282] The system library can include multiple functional modules. For example: surface manager, media library, three-dimensional graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL) and the like.

[0283] The surface manager is used to manage the display subsystem, and provides 2D and 3D layer fusion for multiple applications.

[0284] The media library supports multiple commonly used audio, video format playback and recording, and static image files and the like. The media library can support multiple audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG and the like.

[0285] The three-dimensional graphics processing library is used to realize three-dimensional graphics drawing, image rendering, synthesis, and layer processing and the like.

[0286] The 2D graphics engine is a drawing engine for 2D drawing.

[0287] The kernel layer is the layer between hardware and software. The kernel layer at least contains display driver, camera driver, audio driver, sensor driver.

[0288] Figure 22 According to the embodiment of the present application, a structural block diagram of an electronic device 200 or an electronic device 300 is shown.

[0289] Specifically, as shown in Figure 22 , the electronic device 200 or the electronic device 300 can include a processor 201, a memory 202, a wireless communication processing module 203, a power switch 204, a wired LAN communication processing module 205, a display screen 206 and an audio module 207. Among them:

[0290] The processor 201 can be configured to read and execute computer readable instructions. In a specific implementation, the processor 201 can mainly include a controller, an arithmetic unit, and a register. The controller is mainly responsible for instruction decoding and sending control signals for the corresponding operations of the instructions. The arithmetic unit is mainly responsible for performing fixed-point or floating-point arithmetic operations, shift operations, and logic operations, etc., and can also perform address operations and conversions. The register is mainly responsible for storing the register operands and intermediate operation results temporarily stored during the instruction execution process, etc. In a specific implementation, the hardware architecture of the processor 201 can be an application-specific integrated circuit (ASIC) architecture, a MIPS architecture, an ARM architecture, or an NP architecture, etc.

[0291] In some embodiments, the processor 201 can be configured to parse the signals received by the wireless communication processing module 203 or the wired LAN communication processing module 205, such as the Android application window parameters, drawing information, etc. transmitted by the electronic device 100. The processor 201 can create a desktop application window according to the Android application window parameters; the processor 201 can also execute the drawing information to render an image, so that the rendered image can be displayed on the desktop application window.

[0292] The memory 202 is coupled to the processor 201 and is configured to store various software programs and / or groups of instructions. In a specific implementation, the memory 202 can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The memory 202 can store an operating system, such as an embedded operating system uCOS, VxWorks, RTLinux, etc. The memory 202 can also store a communication program that can be used to communicate with the electronic device 100, one or more servers, or additional devices.

[0293] The power switch 204 can be configured to control the power supply to the various components of the electronic device 200 or the electronic device 300.

[0294] The wired LAN communication processing module 205 can be configured to communicate with other devices in the same LAN through the wired LAN, and can also be configured to connect to the WAN through the wired LAN and communicate with devices in the WAN.

[0295] The display screen 206 can be used to display images, videos, etc. The display screen 206 can be a liquid crystal display (LCD), an organic light-emitting diode (OLED) display screen, an active-matrix organic light emitting diode (AMOLED) display screen, a flexible light-emitting diode (FLED) display screen, a quantum dot light emitting diodes (QLED) display screen, etc. It can be understood that the display content of the desktop application window of the Android application can be displayed on the display screen 206.

[0296] The audio module 210 can be used to output audio signals through an audio output interface, so that the electronic device 200 or the electronic device 300 supports audio playback. The audio module can also be used to receive audio data through an audio input interface. The audio module 207 includes, but is not limited to, a microphone, a speaker, a receiver, etc.

[0297] It can be understood that, Figure 22 The schematic structure does not constitute a specific limitation on the electronic device 200 or the electronic device 300. In some other embodiments of the present application, the electronic device 200 or the electronic device 300 can include more or fewer components than the schematic, or combine certain components, or split certain components, or different component arrangements. The components of the schematic can be implemented in hardware, software, or a combination of software and hardware.

[0298] Figure 23 is a software structure block diagram of the electronic device 200 or the electronic device 300 of the embodiment of the present application.

[0299] In some embodiments, the software structure of the electronic device 200 or the electronic device 300 is from top to bottom, respectively, an application layer, a framework layer, a core library layer, and a driver layer.

[0300] As Figure 23As shown, the driver layer can include, but is not limited to, a CPU driver, a GPU driver, a display controller driver, and the like. Above the driver layer is the core library layer, which is the core part of the operating system, and includes, but is not limited to, input / output services, graphics services for implementing CPU, GPU graphics, and the like. The graphics services can include, but are not limited to, a 2D engine, a 3D engine, a frame buffer, OpenGL, EGL, and the like. The framework layer includes, but is not limited to, graphics services, system services, web services, and user services, and the like; the graphics services can include, but are not limited to, a canvas, a view, and the like. The application layer includes, but is not limited to, a desktop, a media player, a browser, and the like.

[0301] In some embodiments, the electronic device 100 performs image rendering of the drawing information displayed on the display screen mainly by drawing each application window into a buffer, and then synthesizing the buffers of the application windows on the display screen, so that there is an additional buffer for screen synthesis. For example, in the application window display of the Android application of the electronic device 100, the display screen displays the synthesized display window, for example, the display screen of the electronic device 100 displays the window 1 of the "Settings" application, the window 2 of the time or battery status bar, and the like. Therefore, the display screen of the electronic device 100 needs a buffer to realize the buffer synthesis of the windows (window 1, window 2, and the like), and the synthesized windows can be displayed on the display screen.

[0302] Compared with the electronic device 100 performing image rendering of the drawing information displayed on the display screen, the electronic device 200 directly presents the drawing result of the window to the display screen of the computer 200. Unlike the display screen of the electronic device 100 displaying the content synthesized by multiple application windows, the display screen of the computer 200 displays the content of only one application window on one desktop application window, and does not need to synthesize multiple application windows, thereby saving the buffer for synthesizing application windows of the display screen.

[0303] In some embodiments, the application program of the desktop agent service 201 transmits the drawing information to the framework layer, and the framework layer applies to the system for drawing the canvas of the layer according to the drawing information, wherein the size of the canvas is determined according to the width and height parameters of the layer to be drawn contained in the drawing information; the system applies to call the layer memory allocation unit to allocate the canvas for drawing the layer; and the system transmits the drawing information obtained by the system to the rendering driver of the core library layer, so as to enable the rendering driver to drive the drawing engine to call OpenGL or EGL to draw the layer based on the drawing information, thereby completing the image drawing of the desktop application window of the Android application. The drawing engine includes the drawing engine of the graphics processing unit GPU, including a 2D drawing engine, a 3D drawing engine, and the like.

[0304] For example, in some embodiments, the electronic device 200 receives the drawing information sent by the electronic device 100, which contains primitive information, primitive coordinates, etc. Among them, the primitive is the most basic material used by graphics software to operate and organize the picture. A picture is composed of primitives, and a primitive is a set of the simplest and most common collection graphics or characters, such as points, line segments, triangles, polygons, etc.

[0305] In the drawings, some structural or methodological features can be shown in a particular arrangement and / or order. However, it should be understood that such a particular arrangement and / or order can not be required. Rather, in some embodiments, these features can be arranged in a manner different from that shown in the illustrative drawings. Additionally, inclusion of a structural or methodological feature in a particular figure does not imply that such feature is required in all embodiments, and in some embodiments, such feature can not be included or can be combined with other features.

[0306] It should be noted that each unit / module mentioned in the device embodiments of the present application is a logical unit / module. In physical terms, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of the logical unit / module itself is not the most important. The combination of the functions implemented by these logical units / modules is the key to solving the technical problems proposed in the present application. In addition, in order to highlight the innovative part of the present application, the above-mentioned device embodiments of the present application do not introduce units / modules that are not closely related to solving the technical problems proposed in the present application. This does not mean that the above-mentioned device embodiments do not have other units / modules.

[0307] It should be noted that in the examples and descriptions of the present patent, relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including one" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0308] While the application has been illustrated and described in connection with certain preferred embodiments, it will be readily apparent to those of ordinary skill in the art that various changes in form and detail can be made without departing from the spirit and scope of the application.

Claims

1. A cross-platform display method for an application, characterized in that, include: The first system platform obtains the first size of the first window of the first application running on the second system platform, and creates a third window corresponding to the first application in the first system platform, wherein the third window is used to display the display interface of the first application running on the second system platform; The first system platform sets a first identifier for the first application, and the first system platform sends the first identifier to the second system platform, wherein the first identifier is the window handle of the third window; The first system platform receives first drawing information from the second system platform, wherein the first drawing information includes a first identifier, wherein the first identifier is used to identify the display interface of the first application displayed by the third window, the first drawing information is drawing information created based on the first identifier, and the first drawing information is created by the second system platform after associating the first identifier with the identifier of the first application running on the second system platform, and the first drawing information is generated after replacing the window handle of the third window with the window handle of the first window during the process of the second system platform creating the first drawing information; The first system platform displays the first display interface of the first application, drawn according to the first drawing information, in the third window based on the first identifier included in the first drawing information. The first system platform obtains the second size of the second window of the second application running on the second system platform, and creates a fourth window corresponding to the second application in the first system platform, wherein the fourth window is used to display the display interface of the second application running on the second system platform.

2. The method according to claim 1, characterized in that, When the first system platform detects that a user has opened a first application or a second application on the first system platform, it obtains the first size or the second size from the second system platform.

3. The method according to claim 1, characterized in that, The second system platform does not display the first window and the second window.

4. The method according to claim 1, characterized in that, Creating a fourth window corresponding to the second application within the first system platform includes: The first system platform sets a second identifier for the second application; The second identifier is used to identify the display interface of the second application in the fourth window.

5. The method according to claim 4, characterized in that, Also includes: The first system platform sends the second identifier to the second system platform.

6. The method according to claim 5, characterized in that, The first system platform displays the second application running on the second system platform in the fourth window in the following manner: The first system platform receives second drawing information from the second system platform, wherein the second drawing information includes the second identifier; The second system platform displays the second display interface of the second application, which is drawn according to the second drawing information, in the fourth window based on the second identifier included in the second drawing information.

7. The method according to claim 1, characterized in that, Also includes: The first system platform detects the user's first operation on the third window on the first system platform; The first system platform generates operation information and sends the operation information to the second system platform, wherein the operation information includes a first identifier; The first system platform receives third drawing information from the second system platform, wherein the third drawing information includes the first identifier, and the third drawing information is generated by the second system platform by performing the operation corresponding to the operation information; The first system platform displays a third display interface in the third window based on the third drawing information, wherein the third display interface is the display interface after the first operation changes to the first display interface.

8. The method according to claim 7, characterized in that, The first operation includes at least one of the following operations: The operation of changing the size of the first display interface; Operations on the controls in the first display interface.

9. The method according to claim 1, characterized in that, The first system platform and the second system platform have different operating systems.

10. The method according to claim 1, characterized in that, The first system platform and the second system platform run on different electronic devices.

11. The method according to claim 1, characterized in that, The first system platform is an operating system running on a virtual machine on an electronic device, and the second system platform is the operating system of the electronic device.

12. A cross-platform display method for an application, characterized in that, include: The first application runs on the second system platform; The first system platform obtains the first size of the first window of the first application from the second system platform; The first system platform creates a third window and a first identifier corresponding to the first application in the first system platform, wherein the first identifier is used to identify that the third window is used to display the display interface of the first application, and the first identifier is the window handle of the third window; The first system platform sends a first identifier to the second system platform; After receiving the first identifier from the first system platform, the second system platform generates first drawing information, wherein the first drawing information includes the first identifier, the first drawing information is drawing information created based on the first identifier, and the first drawing information is created by the second system platform after associating the first identifier with the identifier of the first application running on the second system platform, and the first drawing information is generated after replacing the window handle of the third window with the window handle of the first window during the process of the second system platform creating the first drawing information. The first system platform receives the first drawing information from the second system platform, and displays the first display interface of the first application, drawn according to the first identifier included in the first drawing information, in the third window.

13. The method according to claim 12, characterized in that, When the first system platform detects that a user has opened the first application on the first system platform, the first system platform sends the first application information to the second system platform; The second system platform runs the first application based on the first application information and creates the first window of the first application in the second system platform; The second system platform sends the first size of the first window of the first application to the first system platform.

14. The method according to claim 12, characterized in that, The second system platform does not display the first window.

15. The method according to claim 12, characterized in that, Also includes: The first system platform detects the user's first operation on the third window on the first system platform; The first system platform generates operation information and sends the operation information to the second system platform, wherein the operation information includes the first identifier; The second system platform generates third drawing information by executing the operation corresponding to the operation information, and sends the third drawing information to the first system platform; The first system platform displays a third display interface in the third window based on the third drawing information, wherein the third display interface is the display interface after the first operation is changed, corresponding to the first display interface.

16. The method according to claim 15, characterized in that, The first operation includes at least one of the following operations: The operation of changing the size of the first display interface; Operations on the controls in the first display interface.

17. The method according to claim 12, characterized in that, The first system platform and the second system platform have different operating systems, wherein the first system platform and the second system include at least one of the following: The first system platform and the second system platform run on different electronic devices; The first system platform is an operating system running on a virtual machine on an electronic device, and the second system platform is the operating system of the electronic device.

18. A readable medium, characterized in that, The readable medium stores instructions that, when executed on an electronic device, cause the electronic device to perform the cross-platform display method of the application described in any one of claims 1 to 17.

19. An electronic device, characterized in that, include: Memory, used to store instructions executed by one or more processors of an electronic device, and A processor is one of the processors in an electronic device, used to execute the cross-platform display method of the application described in any one of claims 1 to 17.

Images