COOPERATIVE DISPLAY METHOD, TERMINAL DEVICE, AND COMPUTER-READABLE STORAGE MEDIUM

Abstract

The present application is applicable to the field of terminal technology, in particular to a cooperative display method, a terminal device, and a computer-readable storage medium. In the method, a first display area of ​​a home screen of a first terminal device and a second display area of ​​a virtual screen of a second terminal device can be determined, so that the second terminal device can display the home screen in the first display area and the virtual screen in the second display area. When the display state of the first terminal device changes, only the content display manner of the home screen, the first display area, and the second display area can be dynamically adjusted, and the content display manner of the virtual screen and the relative position of the virtual screen in the second display area do not need to be adjusted. Therefore, the content of the application displayed on the virtual screen does not need to be laid out or refreshed again, which reduces the delay of displaying the content displayed in the second display area, and improves and enhances the user experience. Thus, the usefulness and feasibility are high.

Description

[Technical field]

[0001] This application claims priority to Chinese Patent Application No. 202011596431.0, entitled “COLLABORATIVE DISPLAY METHOD, TERMINAL DEVICE, AND COMPUTER-READABLE STORAGE MEDIUM,” filed with the State Intellectual Property Office of the People's Republic of China on December 28, 2020, the entire contents of which are incorporated herein by reference.

[0002] The present application relates to the field of terminal technology, and in particular to a collaborative display method, a terminal device, and a computer-readable storage medium. [Background technology]

[0003] Collaborative display is a process in which two or more terminal devices collaboratively complete a display service consistently. In the existing collaborative display, multiple display windows may be set on a second terminal device (e.g., a tablet computer) to display multiple applications of a first terminal device (e.g., a mobile phone). However, when the display state of the first terminal device changes (e.g., from a portrait mode display state to a landscape mode display state), the display windows of the second terminal device also change correspondingly (e.g., from a portrait window to a landscape window). Therefore, the content displayed in each display window needs to be laid out and refreshed again. As a result, a delay occurs in displaying the content on the second terminal device, which affects the user experience. Summary of the Invention

[0004] The embodiments of the present application provide a collaborative display method, a terminal device, and a computer-readable storage medium, so that in collaborative display, when the display state of a first terminal device changes, the number of times content is laid out and refreshed again can be reduced, thereby reducing the delay in displaying content on a second terminal device and improving the user experience.

[0005] According to a first aspect, an embodiment of the present application provides a cooperative display method applied to a first terminal device, the method comprising: determining a first display area of ​​a first home screen of the first terminal device on the second terminal device and a second display area of ​​a virtual screen of the first terminal device on the second terminal device based on a first display state of the first terminal device; projecting the first home screen on the second terminal device, and transmitting first position information of the first display area to the second terminal device, where the first position information is used by the second terminal device to display the first home screen within the first display area; projecting the virtual screen onto the second terminal device, and transmitting second position information of the second display area to the second terminal device, where the second position information is used by the second terminal device to display the virtual screen within the second display area; When it is detected that the first terminal device is adjusted from the first display state to the second display state, adjusting a content display manner of the first home screen according to the second display state to obtain a second home screen, and rearranging the first display area and the second display area according to the second display state; and projecting the second home screen and the virtual screen on the second terminal device, and transmitting third position information of the reorganized first display area and fourth position information of the reorganized second display area to the second terminal device, where the third position information is used by the second terminal device to display the second home screen within the reorganized first display area, and the fourth position information is used by the second terminal device to display the virtual screen within the reorganized second display area. It may comprise:

[0006] In the cooperative display method, the first terminal device may determine a first display area of ​​the home screen of the first terminal device in the second terminal device and a second display area of ​​the virtual screen in the second terminal device based on the display state of the first terminal device, so that the second terminal device can display the home screen in the first display area and the virtual screen in the second display area. When the display state of the first terminal device changes, the first terminal device may only dynamically adjust the content display manner of the home screen, the first display area, and the second display area, and does not need to adjust the content display manner of the virtual screen and the relative position of the virtual screen in the second display area. Therefore, the content of the application displayed on the virtual screen does not need to be laid out or refreshed again, reducing the delay in displaying the content displayed in the second display area and improving the user experience. In this way, the usefulness and feasibility are high.

[0007] Optionally, prior to the step of projecting the virtual screen onto the second terminal device, the method further comprises: creating a virtual screen based on the first home screen when a start command for the application and sent from the second terminal device is obtained, and displaying an application interface of the application on the virtual screen, where the virtual screen is not displayed on the first terminal device. It may comprise:

[0008] In the collaborative display method provided in this implementation, when a user taps an application on a first home screen of a second terminal device, the second terminal device may send an instruction to open the application to the first terminal device. In this case, the first terminal device may create a virtual screen, display an application interface of the application on the virtual screen, and project the virtual screen onto a second display area of ​​the second terminal device, so that the second terminal device can simultaneously open multiple windows of the first terminal device to facilitate user browsing and improve user experience.

[0009] In a possible implementation form of the first aspect, the step of determining a first display area of ​​a first home screen of the first terminal device on the second terminal device and a second display area of ​​a virtual screen of the first terminal device on the second terminal device based on a first display state of the first terminal device includes: obtaining interface information of a second terminal device; and determining a first display area and a second display area based on a first display state of the first terminal device and interface information of the second terminal device; may have:

[0010] In the collaborative display method provided in this implementation, the first terminal device can accurately determine the first display area and the second display area based on the interface information of the second terminal device and the first display state of the first terminal device, to ensure that the second terminal device can display more windows of the first terminal device, improve the utilization of the display interface of the second terminal device, and improve the user's browsing experience.

[0011] For example, the interface information includes an interface height and an interface width, and the step of determining the first display area and the second display area based on the first display state of the first terminal device and the interface information of the second terminal device includes: determining, when the first display state is a portrait mode display state, that a first region height of the first display region is equal to a second region height of the second display region, and that the first region height and the second region height are equal to an interface height; and determining that a first region width of the first display region is the same as a screen width of the first home screen; may have:

[0012] For example, the interface information includes an interface height and an interface width, and the step of determining the first display area and the second display area based on the first display state of the first terminal device and the interface information of the second terminal device includes: determining, when the first display state is a landscape mode display state, that a first region width of the first display region is equal to a second region width of the second display region, and that the first region width and the second region width are equal to an interface width; and determining that a first region height of the first display region is the same as a screen height of the first home screen; may have:

[0013] In another possible implementation form of the first aspect, the first terminal device is a terminal device having a foldable display, and the interface information includes an interface height and an interface width; and The step of determining the first display area and the second display area based on the first display state of the first terminal device and the interface information of the second terminal device includes: determining a third display state of the second terminal device based on an interface height and an interface width when the first display state is a display state of an unfolded screen; and determining a first display region and a second display region based on the third display state and the interface information; may have:

[0014] In the cooperative display method provided in this implementation, when a first terminal device having a foldable display is in a display state of an unfolded screen, the home screen of the first terminal device needs to be correspondingly enlarged, and the area of ​​the first display region also needs to be correspondingly enlarged. The first terminal device can further determine the first display region and the second display region by referring to the display state and the interface information of the second terminal device, so as to fully use the display interface of the second terminal device, and improve the utilization of the display interface of the second terminal device.

[0015] Optionally, determining the first and second display regions based on the third display state and the interface information further comprises: determining, when the third display state is a landscape mode display state, that a first region height of the first display region is equal to a second region height of the second display region, and that the first region height and the second region height are equal to an interface height; and Determining that a first region width of the first display region is equal to a screen width of the first home screen or a screen height of the first home screen. may have:

[0016] Optionally, determining the first and second display regions based on the third display state and the interface information further comprises: determining, when the third display state is a portrait mode display state, that a first region width of the first display region is equal to a second region width of the second display region, and that the first region width and the second region width are equal to an interface width; and determining that a first region height of the first display region is equal to a screen height of the first home screen or a screen width of the first home screen; may have:

[0017] According to a second aspect, an embodiment of the present application provides a cooperative display method applied to a second terminal device, the method comprising: obtaining a first display state of the first terminal device, and determining, based on the first display state, a first display area of ​​a first home screen of the first terminal device on the second terminal device and a second display area of ​​a virtual screen of the first terminal device on the second terminal device; obtaining a first home screen projected by a first terminal device, and displaying the first home screen within a first display area; obtaining a virtual screen projected by a first terminal device, and displaying the virtual screen within a second display area; when it is determined that the first terminal device has been adjusted from the first display state to the second display state, rearranging the first display area and the second display area based on the second display state; and Obtaining a second home screen and a virtual screen projected by a first terminal device, displaying the second home screen within the rearranged first display area, and displaying the virtual screen within the rearranged second display area, where a content display manner of the second home screen corresponds to a second display state. It may comprise:

[0018] In the cooperative display method, the second terminal device may determine a first display area of ​​the home screen of the first terminal device in the second terminal device and a second display area of ​​the virtual screen in the second terminal device based on the display state of the first terminal device, and may display the home screen in the first display area and display the virtual screen in the second display area. When the display state of the first terminal device changes, the second terminal device may only dynamically adjust the first display area and the second display area, and does not need to adjust the content display manner of the virtual screen and the relative position of the virtual screen in the second display area. Therefore, the content of the application displayed on the virtual screen does not need to be laid out or refreshed again, which reduces the delay of displaying the content displayed in the second display area and improves the user experience. Thus, it is highly useful and feasible.

[0019] Optionally, prior to the step of acquiring the virtual screen projected by the first terminal device, the method further comprises: sending a start command to the first terminal device in response to a launch operation performed on the application on the first home screen, where the start command is used by the first terminal device to start the application and display an application interface of the application on the virtual screen; It may comprise:

[0020] In a possible implementation form of the second aspect, the step of determining a first display area of ​​a first home screen of the first terminal device on the second terminal device and a second display area of ​​a virtual screen of the first terminal device on the second terminal device based on the first display state includes: obtaining interface information of a second terminal device; and determining a first display region and a second display region based on the first display state and interface information of the second terminal device; may have:

[0021] For example, the interface information includes an interface height and an interface width, and the step of determining the first display region and the second display region based on the first display state and the interface information of the second terminal device includes: determining, when the first display state is a portrait mode display state, that a first region height of the first display region is equal to a second region height of the second display region, and that the first region height and the second region height are equal to an interface height; and determining that a first region width of the first display region is the same as a screen width of the first home screen; may have:

[0022] For example, the interface information includes an interface height and an interface width, and the step of determining the first display region and the second display region based on the first display state and the interface information of the second terminal device includes: determining, when the first display state is a landscape mode display state, that a first region width of the first display region is equal to a second region width of the second display region, and that the first region width and the second region width are equal to an interface width; and determining that a first region height of the first display region is the same as a screen height of the first home screen; may have:

[0023] In another possible implementation form of the second aspect, the first terminal device is a terminal device having a foldable display, and the interface information includes an interface height and an interface width; and The step of determining the first display region and the second display region based on the first display state and the interface information of the second terminal device includes: determining a third display state of the second terminal device based on an interface height and an interface width when the first display state is a display state of an unfolded screen; and determining a first display region and a second display region based on the third display state and the interface information; may have:

[0024] Optionally, determining the first and second display regions based on the third display state and the interface information further comprises: determining, when the third display state is a landscape mode display state, that a first region height of the first display region is equal to a second region height of the second display region, and that the first region height and the second region height are equal to an interface height; and Determining that a first region width of the first display region is equal to a screen width of the first home screen or a screen height of the first home screen. may have:

[0025] Optionally, determining the first and second display regions based on the third display state and the interface information further comprises: determining, when the third display state is a portrait mode display state, that a first region width of the first display region is equal to a second region width of the second display region, and that the first region width and the second region width are equal to an interface width; and determining that a first region height of the first display region is equal to a screen height of the first home screen or a screen width of the first home screen; may have:

[0026] According to a third aspect, an embodiment of the present application provides a cooperative display device adapted for a first terminal device, the device comprising: a display area determination module configured to determine, based on a first display state of the first terminal device, a first display area of ​​a first home screen of the first terminal device on the second terminal device and a second display area of ​​a virtual screen of the first terminal device on the second terminal device; a homescreen projection module configured to project a first homescreen onto a second terminal device and transmit first position information of the first display area to the second terminal device, where the first position information is used by the second terminal device to display the first homescreen within the first display area; a virtual screen projection module configured to project a virtual screen onto the second terminal device and transmit second position information of the second display area to the second terminal device, where the second position information is used by the second terminal device to display the virtual screen within the second display area; a display area reorganization module configured to, when it is detected that the first terminal device is adjusted from the first display state to the second display state, adjust a content display manner of the first home screen based on the second display state to obtain a second home screen, and rearrange the first display area and the second display area based on the second display state; and a cooperative display module configured to project a second home screen and a virtual screen onto a second terminal device and to transmit third position information of the reorganized first display area and fourth position information of the reorganized second display area to the second terminal device, where the third position information is used by the second terminal device to display the second home screen within the reorganized first display area and the fourth position information is used by the second terminal device to display the virtual screen within the reorganized second display area. It may comprise:

[0027] Optionally, the apparatus may further comprise: a virtual screen creation module configured to create a virtual screen based on the first home screen when a start command for the application and sent from the second terminal device is obtained, and display an application interface of the application on the virtual screen, where the virtual screen is not displayed on the first terminal device; It may comprise:

[0028] In a possible implementation form of the third aspect, the display area determination module: an interface information obtaining unit configured to obtain interface information of a second terminal device; and a display area determining unit configured to determine a first display area and a second display area based on a first display state of the first terminal device and interface information of the second terminal device; may have:

[0029] For example, the interface information includes an interface height and an interface width; and The display area determination unit is configured to determine, when the first display state is a portrait mode display state, that a first area height of the first display area is the same as a second area height of the second display area, and that the first area height and the second area height are the same as an interface height; and determine that a first area width of the first display area is the same as a screen width of the first home screen.

[0030] For example, the interface information includes an interface height and an interface width; and The display area determination unit is further configured to determine, when the first display state is a landscape mode display state, that a first area width of the first display area is the same as a second area width of the second display area, and that the first area width and the second area width are the same as an interface width; and determine that a first area height of the first display area is the same as a screen height of the first home screen.

[0031] In another possible implementation form of the third aspect, the first terminal device is a terminal device having a foldable display, and the interface information includes an interface height and an interface width; and The display area determination unit includes: a display state determination subunit configured to determine a third display state of the second terminal device based on an interface height and an interface width when the first display state is a display state of an unfolded screen; and a display area determining subunit configured to determine the first display area and the second display area based on the third display state and the interface information; may have:

[0032] Optionally, the display area determination subunit is configured to determine, when the third display state is a landscape mode display state, that the first area height of the first display area is the same as the second area height of the second display area, and that the first area height and the second area height are the same as the interface height; and determine that the first area width of the first display area is the same as the screen height of the first terminal device or the screen width of the first terminal device.

[0033] Optionally, the display area determination subunit is further configured to determine, when the third display state is a portrait mode display state, that the first area width of the first display area is the same as the second area width of the second display area, and that the first area width and the second area width are the same as the interface width; and determine that the first area height of the first display area is the same as the screen height of the first terminal device or the screen width of the first terminal device.

[0034] According to a fourth aspect, an embodiment of the present application provides a cooperative display device adapted for a second terminal device, the device comprising: a display area determination module configured to obtain a first display state of the first terminal device, and determine, based on the first display state, a first display area of ​​a first home screen of the first terminal device on the second terminal device and a second display area of ​​a virtual screen of the first terminal device on the second terminal device; a homescreen acquisition module configured to acquire a first homescreen projected by the first terminal device and display the first homescreen within the first display area; a virtual screen acquisition module configured to acquire a virtual screen projected by the first terminal device and display the virtual screen within the second display area; a display area reorganization module configured to, when it is determined that the first terminal device is adjusted from the first display state to the second display state, reorganize the first display area and the second display area based on the second display state; and A cooperative display module configured to obtain a second home screen and a virtual screen projected by a first terminal device, display the second home screen within the reorganized first display area, and display the virtual screen within the reorganized second display area, where a content display manner of the second home screen corresponds to a second display state. It may comprise:

[0035] Optionally, the apparatus further comprises: a start command sending module configured to send a start command to the first terminal device to start the application in response to a launch operation performed on the application on the first home screen, where the start command is used by the first terminal device to start the application and display an application interface of the application on the virtual screen; It may comprise:

[0036] According to a fifth aspect, an embodiment of the present application provides a terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, such that when the processor executes the computer program, the terminal device is capable of implementing a method according to any executable implementation of the first aspect.

[0037] According to a sixth aspect, an embodiment of the present application provides a computer readable storage medium storing a computer program which, when executed by a computer, is capable of causing the computer to implement the method according to any executable implementation of the first aspect.

[0038] According to a seventh aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, is capable of causing the terminal device to perform a method according to any executable implementation of the first aspect. [Brief description of the drawings]

[0039] [Figure 1] 1 is a schematic diagram of the structure of a terminal device to which a collaborative display method according to an embodiment of the present application can be applied;

[0040] [Diagram 2] FIG. 1 is a schematic diagram of a software architecture to which a collaborative display method according to an embodiment of the present application can be applied.

[0041] [Diagram 3] FIG. 1 is a schematic diagram of a system according to an embodiment of the present application.

[0042] [Figure 4] FIG. 2 is a schematic diagram of establishing a connection between a first terminal device and a second terminal device. [Figure 5-1] FIG. 2 is a schematic diagram of establishing a connection between a first terminal device and a second terminal device. [Figure 5-2] FIG. 2 is a schematic diagram of establishing a connection between a first terminal device and a second terminal device.

[0043] [Figure 6] 1 is an exemplary diagram 1 for determining a first display area and a second display area.

[0044] [Figure 7] 2 is an exemplary diagram 2 for determining a first display area and a second display area.

[0045] [Figure 8] 3 is an exemplary diagram 3 for determining a first display area and a second display area.

[0046] [Figure 9] 4 is an exemplary diagram 4 for determining a first display area and a second display area.

[0047] [Figure 10] 1 is an exemplary diagram 1 of rearranging a first display area and a second display area.

[0048] [Figure 11] 2 is an exemplary diagram 2 of rearranging the first and second display areas;

[0049] [Figure 12] 3 is an exemplary diagram 3 of rearranging the first and second display areas;

[0050] [Figure 13] 4 is an exemplary diagram 4 of rearranging the first and second display areas;

[0051] [Figure 14] 5 is an exemplary diagram 5 of rearranging the first and second display areas.

[0052] [Figure 15-1] FIG. 1 is a schematic diagram of an application scenario according to an embodiment of the present application. [Figure 15-2] FIG. 1 is a schematic diagram of an application scenario according to an embodiment of the present application. [Figure 16-1] FIG. 1 is a schematic diagram of an application scenario according to an embodiment of the present application. [Figure 16-2] FIG. 1 is a schematic diagram of an application scenario according to an embodiment of the present application. [Figure 17] FIG. 1 is a schematic diagram of an application scenario according to an embodiment of the present application.

[0053] [Figure 18] FIG. 2 is a schematic diagram of an application scenario according to an embodiment of the present application. [Figure 19] FIG. 2 is a schematic diagram of an application scenario according to an embodiment of the present application. [Figure 20] FIG. 2 is a schematic diagram of an application scenario according to an embodiment of the present application. [Figure 21] FIG. 2 is a schematic diagram of an application scenario according to an embodiment of the present application.

[0054] [Figure 22] 1 is a schematic flowchart of a cooperative display method according to an embodiment of the present application; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0055] The term "include" as used in this specification and the appended claims should be understood to indicate the presence of described features, wholes, steps, operations, elements, and / or components without excluding the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or combinations thereof.

[0056] It is to be further understood that the term "and / or" as used in the specification and appended claims refers to and includes any and all possible combinations of one or more of the associated listed elements.

[0057] As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if determined to" or "if (a described condition or event) is detected" may be interpreted to mean "upon determined to" or "in response to determining" or "upon (a described condition or event) is detected" or "in response to detecting (a described condition or event)," depending on the context.

[0058] Furthermore, in the description of the present specification and the appended claims, the terms "first," "second," "third," etc. are used only to distinguish the descriptions and cannot be understood as indicating or suggesting relative importance.

[0059] References to "one embodiment" or "some embodiments" or the like described in the specification of this application indicate that one or more embodiments of the application include the particular features, structures, or characteristics described with reference to the embodiment. Thus, statements such as "in one embodiment," "in some embodiments," "in some other embodiments," and "in other embodiments" appearing in various places in this specification are not necessarily intended to refer to the same embodiment. Instead, such statements mean "one or more, but not all, of the embodiments," unless otherwise specifically emphasized in a different manner. Unless otherwise specifically emphasized in a different manner, the terms "include," "comprise," "have," and variations of these terms all mean "including, but not limited to."

[0060] Furthermore, "plurality" as used herein in the embodiments should be interpreted as two or more.

[0061] The steps in the cooperative display method provided in the embodiments of the present application are merely examples. Not all steps are mandatory, nor is the content in each information or message mandatory. In the use process, steps or content may be added or removed according to requirements. In different embodiments, cross-references are made to steps or messages that have the same steps or the same functions in the embodiments of the present application.

[0062] The service scenarios described in the embodiments of the present application are intended to more clearly explain the technical solutions in the embodiments of the present application, and do not limit the technical solutions provided in the embodiments of the present application. Those skilled in the art can understand that with the development of network architecture and the emergence of new service scenarios, the technical solutions provided in the embodiments of the present application can also be applied to similar technical problems.

[0063] As described in the background section, in the existing collaborative display, when multiple display windows are set on a second terminal device (e.g., a tablet computer) to display multiple applications of a first terminal device (e.g., a mobile phone), if the display state of the first terminal device changes (e.g., from a portrait mode display state to a landscape mode display state), the display window of the second terminal device also changes correspondingly (e.g., from a portrait mode window to a landscape mode window). Therefore, the content displayed in each display window needs to be laid out and refreshed again. As a result, a delay occurs in displaying the content on the second terminal device, which affects the user experience.

[0064] In order to solve the above-mentioned problem, an embodiment of the present application provides a collaborative display method. During collaborative display, the first terminal device may create a virtual screen independent of the home screen, and send the display information of the home screen and the virtual screen to the second terminal device separately. The second terminal device may display the home screen in the first display area, and display the virtual screen in the second display area. When the display state of the first terminal device changes, only the content display manner of the home screen, the first display area, and the second display area may be dynamically adjusted, and the content display manner of the virtual screen and the relative position of the virtual screen in the second display area do not need to be adjusted. Therefore, the content of the application displayed on the virtual screen does not need to be laid out or refreshed again, which reduces the delay of displaying the content displayed in the second display area and improves the user experience. Thus, it is highly useful and feasible.

[0065] The cooperative display method provided in this embodiment of the present application can be applied to a terminal device having a display, such as a mobile phone, a tablet computer, a wearable device, an in-vehicle device, an augmented reality (AR) / virtual reality (VR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a desktop computer, or a smart TV.

[0066] 1 is a schematic diagram of the structure of a terminal device 100. The terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) port 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headset jack 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identity module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyro sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, an optical proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, and a bone conduction sensor 180M, etc.

[0067] It may be understood that the structure shown in this embodiment of the present application is not specifically limited to the terminal device 100. In some other embodiments of the present application, the terminal device 100 may include more or fewer components than those shown in the figures, or some components may be combined, or some components may be divided, or a different component arrangement may be used. The components shown in the figures may be implemented by hardware, software, or a combination of software and hardware.

[0068] Processor 110 may include one or more processing units. For example, processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. The different processing units may be separate components or may be integrated into one or more processors.

[0069] The controller can generate an operation control signal according to the instruction operation code and the time series signal to complete the control of reading the instruction and executing the instruction.

[0070] A memory may further be disposed within the processor 110 and configured to store instructions and data. In some embodiments, the memory in the processor 110 is a cache. The memory may store instructions or data that have just been used or that are used periodically by the processor 110. If the processor 110 needs to use the instructions or data again, the processor 110 may retrieve the instructions or data directly from the memory. This improves system efficiency by avoiding repeated accesses and reducing the waiting time of the processor 110.

[0071] In some embodiments, the processor 110 may include one or more interfaces, which may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) port, etc.

[0072] The I2C interface is a bidirectional synchronous serial bus and includes one serial data line (SDA) and one serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be individually coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K through an I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface to implement the touch function of the terminal device 100.

[0073] The I2S interface may be configured to perform audio communication. In some embodiments, the processor 110 may include multiple groups of I2S buses. The processor 110 may be coupled to the audio module 170 through the I2S bus and implement communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 may transfer an audio signal to the wireless communication module 160 through the I2S interface and implement a function of answering an incoming call through a Bluetooth headset.

[0074] The PCM interface may also be used for audio communication, analog signal sampling, quantization, and encoding. In some embodiments, audio module 170 may be coupled to wireless communication module 160 through a PCM bus interface. In some embodiments, audio module 170 may alternatively transmit audio signals to wireless communication module 160 through the PCM interface and implement the functionality of answering an incoming call via a Bluetooth headset. Both the I2S interface and the PCM interface may be used for audio communication.

[0075] The UART interface is a general-purpose serial data bus and is used for asynchronous communication. The bus may be a bidirectional communication bus. The bus converts data to be transmitted between serial and parallel communication. In some embodiments, the UART interface is generally configured to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with a Bluetooth module in the wireless communication module 160 through the UART interface to implement a Bluetooth function. In some embodiments, the audio module 170 may transmit an audio signal to the wireless communication module 160 through the UART interface to implement a function of playing music by using a Bluetooth headset.

[0076] The MIPI interface may be configured to connect the processor 110 and a peripheral component such as a display 194 or a camera 193. The MIPI interface includes a camera serial interface (CSI) and a display serial interface (DSI), etc. In some embodiments, the processor 110 communicates with the camera 193 through the CSI to implement a photographing function of the terminal device 100. The processor 110 communicates with the display 194 through the DSI interface to implement a display function of the terminal device 100.

[0077] The GPIO interface may be configured in software. The GPIO interface may be configured as a control signal or a data signal. In some embodiments, the GPIO interface may be configured to connect the processor 110 to the camera 193, the display 194, the wireless communication module 160, the audio module 170, or the sensor module 180, etc. The GPIO interface may alternatively be configured as an I2C interface, an I2S interface, a UART interface, or a MIPI interface, etc.

[0078] The USB port 130 is an interface conforming to the USB standard specification, and may be, for example, a mini USB port, a micro USB port, or a USB Type C port. The USB port 130 may be configured to connect to a charger to charge the terminal device 100, or to transmit data between the terminal device 100 and a peripheral device, or to connect to a headset and play audio by using the headset. The interface may be further configured to connect to another terminal device, for example, an AR device.

[0079] It can be understood that the interface connection relationship between modules shown in this embodiment of the present application is merely an example for explanation, and is not a limitation on the structure of the terminal device 100. In some other embodiments of the present application, different interface connection methods or combinations of multiple interface connection methods in the above embodiments can be used for the terminal device 100 instead.

[0080] The charging management module 140 is configured to receive a charging input from a charger. The charger may be a wireless charger or a wired charger. In some embodiments of wired charging, the charging management module 140 may receive the charging input from the wired charger through the USB port 130. In some embodiments of wireless charging, the charging management module 140 may receive the wireless charging input through a wireless charging coil of the terminal device 100. While charging the battery 142, the charging management module 140 may further provide power to the terminal device by using the power management module 141.

[0081] The power management module 141 is configured to connect to the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives inputs from the battery 142 and / or the charging management module 140 and provides power to the processor 110, the internal memory 121, the display 194, the camera 193, the wireless communication module 160, etc. The power management module 141 may be configured to monitor parameters such as battery capacity, battery cycle count, and battery health (electrical leakage or impedance). In some other embodiments, the power management module 141 may alternatively be located in the processor 110. In some other embodiments, the power management module 141 and the charging management module 140 may alternatively be located in the same device.

[0082] The wireless communication function of the terminal device 100 may be implemented through an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, a modem processor, a baseband processor, and the like.

[0083] Antenna 1 and Antenna 2 are configured to transmit and receive electromagnetic wave signals. Each antenna in the terminal device 100 may be configured to cover one or more communication frequency bands. Different antennas may be multiplexed to improve antenna utilization. For example, Antenna 1 may be multiplexed as a diversity antenna in a wireless local area network. In some other embodiments, the antennas may be used in combination with tuning switches.

[0084] The mobile communication module 150 may be applied to the terminal device 100 and provide wireless communication solutions including 2G / 3G / 4G / 5G. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 may receive electromagnetic waves through the antenna 1, perform processing such as filtering and amplification on the received electromagnetic waves, and transmit the received electromagnetic waves to the modem processor for demodulation. The mobile communication module 150 may further amplify the signal modulated by the modem processor and convert the signal to electromagnetic waves for emission through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be disposed in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 may be disposed in the same device as at least some of the modules in the processor 110.

[0085] The modem processor may include a modulator and a demodulator. The modulator is configured to modulate a low-frequency baseband signal to be transmitted into a medium or high-frequency signal. The demodulator is configured to demodulate a received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transfers the low-frequency baseband signal obtained through demodulation to the baseband processor for processing. The low-frequency baseband signal is processed by the baseband processor and then transmitted to the application processor. The application processor outputs a sound signal by using an audio device (such as but not limited to the speaker 170A and the receiver 170B) or displays an image or video on the display 194. In some embodiments, the modem processor may be an independent component. In some other embodiments, the modem processor may be independent from the processor 110 and be located in the same device as the mobile communication module 150 or another functional module.

[0086] The wireless communication module 160 may be applied to the terminal device 100 and provide wireless communication solutions including wireless local area network (WLAN) (e.g. wireless fidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC) technology, or infrared (IR) technology, etc. The wireless communication module 160 may be one or more components integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signal, and sends the processed signal to the processor 110. The wireless communication module 160 may further receive signals to be transmitted from the processor 110, perform frequency modulation and amplification on the signals, and convert the signals into electromagnetic waves for radiation through the antenna 2.

[0087] In some embodiments, antenna 1 and mobile communication module 150 in terminal device 100 are coupled, and antenna 2 and wireless communication module 160 in terminal device 100 are coupled, so that terminal device 100 can communicate with a network and another device by using a wireless communication technology. The wireless communication technology can include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS), and / or a satellite-based augmentation system (SBAS).

[0088] The terminal device 100 implements a display function by using a GPU, a display 194, an application processor, and the like. The GPU is a microprocessor for image processing and is connected to the display 194 and the application processor. The GPU is configured to perform mathematical and geometric calculations and render images. The processor 110 may include one or more GPUs and execute program instructions to generate or modify display information.

[0089] The display 194 is configured to display images, videos, and the like. The display 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a mini-LED, a micro-LED, a micro-OLED, or a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0090] The terminal device 100 may implement a photographing function by using an ISP, a camera 193, a video codec, a GPU, a display 194, an application processor, and the like.

[0091] The ISP is configured to process data fed back by the camera 193. For example, during shooting, the shutter is pressed and light is propagated through the lens to the camera's photosensitive elements. The light signal is converted to an electrical signal, which the camera's photosensitive elements transmit to the ISP for processing to convert the electrical signal into a visible image. The ISP may further perform algorithmic optimization on image noise, brightness, and color. The ISP may further optimize parameters such as exposure and color temperature for the shooting scenario. In some embodiments, the ISP may be located within the camera 193.

[0092] The camera 193 is configured to capture still images or video. An optical image of an object is generated through a lens and projected onto a photosensitive element. The photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, which is then transmitted to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard format, such as RGB or YUV. In some embodiments, the terminal device 100 may include one or N cameras 193, where N is a positive integer greater than one.

[0093] The digital signal processor is configured to process digital signals and may process other digital signals in addition to the digital image signal. For example, if the terminal device 100 selects a frequency, the digital signal processor is configured to perform a Fourier transform on the frequency energy.

[0094] A video codec is configured to compress or decompress digital video. The terminal device 100 may support one or more video codecs. Thus, the terminal device 100 may play or record video in multiple coding formats, for example, Moving Picture Experts Group (MPEG)-1, MPEG-2, MPEG-3, and MPEG-4.

[0095] The NPU is a neural-network (NN) computing processor that quickly processes input information by referring to the structure of biological neural networks, for example, by referring to the transmission mode between human brain neurons, and can also perform self-learning continuously. The NPU can be used to implement intelligent cognition of the terminal device 100, such as image recognition, face recognition, speech recognition, and text understanding.

[0096] The external memory interface 120 may be configured to connect to an external storage card, for example, a micro SD card, to expand the storage capabilities of the terminal device 100. The external memory card communicates with the processor 110 via the external memory interface 120 to implement data storage functions. For example, files such as music and videos are stored on the external storage card.

[0097] The internal memory 121 may be configured to store computer executable program code, the executable program code including instructions. The internal memory 121 may include a program storage area and a data storage area. The program storage area may store an operating system and applications necessary for at least one function (e.g., audio playback function or image playback function), etc. The data storage area may store data (e.g., audio data or an address book) created in a process using the terminal device 100, etc. Furthermore, the internal memory 121 may include a high-speed random access memory, and may further include a non-volatile memory, such as at least one magnetic disk storage device, a flash storage device, or a universal flash storage (UFS). The processor 110 executes instructions stored in the internal memory 121 and / or instructions stored in a memory located in the processor to perform various functional applications and data processing of the terminal device 100.

[0098] The terminal device 100 may implement audio functions such as music playback or recording, for example, by using an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headset jack 170D, an application processor, and the like.

[0099] Audio module 170 is configured to convert digital audio information into analog audio signals for output, and is further configured to convert analog audio input to digital audio signals. Audio module 170 may be further configured to encode and decode audio signals. In some embodiments, audio module 170 may be located in processor 110, or some functional modules in audio module 170 are located in processor 110.

[0100] The speaker 170A, also referred to as a "horn", is configured to convert electrical audio signals into sound signals. The terminal device 100 may be configured to listen to music or answer incoming calls in a hands-free mode by using the speaker 170A.

[0101] The receiver 170B, also referred to as an "earpiece," is configured to convert an electrical audio signal into a sound signal. When answering an incoming call or receiving voice information by using the terminal device 100, the receiver 170B may be brought close to a person's ear to listen to the sound.

[0102] The microphone 170C, also referred to as a "mike" or "mic", is configured to convert a sound signal into an electrical signal. When a user makes a call or transmits audio information, the user may emit a sound near the microphone 170C through the user's mouth to input the sound signal into the microphone 170C. At least one microphone 170C may be disposed in the terminal device 100. In some other embodiments, two microphones 170C may be disposed in the terminal device 100 to implement a noise reduction function in addition to collecting sound signals. In some other embodiments, three, four, or more microphones 170C may alternatively be disposed in the terminal device 100 to collect sound signals, implement noise reduction, identify sound sources, implement directional recording functions, and the like.

[0103] Headset jack 170D is configured to connect to a wired headset, and may be a USB port 130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0104] The pressure sensor 180A is configured to detect a pressure signal and may convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A may be disposed on the display 194. There are multiple types of pressure sensors 180A, such as a resistive pressure sensor, an inductive pressure sensor, and a capacitive pressure sensor. The capacitive pressure sensor may include at least two parallel plates made of a conductive material. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The terminal device 100 determines the strength of the pressure based on the change in capacitance. When a touch operation is performed on the display 194, the terminal device 100 detects the strength of the touch operation through the pressure sensor 180A. The terminal device 100 may also calculate a touch position based on the detection signal of the pressure sensor 180A. In some embodiments, touch operations performed at the same touch position but with different touch operation strengths may correspond to different operation commands. For example, when a touch operation with a touch operation strength lower than a first pressure threshold is performed on a message application icon, a command to view an SMS message is executed. When a touch having a touch strength greater than or equal to a first pressure threshold is performed on the message application icon, a command is executed to compose a new SMS message.

[0105] The gyro sensor 180B may be configured to determine the movement attitude of the terminal device 100. In some embodiments, the angular velocity of the terminal device 100 around three axes (i.e., x, y, z axes) may be determined by using the gyro sensor 180B. The gyro sensor 180B may be configured to implement image stabilization during shooting. For example, when the shutter is pressed, the gyro sensor 180B detects the angle at which the terminal device 100 shakes, and calculates the distance that the lens module needs to compensate based on the angle, allowing the lens to offset the shaking of the terminal device 100 through a reverse movement, thereby implementing image stabilization. The gyro sensor 180B may also be used in navigation scenarios and motion sensing game scenarios.

[0106] The air pressure sensor 180C is configured to measure air pressure. In some embodiments, the terminal device 100 uses the air pressure values ​​measured by the air pressure sensor 180C to calculate altitude and assist in positioning and navigation.

[0107] The magnetic sensor 180D includes a Hall sensor. The terminal device 100 can detect the opening and closing of the flip leather case by using the magnetic sensor 180D. In some embodiments, when the terminal device 100 is a flip phone, the terminal device 100 can detect the opening and closing of the flip cover based on the magnetic sensor 180D, and further set a function such as automatic unlocking of the flip cover based on the detected opening and closing state of the flip leather case or the detected opening and closing state of the flip cover.

[0108] The acceleration sensor 180E may detect the value of acceleration of the terminal device 100 in various directions (usually three axes). When the terminal device 100 is stationary, the magnitude and direction of gravity may be detected. The acceleration sensor 180E may be further configured to recognize the orientation of the terminal device, and is applied to applications such as switching between landscape mode and portrait mode or a pedometer.

[0109] The distance sensor 180F is configured to measure distance. The terminal device 100 may measure distance by infrared or laser. In some embodiments, in a shooting scenario, the terminal device 100 may measure distance by using the distance sensor 180F to implement quick focusing.

[0110] The optical proximity sensor 180G may include, for example, a light-emitting diode (LED) and a photodetector, such as a photodiode. The light-emitting diode may be an infrared light-emitting diode. The terminal device 100 emits infrared light by using the light-emitting diode. The terminal device 100 detects infrared reflected light from a nearby object by using the photodiode. If sufficient reflected light is detected, the terminal device 100 may determine that an object exists near the terminal device 100. When the detected reflected light is insufficient, the terminal device 100 may determine that there is no object near the terminal device 100. By using the optical proximity sensor 180G, the terminal device 100 may detect that a user holds the terminal device 100 close to the ear to make a phone call and automatically perform screen off to save power. The optical proximity sensor 180G may also be used for smart cover mode or pocket mode to automatically perform screen unlock or lock.

[0111] The ambient light sensor 180L is configured to detect ambient light brightness. The terminal device 100 may adaptively adjust the brightness of the display 194 based on the detected ambient light brightness. The ambient light sensor 180L may also be configured to automatically adjust white balance when taking a picture. The ambient light sensor 180L may further cooperate with the optical proximity sensor 180G to detect whether the terminal device 100 is in a pocket and avoid accidental touch.

[0112] The fingerprint sensor 180H is configured to collect a fingerprint. The terminal device 100 may use the collected fingerprint characteristics to implement fingerprint-based unlocking, application lock access, fingerprint-based photography, fingerprint-based answering of incoming calls, and the like.

[0113] The temperature sensor 180J is configured to detect temperature. In some embodiments, the terminal device 100 executes a temperature handling policy based on the temperature detected by the temperature sensor 180J. For example, if the temperature reported by the temperature sensor 180J exceeds a threshold, the terminal device 100 reduces the performance of the processor near the temperature sensor 180J to reduce power consumption and implement thermal protection. In some other embodiments, if the temperature is less than another threshold, the terminal device 100 heats the battery 142 to prevent the terminal device 100 from abnormally shutting down due to low temperature. In some other embodiments, if the temperature is less than another threshold, the terminal device 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown due to low temperature.

[0114] The touch sensor 180K may also be referred to as a "touch device." The touch sensor 180K may be disposed on the display 194. The touch sensor 180K and the display 194 form a touch screen, also referred to as a "touch screen." The touch sensor 180K is configured to detect a touch operation performed on or near the touch sensor 180K. The touch sensor may forward the detected touch operation to an application processor to determine a touch event type. The display 194 may be configured to provide a visual output associated with the touch operation. In some other embodiments, the touch sensor 180K may also be disposed on a surface of the terminal device 100 at a position different from the display 194.

[0115] The bone conduction sensor 180M may obtain a vibration signal. In some embodiments, the bone conduction sensor 180M may obtain a vibration signal of the vibrating bone of the vocal cord part of the person. The bone conduction sensor 180M may also contact the pulse of the person to receive a blood pressure pulse signal. In some embodiments, the bone conduction sensor 180M may alternatively be disposed in a headset and combined into a bone conduction headset. The audio module 170 may obtain a speech signal through analysis based on the vibration signal of the vibrating bone of the vocal cord part obtained by the bone conduction sensor 180M, and implement a speech function. The application processor may analyze heart rate information based on the blood pressure pulse signal obtained by the bone conduction sensor 180M, and implement a heart rate detection function.

[0116] The buttons 190 include a power button, a volume button, etc. The buttons 190 may be mechanical buttons or touch buttons. The terminal device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the terminal device 100.

[0117] The motor 191 may generate a vibration prompt. The motor 191 may be configured to provide an incoming vibration prompt and touch vibration feedback. For example, touch operations performed in different applications (e.g., taking pictures or playing audio) may correspond to different vibration feedback effects. The motor 191 may also correspond to different vibration feedback effects for touch operations performed on different regions of the display 194. Different application scenarios (e.g., time reminders, incoming information, alarm clocks, and games) may also correspond to different vibration feedback effects. The touch vibration feedback effects may alternatively be customized.

[0118] Indicator 192 may be an indicator light and may be configured to indicate charging status and power changes, as well as to indicate messages, missed calls, notifications, and the like.

[0119] The SIM card interface 195 is configured to connect to a SIM card. The SIM card may be inserted into or removed from the SIM card interface 195 to implement contact with or separation from the terminal device 100. The terminal device 100 may support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 may support a nano-SIM card, a micro-SIM card, a SIM card, etc. Multiple cards may be inserted together into the same SIM card interface 195. The multiple cards may be of the same type or different types. The SIM card interface 195 may be compatible with different types of SIM cards. The SIM card interface 195 may also be compatible with an external memory card. The terminal device 100 interacts with the network by using a SIM card to implement functions such as calling and data communication. In some embodiments, an eSIM, i.e. an embedded SIM card, is used for the terminal device 100. The eSIM card may be integrated into the terminal device 100 and cannot be separated from the terminal device 100.

[0120] A layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture may be used for the software system of the terminal device 100. In this embodiment of the present application, the Android system having a layered architecture is used as an example to illustrate the software structure of the terminal device 100.

[0121] FIG. 2 is a block diagram of the software structure of the terminal device 100 according to this embodiment of the present application.

[0122] In a layered architecture, software is divided into layers, each with a distinct role and task. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: application layer, application framework layer, Android runtime and system libraries, and kernel layer.

[0123] The application layer may include a series of application packages.

[0124] As shown in FIG. 2, the application package may include applications such as camera, gallery, calendar, phone, maps, navigation, WLAN, Bluetooth, music, video, and messaging.

[0125] The application framework layer provides an application programming interface (API) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0126] As shown in FIG. 2, the application framework layer may include a window manager, a content provider, a view system, a telephony manager, a resource manager, a notification manager, and the like.

[0127] A window manager is configured to manage window programs. The window manager may obtain the size of the display and determine whether a status bar, lock screen, screen capture, etc. are present.

[0128] Content providers are configured to store and retrieve data so that the data can be accessed by applications. Data may include video, images, audio, incoming and answered calls, browsing history and bookmarks, phone books, etc.

[0129] The view system includes visual controls, such as controls for displaying text and controls for displaying images. The view system may be configured to build applications. A display interface may include one or more views. For example, a display interface that includes a message notification icon may include a text display view and an image display view.

[0130] The telephone manager is configured to provide the terminal device 100 with communication functions, such as managing the status of incoming calls (including answering or rejecting calls, etc.).

[0131] The resource manager provides various resources for an application, such as localized strings, icons, pictures, layout files, and video files.

[0132] The notification manager may be configured to allow an application to display notification information in the status bar and to convey a notification type message. The displayed notification information may automatically disappear after a short pause without the need for user interaction. For example, the notification manager is configured to notify the completion of a download, provide a message notification, etc. Furthermore, the notification manager may be a notification that appears in the top status bar of the system in the form of a graph or scrollbar text, for example a notification of an application running in the background, or a notification that appears on the screen in the form of a dialog window. For example, the text information is prompted in the status bar, a prompt sound is generated, the terminal device vibrates, or an indicator flashes.

[0133] The Android runtime includes core libraries and a virtual machine, and is responsible for scheduling and managing the Android system.

[0134] The core library contains two parts, one part is the functions that need to be called in the java language, the other part is the core library of Android.

[0135] The application layer and the application framework layer run on a virtual machine. The virtual machine converts java files in the application layer and the application framework layer into binary files for execution. The virtual machine is configured to implement functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0136] The system library may include multiple functional modules, such as a surface manager, a media library, a 3D graphics processing library (eg, OpenGL ES), and a 2D graphics engine (eg, SGL).

[0137] The surface manager is configured to manage the display subsystem and provide a blend of 2D and 3D layers for multiple applications.

[0138] The media library supports playback and recording of multiple common audio and video formats, still image files, etc. The media library can support multiple audio and video encoding formats, e.g., MPEG-4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0139] The 3D graphics processing library is configured to implement 3D graphics drawing, image rendering, compositing, layer processing, and the like.

[0140] The 2D graphics engine is a drawing engine for 2D drawing.

[0141] The kernel layer is a layer between the hardware and the software, and includes at least a display driver, a camera driver, an audio driver, and a sensor driver.

[0142] With reference to a scenario in which imaging and photography are performed, an example of the operating procedures of the software and hardware of the terminal device 100 is described below.

[0143] When the touch sensor 180K receives a touch operation, a corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into an original input event (including information such as the touch coordinates and timestamp of the touch operation). The original input event is stored in the kernel layer. The application framework layer obtains the original input event from the kernel layer and identifies a control corresponding to the input event. For example, the touch operation is a single tap touch operation, and the control corresponding to the single tap operation is a control of a camera application icon. The camera application calls an interface in the application framework layer, so that the camera application is opened. Then, the camera driver is started by calling the kernel layer, and a still image or video is captured through the camera 193.

[0144] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The collaborative display method provided in the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0145] As shown in Fig. 3, the collaborative display method provided in one embodiment of the present application can be applied to a communication system 300. The communication system 300 can include a first terminal device 301 and a second terminal device 302. The first terminal device 301 and the second terminal device 302 can each be a terminal device having a display, such as a mobile phone, a tablet computer, a wearable device, an in-vehicle device, an AR / VR device, a notebook computer, a UMPC, a netbook, a PDA, a desktop computer, or a smart TV. The size of the display of the second terminal device 302 is larger than the size of the display of the first terminal device 301, so that the second terminal device 302 can simultaneously and collaboratively display multiple applications on the first terminal device 301.

[0146] During collaborative display, a connection between the first terminal device 301 and the second terminal device 302 needs to be established to implement data exchange between the first terminal device 301 and the second terminal device 302, so that the second terminal device 302 can collaboratively display the application on the first terminal device 301. For example, the first terminal device 301 and the second terminal device 302 may establish a connection through one tap, close-range auto-discovery, scanning, or the like.

[0147] 4 and 5-1 and 5-2 are schematic diagrams of establishing a connection between a first terminal device and a second terminal device. In a connection established through one tap, the first terminal device 301 and the second terminal device 302 are terminal devices having NFC chips, respectively. When both the NFC function of the first terminal device 301 and the NFC function of the second terminal device 302 are available, a user may use a first pre-set area where the NFC chip of the first terminal device 301 is located to touch a second pre-set area where the NFC chip of the second terminal device 302 is located. As shown in FIG. 4, in this case, the first terminal device 301 may pop up a pop-up connection box 400 saying "Are you sure you want to establish a connection to the second terminal device 302?", and the pop-up connection box 400 may further display a "Connect" button 401 and a "Cancel" button 402. The "Connect" button 401 may be configured to send a connection request to the second terminal device 302. The "Cancel" button 402 may be configured to cancel the connection operation between the first terminal device 301 and the second terminal device 302. When a user taps or touches the "Connect" button 401 on the first terminal device 301, the first terminal device 301 may send a connection request to the second terminal device 302. As shown in (a) in FIG. 5-1 and FIG. 5-2, the second terminal device 302 may pop up a pop-up confirmation box 500 saying "Are you sure you want to allow the connection to the first terminal device 301?", and the pop-up confirmation box 500 may further display an "Allow" button 501 and a "Reject" button 502. The "Allow" button 501 may be configured to establish a connection to the first terminal device 301. The "Reject" button 502 may be configured to reject the connection to the first terminal device 301. When a user taps or touches the "Allow" button 501 on the second terminal device 302, the second terminal device 302 confirms to establish a connection to the first terminal device 301.After the first terminal device 301 establishes a connection to the second terminal device 302, as shown in FIG. 5-1 and FIG. 5-2 (b), the display interface of the second terminal device 302 may display the content displayed in the first terminal device 301, for example, the home screen of the first terminal device 301. The home screen may include application icons of applications such as clock, calendar, gallery, notepad, file, mail, music, calculator, Huawei video, health, weather, camera, and message. Furthermore, the user may collaboratively operate the first terminal device 301 on the second terminal device 302, for example, tap or touch the application icon of the first terminal device 301 on the display interface of the second terminal device 302 to open the application of the first terminal device 301.

[0148] Similarly, in a connection established through short-distance autodiscovery, if the distance between the first terminal device 301 and the second terminal device 302 is less than a preset distance, the connection between the first terminal device 301 and the second terminal device 302 is established based on the operations in Figures 4, 5-1, and 5-2. Similarly, in a connection established through scanning, after the first terminal device 301 scans a two-dimensional code used for connection in the second terminal device 302, the connection between the first terminal device 301 and the second terminal device 302 is established based on the operations in Figures 4, 5-1, and 5-2.

[0149] In one example, after a user enables the multi-screen collaboration of the first terminal device 301 to establish a connection between the first terminal device 301 and the second terminal device 302, the first terminal device 301 may determine a first display area of ​​the home screen of the first terminal device 301 in the display interface of the second terminal device 302, and may transmit display information of the home screen and position information of the first display area to the second terminal device 302. The second terminal device 302 may display the home screen of the first terminal device 301 in the first display area in the display interface of the second terminal device 302. In other words, the first terminal device 301 may project the home screen onto the first display area in the second terminal device 302. The home screen is a screen on which the first terminal device 301 displays content. In other words, the first display area in the display interface of the second terminal device 302 may synchronously display the content currently displayed in the display interface of the first terminal device 301, for example, synchronously display the home screen of the first terminal device 301.

[0150] When a user opens an application (e.g., a Huawei video application) of the first terminal device 301 on the second terminal device 302, the second terminal device 302 may send the user's instruction to open the Huawei video application to the first terminal device 301. After receiving the instruction, the first terminal device 301 may create a virtual screen and determine a second display area of ​​the virtual screen in the display interface of the second terminal device 302. Furthermore, the first terminal device 301 may obtain application data of the Huawei video application, render the application data of the Huawei video application to obtain window data, combine the window data, and then display the window data on the virtual screen, that is, open the Huawei video application on the virtual screen of the first terminal device 301. Then, the first terminal device 301 may send the display information of the virtual screen and the position information of the second display area to the second terminal device 302. The second terminal device 302 may display the virtual screen in the second display area in the display interface of the second terminal device 302. For example, the application interface of the Huawei video application may be displayed in the second display area on the display interface of the second terminal device 302. In other words, the first terminal device 301 may project a virtual screen onto the second display area on the second terminal device 302.

[0151] In another example, after a user enables multi-screen collaboration on the first terminal device 301 to establish a connection between the first terminal device 301 and the second terminal device 302, the first terminal device 301 may send display information of a home screen to the second terminal device 302. The second terminal device 302 may determine a first display area of ​​the home screen of the first terminal device 301 in a display interface of the second terminal device 302, and then display the home screen within the first display area in the display interface of the second terminal device 302.

[0152] When a user opens an application of the first terminal device 301 on the second terminal device 302, the second terminal device 302 may send the user's instruction to open the application to the first terminal device 301. After receiving the instruction, the first terminal device 301 may create a virtual screen, obtain application data of the application, render the application data of the application to obtain window data, combine the window data, and then display the combined window data on the virtual screen, that is, open the application on the virtual screen of the first terminal device 301. Then, the first terminal device 301 may send display information of the virtual screen to the second terminal device 302. The second terminal device 302 may determine a second display area of ​​the virtual screen of the first terminal device 301 in the display interface of the second terminal device 302, and then display the virtual screen in the second display area in the display interface of the second terminal device 302, that is, display the application interface of the application in the second display area in the display interface of the second terminal device 302.

[0153] The content displayed on the virtual screen may not be displayed on the home screen of the first terminal device 301. In other words, the virtual screen may not be displayed on the first terminal device 301. The first terminal device 301 may create a virtual screen based on the current screen parameters of the home screen. The screen parameters may include a screen size, a display state of the screen, and the like. The current screen parameters of the home screen are the screen parameters of the home screen that exist when the virtual screen is created. However, the virtual screen is independent of the home screen. Specifically, the virtual screen is created based on the screen parameters of the home screen, but the screen parameters of the virtual screen are independent of the screen parameters of the home screen. In other words, the screen parameters of the virtual screen do not change according to the screen parameters of the home screen. For example, after a virtual screen in a portrait mode display state is created based on a home screen in a portrait mode display state, when the home screen changes from the portrait mode display state to the landscape mode display state, the screen parameters of the virtual screen may still hold the current screen parameters of the virtual screen. In other words, instead of changing to a landscape mode display state, the virtual screen may still remain in its current portrait mode display state.

[0154] It should be noted that there is no overlapping relationship between the first display area and the second display area, in order to ensure that the home screen and the virtual screen do not interfere with each other when displayed on the second terminal device 302, improve the display effect of performing cooperative display by the second terminal device 302, and improve the user experience.

[0155] It can be seen from the above description that in this embodiment of the present application, the first display area and the second display area may be determined by the first terminal device 301 or may be determined by the second terminal device 302. The principle of determining the first display area and the second display area by the first terminal device 301 is basically the same as the principle of determining the first display area and the second display area by the second terminal device 302. The following provides an explanation by using an example in which the first terminal device 301 determines the first display area and the second display area.

[0156] In this embodiment of the present application, the first terminal device 301 may determine the first display area and the second display area based on the display state of the first terminal device 301 and the interface information of the display interface of the second terminal device 302. For a non-foldable first terminal device 301, the display state may include a landscape mode display state and a portrait mode display state. For a foldable first terminal device 301, the display state may further include a folded screen display state and an unfolded screen display state. The following steps are described individually below: (1) determining the first display area and the second display area when the non-foldable first terminal device 301 is in a landscape mode display state; (2) determining the first display area and the second display area when the non-foldable first terminal device 301 is in a portrait mode display state; (3) determining the first display area and the second display area when the foldable first terminal device 301 is in a folded screen display state. (4) determining a first display area and a second display area when the foldable first terminal device 301 is in an unfolded screen display state;

[0157] The display state of the first terminal device 301 may be obtained by an existing method. In the present embodiment of the present application, there is no particular limitation to this. For example, the display state of the first terminal device 301 may be obtained from an activity task manager service (ATMS) in the first terminal device 301 by using a window manager in the first terminal device 301. The ATMS is configured to manage the state of all activities in the system.

[0158] (1) When the non-foldable first terminal device 301 is in a portrait mode display state, a first display area and a second display area are determined.

[0159] Specifically, when the non-foldable first terminal device 301 is in a portrait mode display state, after the first terminal device 301 establishes a connection to the second terminal device 302, the first terminal device 301 may obtain interface information of the display interface of the second terminal device 302, where the interface information of the display interface may include a current interface height and a current interface width of the display interface, etc., and may vertically divide the display interface of the second terminal device 302 to obtain a first display area and a second display area arranged on the left and right based on the current screen width of the home screen of the first terminal device 301 and the interface information of the second terminal device 302. The area height of the first display area and the area height of the second display area are the same, both of which are the same as the current interface height of the second terminal device 302, and the sum of the area width of the first display area and the area width of the second display area is the current interface width of the second terminal device 302. If necessary, the area width of the first display area may be the same as the current screen width of the home screen of the first terminal device 301.

[0160] It should be understood that the first and second display areas arranged left and right, obtained by vertically dividing the display interface of the second terminal device 302 based on the current screen width of the home screen of the first terminal device 301 and the interface information of the second terminal device 302, may be as shown in (a) of FIG. 6 and (a) of FIG. 7, where the first display area is on the right side and the second display area is on the left side; or as shown in (b) of FIG. 6 and (b) of FIG. 7, where the first display area is on the left side and the second display area is on the right side. (a) of FIG. 6 and (b) of FIG. 6 are exemplary diagrams of the first and second display areas when the first terminal device 301 is in a portrait mode display state and the second terminal device 302 is in a landscape mode display state. (a) of FIG. 7 and (b) of FIG. 7 are exemplary diagrams of the first and second display areas when the first terminal device 301 is in a portrait mode display state and the second terminal device 302 is in a portrait mode display state.

[0161] After obtaining the first display area and the second display area through division, the first terminal device 301 may represent the position information of the first display area by using the coordinate information of the upper left vertex and the lower right vertex of the first display area, and may represent the position information of the second display area by using the coordinate information of the upper left vertex and the lower right vertex of the second display area. The coordinate information is coordinate information in a coordinate system established by using the upper left vertex of the display interface of the second terminal device 302 as the origin of the coordinate (or using the intersection between the upper edge of the display interface of the second terminal device 302 and the left edge of the display interface of the second terminal device 302 as the origin of the coordinate), using the upper edge of the display interface of the second terminal device 302 as the X axis, and using the left edge of the display interface of the second terminal device 302 as the Y axis.

[0162] Therefore, the position information of the first display area may be [(area width of the second display area, 0), (interface width of the display interface, interface height of the display interface)], and the position information of the second display area is [(0,0), (area width of the second display area, interface height of the display interface)]. Alternatively, the position information of the first display area may be [(0,0), (area width of the first display area, interface height of the display interface)], and the position information of the second display area is [(area width of the first display area, 0), (interface width of the display interface, interface height of the display interface)]. Here, the display interface is the display interface of the second terminal device 302.

[0163] For example, as shown in (a) of FIG. 6, when the first terminal device 301 is in a portrait mode display state and the second terminal device 302 is in a landscape mode display state, the first display area may be the right area in the display interface of the second terminal device 302, and the second display area may be the left area in the display interface of the second terminal device 302. In other words, the home screen and the virtual screen may be arranged horizontally from left to right in the display interface of the second terminal device 302. The area height height1 of the first display area = the area height height2 of the second display area = the interface height height of the display interface, the area width width1 of the first display area + the area width width2 of the second display area = the interface width width of the display interface, and the area width width1 of the first display area = the current screen width of the home screen of the first terminal device 301. Therefore, the position information of the first display area may be [(width2,0),(width,height)], and the position information of the second display area may be [(0,0),(width2,height)].

[0164] 7(b), when the first terminal device 301 is in a portrait mode display state and the second terminal device 302 is in a portrait mode display state, the first display area may be a left area in a display interface of the second terminal device 302, and the second display area may be a right area in a display interface of the second terminal device 302. Therefore, the position information of the first display area may be [(0,0),(width1,height)], and the position information of the second display area may be [(width1,0),(width,height)].

[0165] It should be noted that the coordinate information of the upper left vertex and the lower right vertex of the display area (including the first display area and the second display area) is used to represent the position information of the display area only for illustrative purposes and should not be construed as a limitation on this embodiment of the present application. In this embodiment of the present application, the position information of the display area can be represented by using the coordinate information of the four vertices of the display area, or the coordinate information of the lower left vertex and the upper right vertex of the display area, etc.

[0166] (2) When the non-foldable first terminal device 301 is in a landscape mode display state, a first display area and a second display area are determined.

[0167] Specifically, when the non-foldable first terminal device 301 is in a landscape mode display state, after the first terminal device 301 establishes a connection to the second terminal device 302, the first terminal device 301 may obtain interface information of the display interface of the second terminal device 302, where the interface information of the display interface may include a current interface height and a current interface width of the display interface, etc., and based on the current screen height of the home screen of the first terminal device 301 and the interface information of the second terminal device 302, the display interface of the second terminal device 302 may be divided horizontally to obtain a first display area and a second display area arranged above and below. The area width of the first display area and the area width of the second display area are the same, both of which are the same as the current interface width of the second terminal device 302, and the sum of the area height of the first display area and the area height of the second display area is the current interface height of the second terminal device 302. If necessary, the area height of the first display area may be the same as the current screen height of the home screen of the first terminal device 301.

[0168] It should be understood that the first and second display areas arranged vertically, obtained by dividing the display interface of the second terminal device 302 horizontally based on the current screen height of the home screen of the first terminal device 301 and the interface information of the second terminal device 302, may be as shown in (a) of Figure 8 and (a) of Figure 9, in which the first display area is on the upper side and the second display area is on the lower side, or as shown in (b) of Figure 8 and (b) of Figure 9, in which the first display area is on the lower side and the second display area is on the upper side. (a) of Figure 8 and (b) of Figure 8 are exemplary diagrams of the first display area and the second display area when the first terminal device 301 is in a landscape mode display state and the second terminal device 302 is in a landscape mode display state. (a) in FIG. 9 and (b) in FIG. 9 are exemplary diagrams of a first display area and a second display area when a first terminal device 301 is in a landscape mode display state and a second terminal device 302 is in a portrait mode display state.

[0169] Similarly to (1), after obtaining the first display area and the second display area through division, the first terminal device 301 may represent the position information of the first display area by using the coordinate information of the upper left vertex and the lower right vertex of the first display area, and may represent the position information of the second display area by using the coordinate information of the upper left vertex and the lower right vertex of the second display area. Similarly, the coordinate information is coordinate information in a coordinate system established by using the upper left vertex of the display interface of the second terminal device 302 as the origin of the coordinate (or using the intersection between the upper edge of the display interface of the second terminal device 302 and the left edge of the display interface of the second terminal device 302 as the origin of the coordinate), using the upper edge of the display interface of the second terminal device 302 as the X axis, and using the left edge of the display interface of the second terminal device 302 as the Y axis.

[0170] Therefore, the position information of the first display area may be [(0,0),(interface width of the display interface, area height of the first display area)], and the position information of the second display area may be [(0,area height of the first display area),(interface width of the display interface, interface height of the display interface)]. Alternatively, the position information of the first display area may be [(0,area height of the second display area),(interface width of the display interface, interface height of the display interface)], and the position information of the second display area may be [(0,0),(interface width of the display interface, area height of the second display area)]. Here, the display interface is the display interface of the second terminal device 302.

[0171] For example, as shown in (a) of FIG. 8, when the first terminal device 301 is in a landscape mode display state and the second terminal device 302 is in a landscape mode display state, the first display area may be an upper area in the display interface of the second terminal device 302, and the second display area may be a lower area in the display interface of the second terminal device 302. In other words, the home screen and the virtual screen may be arranged vertically up and down in the display interface of the second terminal device 302. The area width width1 of the first display area = the area width width2 of the second display area = the interface width width of the display interface, the area height height1 of the first display area + the area height height2 of the second display area = the interface height height of the display interface, and the area height height1 of the first display area = the current screen height of the home screen of the first terminal device 301. Therefore, the position information of the first display area may be [(0,0),(width,height1)], and the position information of the second display area may be (0,height1),(width,height).

[0172] 9(b), when the first terminal device 301 is in a landscape mode display state and the second terminal device 302 is in a portrait mode display state, the first display area may be a lower area in a display interface of the second terminal device 302, and the second display area may be an upper area in a display interface of the second terminal device 302. Therefore, the position information of the first display area may be [(0,height2),(width,height)], and the position information of the second display area may be [(0,0),(width,height2)].

[0173] (3) When the foldable first terminal device 301 is in a folded screen display state, a first display area and a second display area are determined.

[0174] It should be understood that in a scenario where the foldable first terminal device 301 is in a folded screen display state, when the foldable first terminal device 301 is in a portrait mode display state, the first terminal device 301 may determine the first display area and the second display area in the manner in (1). Specifically, the first terminal device 301 may vertically divide the display interface of the second terminal device 302 based on the current screen width of the home screen of the first terminal device 301 and the interface information of the second terminal device 302 to obtain the first display area and the second display area arranged on the left and right.

[0175] When the foldable first terminal device 301 is in a landscape mode display state, the first terminal device 301 may determine the first and second display areas in the manner in (2). Specifically, the first terminal device 301 may horizontally divide the display interface of the second terminal device 302 based on the current screen height of the home screen of the first terminal device 301 and the interface information of the second terminal device 302 to obtain the first and second display areas arranged vertically.

[0176] (4) When the foldable first terminal device 301 is in an unfolded screen display state, a first display area and a second display area are determined.

[0177] Specifically, when the foldable first terminal device 301 is in a display state of an unfolded screen, the home screen of the first terminal device 301 needs to be correspondingly larger, and the area of ​​the first display region also needs to be correspondingly larger. The first terminal device 301 may determine the first display region and the second display region by referring to the display state of the second terminal device 302, in order to fully use the display interface of the second terminal device 302 and improve the utilization of the display interface of the second terminal device 302.

[0178] The display state of the second terminal device 302 may be determined based on the interface information of the display interface of the second terminal device 302. For example, if the current screen width of the display interface is greater than the current screen height, it may be determined that the second terminal device 302 is in a landscape mode display state. If the current screen width of the display interface is less than the current screen height, it may be determined that the second terminal device 302 is in a portrait mode display state.

[0179] In one example, when the second terminal device 302 is in a landscape mode display state, the first terminal device 301 may determine the first display area and the second display area by referring to the method in (1). Specifically, the first terminal device 301 vertically divides the display interface of the second terminal device 302 based on the current screen width of the home screen of the first terminal device 301 and the interface information of the second terminal device 302 to obtain the first display area and the second display area arranged on the left and right, so that the display area of ​​the second display area is guaranteed, and more virtual screens can be displayed in the second display area, which can improve the utilization of the display interface of the second terminal device 302 and improve the user experience. Similar to (1), the first and second display areas arranged left and right, obtained by vertically dividing the display interface of the second terminal device 302 based on the current screen width of the home screen of the first terminal device 301 and the interface information of the second terminal device 302, may be such that the first display area is on the left side and the second display area is on the right side; or the first display area may be on the right side and the second display area may be on the left side.

[0180] Thus, the position information of the first display area may be [(0,0), (area width of the first display area, interface height of the display interface)], and the position information of the second display area may be [(area width of the first display area, 0), (interface width of the display interface, interface height of the display interface)]. Alternatively, the position information of the first display area may be [(area width of the second display area, 0), (interface width of the display interface, interface height of the display interface)], and the position information of the second display area may be [(0,0), (area width of the second display area, interface height of the display interface)].

[0181] In another example, when the second terminal device 302 is in a portrait mode display state, the first terminal device 301 may refer to the method in (2) to determine the first display area and the second display area. Specifically, the first terminal device 301 may horizontally divide the display interface of the second terminal device 302 based on the current screen height of the home screen of the first terminal device 301 and the interface information of the second terminal device 302 to obtain the first display area and the second display area arranged above and below, so that the display area of ​​the second display area is guaranteed and more virtual screens can be displayed in the second display area, improving the utilization of the display interface of the second terminal device 302 and improving the user experience. Similar to (2), the first and second display areas arranged vertically and obtained by horizontally dividing the display interface of the second terminal device 302 based on the current screen height of the home screen of the first terminal device 301 and the interface information of the second terminal device 302 may be such that the first display area is located at the upper side of the display interface and the second display area is located at the lower side of the display interface; or the first display area may be located at the lower side of the display interface and the second display area is located at the upper side of the display interface.

[0182] Thus, the position information of the first display area may be [(0,0),(interface width of the display interface, area height of the first display area)], and the position information of the second display area may be [(0,area height of the first display area),(interface width of the display interface, interface height of the display interface)]. Alternatively, the position information of the first display area may be [(0,area height of the second display area),(interface width of the display interface, interface height of the display interface)], and the position information of the second display area may be [(0,0),(interface width of the display interface, area height of the second display area)].

[0183] In a possible implementation, in a scenario in which the first terminal device 301 is in a display state of an unfolded screen, if the difference between the screen width of the home screen and the screen height of the home screen is smaller than a preset threshold, the first terminal device 301 may directly determine the area width of the first display area based on the larger one of the screen width of the home screen and the screen height of the home screen. In other words, when the first terminal device 301 is in a landscape mode display state, the first terminal device 301 may determine the larger value of the current screen height of the home screen and the current screen height (e.g., the current screen height of the home screen) as the area width of the first display area. When the first terminal device 301 is in a portrait mode display state, the first display area and the second display area remain the same as the first display area and the second display area that exist when the first terminal device 301 is in a landscape mode display state. Therefore, when the first terminal device 301 changes from a landscape mode display state to a portrait mode display state, the first display area and the second display area do not need to be rearranged, so as to reduce operation delay and improve the display effect. The preset threshold value may be specifically set based on the actual situation.

[0184] In this embodiment of the present application, after the first display area and the second display area are determined, the second terminal device 302 can display the home screen in the first display area and display the virtual screen in the second display area. It should be understood that when the home screen is displayed in the first display area, the relative position of the home screen in the first display area needs to be determined first, so that the second terminal device 302 can display the home screen in the first display area based on the relative position.

[0185] The relative position of the home screen in the first display area may be set by default by the first terminal device 301 or the second terminal device 302, or may be customized by a user in the first terminal device 301 or the second terminal device 302. If the relative position of the home screen in the first display area is set by default by the first terminal device 301 or customized by a user in the first terminal device 301, the first terminal device 301 may transmit the relative position of the home screen in the first display area to the second terminal device 302 when transmitting display information of the home screen or display information of the home screen and position information of the first display area to the second terminal device 302.

[0186] Similarly, when the virtual screen is displayed in the second display area, the relative position of the virtual screen in the second display area needs to be determined first, so that the second terminal device 302 can display the virtual screen in the second display area based on the relative position. The relative position of the virtual screen in the second display area may be set by default by the first terminal device 301 or the second terminal device 302, or may be customized by a user in the first terminal device 301 or the second terminal device 302. If the relative position of the virtual screen in the second display area is set by default by the first terminal device 301 or customized by a user in the first terminal device 301, the first terminal device 301 may transmit the relative position of the virtual screen in the second display area when transmitting the display information of the virtual screen or the display information of the virtual screen and the position information of the second display area to the second terminal device 302.

[0187] In a possible implementation, the relative positions of the virtual screens in the second display area may alternatively be determined based on the arrangement order of the virtual screens in the second display area. The arrangement order may be set by the first terminal device 301 or the second terminal device 302 as a default, or may be customized by a user in the first terminal device 301 or the second terminal device 302. For example, the second terminal device 302 may set by default that the virtual screens are displayed in an arrangement order from left to right. Thus, when the second terminal device 302 receives display information of the first virtual screen transmitted from the first terminal device 301, the second terminal device 302 may display the first virtual screen at the leftmost position in the second display area. Subsequently, when the second terminal device 302 receives the display information of the second virtual screen transmitted from the first terminal device 301, the second terminal device 302 may display the second virtual screen on the right side of the first virtual screen, and this continues in the same manner until there are no more virtual screens that can be added to the second display area for display, or until the number of virtual screens displayed in the second display area reaches a preset maximum number. The preset maximum number may be customized by the user.

[0188] In this embodiment of the present application, during performing the cooperative display by using the second terminal device 302, if the display state of the first terminal device 301 changes, the state of the home screen displayed on the second terminal device 302 will also change correspondingly. Therefore, in order to improve the display effect of the home screen on the second terminal device 302 and improve the user experience, the first display area and the second display area can be rearranged, the home screen can be displayed in the rearranged first display area, and the virtual screen can be displayed in the rearranged second display area. The first display area and the second display area can be rearranged by the first terminal device 301 or the second terminal device 302. The following provides an explanation by using an example in which the first terminal device 301 rearranges the first display area and the second display area.

[0189] Specifically, when the first terminal device 301 changes from a portrait mode display state to a landscape mode display state, the first terminal device 301 may rearrange the first display region and the second display region by referring to the method in (2). Specifically, the first display region and the second display region arranged left and right may be rearranged into the first display region and the second display region arranged up and down. The region width of the first display region existing before the rearrangement is equal to the region height of the rearranged first display region. For example, the first terminal device 301 may rearrange the first display region and the second display region arranged left and right in (a) of FIG. 10 into the first display region and the second display region arranged up and down in (b) of FIG. 10. The region height height1 of the first display region shown in (b) of FIG. 10 is equal to the region width width1 of the first display region shown in (a) of FIG. 10.

[0190] When the first terminal device 301 changes from a landscape mode display state to a portrait mode display state, the first terminal device 301 may rearrange the first display region and the second display region by referring to the method in (1). Specifically, the first display region and the second display region arranged vertically are rearranged into the first display region and the second display region arranged horizontally. The region height of the first display region that existed before the rearrangement is equal to the region width of the rearranged first display region. For example, the first terminal device 301 may rearrange the first display region and the second display region arranged vertically in (a) of FIG. 11 into the first display region and the second display region arranged horizontally in (b) of FIG. 11. The region width width1 of the first display region shown in (b) of FIG. 11 is equal to the region height height1 of the first display region shown in (a) of FIG. 11.

[0191] When the first terminal device 301 changes from a folded screen display state to an unfolded screen display state, the first terminal device 301 may rearrange the first display area and the second display area by referring to the method in (4). Specifically, when the second terminal device 302 is in a landscape mode display state, the first terminal device 301 may vertically divide the display interface of the second terminal device 302 based on the screen width of the unfolded home screen and the interface information of the second terminal device 302 to obtain the first display area and the second display area arranged on the left and right. When the second terminal device 302 is in a portrait mode display state, the first terminal device 301 may horizontally divide the display interface of the second terminal device 302 based on the screen height of the unfolded home screen and the interface information of the second terminal device 302 to obtain the first display area and the second display area arranged on the top and bottom.

[0192] For example, when the second terminal device 302 is in a landscape mode display state, the first terminal device 301 may rearrange the first and second display regions in (a) of FIG. 12 into the first and second display regions in (b) of FIG. 12. The region width width1 of the first display region that existed before the rearrangement is equal to the screen width of the home screen that exists after the first terminal device 301 is folded, and the region width width1' of the rearranged first display region is equal to the screen width of the home screen that exists after the first terminal device 301 is unfolded. For example, when the second terminal device 302 is in a portrait mode display state, the first terminal device 301 may rearrange the first and second display regions in (a) of FIG. 13 into the first and second display regions in (b) of FIG. 13. The area width width1 of the first display area that existed before the reorganization is equal to the screen width of the home screen that exists after the first terminal device 301 is folded, and the area height height1 of the reorganized first display area is equal to the screen height of the home screen that exists after the first terminal device 301 is unfolded.

[0193] In a possible implementation, in a scenario in which the first terminal device 301 is in a display state of an unfolded screen, when the first terminal device 301 changes from a landscape mode display state to a portrait mode display state, the first terminal device 301 may reorganize the first display region and the second display region with reference to the method in (1). When the first terminal device 301 changes from a portrait mode display state to a landscape mode display state, the first terminal device 301 may reorganize the first display region and the second display region with reference to the method in (2).

[0194] In another possible implementation form, in a scenario in which the first terminal device 301 is in an unfolded screen display state, if the first terminal device 301 determines the first display area and the second display area based on the larger of the screen width and screen height of the unfolded home screen, for example, if the first terminal device 301 determines the area width of the first display area based on the screen height of the home screen, i.e., the larger one, when the unfolded first terminal device 301 is in a landscape mode display state, when the first terminal device 301 changes from the landscape mode display state to the portrait mode display state, in order to reduce the operation delay of the cooperative display and improve the display effect, the first terminal device 301 does not need to reorganize the first display area and the second display area, and can directly display the content in the first terminal device 301 in the portrait mode display state within the first display area and the second display area determined in the landscape mode display state.

[0195] When the first terminal device 301 changes from an unfolded screen display state to a folded screen display state, the first terminal device 301 may rearrange the first display area and the second display area by referring to the method in (3). Specifically, when the first terminal device 301 is in a landscape mode display state, the first terminal device 301 may horizontally divide the display interface of the second terminal device 302 based on the screen height of the home screen of the folded first terminal device 301 and the interface information of the second terminal device 302 to obtain the first display area and the second display area arranged on the top and bottom. When the first terminal device 301 is in a portrait mode display state, the first terminal device 301 may vertically divide the display interface of the second terminal device 302 based on the screen width of the home screen of the folded first terminal device 301 and the interface information of the second terminal device 302 to obtain the first display area and the second display area arranged on the left and right.

[0196] For example, when the unfolded first terminal device 301 and the folded first terminal device 301 are in a landscape mode display state, the first terminal device 301 may rearrange the first display region and the second display region in (a) of FIG. 14 into the first display region and the second display region in (b) of FIG. 14. The region height height1 of the first display region existing before the rearrangement is equal to the screen height of the unfolded home screen of the first terminal device 301, and the region height height1' of the rearranged first display region is equal to the screen height of the folded home screen of the first terminal device 301. For example, when the unfolded first terminal device 301 and the folded first terminal device 301 are in a portrait mode display state, the first terminal device 301 may rearrange the first display region and the second display region in (c) of FIG. 14 into the first display region and the second display region in FIG. 14. The area height height1 of the first display area that existed before the reorganization is equal to the screen height of the unfolded home screen of the first terminal device 301, and the area width width1 of the reorganized first display area is equal to the screen width of the folded home screen of the first terminal device 301.

[0197] If necessary, when the display state of the first terminal device 301 changes, the content display manner of the home screen of the first terminal device 301 does not change correspondingly, and when the first display area and the second display area are rearranged, the display effect of the content of the home screen in the second terminal device 302 is affected, which is inconvenient for the user to view. Therefore, when the display state of the first terminal device 301 changes, if the content display manner of the home screen changes with the display state of the first terminal device 301, the first terminal device 301 may rearrange the first display area and the second display area. If the content display manner of the home screen does not change with the display state of the first terminal device 301, the first terminal device 301 does not need to rearrange the first display area and the second display area.

[0198] The content display manner of the home screen may be determined by an existing method. In the present embodiment of the present application, there is no particular limitation thereon. For example, an activity manager service (AMS) in the first terminal device 301 may be used to obtain the landscape / portrait mode state of an activity corresponding to the home screen, and determine the content display manner of the home screen based on the landscape / portrait mode state of the activity.

[0199] If necessary, when the display state of the first terminal device 301 does not change but the content display manner of the home screen of the first terminal device 301 changes, for example, when a user taps a landscape / portrait mode switching button of an application displayed on the home screen to change the content display manner of the home screen, but the first terminal device 301 is not rotated correspondingly, the first terminal device 301 may alternatively reorganize the first display area and the second display area based on the change in the content display manner, so as to improve the display effect of the content of the home screen on the second terminal device 302, help the user view the content on the second terminal device 302, and improve the user's viewing experience.

[0200] The content display manner may include landscape mode display and portrait mode display. In this embodiment of the present application, in the case of rearranging the first display area and the second display area based on the content display manner, please refer to the manners in (1) and (2). Specifically, when the content display manner of the home screen changes from portrait mode display to landscape mode display, the first terminal device 301 may rearrange the first display area and the second display area by referring to the method in (2). Specifically, the first terminal device 301 may obtain interface information of the display interface of the second terminal device 302, and may horizontally divide the display interface of the second terminal device 302 based on the current screen width of the home screen of the first terminal device 301 (i.e., the screen width in the portrait mode display state) and the interface information of the second terminal device 302 to obtain the first display area and the second display area arranged above and below. When the content display manner of the home screen changes from landscape mode display to portrait mode display, the first terminal device 301 may rearrange the first display area and the second display area by the method in (1). Specifically, the first terminal device 301 may obtain interface information of the display interface of the second terminal device 302, and based on the current screen height of the home screen of the first terminal device 301 (i.e., the screen height in the landscape mode display state) and the interface information of the second terminal device 302, may vertically divide the display interface of the second terminal device 302 to obtain a first display area and a second display area arranged on the left and right.

[0201] In this embodiment of the present application, after the first display area and the second display area are rearranged, the content display manner and the relative position of the home screen in the first display area may change with the display state of the first terminal device 301 or the content display manner of the home screen, but the virtual screen in the second display area is still displayed based on the content display manner and relative position that existed before the rearrangement. For example, if the first virtual screen and the second virtual screen in the second display area are displayed in the second display area in portrait mode with a left-to-right arrangement order before the rearrangement, after the first display area and the second display area are rearranged, the first virtual screen and the second virtual screen are still displayed in the second display area rearranged in portrait mode with a left-to-right arrangement order, thereby reducing the number of times to refresh and lay out the content in the virtual screen again, reducing the display delay in the second display area, improving the display effect, and improving the user experience.

[0202] It can be understood that if the area height of the reorganized second display area is smaller than the screen height of the virtual screen or the area width of the reorganized second display area is smaller than the screen width of the virtual screen, before displaying the virtual screen in the second display area, the second terminal device 302 may proportionally reduce the virtual screen based on the reorganized second display area, and then may completely display the proportionally reduced virtual screen in the second display area of ​​the second terminal device 302 based on the relative position of the virtual screen in the second display area that existed before the reorganization; or the first terminal device 301 may proportionally reduce the virtual screen based on the reorganized second display area, and may send the display information of the proportionally reduced virtual screen to the second terminal device 302, and the second terminal device 302 may completely display the virtual screen in the second display area based on the relative position of the virtual screen in the second display area that existed before the reorganization. In this way, it can be guaranteed that the virtual screen can be completely displayed on the second terminal device 302.

[0203] The following provides an exemplary description of the cooperative display method provided in the embodiments of the present application with reference to a specific application scenario. In the following application scenario, an exemplary description is provided by using an example in which the first terminal device 301 determines and rearranges the first display area and the second display area.

[0204] For example, in a scenario in which the first terminal device 301 is in a portrait mode display state and the second terminal device 302 is in a landscape mode display state, after the first terminal device 301 and the second terminal device 302 perform multi-screen collaboration, the first terminal device 301 may determine that the first display area and the second display area are arranged horizontally left and right. For example, the first display area is on the right side and the second display area is on the left side. The first terminal device 301 may transmit display information of the home screen of the first terminal device 301 and position information of the first display area to the second terminal device 302. As shown in (a) of FIG. 15-1 and FIG. 15-2, after receiving the display information of the home screen and the position information of the first display area, the second terminal device 302 may display the home screen 1500 in the first display area in the display interface of the second terminal device 302.

[0205] When a user taps or touches a message application 1501 on the home screen 1500 of the second terminal device 302, the second terminal device 302 may send an instruction to the first terminal device 301 indicating that the user taps or touches the message application. After receiving the instruction, the first terminal device 301 may create a first virtual screen, obtain application data of the message application, render the application data of the message application to obtain window data, combine the window data, and then display the combined window data on the first virtual screen. Then, the first terminal device 301 may send display information of the first virtual screen and position information of the second display area to the second terminal device 302. As shown in (b) of FIG. 15-1 and FIG. 15-2, after receiving the display information of the first virtual screen and the position information of the second display area, the second terminal device 302 may display the first virtual screen in the second display area in the display interface of the second terminal device 302 in a left-to-right arrangement order, in other words, may display the content of the message application at the leftmost position in the second display area.

[0206] If the user subsequently taps or touches the Huawei video application 1502 on the home screen 1500 of the second terminal device 302, the second terminal device 302 may send an instruction to the first terminal device 301 indicating that the user taps or touches the Huawei video application. After receiving the instruction, the first terminal device 301 may create a second virtual screen, obtain application data of the Huawei video application, render the application data of the Huawei video application to obtain window data, combine the window data, and then display the combined window data on the second virtual screen. Then, the first terminal device 301 may send display information of the second virtual screen to the second terminal device 302. As shown in (a) in FIG. 16-1 and FIG. 16-2, after receiving the display information of the second virtual screen, the second terminal device 302 may display the second virtual screen to the right of the message application in the second display area in a left-to-right display order.

[0207] When a user taps or touches a gallery application on the first terminal device 301, the first terminal device 301 may open the gallery application on a home screen and transmit display information of the home screen to the second terminal device 302. As shown in (b) of FIG. 16-1 and FIG. 16-2, after receiving the display information of the home screen, the second terminal device 302 may display the content of the gallery application in the first display area.

[0208] In a scenario in which the message application, the Huawei video application, and the gallery application are opened, when the user rotates the first terminal device 301 and the content display manner of the home screen of the first terminal device 301 changes accordingly, specifically, when the user switches the first terminal device 301 from a portrait mode display state to a landscape mode display state, and the content displayed on the first terminal device 301 also changes from portrait mode display to landscape mode display, the first terminal device 301 may rearrange the first display area and the second display area arranged horizontally from left to right into the first display area and the second display area arranged vertically from top to bottom. For example, the rearranged first display area is on the upper side, and the rearranged second display area is on the lower side. Then, the first terminal device 301 may transmit the display information of the rotated home screen and the position information of the rearranged first display area to the second terminal device 302, and transmit the position information of the rearranged second display area and the display information of the first virtual screen and the second virtual screen to the second terminal device 302. The first virtual screen and the second virtual screen do not change with the display state of the first terminal device 301, and the relative positions of the first virtual screen and the second virtual screen in the second display area also remain unchanged. As shown in Fig. 17, after receiving the display information of the rotated home screen, the position information of the reorganized first display area, the position information of the reorganized second display area, and the display information of the first virtual screen and the second virtual screen, the second terminal device 302 may display the home screen in the reorganized first display area, and may still display the first virtual screen and the second virtual screen in the reorganized second display area in the portrait mode display manner based on the relative positions of the first virtual screen and the second virtual screen in the second display area that existed before the reorganization.

[0209] It may be understood that when a user rotates the first terminal device 301 and the content of the virtual screen (including at least one of the first virtual screen and the second virtual screen) does not change, the first terminal device 301 does not need to transmit display information of the virtual screen to the second terminal device 302, and the second terminal device 302 may directly display the virtual screen in the reorganized second display area based on the previously acquired display information of the virtual screen and the relative position of the virtual screen in the second display area, thereby improving the display efficiency of the virtual screen.

[0210] For example, in a scenario in which the first terminal device 301 is folded and in a portrait mode display state, and the second terminal device 302 is in a portrait mode display state, after the first terminal device 301 and the second terminal device 302 perform multi-screen collaboration, the first terminal device 301 may determine that the first display area and the second display area are arranged horizontally left and right. For example, the first display area is on the right side, and the second display area is on the left side. The first terminal device 301 may transmit display information of the home screen 1800 of the first terminal device 301 and position information of the first display area to the second terminal device 302. As shown in FIG. 18, after receiving the display information of the home screen 1800 and the position information of the first display area, the second terminal device 302 may display the home screen 1800 in the first display area in the display interface of the second terminal device 302.

[0211] When a user taps the Huawei video application 1801 on the home screen 1800 of the second terminal device 302, the second terminal device 302 may send an instruction to the first terminal device 301 indicating that the user tapped the Huawei video application. After receiving the instruction, the first terminal device 301 may create a first virtual screen, obtain application data of the Huawei video application, render the application data of the Huawei video application to obtain window data, combine the window data, and then display the combined window data on the first virtual screen. Then, the first terminal device 301 may send the display information of the first virtual screen and the position information of the second display area to the second terminal device 302. As shown in FIG. 19, after receiving the display information of the first virtual screen and the position information of the second display area, the second terminal device 302 may display the first virtual screen in the second display area in the display interface of the second terminal device 302 in a right-to-left display order, specifically, display the Huawei video application on the right side in the second display area.

[0212] When a user taps or touches a message application on the first terminal device 301, the first terminal device 301 may open the message application on a home screen and send display information of the home screen to the second terminal device 302. As shown in FIG. 20, after receiving the display information of the home screen, the second terminal device 302 may display the content of the message application in the first display area.

[0213] In a scenario in which the message application and the Huawei video application are opened, when the user unfolds the first terminal device 301, the first terminal device 301 may rearrange the first and second display areas arranged horizontally from left to right into the first and second display areas arranged vertically from top to bottom. For example, the rearranged first display area is at the top, and the rearranged second display area is at the bottom. Then, the first terminal device 301 may send the display information of the unfolded home screen and the position information of the rearranged first display area to the second terminal device 302, and send the position information of the rearranged second display area and the display information of the first virtual screen to the second terminal device 302. The first virtual screen does not change with the display state of the first terminal device 301, and the relative position of the first virtual screen in the second display area also remains unchanged. As shown in FIG. 21 , after receiving the display information of the unfolded home screen, the position information of the reorganized first display area, the position information of the reorganized second display area, and the display information of the first virtual screen, the second terminal device 302 may display the home screen in the reorganized first display area, and display the first virtual screen in the second display area still reorganized in the portrait mode display manner based on the relative position of the first virtual screen in the second display area that existed before the reorganization, that is, display the first virtual screen on the right side in the second display area still reorganized in the portrait mode display manner.

[0214] It may be understood that when a user rotates the first terminal device 301 and the content of the first virtual screen does not change, the first terminal device 301 does not need to transmit display information of the first virtual screen to the second terminal device 302, and the second terminal device 302 may directly display the first virtual screen in the reorganized second display area based on the previously acquired display information of the first virtual screen and the relative position of the first virtual screen in the second display area, thereby improving the display efficiency of the virtual screen.

[0215] Based on the above description, the following briefly describes the cooperative display method provided in the embodiment of the present application. Figure 22 is a schematic flowchart of the cooperative display method according to an embodiment of the present application. As shown in Figure 22, the method may include the following steps:

[0216] 1. A user enables multi-screen collaboration on a first terminal device.

[0217] 2. The first terminal device establishes a connection to the second terminal device.

[0218] It can be understood that a user can enable multi-screen collaboration by using a first preset area where the NFC chip is located on a first terminal device and touching a second preset area where the NFC chip is located on a second terminal device. In this case, the first terminal device can pop up a pop-up connection box saying "Are you sure you want to establish a connection to the second terminal device", which may include a "Connect" button and a "Cancel" button. If the user taps the "Connect" button on the first terminal device, the first terminal device can send a connection request to the second terminal device. After receiving the connection request, the second terminal device can pop up a pop-up confirmation box saying "Are you sure you want to allow a connection to the first terminal device", which may include a "Allow" button and a "Reject" button. If the user taps the "Allow" button on the second terminal device, the second terminal device establishes a connection to the first terminal device. In this case, the second terminal device can collaboratively display the content of the first terminal device.

[0219] 3. The first terminal device determines a first display area of ​​the home screen on the second terminal device based on the first display state of the first terminal device, and sends display information of the home screen and first position information of the first display area to the second terminal device.

[0220] 4. The second terminal device displays the home screen within the first display area.

[0221] 5. The user taps on the application on the first terminal device to open it on the second terminal device.

[0222] 6. The second terminal device sends an instruction to the first terminal device to open the application.

[0223] 7. After receiving the command, the first terminal device creates a virtual screen, displays the application on the virtual screen, determines a second display area of ​​the virtual screen in the second terminal device, and sends display information of the virtual screen and second position information of the second display area to the second terminal device.

[0224] Specifically, the first terminal device may obtain a first display state of the first terminal device and interface information of the display interface of the second terminal device, and based on the first display state of the first terminal device and the interface information of the second terminal device, may determine a first display area of ​​the home screen in the display interface of the second terminal device and a second display area of ​​the virtual screen in the display interface of the second terminal device.

[0225] The display state of the first terminal device 301 can be obtained by an existing method. This embodiment of the present application is not particularly limited thereto. For example, the display state of the first terminal device 301 can be obtained from the ATMS in the first terminal device 301 by using a window manager in the first terminal device 301. Furthermore, after determining the first display area corresponding to the home screen and the second display area corresponding to the virtual screen, the window manager can obtain and store the position information of the first display area and the position information of the second display area.

[0226] For example, when the first terminal device is a non-foldable terminal device and the first terminal device is in a portrait mode display state, or when the first terminal device is a foldable terminal device and the first terminal device is folded and in a portrait mode display state, the first terminal device may determine that the first display area and the second display area are arranged horizontally left and right, both of the area heights of the first display area and the second display area are the same as the interface height of the display interface of the second terminal device, and the sum of the area widths of the first display area and the second display area is equal to the interface width of the display interface of the second terminal device.

[0227] For example, when the first terminal device is a non-foldable terminal device and the first terminal device is in a landscape mode display state, or when the first terminal device is a foldable terminal device and the first terminal device is folded and in a landscape mode display state, the first terminal device may determine that the first display area and the second display area are arranged vertically above and below, both of the area widths of the first display area and the second display area are the same as the interface width of the display interface of the second terminal device, and the sum of the area heights of the first display area and the second display area is equal to the interface height of the display interface of the second terminal device.

[0228] For example, when the first terminal device is a foldable terminal device and the first terminal device is in a display state of an unfolded screen, the first terminal device may determine the first display region and the second display region based on the interface information of the second terminal device. When the second terminal device is determined to be in a landscape mode display state based on the interface information of the second terminal device, the first terminal device may determine that the first display region and the second display region are arranged horizontally from left to right, both the region heights of the first display region and the second display region are equal to the interface height of the display interface of the second terminal device, and the sum of the region widths of the first display region and the second display region are equal to the interface width of the display interface of the second terminal device. When the second terminal device is determined to be in a portrait mode display state based on the interface information of the second terminal device, the first terminal device may determine that the first display region and the second display region are arranged vertically from top to bottom, both the region widths of the first display region and the second display region are equal to the interface width of the display interface of the second terminal device, and the sum of the region heights of the first display region and the second display region are equal to the interface height of the display interface of the second terminal device.

[0229] 8. The second terminal device displays the virtual screen within the second display area.

[0230] 9. The user adjusts the first terminal device from the first display state to the second display state.

[0231] 10. The first terminal device rearranges the first display area and the second display area based on the second display state, and sends display information of the home screen, third position information of the rearranged first display area, display information of the virtual screen, and fourth position information of the rearranged second display area to the second terminal device.

[0232] In this embodiment of the present application, a user may rotate the first terminal device to adjust the first terminal device from a first display state to a second display state. The second display state is any one of a portrait mode display state, a landscape mode display state, a folded screen display state, or an unfolded screen display state. For example, the first terminal device may be adjusted from a portrait mode display state to a landscape mode display state, or from a landscape mode display state to a portrait mode display state; or from a folded screen display state to an unfolded screen display state; or from an unfolded screen display state to a folded screen display state, etc.

[0233] When the display state of the first terminal device changes, the configuration file of the AMS in the first terminal device changes accordingly. The AMS may send a message to the window manager in the first terminal device indicating that the display state of the first terminal device has changed. After receiving the message, the window manager may again obtain and store the position information of the first display area and the position information of the second display area.

[0234] After the first display area and the second display area are rearranged, the first terminal device may transmit display information of the home screen of the first terminal device, third position information of the rearranged first display area, display information of the virtual screen, and fourth position information of the rearranged second display area to the second terminal device, so that the second terminal device displays the home screen and virtual screen of the first terminal device.

[0235] 11. The second terminal device displays a home screen within the reorganized first display area and displays a virtual screen within the reorganized second display area.

[0236] The content display manner of the home screen may change with the display state of the first terminal device, but the content display manner of the virtual screen remains unchanged. That is, the content display manner of the virtual screen does not change with the display state of the first terminal device. For example, when both the home screen and the virtual screen display portrait mode and the first terminal device changes from the portrait mode display state to the landscape mode display state, the home screen may change from the portrait mode display to the landscape mode display, but the virtual screen may still be displayed in portrait mode.

[0237] Moreover, the relative position of the virtual screen in the rearranged second display area is the same as the relative position of the virtual screen in the second display area before the rearrangement. In other words, when the display state of the first terminal device changes, the content of the application displayed on the virtual screen does not need to be refreshed and laid out again. In other words, the second terminal device can still continue to display the virtual screen based on the relative position of the virtual screen in the second display area before the rearrangement and the original content display manner of the virtual screen.

[0238] In this embodiment of the present application, the first display area of ​​the home screen of the first terminal device and the second display area of ​​the virtual screen of the second terminal device can be determined based on the display state of the first terminal device, so that the second terminal device can display the home screen in the first display area and the virtual screen in the second display area. However, when the display state of the first terminal device changes, only the content display manner of the home screen, the first display area, and the second display area can be dynamically adjusted, and the content display manner of the virtual screen and the relative position of the virtual screen in the second display area do not need to be adjusted. Therefore, the content of the application displayed on the virtual screen does not need to be laid out or refreshed again, the delay in displaying the content displayed in the second display area is reduced, and the user experience is improved. In this way, the usefulness and feasibility are high.

[0239] It should be understood that the sequence numbers of steps do not refer to the execution order in the embodiments. The execution order of the processes needs to be determined based on the functions and internal logic of the processes, and should not be any limitation on the implementation process of the embodiments of the present application.

[0240] An embodiment of the present application further provides a terminal device. The terminal device includes at least one memory, at least one processor, and a computer program stored in the at least one memory and executable on the at least one processor. When the processor executes the computer program, the terminal device can implement the steps in any one of the embodiments of the above-mentioned method. For example, the structure of the terminal device can be shown in FIG. 1.

[0241] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program, which, when executed by a computer, enables the computer to implement the steps of any one of the above-mentioned method embodiments.

[0242] An embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to implement the steps in any one of the above method embodiments.

[0243] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, the integrated unit may be stored in a computer-readable storage medium. Based on such understanding, in this application, all or some of the procedures of the method in the above-mentioned embodiments may be implemented by a computer program that instructs related hardware. The computer program may be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps in the method embodiments may be implemented. The computer program includes a computer program code. The computer program code may be in source code format, object code format, executable file format, or some intermediate format, etc. The computer-readable storage medium may include at least any entity or device that can carry a computer program code to an apparatus / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electric carrier signal, an electric communication signal, a software distribution medium, such as a USB flash drive, a removable hard disk, a magnetic disk, or an optical disk. In some jurisdictions, a computer readable medium may not be an electrical carrier signal or an electrical communication signal according to statute and patent practice.

[0244] In the above embodiments, all the descriptions of the embodiments have their own focus. For parts not described in detail in one embodiment, please refer to the relevant descriptions in another embodiment.

[0245] Those skilled in the art may recognize that, in combination with the examples described in the embodiments disclosed herein, the units and algorithm steps may be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented by hardware or software is determined by the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions of each specific application, but the implementation should not be considered to go beyond the scope of the present application.

[0246] In the embodiments provided in the present application, it should be understood that the disclosed apparatus / terminal device and method may be implemented in other ways. For example, the embodiment of the described apparatus / terminal device is merely an example. For example, the division into modules or units is merely a logical functional division, and may be other divisions in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some functions may be ignored or not performed. Furthermore, the shown or described mutual couplings or direct couplings or communication connections may be implemented through some interfaces. Indirect couplings or communication connections between devices or units may be implemented in electronic, mechanical, or other forms.

[0247] The units described as separate parts may or may not be physically separate, and the parts shown as units may or may not be physical units, and may be located in one place or distributed among multiple network units. Some or all of these units may be selected based on actual requirements to achieve the objectives of the solutions of these embodiments.

[0248] The above embodiments are only used to describe the technical solutions of the present application, and are not intended to limit the technical solutions. Although the present application has been described in detail in relation to the above embodiments, those skilled in the art should further understand that, without departing from the spirit and scope of the technical solutions of the embodiments of the present application, they may make modifications to the technical solutions described in the above embodiments, or make equivalent replacements for some technical features thereof. Any modifications or replacements shall fall within the protection scope of the present application.

Claims

1. A cooperative display method applied to a first terminal device, comprising: determining a first display area of ​​a first home screen of the first terminal device on a second terminal device and a second display area of ​​a virtual screen of the first terminal device on the second terminal device based on a first display state of the first terminal device; projecting the first home screen onto the second terminal device and transmitting first position information of the first display area to the second terminal device, where the first position information is used by the second terminal device to display the first home screen within the first display area; projecting the virtual screen onto the second terminal device and transmitting second position information of the second display area to the second terminal device, where the second position information is used by the second terminal device to display the virtual screen within the second display area, and the virtual screen is not displayed on the first terminal device; When it is detected that the first terminal device is adjusted from the first display state to a second display state, adjusting a content display manner of the first home screen based on the second display state to obtain a second home screen, and rearranging the first display area and the second display area based on the second display state; and projecting the second home screen and the virtual screen onto the second terminal device and transmitting third position information of the reorganized first display area and fourth position information of the reorganized second display area to the second terminal device, where the third position information is used by the second terminal device to display the second home screen within the reorganized first display area and the fourth position information is used by the second terminal device to display the virtual screen within the reorganized second display area. A cooperative display method comprising:

2. A cooperative display method as described in claim 1, wherein the second home screen is displayed within the reorganized first display area in the second display state, and the virtual screen is displayed within the reorganized second display area while remaining in the first display state.

3. A cooperative display method as described in claim 1 or 2, wherein if the content of the virtual screen does not change while the first terminal device is adjusted from the first display state to the second display state, the first terminal device does not include display information of the virtual screen when transmitting the fourth position information of the reorganized second display area to the second terminal device.

4. A cooperative display method described in any one of claims 1 to 3, wherein the virtual screen is created based on screen parameters of the first home screen, and the screen parameters of the virtual screen do not change together with the screen parameters of the first home screen.

5. Prior to the step of projecting the virtual screen onto the second terminal device, generating the virtual screen based on the first home screen when a start command for the application transmitted from the second terminal device is obtained, and displaying an application interface of the application on the virtual screen; The cooperative display method according to claim 1 , comprising:

6. The step of determining a first display area of ​​a first home screen of the first terminal device on a second terminal device and a second display area of ​​a virtual screen of the first terminal device on the second terminal device based on a first display state of the first terminal device includes: obtaining interface information of the second terminal device; and determining the first display area and the second display area based on the first display state of the first terminal device and the interface information of the second terminal device; The cooperative display method according to claim 1 , comprising:

7. The interface information includes an interface height and an interface width, and the step of determining the first display area and the second display area based on the first display state of the first terminal device and the interface information of the second terminal device includes: determining, when the first display state is a portrait mode display state, that a first region height of the first display region is equal to a second region height of the second display region, and that the first region height and the second region height are equal to the interface height; The cooperative display method of claim 6, comprising:

8. determining that a first region width of the first display region is equal to a screen width of the first home screen; The cooperative display method of claim 7 further comprising:

9. The interface information includes an interface height and an interface width, and the step of determining the first display area and the second display area based on the first display state of the first terminal device and the interface information of the second terminal device includes: determining, when the first display state is a landscape mode display state, that a first region width of the first display region is equal to a second region width of the second display region, and that the first region width and the second region width are equal to the interface width; The cooperative display method of claim 6, comprising:

10. determining that a first region height of the first display region is equal to a screen height of the first home screen; The cooperative display method of claim 9 further comprising:

11. The first terminal device is a terminal device having a foldable display, and the interface information includes an interface height and an interface width; and The step of determining the first display area and the second display area based on the first display state of the first terminal device and the interface information of the second terminal device includes: determining a third display state of the second terminal device based on the interface height and the interface width when the first display state is a display state of an unfolded screen; and determining the first display area and the second display area based on the third display state and the interface information; The cooperative display method of claim 6, comprising:

12. A cooperative display method applied to a first terminal device, comprising: determining a first display area of ​​a first home screen of the first terminal device on a second terminal device and a second display area of ​​a virtual screen of the first terminal device on the second terminal device based on a first display state of the first terminal device; projecting the first home screen onto the second terminal device and transmitting first position information of the first display area to the second terminal device, where the first position information is used by the second terminal device to display the first home screen within the first display area; projecting the virtual screen onto the second terminal device and transmitting second position information of the second display area to the second terminal device, where the second position information is used by the second terminal device to display the virtual screen within the second display area; When it is detected that the first terminal device is adjusted from the first display state to a second display state, adjusting a content display manner of the first home screen based on the second display state to obtain a second home screen, and rearranging the first display area and the second display area based on the second display state; and projecting the second home screen and the virtual screen onto the second terminal device and transmitting third position information of the reorganized first display area and fourth position information of the reorganized second display area to the second terminal device, where the third position information is used by the second terminal device to display the second home screen within the reorganized first display area and the fourth position information is used by the second terminal device to display the virtual screen within the reorganized second display area; The step of determining a first display area of ​​a first home screen of the first terminal device on a second terminal device and a second display area of ​​a virtual screen of the first terminal device on the second terminal device based on a first display state of the first terminal device includes: obtaining interface information of the second terminal device; and determining the first display area and the second display area based on the first display state of the first terminal device and the interface information of the second terminal device; having The first terminal device is a terminal device having a foldable display, and the interface information includes an interface height and an interface width; and The step of determining the first display area and the second display area based on the first display state of the first terminal device and the interface information of the second terminal device includes: determining a third display state of the second terminal device based on the interface height and the interface width when the first display state is a display state of an unfolded screen; and determining the first display area and the second display area based on the third display state and the interface information; A cooperative display method comprising:

13. determining the first display area and the second display area based on the third display state and the interface information, determining, when the third display state is a landscape mode display state, that a first region height of the first display region is equal to a second region height of the second display region, and that the first region height and the second region height are equal to the interface height; 13. The cooperative display method according to claim 11 or 12, comprising:

14. determining that a first region width of the first display region is equal to a screen width of the first home screen or a screen height of the first home screen; The cooperative display method of claim 13 further comprising:

15. determining the first display area and the second display area based on the third display state and the interface information, determining, when the third display state is a portrait mode display state, that a first region width of the first display region is equal to a second region width of the second display region, and that the first region width and the second region width are equal to the interface width; 13. The cooperative display method according to claim 11 or 12, comprising:

16. determining that a first region height of the first display region is equal to a screen height of the first home screen or a screen width of the first home screen; The cooperative display method of claim 15 further comprising:

17. A cooperative display method applied to a second terminal device, comprising: Obtaining a first display state of a first terminal device, and determining a first display area of ​​a first home screen of the first terminal device on the second terminal device and a second display area of ​​a virtual screen of the first terminal device on the second terminal device based on the first display state; Obtaining the first home screen projected by the first terminal device, and displaying the first home screen within the first display area; acquiring the virtual screen projected by the first terminal device and displaying the virtual screen within the second display area; when it is determined that the first terminal device has been adjusted from the first display state to a second display state, rearranging the first display area and the second display area based on the second display state; and obtaining a second home screen and the virtual screen projected by the first terminal device, displaying the second home screen within the reorganized first display area, and displaying the virtual screen within the reorganized second display area, where a content display manner of the second home screen corresponds to the second display state, and a content display manner of the virtual screen maintains the first display state; A cooperative display method comprising:

18. Prior to the step of acquiring the virtual screen projected by the first terminal device, sending a start command to the first terminal device in response to a launch operation performed on an application on the first home screen, where the start command is used by the first terminal device to start the application and display an application interface of the application on the virtual screen.

20. The cooperative display method of claim 17, comprising:

19. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, the terminal device being capable of implementing a cooperative display method according to any one of claims 1 to 18 when the processor executes the computer program.

20. A computer program product for causing a computer to implement the cooperative display method according to any one of claims 1 to 18.

Images