Screen mirroring method, and device

Abstract

Provided in the present application are a screen mirroring method, and a device. The method comprises: detecting a screen mirroring operation of a user for a first interface; and in response to the screen mirroring operation, sending first screen mirroring data to a second device, wherein the first screen mirroring data comprises screen mirroring data of the first interface and N interfaces associated with the first interface, N being an integer greater than or equal to 1. In the present application, during screen mirroring from the first device to the second device, the sent screen mirroring data may comprise screen mirroring data of a plurality of interfaces, such that the second device can simultaneously display interface content of the plurality of interfaces, thereby fully utilizing the display area of a display screen without black border padding, and thus facilitating an improvement in the user experience.

Description

A screen projection method and device

[0001] The present application claims priority to the Chinese patent application No. 202411107161.0, filed on August 13, 2024, and entitled "A screen projection method and device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the field of devices, and more particularly, to a screen projection method and device. BACKGROUND

[0003] Currently, an interface displayed by one device can be projected onto another device through distributed technology, for example, a mobile phone can project the interface being displayed onto a television. For another example, a mobile phone can project the interface being displayed onto a cockpit screen of a vehicle. Since the sizes and shapes of display screens of different devices can be different, when a small-screen device projects an interface onto a large-screen device, the large-screen device can not fully utilize the display screen, and only displays the projected interface in a partial area, and other areas are filled with black edges, resulting in a decrease in user experience. Therefore, how to ensure user experience during screen projection has become a technical problem to be solved. SUMMARY

[0004] The present application provides a screen projection method and device. When a first device projects onto a second device, the projection data sent by the first device can include projection data of multiple interfaces, so that the second device can simultaneously display interface content of the multiple interfaces, fully utilize the display area of the display screen, and will not appear black edge filling, which helps to improve user experience.

[0005] In a first aspect, a screen projection method is provided. The method includes detecting a user's screen projection operation on a first interface, and in response to the screen projection operation, sending first projection data to a second device, the first projection data including projection data of the first interface and N interfaces associated with the first interface, N being an integer greater than or equal to 1.

[0006] In the present application, when a first device projects onto a second device, the projection data sent by the first device can include projection data of multiple interfaces, so that the second device can simultaneously display interface content of the multiple interfaces, fully utilize the display area of the display screen, and will not appear black edge filling, which helps to improve user experience.

[0007] With reference to the first aspect, in some implementations of the first aspect, the method further includes: receiving first indication information sent by the second device, the first indication information being used to indicate the first operation of the user on the second device; and sending second screen projection data to the second device according to the first indication information, wherein the second screen projection data comprises screen projection data of a second interface, and the second interface is associated with the first operation.

[0008] In the implementation, when the second device detects the operation of the user, the second device can send indication information to the first device to indicate the operation of the user, and the first device can generate new screen projection data according to the indication information and send the new screen projection data to the second device, so that the second device can update the displayed interface, and the user can interact with the second device, thereby improving the user experience.

[0009] With reference to the first aspect, in some implementations of the first aspect, the method further includes: controlling the display device to display the second interface to replace the first interface.

[0010] For example, the display device can be a display screen.

[0011] With reference to the first aspect, in some implementations of the first aspect, the first interface comprises an icon of a first application program, and the first operation is an operation of clicking the icon of the first application program, wherein the second screen projection data further comprises screen projection data of X interfaces associated with the second interface, X is an integer greater than or equal to 1.

[0012] With reference to the first aspect, in some implementations of the first aspect, when the second device is in a horizontal screen state and the second interface supports a horizontal screen layout mode, the second screen projection data comprises screen projection data corresponding to the horizontal screen layout mode of the second interface.

[0013] With reference to the first aspect, in some implementations of the first aspect, the first interface is an interface of a first application program, the operation of the user is an operation of returning to a desktop, and the second interface is a main interface of the desktop.

[0014] With reference to the first aspect, in some implementations of the first aspect, the first interface comprises a first control, and the first control corresponds to a first service, the method further includes: receiving second indication information sent by the second device, the second indication information being used to indicate an operation of the user clicking the first control; and sending service data of the first service to the second device.

[0015] In a second aspect, a screen projection method is provided. The method comprises: receiving first screen projection data sent by a first device, the first screen projection data comprising screen projection data of a first interface and N interfaces associated with the first interface, N is an integer greater than or equal to 1; and controlling a display device to simultaneously display interface content included in the first interface and M interfaces of the N interfaces according to the first screen projection data, N is greater than or equal to M, M is an integer greater than or equal to 1.

[0016] Exemplarily, the display device can be a display screen.

[0017] In the embodiments of the present application, when the first device projects a screen to the second device, the screen projection data sent by the first device can comprise screen projection data of multiple interfaces, so that the second device can simultaneously display interface content of the multiple interfaces, fully utilize the display area of the display screen, and avoid black edge filling, which helps to improve the user experience.

[0018] In combination with the second aspect, in some implementations of the second aspect, the method further comprises: in response to a first operation of a user, sending first indication information to the first device, the first indication information being used to indicate the first operation of the user; receiving second screen projection data sent by the first device, wherein the second screen projection data comprises screen projection data of a second interface associated with the first operation of the user; and updating interface content according to the second screen projection data.

[0019] In the embodiments of the present application, when the second device detects an operation of a user, the second device can send indication information to the first device to indicate the operation of the user, the first device can generate new screen projection data according to the indication information and send the new screen projection data to the second device, so that the second device can update the displayed interface, so that the user can interact with the second device, and the user experience is improved.

[0020] In combination with the second aspect, in some implementations of the second aspect, the first interface comprises an icon of a first application program, the first operation is clicking the icon of the first application program, the second interface is an interface of the first application program, the second screen projection data further comprises screen projection data of X interfaces associated with the second interface, X is an integer greater than or equal to 1, and the updating of the interface content according to the second screen projection data comprises: controlling the display device to simultaneously display interface content included in the second interface and Y interfaces of the X interfaces, and no longer display interface content included in the first interface and the M interfaces, wherein Y is an integer greater than or equal to 1.

[0021] With reference to the second aspect, in some implementations of the second aspect, the first interface includes an icon of the first application program, the first operation is clicking the icon of the first application program, the second interface is an interface of the first application program, and the updating the interface content according to the second screen projection data includes: controlling the display device to display interface content included in the second interface to replace interface content included in the first interface.

[0022] With reference to the second aspect, in some implementations of the second aspect, the first interface includes an icon of the first application program, the second interface supports a horizontal screen layout mode, the first operation is clicking the icon of the first application program, the second interface is an interface of the first application program, and the updating the interface content according to the second screen projection data includes: controlling the display device to display interface content included in the second interface in full screen.

[0023] With reference to the second aspect, in some implementations of the second aspect, the first interface is an interface of the first application program, the first operation is an operation of returning to a desktop, the second interface is a main interface of the desktop, and the updating the interface content according to the second screen projection data includes: controlling the display device to simultaneously display interface content included in the second interface and B interfaces of the V interfaces, and no longer display interface content included in the first interface and the M interfaces, where V≥B≥0, and Y is an integer.

[0024] With reference to the second aspect, in some implementations of the second aspect, the first interface includes a first control corresponding to a first service, and the method further includes: in response to an operation of the user clicking the first control, sending second indication information to the first device; and receiving first service data sent by the first device.

[0025] With reference to the second aspect, in some implementations of the second aspect, the method further includes: receiving third screen projection data sent by a third device; and according to the first screen projection data and the third screen projection data, controlling the display device to simultaneously display interface content included in the first interface and a third interface, and no longer display interface content included in the M interfaces, where the third interface is an interface currently displayed by the third device.

[0026] In a third aspect, a system is provided, including a first device and a second device, where the first device is configured to, in response to a screen projection operation of a user on a first interface, send first screen projection data to the second device, the first screen projection data including screen projection data of the first interface and N interfaces associated with the first interface, N≥1 and N is an integer; and the second device is configured to, according to the first screen projection data, simultaneously display interface content included in the first interface and M interfaces of the N interfaces, N≥M≥1, and M is an integer.

[0027] With reference to the third aspect, in some implementations of the third aspect, the second device is further configured to: in response to the first operation of the user, send, to the first device, the first indication information, the first indication information being used to indicate the first operation of the user on the second device; and the first device is further configured to: send, to the second device, second casting data, the second casting data comprising casting data of a second interface, the second interface being associated with the first operation; and the second device is further configured to update the interface content according to the second casting data.

[0028] With reference to the third aspect, in some implementations of the third aspect, the first interface comprises an icon of a first application program, the first operation is clicking the icon of the first application program, and the second interface is an interface of the first application program; and the second device is specifically configured to display interface content included in the second interface to replace interface content included in the first interface.

[0029] With reference to the third aspect, in some implementations of the third aspect, the first interface comprises an icon of a first application program, the first operation is clicking the icon of the first application program, and the second interface is an interface of the first application program; and the second device is specifically configured to display interface content included in the second interface to replace interface content included in the first interface.

[0030] With reference to the third aspect, in some implementations of the third aspect, the second device is in a landscape state, the first interface comprises an icon of a first application program, the second interface supports a landscape layout mode, the first operation is clicking the icon of the first application program, and the second interface is an interface of the first application program; and the second device is specifically configured to display interface content included in the second interface in full screen.

[0031] With reference to the third aspect, in some implementations of the third aspect, the first interface is an interface of a first application program, the first operation is a return-to-desktop operation, and the second interface is a main interface of a desktop; and the second device is specifically configured to display interface content included in the second interface and interface content included in B interfaces of V interfaces, and no longer display interface content included in the first interface and interface content included in M interfaces, where V ≥ B ≥ 0, and Y is an integer.

[0032] With reference to the third aspect, in some implementations of the third aspect, the first interface comprises a first control, the first control corresponding to a first service, and the second device is further configured to: in response to an operation of the user clicking the first control, send, to the first device, second indication information; and the first device is further configured to: send, to the second device, service data of the first service.

[0033] With reference to the third aspect, in some implementations of the third aspect, the second device further functions to: receive third casting data sent by a third device; and control the display apparatus to simultaneously display interface content included in the first interface and a third interface according to the first casting data and the third casting data, and no longer display interface content included in the M interfaces, wherein the third interface is an interface currently displayed by the third device.

[0034] A fourth aspect provides a casting method, including: detecting a casting operation of a first interface by a user; in response to the casting operation, sending first casting data to a second device, the first casting data including casting data of the first interface; receiving first indication information sent by the second device, the first indication information being used to indicate an operation of clicking an icon of a first application on the first interface by the user on the second device; and sending second casting data to the second device according to the first indication information, wherein the second casting data includes casting data of the first interface and a second interface, and the second interface is an interface of the first application.

[0035] A fifth aspect provides a casting method, including: receiving first casting data sent by a first device, the first casting data including casting data of a first interface; controlling a display apparatus to display the first interface according to the first casting data; in response to an operation of clicking an icon of a first application on the first interface by a user, sending first indication information to the first device; receiving second casting data sent by the first device, wherein the second casting data includes casting data of the first interface and a second interface, and the second interface is an interface of the first application; and simultaneously displaying the first interface and the second interface according to the second casting data.

[0036] A sixth aspect provides a device, including one or more processors; one or more memories; and one or more programs stored in the one or more memories, the one or more programs including instructions which, when executed on the one or more processors, cause the above aspect or any possible implementation of the above aspect to be performed.

[0037] A seventh aspect provides a readable storage medium, including a program or instructions, which, when executed on a device, cause the above aspect or any possible implementation of the above aspect to be performed.

[0038] An eighth aspect provides a program product, including a program or instructions, which, when executed on a device, cause the above aspect or any possible implementation of the above aspect to be performed.

[0039] In a ninth aspect, a program is provided, which, when running on a device, causes the above aspect or any possible implementation manner of the above aspect to be performed.

[0040] In a tenth aspect, a device is provided, which comprises modules / units for performing the method of the above aspect or any possible implementation manner of the above aspect; these modules / units can be implemented by hardware, or by hardware executing corresponding software. BRIEF DESCRIPTION OF DRAWINGS

[0041] FIG. 1 is a structural schematic diagram of a device provided by an embodiment of the present application.

[0042] FIG. 2 is a software structure block diagram of a device provided by an embodiment of the present application.

[0043] FIG. 3 is a set of GUIs provided by an embodiment of the present application.

[0044] FIG. 4-FIG. 6 are a set of GUIs provided by an embodiment of the present application.

[0045] FIG. 7-FIG. 8 are a set of GUIs provided by an embodiment of the present application.

[0046] FIG. 9-FIG. 11 are a set of GUIs provided by an embodiment of the present application.

[0047] FIG. 12 is a set of GUIs provided by an embodiment of the present application.

[0048] FIG. 13-FIG. 16 are a set of GUIs provided by an embodiment of the present application.

[0049] FIG. 17 is a set of GUIs provided by an embodiment of the present application.

[0050] FIG. 18 is a schematic flowchart of a screen projection method provided by an embodiment of the present application.

[0051] FIG. 19 is a schematic flowchart of a screen projection method provided by an embodiment of the present application. DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings.

[0053] The terminology used in the following description merely for the purpose of describing particular embodiments and is not intended to limit the application. As used in this description and the accompanying claims, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. The term "or" as used herein is used to denote exclusive, at least, or other (possibly inclusive) options, using either the definite term "or" or an open term such as "at least," "including," and "that involves." The term "a kind of" is used to denote a "or" relationship between the preceding and following terms.

[0054] Reference throughout this specification to "one embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. Thus, the appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in additional embodiments," and the like in various places throughout this specification are not necessarily referring to the same embodiment, unless otherwise expressly specified. The terms "including," "containing," "having," and variations thereof are meant to encompass the terms "including," "containing," "having," and variations thereof, unless otherwise expressly specified.

[0055] The following describes devices, user interfaces for such devices, and embodiments for using such devices. In some embodiments, a device can be a portable device that also contains other functionality such as personal digital assistant and / or music player functionality, such as a cell phone, a tablet, a wearable device with wireless communication capabilities (e.g., a smart watch), etc. Exemplary embodiments of portable devices include, but are not limited to, portable devices running the iOS® operating system, the Android® operating system, the Windows® operating system, the or other operating systems. The portable devices described above can also be other portable devices, such as a laptop, etc. It will also be appreciated that in other embodiments, the devices described above also include desktop computers.

[0056] Exemplarily, FIG. 1 shows a structural schematic diagram of a device 100. The device 100 can include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a loudspeaker 170A, a receiver 170B, a microphone 170C, a headset jack 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 can include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0057] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the device 100. In other embodiments of the present application, the device 100 can include more or fewer components than shown, or combine certain components, or split certain components, or different arrangement of components. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0058] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units can be independent devices, or can be integrated in one or more processors.

[0059] The controller can be the nerve center and command center of the device 100. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching and executing instructions.

[0060] The processor 110 can also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can hold instructions or data that the processor 110 has just used or is using repeatedly. If the processor 110 needs to use the instructions or data again, it can call them directly from the memory. This avoids repeated access and reduces the latency of the processor 110, thus improving the efficiency of the system.

[0061] In some embodiments, the processor 110 can include one or more interfaces. The interfaces can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0062] The wireless communication function of the device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor, etc.

[0063] The antenna 1 and the antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example: the antenna 1 can be multiplexed as a diversity antenna of a wireless local area network. In some other embodiments, the antennas can be used in combination with tuning switches.

[0064] The mobile communication module 150 can provide a solution including 2G / 3G / 4G / 5G, etc. wireless communication applied on the device 100. The mobile communication module 150 can include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves by the antenna 1, and perform filtering, amplification, etc. on the received electromagnetic waves, and transfer to the modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor, and radiate as electromagnetic waves by the antenna 1. In some embodiments, at least part of the functional modules of the mobile communication module 150 can be disposed in the processor 110. In some embodiments, at least part of the functional modules of the mobile communication module 150 can be disposed in the same device as at least part of the modules of the processor 110.

[0065] The modem processor can include a modulator and a demodulator. The modulator is configured to modulate a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is configured to demodulate a received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal processed by the baseband processor is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the microphone 170B, etc.), or displays an image or a video through the display screen 194. In some embodiments, the modem processor can be a separate device. In other embodiments, the modem processor can be independent of the processor 110, and disposed in the same device as the mobile communication module 150 or other functional modules.

[0066] The wireless communication module 160 can provide solutions for wireless communication including wireless local area networks (WLAN) (e.g., wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, etc. The wireless communication module 160 can be one or more devices that integrate at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency-modulates and filters the electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 160 can also receive signals to be transmitted from the processor 110, frequency-modulate them, amplify them, and radiate them as electromagnetic waves via the antenna 2.

[0067] In some embodiments, the antenna 1 and the mobile communication module 150 of the device 100 are coupled, and the antenna 2 and the wireless communication module 160 are coupled, so that the device 100 can communicate with networks and other devices through wireless communication technologies.

[0068] The device 100 implements display functions through a GPU, a display screen 194, and an application processor, etc. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs that execute program instructions to generate or change display information.

[0069] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. In some embodiments, the device 100 can include 1 or N display screens 194, N being a positive integer greater than 1.

[0070] The internal memory 121 can be used to store program codes including instructions. The processor 110 performs various functional applications and data processing of the device 100 by running the instructions stored in the internal memory 121. The internal memory 121 can include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program (such as an image playing function) required by a function, and the like. The data storage area can store data (such as audio data) created during the use of the device 100, and the like. In addition, the internal memory 121 can include a high-speed random access memory, and can further include a non-volatile memory such as at least one disk storage device, a flash memory device, and the like.

[0071] The device 100 can realize an audio function through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, the application processor, and the like. For example, music playing, and the like.

[0072] The audio module 170 is used to convert digital audio information into an analog audio signal output, and is also used to convert an analog audio input into a digital audio signal. The audio module 170 can also be used to encode and decode an audio signal. In some embodiments, the audio module 170 can be arranged in the processor 110, or part of the function modules of the audio module 170 can be arranged in the processor 110.

[0073] The speaker 170A, also referred to as a "loudspeaker", is used to convert an audio electrical signal into a sound signal.

[0074] The receiver 170B, also referred to as a "earpiece", is used to convert an audio electrical signal into a sound signal.

[0075] The microphone 170C, also referred to as a "microphone", "sound transducer", is used to convert a sound signal into an electrical signal.

[0076] FIG. 2 is a software structure block diagram of the device 100 according to an embodiment of the present application. A layered architecture divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through a software interface. In some embodiments, the system is divided into four layers, from top to bottom, an application layer, an application framework layer, a system service layer, and a kernel layer.

[0077] As shown in FIG. 2, the application layer can include a launcher, a super desktop assistant module, a screen projection display module, settings, a calendar, a map, a WLAN, music, a short message, and the like.

[0078] The launcher is responsible for managing and presenting interfaces, including desktops, interfaces of applications, and the like. The launcher can obtain application icons, service cards, folders, and layout information of a desktop, and can draw and render one or more interfaces into an interface ability when casting. The ability can be understood as an abstraction of the capabilities of an application, and is an important component of an application. An application can have multiple capabilities, i.e., can include multiple abilities. The interface ability is used to provide the ability to interact with the user. In other words, the interface ability can be understood as a screen, and the device can draw and render one or more interfaces into the screen.

[0079] It can be understood that when the launcher of the device draws and renders one or more interfaces into an interface ability and casts to another device, the other device can simultaneously display the one or more interfaces, and the user can interact with the one or more interfaces through the other device. For specific descriptions, please refer to the following.

[0080] It should be noted that the ability is only a name, and different operating systems can have different names, for example, it can also be named as activity in other operating systems.

[0081] The launcher at the sink end is also used to build a casting page (surface).

[0082] Casting display module: when the device is a sink, the application layer can include a casting display module, which is used to display the casting content of the virtual screen of the source, and can interact with the virtual screen.

[0083] Super desktop helper module: the super desktop helper module is used to be responsible for the connection process of the source, maintain the communication ability of the source and the sink, communicate with the launcher of the sink and notify the layout related events of the source.

[0084] The application framework layer provides an application programming interface (API) and a programming framework for the applications of the application layer. The application framework layer can include some pre-defined functions.

[0085] As shown in FIG. 2, the application framework layer can include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0086] The window manager is used to manage window programs. The window manager can acquire the display screen size, determine whether there is a status bar, lock the screen, and capture the screen, etc. The content provider is used to store and acquire data, and make the data accessible to the application program. The data can include video, image, etc.

[0087] The view system includes visual controls, such as a control for displaying text, a control for displaying pictures, etc., such as the indication information for prompting the virtual shutter key in the embodiment of the present application. The view system can be used to build an application program. The display interface can be composed of one or more views. For example, a view for displaying text and a view for displaying pictures.

[0088] The phone manager is used to provide the communication function of the device 100. For example, the management of the call state (including connection, hang-up, etc.).

[0089] The resource manager provides various resources for the application program, such as icons, pictures, layout files, video files, etc.

[0090] The notification manager makes the application program display notification information in the status bar, which can be used to convey the notification type of message, which can automatically disappear after a short stay, without user interaction. For example, the notification manager is used to notify the completion of downloading, message reminder, etc. The notification manager can also be a notification in the form of a chart or a scroll bar text appearing in the top status bar of the system, such as the notification of the application program running in the background, and can also be a notification in the form of a dialogue window appearing on the screen. For example, prompting text information in the status bar, issuing a prompt sound, etc.

[0091] The application program layer and the application program framework layer run in the virtual machine. The virtual machine executes the java files of the application program layer and the application program framework layer into binary files. The virtual machine is used to perform the management of the object life cycle, the management of the stack, the management of the thread, the management of the security and the exception, and the garbage collection, etc.

[0092] The system library can include a plurality of functional modules. For example: the surface manager, the media library, the three-dimensional graphics processing library (for example: OpenGL ES), the two-dimensional graphics engine (for example: SGL), the collaborative configuration management module, the connection module, the distributed media module, the distributed hardware module, the projection module, and the super desktop module.

[0093] The surface manager is used to manage the display subsystem, and provides the fusion of two-dimensional and three-dimensional layers for a plurality of application programs.

[0094] The media library supports multiple commonly used audio, video format playback and recording, and static image files, etc. The media library can support multiple audio and video encoding formats, such as: MPEG4, H.264, MP3, etc.

[0095] The three-dimensional graphics processing library is used to implement three-dimensional graphics drawing, image rendering, synthesis, and layer processing, etc.

[0096] The two-dimensional graphics engine is a drawing engine for two-dimensional drawing.

[0097] The collaborative configuration management module is used to manage the strategy of cross-device application migration and hardware migration.

[0098] The connection module is used to be responsible for distributed business and processing business conflicts.

[0099] The distributed media module is used to support multiple audio, audio distributed capabilities, cross-device camera capability construction, etc.

[0100] The distributed hardware module is used to provide hardware virtualization.

[0101] The screen projection module is used to project the interface content of the virtual screen into a specified container.

[0102] The super desktop module is used to be responsible for interacting with other modules and providing the super desktop assistant module with the ability of double-end communication.

[0103] In addition, the system library can also include a state monitoring service module, such as a physical state identification module, which is used to analyze and identify user gestures; a sensor service module, which is used to monitor sensor data uploaded by various sensors in the hardware layer to determine the physical state of the device 100.

[0104] The kernel layer is the layer between hardware and software. The kernel layer at least includes display drivers, camera drivers, audio drivers, and sensor drivers.

[0105] The hardware layer can include various sensors, such as the various sensors introduced in FIG. 1, the acceleration sensor, the gyroscope sensor, the touch sensor, etc. involved in the embodiments of the present application.

[0106] It should be noted that only one way of dividing the system framework is taken as an example in FIG. 2, and should not be understood as a specific limitation on the embodiments of the present application. In the embodiments of the present application, when the operating system carried by the device is different, different frameworks can be used for different operating systems. It can be understood that when different frameworks are adopted, the division manner of each layer of the framework, the specific naming, and which layer the above-mentioned modules are located in can be different.

[0107] Currently, an interface displayed by one device can be projected to another device through distributed technology. For example, a mobile phone can project the interface being displayed to a television. For another example, a mobile phone can project the interface being displayed to a cockpit screen of a vehicle. Since the sizes and shapes of display screens of different devices can be different, when one device projects an interface to another device, the other device can not fully utilize the display screen and only display the projected interface in a partial area and fill other areas with black edges, resulting in a decrease in user experience, which will be described below in combination with FIG. 3.

[0108] FIG. 3 shows a set of graphical user interfaces (GUIs) provided by an embodiment of the present application.

[0109] As shown in (a) of FIG. 3, device 1 displays interface 301, which is a desktop. Device 1 can project interface 301 to device 2. After receiving the projection data sent by device 1, device 2 can display a GUI as shown in (b) of FIG. 3. In the GUI shown in FIG. 3, device 1 is a mobile phone and device 2 is a television.

[0110] As shown in (b) of FIG. 3, after receiving the projection data of device 1, device 2 can display interface 302 on the display screen, where interface 302 has the same interface content as interface 301. As can be seen from (b) of FIG. 3, since the display screen of the television is in a landscape state and the display screen of the mobile phone is in a portrait state when the mobile phone displays interface 301, device 2 only displays interface 302 in the middle area of the display screen and fills the partial areas on both sides of the display screen with black edges when displaying interface 302, so as to ensure the normal proportion of the projected interface.

[0111] It should be noted that the landscape state refers to the state of a device when the device is in a landscape display state. The portrait state refers to the state of a device when the device is in a portrait display state. In the landscape state, the display screen of the device is in a horizontal strip shape. In the portrait state, the display screen of the device is in a vertical strip shape.

[0112] As can be known from the description of FIG. 3, when device 1 projects an interface to device 2, since the sizes and shapes of the display screens of the two devices are different, device 2 cannot fully utilize the display area of the display screen when displaying the projected interface and only displays the interface content of the projected interface in the middle area, which wastes the display area and reduces the user experience.

[0113] In addition, the above-mentioned screen projection manner is only that the device 1 sends the screen projection data of the interface 301 to the device 2, and then the device 2 displays the interface 302 according to the screen projection data, but the interface 302 displayed by the device 2 includes controls that are not controllable by the user. In other words, the user cannot interact with the interface 302 through the device 2.

[0114] In summary, when one device projects the displayed interface to another device, the other device only displays the projected interface in a partial area, and other areas are filled with black edges, and the user cannot interact with the interface through the other device, resulting in a decline in user experience. Based on this, how to provide a better screen projection experience has become a technical problem to be solved, and the following will first introduce the screen projection method provided by the embodiments of the present application in combination with the GUI.

[0115] FIGS. 4-6 show a set of GUIs provided by the embodiments of the present application.

[0116] As shown in (a) of FIG. 4, the device 1 displays an interface 401, which is a desktop. As can be seen from the figure, the desktop of the device 1 includes four interfaces in total, and the interface 401 displayed by the device 1 is the second interface in the desktop, which can also be referred to as the main interface. When the device 1 responds to the right swipe operation of the user, a GUI as shown in (b) of FIG. 4 can be displayed, that is, the interface 402 can be displayed, wherein the interface 402 is the first interface of the desktop, and the interface 402 can also be referred to as the minus one screen interface.

[0117] Similarly, when the device 1 responds to the left swipe operation of the user, a GUI as shown in (c) of FIG. 4 or a GUI as shown in (d) of FIG. 4 can be displayed, that is, the interface 403 and the interface 404 are displayed in turn, wherein the interface 403 is the third interface of the desktop, and the interface 404 is the fourth interface of the desktop.

[0118] In summary, it can be understood that the interface 402 is on the left side of the interface 401, the interface 401 is on the left side of the interface 403, and the interface 403 is on the left side of the interface 404.

[0119] In some embodiments, when the device 1 displays the interface 401, the interface 403 and the interface 404, the device 1 can divide the interface into three areas, namely a work area, a dock area and a navigation point area. The work area can include one or more application icons, service cards, folders, etc. It can be understood that the application icons, service cards, folders included in the work area of the interface 401, the interface 403 and the interface 404 can be different. Since the interface 401 is the main interface, the work area of the interface 401 can further include time, date, week, etc.

[0120] The dock area can include one or more application icons, and the application icons in the dock area can be understood as icons of resident applications. The dock area can also be referred to as a resident application area. In other words, the dock areas of the interfaces 401, 403, and 404 can include the same application icons.

[0121] The navigation point area can include a plurality of navigation points, and the device 1 can indicate the current display interface as the mth interface of the desktop by marking the navigation points and the like.

[0122] It should be noted that the division of the interfaces 401, 403, and 404 into three areas is only an example, and in other embodiments, the interfaces 401, 403, and 404 can include more or fewer areas.

[0123] With reference to (a) of FIG. 4, when the device 1 detects the user operation of swiping down the status bar, in response to the operation, a GUI as shown in (a) of FIG. 5 can be displayed.

[0124] As shown in (a) of FIG. 5, in response to the user operation of swiping down the status bar, the device 1 can display a control center interface 405 including a screen projection control 406. When the device 1 detects the user operation of clicking the screen projection control 406, in response to the operation, a GUI as shown in (b) of FIG. 5 can be displayed.

[0125] As shown in (b) of FIG. 5, in response to the user operation of clicking the screen projection control 406, the device 1 can display an option box 407 including a control 408 corresponding to the device 2. When the device 1 detects the user operation of clicking the control 408, in response to the operation, the device 1 can generate screen projection data 1 according to the display parameters of the device 2, the interfaces 401, 402, 403, and 404, and then send the screen projection data 1 to the device 2, so that the device 2 can display a GUI as shown in (c) of FIG. 5 or a GUI as shown in (b) of FIG. 6.

[0126] As shown in (c) of FIG. 5, the device 2 receives the screen projection data sent by the device 1, and can display a plurality of interfaces including interfaces 409, 410, and 411 at the same time, wherein the interface 409 is on the left side of the interface 410, the interface 410 is on the left side of the interface 411, the interface 409 is associated with the interface 402, the interface 410 is associated with the interface 401, and the interface 411 is associated with the interface 403. Two interface associations can be understood as two interfaces having the same or partially same interface content.

[0127] In some embodiments, the device 2 can identify the interface currently displayed by the device 1.

[0128] For example, taking the interface 401 as an example, which is the interface currently displayed by the device 1, the device 2 can highlight the interface 410.

[0129] It should be noted that (c) in FIG. 5 only takes the device 2 simultaneously displaying the interface 409, the interface 410 and the interface 411 as an example, but the embodiments of the present application are not limited thereto.

[0130] It should be further noted that when the device 2 simultaneously displays multiple interfaces, the positional relationship between the multiple interfaces can be the same as the positional relationship between the multiple interfaces in the device 1.

[0131] For example, the device 2 can simultaneously display the interface 409 and the interface 410, where the interface 409 is on the left side of the interface 410.

[0132] For another example, the device 2 can simultaneously display the interface 410 and the interface 411, where the interface 410 is on the left side of the interface 411.

[0133] For another example, the device 2 can simultaneously display the interface 410, the interface 411 and the interface 412, where the interface 410 is on the left side of the interface 411, and the interface 411 is on the left side of the interface 412.

[0134] For another example, the device 2 can simultaneously display the interface 409, the interface 410, the interface 411 and the interface 412, where the interface 409 is on the left side of the interface 410, the interface 410 is on the left side of the interface 411, and the interface 411 is on the left side of the interface 412.

[0135] Continuing to refer to (c) in FIG. 5, when the device 2 detects the switching interface operation of the user, in response to the operation, the device 2 can display a GUI as shown in (a) in FIG. 6.

[0136] The device 2 detecting the switching interface operation of the user can include the following possible implementation manners:

[0137] In one possible implementation manner, the display screen of the device 2 is a touchable display screen, and the switching interface operation of the user can be a sliding operation of the user on the display screen of the device 2, that is, the device 2 can detect the switching interface operation of the user through the display screen.

[0138] In one possible implementation manner, the device 2 is configured with a camera, and the switching interface operation of the user can be a gesture of the user in the air, that is, the device 2 can detect the switching interface operation of the user through the camera.

[0139] In one possible implementation manner, the device 2 is configured with a remote controller, and the switching interface operation of the user can be the user pressing a control button on the remote controller, that is, the device 2 can detect the switching interface operation of the user through the remote controller.

[0140] In a possible implementation, the operation of the switching interface of the user can be a voice instruction of the user, that is, the device 2 can detect the operation of the switching interface of the user through a microphone.

[0141] As shown in (a) of FIG. 6, in response to the operation of the switching interface of the user, the device 2 can adjust the displayed interface, that is, display the interface 412 and no longer display the interface 409.

[0142] It should be noted that (c) of FIG. 5 and (a) of FIG. 6 are only examples in which the device 2 simultaneously displays the interface 409, the interface 410 and the interface 411 and displays the interface 412 and no longer displays the interface 409 after detecting the leftward sliding operation of the user, but should not be understood as a specific limitation on the embodiments of the present application.

[0143] For example, the device 2 can simultaneously display the interface 409 and the interface 410, where the interface 409 is on the left side of the interface 410. When the device 2 detects the leftward sliding operation of the user, the device 2 can display the interface 411 and the interface 412 in sequence according to the magnitude of the leftward sliding operation of the user. When the device 2 displays the interface 411, the device 2 can no longer display the interface 409, that is, the device 2 simultaneously displays the interface 410 and the interface 411. When the device 2 displays the interface 412, the device 2 can no longer display the interface 410, that is, the device 2 simultaneously displays the interface 411 and the interface 412.

[0144] For another example, the device 2 can simultaneously display the interface 410 and the interface 411, where the interface 410 is on the left side of the interface 411. When the device 2 detects the leftward sliding operation of the user, the device 2 can display the interface 412 and no longer display the interface 410, that is, the device 2 simultaneously displays the interface 411 and the interface 412. When the device 2 detects the rightward sliding operation of the user, the device 2 can display the interface 409 and no longer display the interface 411, that is, the device 2 can simultaneously display the interface 409 and the interface 410.

[0145] For another example, the device 2 can simultaneously display the interface 410, the interface 411 and the interface 412, where the interface 410 is on the left side of the interface 411 and the interface 411 is on the left side of the interface 412. When the device 2 detects the rightward sliding operation of the user, the device 2 can display the interface 409 and no longer display the interface 412, that is, the device 2 can simultaneously display the interface 409, the interface 410 and the interface 411.

[0146] As described above, the interface 401, the interface 403 and the interface 404 can be divided into three areas, i.e., a work area, a dock area and a navigation point area. Since the icons of the applications in the dock area of the interface 401, the interface 403 and the interface 404 can be the same, when the device 2 displays multiple interfaces among the interface 401, the interface 403 and the interface 404 at the same time, if each interface also includes the dock area, part of the display area will be wasted. Similarly, if each interface also includes the navigation point area, part of the display area will also be wasted. Based on this, the device 1 can separate the dock area and the navigation point area of the interface 401, the interface 403 and the interface 404 when generating the screen projection data according to the interface 401, the interface 402, the interface 403 and the interface 404, i.e., the device 1 can display the GUI as shown in (b) of FIG. 6.

[0147] As shown in (b) of FIG. 6, the device 2 can simultaneously display the interface 413, the interface 414 and the interface 415, wherein the interface 413 includes the interface content of the interface 402, the interface 414 includes the interface content of the work area of the interface 401, and the interface 415 includes the interface content of the work area of the interface 403. In addition, the device 2 can also separately display the dock area 416 and the navigation bar 417, wherein the application icons included in the dock area 416 are the same as the icons of the applications included in the dock area of the interface 401, the interface 403 and the interface 404.

[0148] It can be understood that the device 1 can render the original navigation point area as the navigation bar when generating the screen projection data, which can better adapt to the application scenario of simultaneously displaying multiple interfaces.

[0149] It should be noted that the interface content displayed in (c) of FIG. 5 and (a) and (b) of FIG. 6 can also be referred to as the interface content of the super desktop of the device 2.

[0150] It should also be noted that in the GUIs shown in FIGS. 4-6, the interface content of the multiple interfaces of the desktop of the device 1 displayed by the device 2 through multiple interfaces is taken as an example, but the embodiments of the present application are not limited thereto. In some other embodiments of the present application, the device 2 can also display the interface content of the multiple interfaces of the desktop of the device 1 through one interface.

[0151] In the embodiments of the present application, the device 1 can include the screen projection data of multiple interfaces in the screen projection data sent to the device 2 when projecting the screen to the device 2, so that the device 2 can simultaneously display the interface content of multiple interfaces, fully utilize the display screen, and will not appear black edge filling, which helps to improve the user experience.

[0152] The device 1 displays the desktop to the device 2 in the examples of FIG. 4-FIG. 6, but the embodiments of the present application are not limited thereto, and in some other embodiments of the present application, the device 1 can also display the interface of an application to the device 2, which will be described below in conjunction with FIG. 7 and FIG. 8.

[0153] FIG. 7-FIG. 8 show a set of GUIs provided by the embodiments of the present application.

[0154] As shown in (a)-(d) of FIG. 7, the device 1 displays the interface 501, which is the interface of the application 1. The device 1 can display one or more controls, such as the control 502, the control 503, and the control 504, on the interface 501, which are used to control the device 1 to display other interfaces. The device 1 can display the interface 505 when detecting the operation of the user clicking the control 502. The device 1 can display the interface 506 when detecting the operation of the user clicking the control 503. The device 1 can display the interface 507 when detecting the operation of the user clicking the control 504.

[0155] It can be understood that the interface 501 has an association relationship with the interface 505, the interface 506, and the interface 507, because the interface 501 includes the controls for displaying the interface 505, the interface 506, and the interface 507.

[0156] The device 1 detects the operation of the user displaying to the device 2 when displaying the interface 501, and in response to the operation, can generate the screen projection data according to the display parameters of the device 2, the interface content of the interface 501, the interface 505, the interface 506, and the interface 507, and then send the screen projection data to the device 2. The device 2 can display the GUIs shown in FIG. 8 after receiving the screen projection data.

[0157] It should be understood that the description of the operation of the user projecting can refer to the description of FIG. 4, and for brevity, will not be repeated here.

[0158] As shown in FIG. 8, the device 2 can display the interface 508, the interface 509, the interface 510, and the interface 511 after receiving the screen projection data sent by the device 1, wherein the interface 508 is associated with the interface 501, the interface 509 is associated with the interface 505, the interface 510 is associated with the interface 506, and the interface 511 is associated with the interface 507.

[0159] In the embodiments of the present application, the device 1 can include the screen projection data of multiple interfaces in the screen projection data when projecting to the device 2, so that the device 2 can simultaneously display the interface content included in multiple interfaces, fully utilize the display screen, and will not appear black edge filling, which helps to improve the user experience.

[0160] In the above, the device 1 detects the user's screen projection operation when displaying the interface of the desktop or the application program, can generate the screen projection data according to the multiple interfaces of the desktop or the multiple interfaces of the application program, and then sends the screen projection data to the device 2, so that the device 2 can display multiple interfaces at the same time. In the following, the user's interaction with the device 2 will be introduced in combination with the GUI.

[0161] FIGS. 9-11 show a set of GUIs provided by the embodiments of the present application.

[0162] As shown in (a) and (b) of FIG. 9, the device 1 displays an interface 601, which is the main interface of the desktop. The device 1 can send the screen projection data 1 to the device 2 in response to the user's screen projection operation. The device 2 can display an interface 602, an interface 603, an interface 604, a dock area 605 and a navigation bar area 606 after receiving the screen projection data 1 sent by the device 1, wherein the interface 603 is associated with the interface 601. The device 2 can display an icon 607, an icon 608, an icon 609 and a folder 610 in the interface 603. The icon 607 corresponds to a music application program including multiple interfaces, the icon 608 corresponds to a calculator application program including one interface, the icon 609 corresponds to a game application program supporting a landscape layout mode, and the folder 610 includes the icons of one or more application programs.

[0163] When the device 2 detects the user's operation of clicking the icon 607, the device 2 can send indication information 1 to the device 1 in response to the operation, wherein the indication information 1 is used to indicate the user's operation of clicking the icon 607. The device 1 can generate the screen projection data 2 according to the indication information 1 after receiving the indication information 1, and then send the screen projection data 2 to the device 2. The device 2 can update the displayed interface according to the screen projection data 2, i.e., display the GUI as shown in (a) of FIG. 10.

[0164] As shown in (a) of FIG. 10, the device 2 can display multiple interfaces of the music application program at the same time after receiving the screen projection data 2, and no longer display the interface 602, the interface 603, the interface 604, the dock area 605 and the navigation bar area 606.

[0165] When the device 2 detects the user's operation of clicking the icon 608, the device 2 can send indication information 2 to the device 1 in response to the operation, wherein the indication information 2 is used to indicate the user's operation of clicking the icon 608. The device 1 can generate the screen projection data 3 according to the indication information 2 after receiving the indication information 2, and then send the screen projection data 3 to the device 2. The device 2 can update the displayed interface according to the screen projection data 3, i.e., display the GUI as shown in (b) of FIG. 10.

[0166] As shown in (b) of FIG. 10, after receiving the screen projection data 3, the device 2 can replace the interface 603 with an interface 611, which is an interface of a calculator application. In other words, the device 2 can simultaneously display the interface of the calculator application and the interface of the desktop.

[0167] It can be understood that, since the calculator application only includes one interface, when generating the screen projection data 3, the device 1 can replace the screen projection data of any one of the interface 602, the interface 603 and the interface 604 with the screen projection data of the interface of the calculator application (for example, the screen projection data of the interface 603 is replaced in (b) of FIG. 10), so that after receiving the screen projection data 3, the device 2 can simultaneously display the interface 611, the interface 602, the interface 604, the dock area 605 and the navigation bar area 606.

[0168] It can also be understood that (b) of FIG. 10 only takes the example that the application includes only one interface, but should not be understood as a specific limitation corresponding to the embodiments of the present application. In other embodiments, the number of interfaces included by the application is greater than 1 but less than the number of interfaces simultaneously displayed by the device 2, and when generating the screen projection data 3, the device 1 can still replace part of the screen projection data in the screen projection data 1 with the screen projection data of the interface of the application, so that the device 2 can simultaneously display the interface of the application and the interface in the desktop.

[0169] When the device 2 detects the operation of the user clicking the icon 609, in response to the operation, the device 2 can send indication information 3 to the device 1, where the indication information 3 is used to indicate the operation of the user clicking the icon 609. The device 1 receives the indication information 3, and can generate screen projection data 4 according to the indication information 3, and then send the screen projection data 4 to the device 2. The device 2 receives the screen projection data 4, and can update the displayed interface according to the screen projection data 4, that is, display the GUI as shown in (a) of FIG. 11.

[0170] As shown in (a) of FIG. 11, after receiving the screen projection data 4, the device 2 can display the interface 612 in full screen, where the interface 612 is an interface of a game application.

[0171] It can be understood that, since the device 2 is in a horizontal screen state, and the icon 609 corresponds to a game application that supports a horizontal screen layout mode, when generating the screen projection data 4, the device 1 includes screen projection data corresponding to the horizontal screen layout mode of the interface of the game application in the screen projection data 4, so that after receiving the screen projection data 4, the device 2 can display the interface 612 in full screen.

[0172] When the device 2 detects the operation of the user clicking the folder 610, in response to the operation, the device 2 can send indication information 4 to the device 1, the indication information 4 being used to indicate the operation of the user clicking the folder 610. The device 1 receives the indication information 4, can generate casting data 5 according to the indication information 4, and then send the casting data 5 to the device 2. The device 2 receives the casting data 5, and can update the displayed interface according to the casting data 5, i.e., display the GUI as shown in (b) of FIG. 11.

[0173] As shown in (b) of FIG. 11, after the device 2 receives the casting data 5, the device 2 can display the interface 613 while displaying the interface 602, the interface 603, the interface 604, the dock area 605 and the navigation bar area 606, the interface 613 can include icons of one or more application programs, and the interface 613 can be overlaid on the interface 602 and the interface 603.

[0174] In some embodiments, the device 2 can also highlight the interface 613.

[0175] In another embodiment, after the device 2 receives the casting data 5, the device 2 can replace the interface 603 with the interface 613.

[0176] The operation of the device 2 detecting the user clicking the icon of the application program or the folder can include the following possible implementations:

[0177] In one possible implementation, the display screen of the device 2 is a touchable display screen, and the operation of the user clicking the icon of the application program or the folder can be a clicking operation of the user on the display screen of the device 2, i.e., the device 2 can detect the operation of the user clicking the icon of the application program or the folder through the display screen.

[0178] In one possible implementation, the device 2 is configured with a remote controller, and the operation of the user clicking the icon of the application program or the folder can be the user pressing a control button on the remote controller, i.e., the device 2 can detect the operation of the user clicking the icon of the application program or the folder through the remote controller.

[0179] In some embodiments, in addition to sending the indication information to the device 1 in response to the operation of the user clicking the icon of the application program or the folder, the device 2 can also send the indication information to the device 1 in response to a voice instruction of the user, the voice instruction being used to indicate starting the application program or opening the folder.

[0180] In the embodiments of the present application, when the device 2 detects the operation of the user enabling the application or the operation of opening the folder while displaying the desktop of the device 1, the device 1 can be sent indication information to indicate the operation of the user, and the device 1 can generate and send new screen projection data to the device 2 according to the indication information, so that the device 2 can update the displayed interface, so that the user can interact with the device 2, and the user experience is improved.

[0181] In the GUIs shown in FIGS. 9-11, the operation of the user is detected while the device 2 simultaneously displays multiple interfaces of the desktop, and in the following, the operation of the user is detected while the device 2 simultaneously displays multiple interfaces of the application will be introduced as an example.

[0182] FIG. 12 shows a set of GUIs provided by the embodiments of the present application.

[0183] As shown in (a) of FIG. 12, the device 2 can simultaneously display the interface 701, the interface 702, the interface 703 and the interface 704 after receiving the screen projection data 1 sent by the device 1, wherein the interface 701, the interface 702, the interface 703 and the interface 704 belong to the same application 1, and the interface content corresponding to the interface 701 is the interface content being displayed by the device 1.

[0184] When the device 2 detects the operation of the user returning to the desktop, the device 2 can send indication information 1 to the device 1 in response to the operation, and the indication information 1 is used to indicate the operation of the user returning to the desktop. After the device 1 receives the indication information 1, the device 1 can generate and send screen projection data 2 to the device 2, and the screen projection data 2 includes the screen projection data of one or more interfaces of the desktop. After the device 2 receives the screen projection data 2, the device 2 can display the GUI as shown in (b) of FIG. 12.

[0185] As shown in (b) of FIG. 12, the device 2 can simultaneously display one or more interfaces of the desktop after receiving the screen projection data 2.

[0186] It should be understood that the detailed description of (b) of FIG. 12 can refer to the description of (b) of FIG. 6, and for the sake of brevity, it will not be repeated here.

[0187] In the embodiments of the present application, when the device 2 detects the operation of the user returning to the desktop while displaying multiple interfaces of the application, the device 1 can be sent indication information to indicate the operation of the user, and the device 1 can generate and send new screen projection data to the device 2 according to the indication information, so that the device 2 can update the displayed interface, so that the user can interact with the device 2, and the user experience is improved.

[0188] It should be noted that the operation of returning to the desktop in FIG. 12 is taken as an example of swiping up, but should not be understood as a specific limitation to the embodiments of the present application. In some other embodiments of the present application, the operation of returning to the desktop can also be a voice instruction of the user, a right swiping operation of the user on the left edge of the display screen of the device 2, a left swiping operation of the user on the right edge of the display screen of the device 2, and the like.

[0189] In some embodiments, the interfaces 701, 702, 703 and 704 can include one or more controls corresponding to one or more services, for example, the interface 701 includes a play control 705 for playing music. When the device 2 detects an operation of the user clicking the play control 705, in response to the operation, the device 2 can send indication information to the device 1, the indication information being used to indicate the operation of the user clicking the play control 705. After the device 1 receives the indication information, the device 1 can send service data to the device 2, the service data including audio data corresponding to a song. After the device 2 receives the service data, the device 2 can play the song through its own speaker. It can be understood that in these embodiments, since the device 1 will have corresponding animation effects (for example, scrolling to display the lyrics of the song being played, displaying the playing progress of the song, and the like) when playing the song, the device 1 can also send corresponding projection data of the animation effects to the device 2, so that the device 2 can display the corresponding animation effects on the interface 701.

[0190] In the embodiments of the present application, when the device 2 detects an operation of the user clicking a control while displaying a plurality of interfaces of an application program, the device 2 can send indication information to the device 1 to indicate the operation of the user, and the device 1 can send corresponding service data to the device 2 according to the indication information, so that the user can execute the service in the device 1 through the device 2, which helps to improve the user experience.

[0191] The above describes an example in which the desktop of the device 1 includes a plurality of interfaces, but the embodiments of the present application are not limited thereto. In some other embodiments of the present application, the desktop of the device 1 can also be one interface, which will be described below in conjunction with FIGS. 13-16.

[0192] FIG. 13 shows a set of GUIs provided by the embodiments of the present application.

[0193] As shown in (a) of FIG. 13, the device 1 displays an interface 801, which is the main interface of the desktop, and the number of interfaces of the desktop of the device 1 is 1. In response to a projection operation of the user, the device 1 can send projection data 1 to the device 2. After receiving the projection data 1 sent by the device 1, the device 2 can display a GUI as shown in (b) of FIG. 13 or (a) of FIG. 14.

[0194] As shown in (b) of FIG. 13, after receiving the screen projection data 1, the device 2 can display an interface 802 on the left side of the display screen, where the interface 802 has the same interface content as the interface 801.

[0195] It should be noted that the device 2 displays the interface 802 on the left side of the display screen is only an example of the embodiments of the present application, but should not be construed as a specific limitation of the embodiments of the present application. In other embodiments of the present application, the device 2 can also display the interface 802 in the middle region or the right side of the display screen.

[0196] As shown in (a) of FIG. 14, after receiving the screen projection data 1, the device 2 can display an interface 803 and a dock area 804, where the interface 803 includes icons of application programs in the working area of the interface 801.

[0197] It should be noted that in the GUIs shown in (b) of FIG. 13 and (a) of FIG. 14, the device 2 does not use black edges to fill the blank area when displaying the interface 802, which is used to display other interfaces, so that the device 2 can also display multiple interfaces simultaneously in response to the user's operation. Hereinafter, taking the device 2 displaying the GUI shown in (a) of FIG. 14 after receiving the screen projection data 1 as an example.

[0198] The interface 803 includes one or more icons of application programs and one or more folders, for example, an icon 805, an icon 806, an icon 807, and a folder 808. The icon 805 corresponds to a calculator application program, which includes an interface. The icon 806 corresponds to a music application program, which includes multiple interfaces. The icon 807 corresponds to a game application program, which supports a landscape layout mode. The folder 808 includes one or more icons of application programs.

[0199] When the device 2 detects the user's operation of clicking the icon 805, in response to the operation, the device 2 can send indication information 1 to the device 1, where the indication information 1 is used to indicate the user's operation of clicking the icon 805. The device 1 receives the indication information 1, and can generate screen projection data 2 according to the indication information 1, and then sends the screen projection data 2 to the device 2. The device 2 receives the screen projection data 2, and can update the displayed interface according to the screen projection data 2, that is, display the GUI shown in (b) of FIG. 14.

[0200] As shown in (b) of FIG. 14, after receiving the screen projection data 2, the device 2 can simultaneously display the interface 803 and an interface 809, where the interface 809 is an interface of the calculator application program.

[0201] It can be understood that (b) in FIG. 14 only takes an example that the application program includes only one interface, but should not be understood as a specific limitation of the embodiments of the present application. In other embodiments, the number of interfaces included by the application program is greater than 1, but the sum of the number of interfaces included by the application program and the number of interfaces of the desktop is less than the number of interfaces that can be simultaneously displayed by the device 2, and the device 2 can still simultaneously display multiple interfaces of the application program and the interfaces in the desktop.

[0202] When the device 2 detects the operation of the user clicking the icon 806, in response to the operation, the device 1 can be sent indication information 2 indicating the operation of the user clicking the icon 806. The device 1 receives the indication information 2, can generate the screen projection data 3 according to the indication information 2, and then send the screen projection data 3 to the device 2. The device 2 receives the screen projection data 3, and can update the displayed interface according to the screen projection data 3, that is, display the GUI as shown in (a) of FIG. 15.

[0203] As shown in (a) of FIG. 15, after the device 2 receives the screen projection data 3, multiple interfaces of the music application program can be simultaneously displayed, and the interface 803 and the dock area 804 are no longer displayed.

[0204] When the device 2 detects the operation of the user clicking the icon 807, in response to the operation, the device 1 can be sent indication information 3 indicating the operation of the user clicking the icon 807. The device 1 receives the indication information 3, can generate the screen projection data 4 according to the indication information 3, and then send the screen projection data 4 to the device 2. The device 2 receives the screen projection data 4, and can update the displayed interface according to the screen projection data 4, that is, display the GUI as shown in (b) of FIG. 15.

[0205] As shown in (b) of FIG. 15, after the device 2 receives the screen projection data 4, the interface 810 which is the interface of the game application program can be displayed in full screen.

[0206] When the device 2 detects the operation of the user clicking the folder 808, in response to the operation, the device 1 can be sent indication information 4 indicating the operation of the user clicking the folder 808. The device 1 receives the indication information 4, can generate the screen projection data 5 according to the indication information 4, and then send the screen projection data 5 to the device 2. The device 2 receives the screen projection data 5, and can update the displayed interface according to the screen projection data 5, that is, display the GUI as shown in FIG. 16.

[0207] As shown in FIG. 16, after the device 2 receives the screen projection data 5, the interface 803, the dock area 804 and the interface 811 which can include one or more icons of application programs can be simultaneously displayed.

[0208] In the implementation, when the device 2 detects an operation of enabling an application or an operation of opening a folder while displaying the desktop of the device 1, the device 1 can be sent indication information indicating the operation of the user, and the device 1 can generate and send new screen projection data to the device 2 according to the indication information, so that the device 2 can update the displayed interface, so that the user can interact with the device 2, and the user experience is improved.

[0209] In the foregoing, the device 2 receives the screen projection data sent by the device 1 is taken as an example, but the implementation of the present application is not limited thereto. In another implementation of the present application, the device 2 can receive screen projection data sent by multiple devices at the same time, so that the device 2 can display interfaces being displayed by the multiple devices at the same time.

[0210] FIG. 17 shows a set of GUIs provided by the implementation of the present application.

[0211] As shown in FIG. 17, the device 2 can receive screen projection data sent by the device 1, the device 3, the device 4 and the device 5, so as to display interfaces 901, 902, 903 and 904 at the same time, wherein the interface content in the interface 901 is the interface content being displayed by the device 1, the interface content in the interface 902 is the interface content being displayed by the device 3, the interface content in the interface 903 is the interface content being displayed by the device 4, and the interface content in the interface 904 is the interface content being displayed by the device 5.

[0212] It can be understood that when the device 2 receives screen projection data sent by more devices, the device 2 can also display a navigation bar when displaying multiple interfaces at the same time.

[0213] In some implementations, when the device 2 receives screen projection data sent by one device, the device 2 can display interface content included in multiple interfaces of the device. When the device 2 receives screen projection data of multiple devices, the device 2 can display interface content included in interfaces being displayed by the multiple devices.

[0214] For example, the device 2 can first receive screen projection data sent by the device 1, and the device 2 can display interface content included in multiple interfaces of the device 1, that is, the device 2 can display a GUI as shown in (c) of FIG. 5 or (b) of FIG. 6. When the device 2 receives screen projection data sent by the device 3, the device 4 and the device 5, the device 2 can display interface content included in interfaces being displayed by the device 1, the device 3, the device 4 and the device 5 at the same time, that is, the device 2 can display a GUI as shown in FIG. 17.

[0215] Similarly, when the device 2 switches from receiving screen projection data of multiple devices to receiving screen projection data of one device, the device 2 can switch from displaying interface content included in interfaces being displayed by the multiple devices to displaying interface content included in multiple interfaces of one device.

[0216] It should be noted that FIG. 9 only takes the example that the device 2 receives the screen projection data sent by four devices, and should not be understood as a specific limitation to the embodiments of the present application. In some other embodiments of the present application, the device 2 can receive screen projection data sent by more or less devices.

[0217] Since the device 2 displays the interface 901, the interface 902, the interface 903 and the interface 904 simultaneously, the user can interact with any one of the interfaces.

[0218] For example, the interface 901 includes an icon 905, which corresponds to a music application. When the device 2 detects that the user clicks the icon 905, in response to the operation, the device 2 can send indication information to the device 1. After the device 1 receives the indication information, the device 1 can generate new screen projection data, and then send the new screen projection data to the device 2. After the device 2 receives the new screen projection data, the device 2 can display multiple interfaces of the music application simultaneously, and no longer display the interface 901, the interface 902, the interface 903 and the interface 904.

[0219] For another example, the interface 901 includes an icon 905, which corresponds to a music application. When the device 2 detects that the user clicks the icon 905, in response to the operation, the device 2 can send indication information to the device 1. After the device 1 receives the indication information, the device 1 can generate new screen projection data, and then send the new screen projection data to the device 2. After the device 2 receives the new screen projection data, the device 2 can replace the interface 901 with a main interface of the music application, and display the interface 902, the interface 903 and the interface 904 simultaneously.

[0220] For another example, the interface 901 includes an icon 906, which corresponds to a calculator application. When the device 2 detects that the user clicks the icon 906, in response to the operation, the device 2 can send indication information to the device 1. After the device 1 receives the indication information, the device 1 can generate new screen projection data, and then send the new screen projection data to the device 2. After the device 2 receives the new screen projection data, the device 2 can replace the interface 901 with a main interface of the calculator application, and display the interface 902, the interface 903 and the interface 904 simultaneously.

[0221] For another example, the interface 901 includes an icon 907, which corresponds to a game application. When the device 2 detects that the user clicks the icon 907, in response to the operation, the device 2 can send indication information to the device 1. After the device 1 receives the indication information, the device 1 can generate new screen projection data, and then send the new screen projection data to the device 2. After the device 2 receives the new screen projection data, the device 2 can display an interface of the game application in full screen.

[0222] For another example, the interface 901 includes a folder 908. When the device 2 detects that the user clicks the folder 908, the device 1 can send indication information to the device 2 in response to the operation. After the device 1 receives the indication information, the device 1 can generate new screen projection data, and then send the new screen projection data to the device 2. After the device 2 receives the new screen projection data, the device 2 can replace the interface 901 with an interface of the folder 908, and simultaneously display the interface 902, the interface 903, and the interface 904.

[0223] In the embodiments of this application, the device 2 can receive screen projection data sent by multiple devices, so that the device 2 can simultaneously display interface contents displayed by the multiple devices, can fully utilize the display screen, and can not appear black edge filling, which helps to improve the user experience.

[0224] The above describes the screen projection method provided by the embodiments of this application in combination with the GUI. In the following, the screen projection method provided by the embodiments of this application is described in combination with a method flowchart.

[0225] FIG. 18 shows a schematic flowchart of a screen projection method provided by the embodiments of this application. The method can be executed by a device or a component (for example, a processor, a chip system, etc.) of the device. In the following, the execution subject is taken as the device for example, and as shown in FIG. 18, the method includes the following steps.

[0226] S1001, the first device detects a screen projection operation of a first interface by a user.

[0227] The first device can detect the screen projection operation of the first interface by the user when displaying the first interface.

[0228] In some embodiments, the first interface is one interface in a desktop.

[0229] For example, as shown in (a) of FIG. 4, the device 1 displays the interface 401.

[0230] In some embodiments, the first interface is one interface of an application program.

[0231] For example, as shown in (a) of FIG. 7, the device 1 displays the interface 501.

[0232] S1002, the first device sends first screen projection data to a second device.

[0233] Correspondingly, the second device receives the first screen projection data sent by the first device.

[0234] Specifically, the first device detects a user's screen projection operation on the first interface, and in response to the screen projection operation, can generate and send first screen projection data to the second device according to display parameters of the second device, the currently displayed first interface, and N interfaces associated with the first interface, where N is an integer greater than or equal to 1. The display parameters of the second device include, but are not limited to, width, height, and dots per inch (DPI), where the width and height are the width and height of a surface created by the second device, which can be used to carry the screen projection data sent by the first device.

[0235] In some embodiments, the display parameters of the second device can be sent by the second device to the first device.

[0236] In some embodiments, the first device can obtain the model of the second device when the first device is connected to the second device, and then the first device can determine the display parameters of the second device according to the model of the second device.

[0237] S1003, the second device simultaneously displays interface content included in the first interface and M interfaces of the N interfaces according to the first screen projection data.

[0238] Specifically, after receiving the first screen projection data, the second device can control the display device (for example, a display screen) to simultaneously display interface content included in the first interface and M interfaces of the N interfaces, where N is greater than or equal to M, M is an integer greater than or equal to 1, and M is an integer.

[0239] It can be understood that due to the size limitation of the display screen of the second device, the second device can not be able to simultaneously display the interface content included in the first interface and the N interfaces, and therefore the second device can display the interface content included in the first interface and M interfaces of the N interfaces.

[0240] For example, as shown in (c) of FIG. 5, device 1 can send screen projection data of interface 401, interface 402, interface 403, and interface 404 to device 2, and device 2 can simultaneously display interface 409, interface 410, and interface 411, where interface 409 is associated with interface 402, interface 410 is associated with interface 401, and interface 411 is associated with interface 403.

[0241] In some embodiments, the first interface and the M interfaces are interfaces in a desktop, the second device can display icons corresponding to resident applications of the desktop at a first position, display icons of applications of working areas of the first interface and the M interfaces at a second position, and display a navigation bar at a third position.

[0242] For example, as shown in (b) of FIG. 6, the device 1 can send the screen projection data of the interface 401, the interface 402, the interface 403 and the interface 404 to the device 2, and the device 2 can simultaneously display the interface 413, the interface 414, the interface 415, the dock area 416 and the navigation bar area 417.

[0243] It can be understood that when the second device can simultaneously display all the interfaces of the desktop of the first device, the second device can also not display the navigation bar.

[0244] In the embodiments of the present application, when the first device screens projects to the second device, the screen projection data sent by the first device can include the screen projection data of multiple interfaces, so that the second device can simultaneously display multiple interfaces, fully utilize the display area of the display screen, and will not appear black edge filling, which helps to improve the user experience.

[0245] In some embodiments, the method further includes:

[0246] S1004, the second device sends first indication information to the first device.

[0247] Correspondingly, the first device receives the first indication information sent by the second device, and the first indication information is used to indicate the first operation of the user on the second device.

[0248] Specifically, when the second device simultaneously displays the first interface and the interface content included in the M interfaces, the second device detects the first operation of the user, and in response to the first operation of the user, the second device can generate the first indication information, which can indicate the first operation of the user on the second device, and then the second device can send the first indication information to the first device.

[0249] S1005, the first device sends second screen projection data to the second device.

[0250] Correspondingly, the second device receives the second screen projection data sent by the first device.

[0251] Specifically, the first device can determine the first operation of the user according to the first indication information, then determine the second interface corresponding to the first operation of the user according to the first operation of the user, and then generate and send the second screen projection data to the second device according to the display parameters of the second device and the second interface.

[0252] S1006, the second device updates the interface content according to the second screen projection data.

[0253] After receiving the second screen projection data, the second device can control the display device to update the interface content being displayed according to the second screen projection data.

[0254] In some embodiments, the second screen projection data further comprises screen projection data of X interfaces associated with the second interface, X≥1 and X is an integer. The first interface comprises an icon of the first application program, the first operation of the user is clicking the icon of the first application program, and the second interface can be a main interface of the first application program, S1006, and the second device updates the interface content according to the second screen projection data, comprising:

[0255] The second device controls the display device to simultaneously display interface content included in the second interface and Y interfaces of the X interfaces, and no longer display interface content included in the first interface and the M interfaces, wherein X≥Y≥1 and Y is an integer.

[0256] For example, as shown in (b) of FIG. 9 and (a) of FIG. 10, the device 2 can send indication information to the device 1 to indicate that the user clicks the icon 607. After receiving the indication information, the device 1 can generate and send new screen projection data to the device 2. After receiving the new screen projection data, the device 2 can simultaneously display multiple interfaces of the music application program.

[0257] For another example, as shown in (a) of FIG. 14 and (a) of FIG. 15, the device 2 can send indication information to the device 1 to indicate that the user clicks the icon 806. After receiving the indication information, the device 1 can generate and send new screen projection data to the device 2. After receiving the new screen projection data, the device 2 can simultaneously display multiple interfaces of the music application program.

[0258] In some embodiments, the first interface comprises an icon of the first application program, the first operation of the user is clicking the icon of the first application program, the second interface is a main interface of the first application program, S1006, and the second device updates the interface content according to the second screen projection data, comprising:

[0259] The second device controls the display device to display interface content included in the second interface, and no longer display interface content included in the first interface.

[0260] Since the second device can have multiple ways to simultaneously display interface content of multiple interfaces of the first device, the second device no longer displaying interface content included in the first interface can also correspond to different implementation manners:

[0261] In one possible implementation manner, the second device no longer displays interface content included in a work area of the first interface.

[0262] For example, as shown in (b) of FIG. 9 and (b) of FIG. 10, the device 2 can display the interface 611 and no longer display the interface 603, i.e., no longer display interface content included in the work area of the interface 601.

[0263] In a possible implementation, the second device no longer displays all interface content included in the first interface.

[0264] For example, as shown in (c) of FIG. 5, the device displays interface 409, interface 410, and interface 411 at the same time, where interface 409 is associated with interface 402, interface 410 is associated with interface 401, and interface 411 is associated with interface 403. When the device 2 detects that the user clicks the icon of the calculator application in interface 410, the device 2 can display the interface corresponding to the calculator application to replace interface 410, that is, no longer display all interface content included in interface 401.

[0265] In some embodiments, the second device is in a landscape state, the first interface includes an icon of a first application, the first application supports a landscape layout mode, the operation of the user is clicking the icon of the first application, and the second interface is an interface of the first application. S1006. The second device updates the interface content according to the second screen projection data, including: the second device controls the display device to display, in full screen, interface content included in the second interface.

[0266] For example, as shown in (b) of FIG. 9 and (a) of FIG. 11, the device 2 can send indication information to the device 1 to indicate that the user clicks the icon 609. After receiving the indication information, the device 1 can generate new screen projection data and send the new screen projection data to the device 2. After receiving the new screen projection data, the device 2 can display the interface 612 in full screen.

[0267] In some embodiments, the first interface is an interface of a first application, the first operation of the user is a return-to-desktop operation, and the second interface is an interface of a desktop. S1006. The second device updates the interface content according to the second screen projection data, including: the second device controls the display device to display, at the same time, interface content included in the second interface and B of V interfaces, and no longer display interface content included in the first interface and M interfaces, where V≥B≥0, and Y is an integer.

[0268] For example, as shown in (a) and (b) of FIG. 12, the device 2 can send indication information to the device 1 to indicate that the user slides up. After receiving the indication information, the device 1 can generate new screen projection data and send the new screen projection data to the device 2. After receiving the new screen projection data, the device 2 can display, at the same time, interface content included in multiple interfaces of the desktop of the device 1.

[0269] In some embodiments, the first electronic device receives first indication information, and can also control the display device to display the second interface according to the first indication information.

[0270] In some embodiments, the first interface includes a first control, and the first control corresponds to a first service. The method further includes:

[0271] The second device sends second indication information to the first device in response to the operation of the user clicking the first control.

[0272] The first device generates first service data according to the second indication information.

[0273] The first device sends the first service data to the second device.

[0274] For example, as shown in (a) of FIG. 12, the device 2 can send indication information to the device 1 to indicate the operation of the user clicking the play control 705. After receiving the indication information, the device 1 can generate corresponding audio data, so that the device 1 can send the audio data to the device 2, so that the device 2 can play the audio data.

[0275] In some embodiments, the method further includes:

[0276] The second device receives third screen projection data sent by a third device.

[0277] The second device controls the display device to simultaneously display interface content included in a first interface and a third interface according to the first screen projection data and the third screen projection data, and no longer display interface content included in M interfaces, wherein the third interface is an interface currently displayed by the third device.

[0278] Specifically, the second device can simultaneously receive screen projection data of multiple devices. When the second device first receives the screen projection data sent by the first device, the first device can simultaneously display interface content included in multiple interfaces of the first device. When the second device further receives the screen projection data sent by the third device, the second device can simultaneously display interface content included in interfaces currently displayed by the first device and the third device, and can no longer display interface content included in M interfaces of the first device.

[0279] For example, the device 2 can first receive the screen projection data sent by the device 1, and the device 2 can display interface content included in multiple interfaces of the device 1, that is, the device 2 can display the GUI as shown in (c) of FIG. 5 or (b) of FIG. 6. When the device 2 further receives the screen projection data sent by the devices 3, 4 and 5, the device 2 can simultaneously display interface content included in interfaces currently displayed by the devices 1, 3, 4 and 5, that is, the device 2 can display the GUI as shown in FIG. 17.

[0280] The method for screen projection provided by the embodiments of the present application is described in detail above. In various embodiments of the present application, the terms and / or descriptions of various embodiments are consistent and can be mutually referenced if there is no special description and logical conflict. The technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0281] The above describes the method for screen projection provided by the embodiments of the present application from the perspective of the device. It can be understood that the device includes hardware structures and / or software modules corresponding to the execution of each function in order to implement the above functions. Those skilled in the art should easily realize that, in combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and software. Whether a certain function is implemented in hardware or in the form of software driving hardware depends on the specific application of the technical solution and the design constraint conditions. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0282] In the case of adopting the division of each function module (or unit) corresponding to each function, the method for screen projection provided by the embodiments of the present application will be described from the perspective of the function module.

[0283] FIG. 19 shows a schematic flowchart of the method for screen projection provided by the embodiments of the present application. As shown in FIG. 19, the method includes the following steps.

[0284] S1101, a second launcher creates a page.

[0285] The second launcher is a launcher of a second device, and the page created by the second launcher can be used to carry screen projection data.

[0286] S1102, the second launcher sends parameters of the page to a second collaborative capability module.

[0287] The second collaborative capability module is a collaborative capability module of the second device, and the parameters of the page include the width and height of the page. The second collaborative capability module includes the collaborative configuration management module, the connection module, etc. shown in FIG. 2.

[0288] S1103, the second collaborative capability module performs initialization processing with a first collaborative capability module.

[0289] The first collaborative capability module is a collaborative capability module of a first device. The initialization processing of the second collaborative capability module with the first collaborative capability module includes that the second collaborative capability module sends the parameters of the page to the first collaborative capability module. In addition, the second collaborative capability module can also send the DPI of the display screen of the second device to the first collaborative capability module.

[0290] S1104, a first screen projection module and a second screen projection module establish a connection.

[0291] The first screen projection module is a screen projection module of the first device, and the second screen projection module is a screen projection module of the second device. The first screen projection module and the second screen projection module can establish a connection after completion of the initialization.

[0292] S1105, the first screen projection module creates a virtual display.

[0293] The first screen projection module can create a virtual display (VD).

[0294] S1106, the first screen projection module starts screen projection.

[0295] It should be noted that the execution order of steps S1105 and S1106 is not limited in the embodiments of the present application. For example, step S1105 is executed before step S1106 is executed by the first screen projection module, or step S1105 can be executed after step S1106 is executed.

[0296] S1107, the first launcher creates a first activity.

[0297] S1108, the first launcher draws a layout.

[0298] After starting screen projection, the first launcher can draw and render the first interface currently displayed and the N interfaces associated with the first interface into the first activity according to the display parameters of the second device, so that the second device can display the interface content included in the multiple interfaces at the same time. The first activity can also be referred to as the first interface capability.

[0299] S1109, the first screen projection module sends first screen projection data to the second screen projection module.

[0300] Correspondingly, the second screen projection module receives the first screen projection data sent by the first screen projection module, so that the second launcher can display the interface content included in the first interface and the M interfaces of the N interfaces.

[0301] S1110, the first launcher and the second launcher are synchronized and refreshed.

[0302] A synchronous refreshing mechanism can be established between the first launcher and the second launcher, when the displayed interface of the first device changes, the displayed interface of the second device will also change.

[0303] S1111, the second launcher sends first indication information to the first launcher.

[0304] Correspondingly, the first launcher receives the first indication information sent by the second launcher, and the first indication information is used to indicate the first operation of the user.

[0305] It should be understood that the detailed description of the first indication information can be referred to in the above, and will not be described here for brevity.

[0306] The embodiment of the present application provides a program product (also referred to as a computer program product), when the program product is run on a device, the device executes the technical solutions in the above-described embodiments. The implementation principle and technical effects are similar to the above-described method-related embodiments, and will not be repeated here.

[0307] The embodiment of the present application provides a readable storage medium (also referred to as a computer readable storage medium), which contains instructions, when the instructions are run on a device, the device executes the technical solutions of the above-described embodiments. The implementation principle and technical effects are similar, and will not be repeated here.

[0308] The embodiment of the present application provides a chip, which is used to execute instructions, when the chip is run, the technical solutions in the above-described embodiments are executed. The implementation principle and technical effects are similar, and will not be repeated here.

[0309] Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solutions. Those skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of the present application.

[0310] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0311] The above is only a specific implementation of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited to this. Any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the embodiments of the present application, which should be covered within the protection scope of the embodiments of the present application. Therefore, the protection scope of the embodiments of the present application should be subject to the protection scope of the claims.

Claims

1. A screen projection method, characterized by, The method comprises: detecting a user's screen projection operation on a first interface; in response to the screen projection operation, sending first screen projection data to a second device, the first screen projection data comprising screen projection data of the first interface and N interfaces associated with the first interface, N being an integer greater than or equal to 1.

2. The method of claim 1, wherein, The method further comprises: receiving first indication information sent by the second device, the first indication information being used to indicate a first operation of a user on the second device; sending second screen projection data to the second device according to the first indication information, wherein the second screen projection data comprises screen projection data of a second interface associated with the first operation.

3. The method of claim 2, wherein, The first interface comprises an icon of a first application program, and the first operation is an operation of clicking the icon of the first application program, wherein the second screen projection data further comprises screen projection data of X interfaces associated with the second interface, X being an integer greater than or equal to 1.

4. The method according to claim 2 or 3, characterized in that, The method further comprises: controlling a display device to display the second interface.

5. The method according to any one of claims 2 to 4, characterized in that, When the second device is in a horizontal screen state and the second interface supports a horizontal screen layout mode, the second screen projection data comprises screen projection data corresponding to the horizontal screen layout mode of the second interface.

6. The method of claim 1, wherein, The first interface comprises a first control corresponding to a first service, and the method further comprises: receiving second indication information sent by the second device, the second indication information being used to indicate an operation of a user clicking the first control; sending service data of the first service to the second device.

7. A screen projection method, characterized by, The method comprises: receiving first screen projection data sent by a first device, the first screen projection data comprising screen projection data of a first interface and N interfaces associated with the first interface, N being an integer greater than or equal to 1; controlling a display device to simultaneously display interface content included in the first interface and M interfaces of the N interfaces according to the first screen projection data, N≥M≥1, and M being an integer.

8. The method of claim 7, wherein, The method further comprises: in response to a first operation of a user, sending first indication information to the first device, the first indication information being used to indicate the first operation; receiving second screen projection data sent by the first device, wherein the second screen projection data comprises screen projection data of a second interface associated with the first operation; updating interface content according to the second screen projection data.

9. The method of claim 8, wherein, The first interface comprises an icon of a first application program, the first operation is an operation of clicking the icon of the first application program, the second interface is an interface of the first application program, the second screen projection data further comprises screen projection data of X interfaces associated with the second interface, X being an integer greater than or equal to 1, and the updating of the interface content according to the second screen projection data comprises: controlling the display device to simultaneously display interface content included in the second interface and Y interfaces of the X interfaces, and no longer display interface content included in the first interface and the M interfaces, Y being an integer greater than or equal to 1.

10. The method of claim 8, wherein, The first interface includes an icon of a first application, the first operation is clicking the icon of the first application, the second interface is an interface of the first application, and updating the interface content according to the second screen projection data includes: controlling the display device to display interface content included in the second interface to replace interface content included in the first interface.

11. The method of claim 8, wherein, The first interface includes an icon of a first application, the second interface supports a landscape layout mode, the first operation is clicking the icon of the first application, the second interface is an interface of the first application, and updating the interface content according to the second screen projection data includes: controlling the display device to display interface content included in the second interface in full screen.

12. The method of claim 7, wherein, The first interface includes a first control, and the first control corresponds to a first service. The method further includes: in response to an operation of the user clicking the first control, sending second indication information to the first device, wherein the second indication information is used to indicate the operation of the user clicking the first control; receiving service data of the first service sent by the first device.

13. The method according to any one of claims 7 to 12, characterized in that, The method further includes: receiving third screen projection data sent by a third device; controlling the display device to simultaneously display interface content included in the first interface and a third interface according to the first screen projection data and the third screen projection data, and no longer display interface content included in the M interfaces, wherein the third interface is an interface currently displayed by the third device.

14. An apparatus, comprising: comprise one or more processors; one or more memories; the one or more memories store one or more programs, the one or more programs include instructions, when the instructions are executed by the one or more processors, cause the method of any one of claims 1 to 6 or the method of any one of claims 7 to 13 to be executed.

15. A readable storage medium, characterized by, The readable storage medium stores instructions, when the instructions run on a device, cause the method of any one of claims 1 to 6 or the method of any one of claims 7 to 13 to be executed.

16. A chip, characterized by The chip includes a processor and a communication interface, the communication interface is used to receive a signal and transmit the signal to the processor, and the processor processes the signal, so that the method of any one of claims 1 to 6 or the method of any one of claims 7 to 13 is executed.

17. A program product, characterized by When the program product runs on a device, the device executes the method of any one of claims 1 to 6 or the method of any one of claims 7 to 13.

Images