Screen projection method and equipment

Abstract

The invention provides a screen projection method and equipment. The method comprises the following steps: detecting a screen projection operation of a user on a first interface; and in response to the screen projection operation, first screen projection data is sent to a second device, the first screen projection data comprises the first interface and screen projection data of N interfaces associated with the first interface, and N is an integer greater than or equal to 1. In the application, when the first device projects the screen to the second device, the transmitted screen projection data can include the screen projection data of the plurality of interfaces, so that the second device can display the interface contents of the plurality of interfaces at the same time, the display area of the display screen is fully utilized, the condition of black edge filling does not occur, and the user experience is improved.

Description

Technical Field

[0001] This application relates to the field of devices, and more specifically, to a method and device for screen projection. Background Technology

[0002] Currently, distributed technologies can be used to project the interface displayed on one device onto another. For example, a mobile phone can project its interface onto a television. Similarly, a mobile phone can project its interface onto a vehicle's cockpit screen. However, because different devices may have different screen sizes and shapes, when a small-screen device projects its interface onto a large-screen device, the large-screen device may not be able to fully utilize its display, only showing the projected interface in a portion of the screen while filling the rest with black borders. This results in a degraded user experience. Therefore, ensuring a good user experience during screen projection has become a pressing technical problem that needs to be solved. Summary of the Invention

[0003] This application provides a screen mirroring method and device. When the first device mirrors its screen to the second device, the screen mirroring data sent can include screen mirroring data of multiple interfaces, so that the second device can display the content of multiple interfaces at the same time, making full use of the display area of ​​the screen and avoiding black border filling, which helps to improve the user experience.

[0004] In a first aspect, a screen mirroring method is provided, the method comprising: detecting a user's screen mirroring operation on a first interface; and in response to the screen mirroring operation, sending first screen mirroring data to a second device, the first screen mirroring data including the first interface and screen mirroring data of N interfaces associated with the first interface, where N≥1 and are integers.

[0005] In this embodiment of the application, when the first device projects its screen to the second device, the projection data sent may include projection data of multiple interfaces, so that the second device can display the content of multiple interfaces at the same time, making full use of the display area of ​​the screen and avoiding black border filling, which helps to improve the user experience.

[0006] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: receiving first instruction information sent by the second device, the first instruction information being used to instruct a user on a first operation on the second device; and sending second projection data to the second device according to the first instruction information, wherein the second projection data includes projection data of a second interface, the second interface being associated with the first operation.

[0007] In this application, when the second device detects a user's operation, it can send an instruction to the first device to instruct the user's operation. The first device can then generate and send new projection data to the second device based on the instruction, thereby allowing the second device to update the displayed interface and enabling the user to interact with the second device, thus improving the user experience.

[0008] In conjunction with the first aspect, in some implementations of the first aspect, the method further includes: controlling the display device to display the second interface to replace the first interface.

[0009] For example, the display device may be a display screen.

[0010] In conjunction with the first aspect, in some implementations of the first aspect, the first interface includes an icon of a first application, and the first operation is an operation of clicking the icon of the first application. The second projection data also includes projection data of X interfaces associated with the second interface, where X ≥ 1 and is an integer.

[0011] In conjunction with the first aspect, in some implementations of the first aspect, when the second device is in landscape mode and the second interface supports landscape layout, the second projection data includes projection data corresponding to the landscape layout of the second interface.

[0012] In conjunction with the first aspect, in some implementations of the first aspect, the first interface is the interface of the first application, the user's operation is to return to the desktop, and the second interface is the main interface of the desktop.

[0013] In conjunction with the first aspect, in some implementations of the first aspect, the first interface includes a first control corresponding to a first service, and the method further includes: receiving second instruction information sent by the second device, the second instruction information being used to instruct the user to click the first control; and sending service data of the first service to the second device.

[0014] Secondly, a screen mirroring method is provided, comprising: receiving first screen mirroring data sent by a first device, the first screen mirroring data including the first interface and screen mirroring data of N interfaces associated with the first interface, where N≥1 and is an integer; controlling a display device to simultaneously display the interface content of the first interface and M interfaces among the N interfaces according to the first screen mirroring data, where N≥M≥1 and M is an integer.

[0015] For example, the display device may be a display screen.

[0016] In this embodiment of the application, when the first device projects its screen to the second device, the projection data sent may include projection data of multiple interfaces, so that the second device can display the content of multiple interfaces at the same time, making full use of the display area of ​​the screen and avoiding black border filling, which helps to improve the user experience.

[0017] In conjunction with the second aspect, in some implementations of the second aspect, the method further includes: responding to a user's first operation by sending first instruction information to the first device, the first instruction information being used to instruct the user's first operation; receiving second projection data sent by the first device, wherein the second projection data includes projection data of a second interface, the second interface being associated with the user's first operation; and updating the interface content according to the second projection data.

[0018] In this application, when the second device detects a user's operation, it can send an instruction to the first device to instruct the user's operation. The first device can then generate and send new projection data to the second device based on the instruction, thereby allowing the second device to update the displayed interface and enabling the user to interact with the second device, thus improving the user experience.

[0019] In conjunction with the second aspect, in some implementations of the second aspect, the first interface includes an icon of a first application, the first operation is clicking the icon of the first application, the second interface is the interface of the first application, and the second projection data also includes projection data of X interfaces associated with the second interface, where X ≥ 1 and is an integer. Updating the interface content according to the second projection data includes: controlling the display device to simultaneously display the interface content included in the second interface and Y of the X interfaces, and no longer displaying the interface content included in the first interface and the M interfaces, where Y ≥ 1 and is an integer.

[0020] In conjunction with the second aspect, in some implementations of the second aspect, the first interface includes an icon of a first application, the first operation is clicking the icon of the first application, the second interface is the interface of the first application, and updating the interface content according to the second projection data includes: controlling the display device to display the interface content included in the second interface to replace the interface content included in the first interface.

[0021] In conjunction with the second aspect, in some implementations of the second aspect, the first interface includes an icon of a first application, the second interface supports a landscape layout, the first operation is clicking the icon of the first application, the second interface is the interface of the first application, and updating the interface content according to the second projection data includes: controlling the display device to display the interface content included in the second interface in full screen.

[0022] In conjunction with the second aspect, in some implementations of the second aspect, the first interface is the interface of the first application, the first operation is the operation of returning to the desktop, the second interface is the main interface of the desktop, and updating the interface content according to the second projection data includes: controlling the display device to simultaneously display the interface content included in the second interface and B of the V interfaces, and no longer displaying the interface content included in the first interface and M interfaces, where V≥B≥0, and Y is an integer.

[0023] In conjunction with the second aspect, in some implementations of the second aspect, the first interface includes a first control corresponding to a first service, and the method further includes: in response to a user clicking the first control, sending second instruction information to the first device; and receiving first service data sent by the first device.

[0024] In conjunction with the second aspect, in some implementations of the second aspect, the method further includes: receiving third projection data sent by a third device; controlling the display device to simultaneously display the interface content included in the first interface and the third interface according to the first projection data and the third projection data, and no longer displaying the interface content included in the M interfaces, wherein the third interface is the interface currently displayed by the third device.

[0025] Thirdly, a system is provided, comprising a first device and a second device, wherein the first device is configured to send first projection data to the second device in response to a user's projection operation on a first interface, the first projection data including projection data of the first interface and N interfaces associated with the first interface, where N≥1 and are integers; the second device is configured to simultaneously display the interface content of the first interface and M of the N interfaces according to the first projection data, where N≥M≥1 and M are integers.

[0026] In conjunction with the third aspect, in some implementations of the third aspect, the second device is further configured to: in response to a user's first operation, send the first instruction information to the first device, the first instruction information being used to instruct the user on the first operation on the second device; the first device is further configured to send second projection data to the second device, the second projection data including projection data of a second interface, the second interface being associated with the first operation; and the second device is further configured to update the interface content according to the second projection data.

[0027] In conjunction with the third aspect, in some implementations of the third aspect, the first interface includes an icon of a first application, the first operation is clicking the icon of the first application, the second interface is the interface of the first application, the second projection data also includes projection data of X interfaces associated with the second interface, where X ≥ 1 and is an integer, and the second device is specifically used to simultaneously display the interface content included in the second interface and Y of the X interfaces, and no longer display the interface content included in the first interface and the M interfaces, where Y ≥ 1 and is an integer.

[0028] In conjunction with the third aspect, in some implementations of the third aspect, the first interface includes an icon of a first application, the first operation is clicking the icon of the first application, the second interface is the interface of the first application, and the second device is specifically used to display the interface content included in the second interface to replace the interface content included in the first interface.

[0029] In conjunction with the third aspect, in some implementations of the third aspect, the second device is in landscape mode, the first interface includes an icon of the first application, the second interface supports landscape layout, the first operation is clicking the icon of the first application, the second interface is the interface of the first application, and the second device is specifically used to display the interface content included in the second interface in full screen.

[0030] In conjunction with the third aspect, in some implementations of the third aspect, the first interface is the interface of the first application, the first operation is the operation of returning to the desktop, the second interface is the main interface of the desktop, the interface content is updated according to the second screen projection data, and the second device is specifically used to simultaneously display the second interface and the interface content included in B of the V interfaces, and no longer display the interface content included in the first interface and the M interfaces, where V≥B≥0, and Y is an integer.

[0031] In conjunction with the third aspect, in some implementations of the third aspect, the first interface includes a first control corresponding to a first service, and the second device is further used to send second instruction information to the first device in response to the user clicking the first control; the first device is also used to send service data of the first service to the second device.

[0032] In conjunction with the third aspect, in some implementations of the third aspect, the second device is further configured to: receive third projection data sent by the third device; control the display device to simultaneously display the interface content included in the first interface and the third interface according to the first projection data and the third projection data, and no longer display the interface content included in the M interfaces, wherein the third interface is the interface currently displayed by the third device.

[0033] Fourthly, a screen mirroring method is provided, the method comprising: detecting a user's screen mirroring operation on a first interface; responding to the screen mirroring operation, sending first screen mirroring data to a second device, the first screen mirroring data including screen mirroring data of the first interface; receiving first instruction information sent by the second device, the first instruction information being used to instruct the user to click an icon of a first application on the first interface on the second device; and sending second screen mirroring data to the second device according to the first instruction information, wherein the second screen mirroring data includes screen mirroring data of the first interface and a second interface, the second interface being the interface of the first application.

[0034] Fifthly, a screen mirroring method is provided, the method comprising: receiving first screen mirroring data sent by a first device, the first screen mirroring data including screen mirroring data of a first interface; controlling a display device to display the first interface according to the first screen mirroring data; responding to an operation by a user clicking an icon of a first application on the first interface, sending first instruction information to the first device; receiving second screen mirroring data sent by the first device, wherein the second screen mirroring data includes screen mirroring data of the first interface and a second interface, the second interface being the interface of the first application; and simultaneously displaying the first interface and the second interface according to the second screen mirroring data.

[0035] In a sixth aspect, an apparatus is provided, the apparatus comprising one or more processors; one or more memories; the one or more memories storing one or more programs, the one or more programs comprising instructions that, when executed by the one or more processors, cause the foregoing aspect or any possible implementation thereof to be performed.

[0036] In a seventh aspect, a readable storage medium is provided, the readable storage medium including a program or instructions that, when executed on a device, cause the foregoing aspects or any possible implementation thereof to be performed.

[0037] Eighthly, a program product is provided, comprising a program or instructions that, when executed on a device, cause the foregoing aspects or any possible implementation thereof to be performed.

[0038] Ninthly, a program is provided that, when run on a device, causes the foregoing aspects or any possible implementation thereof to be executed.

[0039] In a tenth aspect, an apparatus is provided, comprising modules / units for performing the above aspects or any possible design of the above aspects; these modules / units may be implemented in hardware or by hardware executing corresponding software. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the device provided in the embodiments of this application.

[0041] Figure 2 This is a software structure block diagram of the device provided in the embodiments of this application.

[0042] Figure 3 This is a set of GUIs provided in the embodiments of this application.

[0043] Figure 4 This is a set of GUIs provided in the embodiments of this application.

[0044] Figure 5 This is a set of GUIs provided in the embodiments of this application.

[0045] Figure 6 This is a set of GUIs provided in the embodiments of this application.

[0046] Figure 7 This is a set of GUIs provided in the embodiments of this application.

[0047] Figure 8 This is a set of GUIs provided in the embodiments of this application.

[0048] Figure 9 This is a set of GUIs provided in the embodiments of this application.

[0049] Figure 10 This is a schematic flowchart of the screen projection method provided in the embodiments of this application.

[0050] Figure 11 This is a schematic flowchart of the screen projection method provided in the embodiments of this application. Detailed Implementation

[0051] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0052] The terminology used in the following embodiments is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to also include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, “at least one” and “one or more” refer to one, two, or more than two. The term “and / or” is used to describe the relationship between related objects, indicating that three relationships may exist; for example, A and / or B can indicate: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character “ / ” generally indicates that the preceding and following related objects are in an “or” relationship.

[0053] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0054] The following describes a device, a user interface for such a device, and embodiments for using such a device. In some embodiments, the device may be a portable device that also includes other functions such as a personal digital assistant and / or music player, such as a mobile phone, tablet computer, wearable device with wireless communication capabilities (such as a smartwatch), etc. Exemplary embodiments of the portable device include, but are not limited to, carrying... Alternatively, it can be a portable device with another operating system. The aforementioned portable device can also be other portable devices, such as laptops. It should also be understood that in some other embodiments, the aforementioned device also includes a desktop computer.

[0055] For example, Figure 1 A schematic diagram of the structure of device 100 is shown. Device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, antenna 1, antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0056] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on device 100. In other embodiments of this application, device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0057] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0058] The controller can serve as the central nervous system and command center of the device 100. The controller can generate operation control signals based on instruction opcodes and timing signals to control the fetching and execution of instructions.

[0059] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0060] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0061] The wireless communication function of device 100 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0062] Antennas 1 and 2 are used to transmit and receive electromagnetic wave signals. Each antenna in device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

[0063] The mobile communication module 150 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low-noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via the antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via the antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0064] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0065] The wireless communication module 160 can provide solutions for wireless communication applications on device 100, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0066] In some embodiments, antenna 1 of device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling device 100 to communicate with networks and other devices via wireless communication technology.

[0067] Device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0068] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. In some embodiments, device 100 may include one or N displays screens 194, where N is a positive integer greater than 1.

[0069] Internal memory 121 can be used to store program code, which includes instructions. Processor 110 executes various functional applications and data processing of device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as image playback function), etc. The data storage area may store data created during the use of device 100 (such as audio data), etc. In addition, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, etc.

[0070] Device 100 can implement audio functions, such as music playback, through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, and an application processor.

[0071] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0072] The loudspeaker 170A, also known as a "loudspeaker", is used to convert audio electrical signals into sound signals.

[0073] The receiver 170B, also known as the "earpiece", is used to convert audio electrical signals into sound signals.

[0074] The microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals.

[0075] Figure 2 This is a software structure block diagram of the device 100 according to an embodiment of this application. The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the system is divided into four layers, from top to bottom: the application layer, the application framework layer, the system service layer, and the kernel layer.

[0076] like Figure 2As shown, the application layer may include a launcher, a super desktop assistant module, a screen mirroring module, settings, calendar, map, WLAN, music, SMS, etc.

[0077] The launcher is responsible for managing and presenting the user interface, which includes the desktop, application interfaces, etc. The launcher can obtain application icons, service cards, folders, and layout information from the desktop, and can render one or more interfaces into a single interface capability during screen mirroring. An ability can be understood as an abstraction of the capabilities an application possesses; it is an important component of the application. An application can have multiple capabilities, meaning it can contain multiple capabilities. Interface capabilities provide the ability to interact with the user. In other words, an interface capability can be understood as a screen on which the device can render one or more interfaces.

[0078] It is understandable that when a device's launcher renders one or more interfaces onto an interface capability and projects it onto another device, that other device can simultaneously display the one or more interfaces, and the user can interact with the one or more interfaces through that other device. Please see below for details.

[0079] It should be noted that "ability" is just a name and can be named differently for different operating systems. For example, it can also be named "activity" in other operating systems.

[0080] The launcher at the sink node is also used to create the projection surface.

[0081] Screen mirroring module: When the device is a sink, the application layer may include a screen mirroring module, which is used to display the screen mirroring content of the source virtual screen and can interact with the virtual screen.

[0082] Super Desktop Assistant Module: This module is responsible for the connection process on the source side, maintaining communication between the source and sink sides, communicating with the launcher on the sink side, and notifying the source side of layout-related events.

[0083] The application framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer may include some predefined functions.

[0084] like Figure 2As shown, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0085] The window manager is used to manage windowed applications. It can obtain the screen size, determine if a status bar is present, lock the screen, and capture screenshots, among other things. The content provider stores and retrieves data, making that data accessible to applications. This data can include videos, images, etc.

[0086] The view system includes visual controls, such as controls for displaying text, controls for displaying images, and such as the indicator information for prompting a virtual shutter button in the embodiments of this application. The view system can be used to build applications. The display interface can consist of one or more views. For example, it may include a view for displaying text and a view for displaying images.

[0087] The phone manager is used to provide communication functions for device 100. This includes managing call status (including connection and disconnection).

[0088] File Explorer provides applications with various resources, such as icons, images, layout files, video files, and more.

[0089] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar and playing sound alerts.

[0090] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0091] The system library can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing library (e.g., OpenGL ES), 2D graphics engine (e.g., SGL), collaborative configuration management module, connectivity module, distributed media module, distributed hardware module, screen projection module, and super desktop module.

[0092] The Surface Manager is used to manage the display subsystem and provides the blending of two-dimensional and three-dimensional layers for multiple applications.

[0093] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, and MP3.

[0094] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0095] A 2D graphics engine is a drawing engine for 2D drawing.

[0096] The Collaborative Configuration Management module is used to manage policies for cross-device application migration and hardware migration.

[0097] The connectivity module is responsible for distributed business operations and handling business conflicts.

[0098] The distributed media module is used to support multi-channel audio, distributed audio capabilities, and cross-device camera capabilities.

[0099] Distributed hardware modules are used to provide hardware virtualization.

[0100] The screen mirroring module is used to project the interface content of a virtual screen into a specified container.

[0101] The Super Desktop module is responsible for interacting with other modules, providing the Super Desktop Assistant module with the ability to communicate between the two ends.

[0102] In addition, the system library may also include status monitoring service modules, such as a physical status recognition module for analyzing and recognizing user gestures; and a sensor service module for monitoring sensor data uploaded by various sensors at the hardware layer to determine the physical status of device 100.

[0103] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0104] The hardware layer can include various types of sensors, such as Figure 1 The various sensors described in the document include accelerometers, gyroscopes, touch sensors, etc., which are involved in the embodiments of this application.

[0105] It should be noted that, Figure 2 This example only illustrates one way of dividing the system framework and should not be construed as a specific limitation on the embodiments of this application. In the embodiments of this application, different frameworks can be used for different operating systems. It is understood that when different frameworks are adopted, the way the framework layers are divided, the specific naming, and the specific layer in which each module is located can be different.

[0106] Currently, distributed technologies can be used to project the interface displayed on one device onto another. For example, a mobile phone can project its interface onto a TV. Another example is projecting a mobile phone's interface onto a car's cockpit screen. However, because different devices may have different screen sizes and shapes, when one device projects its interface onto another, the other device may not be able to fully utilize its screen, displaying only a portion of the projected interface while filling other areas with black borders. This leads to a degraded user experience. The following section will discuss this further. Figure 3 Please provide an explanation.

[0107] Figure 3 A set of graphical user interfaces (GUIs) provided in embodiments of this application are shown.

[0108] like Figure 3 As shown in (a), device 1 displays interface 301, which is the desktop. Device 1 can project interface 301 onto device 2. After receiving the projection data sent by device 1, device 2 can display as shown in (a). Figure 3 The GUI shown in (b) is shown in the image. Figure 3 In the GUI shown, Device 1 is a mobile phone, and Device 2 is a TV.

[0109] like Figure 3 As shown in (b), after receiving the projection data from device 1, device 2 can display interface 302 on the screen, where the content of interface 302 is the same as that of interface 301. Through Figure 3 As can be seen from (b) in the figure, since the TV screen is in landscape mode and the mobile phone screen is in portrait mode when displaying interface 301, in order to ensure that the screen projection interface is in the correct proportion, when displaying interface 302, device 2 will only display interface 302 in the middle area of ​​the screen and use black borders to fill part of the area on both sides of the screen.

[0110] It's important to note that landscape mode refers to the device's state when displayed in landscape mode, while portrait mode refers to the device's state when displayed in portrait mode. In landscape mode, the device's screen displays horizontal bars, and in portrait mode, the screen displays vertical bars.

[0111] Combined with targeting Figure 3 As described above, when device 1 projects its interface onto device 2, the two devices have different screen sizes and shapes, so device 2 cannot fully utilize its own screen area when displaying the projected interface. It only displays the content of the projected interface in the middle area, wasting display area and reducing user experience.

[0112] In addition, the above-described screen mirroring method only involves device 1 sending the mirroring data of interface 301 to device 2, and then device 2 displaying interface 302 based on the mirrored data. However, the controls included in the interface 302 displayed by device 2 are not operable by the user. In other words, the user cannot interact with interface 302 through device 2.

[0113] In summary, when one device projects its interface onto another, the other device only displays a portion of the projected interface, with the remaining areas filled with black borders. Furthermore, the user cannot interact with the interface through the other device, resulting in a degraded user experience. Therefore, providing a better projection experience has become a pressing technical issue. The following section will first introduce the projection method provided in this application's embodiments, using a GUI as an example.

[0114] Figure 4 A set of GUIs provided in embodiments of this application is shown.

[0115] like Figure 4 As shown in (a) of Figure 4, device 1 displays interface 401, which is the desktop. As can be seen from the figure, the desktop of device 1 includes four interfaces. The currently displayed interface 401 is the second interface on the desktop, and interface 401 can also be called the main interface. When device 1 responds to the user's right swipe operation, it can display the GUI as shown in (b) of Figure 4, that is, it can display interface 402. Interface 402 is the first interface on the desktop, and interface 402 can also be called the negative one screen interface.

[0116] Similarly, when device 1 responds to the user's left swipe, it can display something like this. Figure 4 The GUI described in (c) of the document, or as follows Figure 4 The GUI shown in (d) displays interface 403 and interface 404 in sequence. Interface 403 is the third interface on the desktop, and interface 404 is the fourth interface on the desktop.

[0117] In summary, it can be understood that interface 402 is to the left of interface 401, interface 401 is to the left of interface 403, and interface 403 is to the left of interface 404.

[0118] In some embodiments, when displaying interfaces 401, 403, and 404, device 1 can divide the interface into three areas: a work area, a dock area, and a navigation point area. The work area may include icons, service cards, folders, etc., of one or more applications. It is understood that the application icons, service cards, and folders included in the work areas of interfaces 401, 403, and 404 may differ. Since interface 401 is the main interface, the work area of ​​interface 401 may also include information such as time, date, and day of the week.

[0119] The dock area can include icons for one or more applications. These icons can be understood as the icons for persistent applications. The dock area can also be called the persistent application area. In other words, the dock areas of interfaces 401, 403, and 404 can contain the same application icons.

[0120] The navigation point area can include multiple navigation points. Device 1 can indicate which screen on the desktop the currently displayed interface is by marking navigation points or other means.

[0121] It should be noted that the above division of interface 401, interface 403 and interface 404 into three areas is only an example. In other embodiments, interface 401, interface 403 and interface 404 may include more or fewer areas.

[0122] Continue to refer to Figure 4 In (a), when device 1 detects a user swiping down the status bar, in response to that action, it can display something like... Figure 4 The GUI shown in (e) is shown in the image.

[0123] like Figure 4 As shown in (e), in response to a user's swipe down on the status bar, device 1 can display a control center interface 405, which includes a screen mirroring control 406. When device 1 detects a user clicking the screen mirroring control 406, in response to this action, it can display as shown in (e). Figure 4 The GUI shown in (f) is shown in the image.

[0124] like Figure 4 As shown in (f), in response to a user clicking the screen mirroring control 406, device 1 can display an option box 407, which includes the corresponding control 408 for device 2. When device 1 detects a user clicking the control 408, in response to this operation, it can generate screen mirroring data 1 based on the display parameters of device 2, interface 401, interface 402, interface 403, and interface 404, and then send screen mirroring data 1 to device 2, enabling device 2 to display as shown in (f). Figure 4 The GUI shown in (g) or as in Figure 4 The GUI shown in (i) is shown in the image.

[0125] like Figure 4As shown in (g), device 2 receives the projection data sent by device 1 and can simultaneously display multiple interfaces, including interface 409, interface 410, and interface 411. Interface 409 is to the left of interface 410, and interface 410 is to the left of interface 411. Interface 409 is associated with interface 402, interface 410 is associated with interface 401, and interface 411 is associated with interface 403. Associating two interfaces means that the content of the two interfaces is completely or partially the same.

[0126] In some embodiments, device 2 may identify the interface currently displayed by device 1.

[0127] For example, if the interface currently displayed on device 1 is interface 401, device 2 can highlight interface 410.

[0128] It should be noted that, Figure 4 (g) in this example only uses the simultaneous display of interface 409, interface 410 and interface 411 on device 2, but the embodiments of this application are not limited to this.

[0129] It should also be noted that when device 2 displays multiple interfaces simultaneously, the positional relationship between these multiple interfaces can be the same as the positional relationship of these multiple interfaces in device 1.

[0130] For example, device 2 can display interface 409 and interface 410 simultaneously, with interface 409 to the left of interface 410.

[0131] For example, device 2 can display interface 410 and interface 411 simultaneously, with interface 410 to the left of interface 411.

[0132] For example, device 2 can simultaneously display interface 410, interface 411 and interface 412, wherein interface 410 is to the left of interface 411 and interface 411 is to the left of interface 412.

[0133] For example, device 2 can simultaneously display interface 409, interface 410, interface 411 and interface 412, wherein interface 409 is to the left of interface 410, interface 410 is to the left of interface 411, and interface 411 is to the left of interface 412.

[0134] Continue to refer to Figure 4 In (g), when device 2 detects a user's interface switching operation, in response to the operation, it can display something like... Figure 4 The GUI shown in (h) is shown in the image.

[0135] Device 2 can detect the user's interface switching operation in several possible ways:

[0136] One possible implementation is that the display screen of device 2 is a touch screen, and the user's operation to switch interfaces can be a swipe operation on the display screen of device 2, that is, device 2 can detect the user's operation to switch interfaces through the display screen.

[0137] One possible implementation is that device 2 is equipped with a camera, and the user's interface switching operation can be a user's air gesture, that is, device 2 can detect the user's interface switching operation through the camera.

[0138] One possible implementation is that device 2 is equipped with a remote control, and the user can switch interfaces by pressing the control buttons on the remote control. In other words, device 2 can detect the user's interface switching operation through the remote control.

[0139] One possible implementation is that the user's interface switching operation can be a voice command, meaning that device 2 can detect the user's interface switching operation through the microphone.

[0140] like Figure 4 As shown in (h), in response to the user's operation of switching interfaces, device 2 can adjust the displayed interface, that is, display interface 412 and no longer display interface 409.

[0141] It should be noted that, Figure 4 In (g) and (h), the example is that device 2 displays interface 409, interface 410 and interface 411 simultaneously, and displays interface 412 after detecting the user's left swipe operation and no longer displays interface 409. However, it should not be construed as a specific limitation on the embodiments of this application.

[0142] For example, device 2 can simultaneously display interface 409 and interface 410, with interface 409 to the left of interface 410. When device 2 detects a user's leftward swipe, it can sequentially display interface 411 and interface 412 based on the extent of the swipe. When device 2 displays interface 411, interface 409 may no longer be displayed; that is, device 2 displays both interface 410 and interface 411 simultaneously. When device 2 displays interface 412, interface 410 may no longer be displayed; that is, device 2 displays both interface 411 and interface 412 simultaneously.

[0143] For example, device 2 can simultaneously display interface 410 and interface 411, with interface 410 to the left of interface 411. When device 2 detects a user's left swipe, it can display interface 412 and stop displaying interface 410, meaning device 2 displays both interfaces 411 and 412 simultaneously. When device 2 detects a user's right swipe, it can display interface 409 and stop displaying interface 411, meaning device 2 can display both interfaces 409 and 410 simultaneously.

[0144] For example, device 2 can simultaneously display interfaces 410, 411, and 412, where interface 410 is to the left of interface 411, and interface 411 is to the left of interface 412. When device 2 detects a user's right swipe operation, device 2 can display interface 409 and no longer display interface 412; that is, device 2 can simultaneously display interfaces 409, 410, and 411.

[0145] As mentioned above, interfaces 401, 403, and 404 can be divided into three areas: a working area, a dock area, and a navigation point area. Since the application icons in the dock areas of interfaces 401, 403, and 404 can be the same, when device 2 displays multiple interfaces from these interfaces simultaneously, if each interface also includes a dock area, some display area will be wasted. Similarly, if each interface also includes a navigation point area, some display area will also be wasted. Therefore, when device 1 generates projection data based on interfaces 401, 402, 403, and 404, it can separate the dock areas and navigation point areas of interfaces 401, 403, and 404, thus allowing for the display of... Figure 4 The GUI shown in (i) is shown in the image.

[0146] like Figure 4 As shown in (i), device 2 can simultaneously display interfaces 413, 414, and 415. Interface 413 includes the interface content of interface 402, interface 414 includes the interface content of the work area of ​​interface 401, and interface 415 includes the interface content of the work area of ​​interface 403. In addition, device 2 can also separately display a dock area 416 and a navigation bar 417. The application icons included in dock area 416 are the same as those of the applications included in the dock areas of interfaces 401, 403, and 404.

[0147] It is understandable that when generating projection data, device 1 can render the original navigation point area as a navigation bar, which can better adapt to application scenarios that display multiple interfaces at the same time.

[0148] It should be noted that device 2 is in Figure 4 The interface content displayed in (g)-(i) can also be referred to as the interface content of the super desktop of device 2.

[0149] It should also be noted that, in Figure 4In the GUI shown, the example is that device 2 displays the interface content of multiple interfaces of the desktop of device 1 through multiple interfaces. However, the embodiments of this application are not limited to this. In other embodiments of this application, device 2 can also display the interface content of multiple interfaces of the desktop of device 1 through one interface.

[0150] In this embodiment, when device 1 projects its screen to device 2, the projection data sent may include projection data for multiple interfaces, so that device 2 can display the content of multiple interfaces at the same time, making full use of the display screen and avoiding black border filling, which helps to improve the user experience.

[0151] Figure 4 The example given is device 1 displaying its desktop and then projecting its screen to device 2. However, this application is not limited to this. In other embodiments of this application, device 1 can also project its screen to device 2 while displaying the interface of an application. The following description, in conjunction with... Figure 5 Please provide an explanation.

[0152] Figure 5 A set of GUIs provided in embodiments of this application is shown.

[0153] like Figure 5 As shown in (a)-(d) in the diagram, device 1 displays interface 501, which is the interface of application 1. Device 1 can display one or more controls on interface 501, such as controls 502, 503, and 504, which are used to control device 1 to display other interfaces. When device 1 detects a user clicking control 502, it can display interface 505. When device 1 detects a user clicking control 503, it can display interface 506. When device 1 detects a user clicking control 504, it can display interface 507.

[0154] It is understandable that since interface 501 includes controls for displaying interface 505, interface 506 and interface 507, interface 501 is associated with interface 505, interface 506 and interface 507.

[0155] When device 1 detects a user's screen mirroring operation to device 2 while displaying interface 501, in response to this operation, it generates screen mirroring data based on the display parameters of device 2 and the content of interfaces 501, 505, 506, and 507, and then sends the screen mirroring data to device 2. After receiving the screen mirroring data, device 2 can display as follows: Figure 5 The GUI shown in (e) is shown in the image.

[0156] It should be understood that a description of the user's screen mirroring operation can be found in the documentation for... Figure 4 For the sake of brevity, the description in the text will not be repeated here.

[0157] like Figure 5 As shown in (e), after receiving the screen projection data sent by device 1, device 2 can display interface 508, interface 509, interface 510 and interface 511. Among them, interface 508 is associated with interface 501, interface 509 is associated with interface 505, interface 510 is associated with interface 506, and interface 511 is associated with interface 507.

[0158] In this embodiment, when device 1 projects its screen to device 2, the projection data sent may include projection data for multiple interfaces. This allows device 2 to display the content of multiple interfaces simultaneously, making full use of the display screen and preventing black borders from appearing, thus improving the user experience.

[0159] The preceding text described how device 1 detects the user's screen mirroring operation when displaying the desktop or application interface. It can generate mirroring data based on multiple desktop or application interfaces and then send this data to device 2, allowing device 2 to display multiple interfaces simultaneously. The following text will describe the interaction between the user and device 2 using the GUI.

[0160] Figure 6 A set of GUIs provided in embodiments of this application is shown.

[0161] like Figure 6 As shown in (a) and (b), device 1 displays interface 601, which is the main desktop interface. In response to a user's screen mirroring operation, device 1 can send screen mirroring data 1 to device 2. After receiving screen mirroring data 1 from device 1, device 2 can display interfaces 602, 603, 604, a dock area 605, and a navigation bar area 606, where interface 603 is associated with interface 601. Device 2 can display icons 607, 608, 609, and a folder 610 on interface 603. Icon 607 corresponds to a music application, which includes multiple interfaces; icon 608 corresponds to a calculator application, which includes one interface; icon 609 corresponds to a game application, which supports landscape layout; and folder 610 includes icons for one or more applications.

[0162] When device 2 detects a user's click on icon 607, in response, it can send instruction information 1 to device 1. Instruction information 1 instructs the user on the click on icon 607. Device 1 receives instruction information 1, generates projection data 2 based on it, and then sends projection data 2 to device 2. Device 2 receives projection data 2 and updates the displayed interface accordingly, displaying, as shown below. Figure 6 The GUI shown in (c) is shown in the image.

[0163] like Figure 6 As shown in (c), after receiving the screen projection data 2, device 2 can simultaneously display multiple interfaces of the music application, and no longer displays interface 602, interface 603, interface 604, dock area 605 and navigation bar area 606.

[0164] When device 2 detects a user's click on icon 608, in response, it can send instruction information 2 to device 1. Instruction information 2 instructs the user on the click on icon 608. Device 1 receives instruction information 2, generates projection data 3 based on it, and then sends projection data 3 to device 1. Device 2 receives projection data 3 and updates the displayed interface accordingly, displaying, as shown below. Figure 6 The GUI shown in (d) is shown in the image.

[0165] like Figure 6 As shown in (d), after receiving the screen projection data 3, device 2 can replace interface 603 with interface 611, which is the interface of the calculator application. In other words, device 2 can simultaneously display the interface of the calculator application and the desktop interface.

[0166] Understandably, since the calculator application only includes one interface, when generating screen projection data 3, device 1 can use the screen projection data of the calculator application's interface to replace the screen projection data of any one of interfaces 602, 603, and 604. Figure 6 (d) Take the screen projection data of replacing interface 603 as an example), so that after receiving screen projection data 3, device 2 can simultaneously display interface 611, interface 602, interface 604, dock area 605 and navigation bar area 606.

[0167] It is also understandable that Figure 6 (c) in this document is only an example of an application having only one interface, but should not be construed as a specific limitation corresponding to the embodiments of this application. In other embodiments, the number of interfaces included in the application is greater than 1 but less than the number of interfaces displayed simultaneously by the device 2. When generating the screen projection data 3, the device 1 can still use the application's interface screen projection data to replace part of the screen projection data in the screen projection data 1, so that the device 2 can display the application's interface and the interface on the desktop simultaneously.

[0168] When device 2 detects a user's click on icon 609, in response, it can send instruction information 3 to device 1. Instruction information 3 instructs the user on the click on icon 609. Device 1 receives instruction information 3, generates projection data 4 based on it, and then sends projection data 4 to device 2. Device 2 receives projection data 4 and updates the displayed interface accordingly, displaying, as shown below. Figure 6 The GUI shown in (e) is shown in the image.

[0169] like Figure 6 As shown in (e), after receiving the screen projection data 4, device 2 can display interface 612 in full screen, which is the interface of the game application.

[0170] It is understandable that since device 2 is in landscape mode and the game application corresponding to icon 609 supports landscape layout, when device 1 generates screen projection data 4, the screen projection data 4 includes the screen projection data corresponding to the landscape layout of the game application interface, so that device 2 can display the interface 612 in full screen after receiving the screen projection data 4.

[0171] When device 2 detects a user's click on folder 610, in response, it can send instruction information 4 to device 1. Instruction information 4 instructs the user on the click on folder 610. Device 1 receives instruction information 4, generates projection data 5 based on it, and then sends projection data 5 to device 2. Device 2 receives projection data 5 and updates the displayed interface accordingly, displaying, as shown below. Figure 6 The GUI shown in (f) is shown in the image.

[0172] like Figure 6 As shown in (f), after receiving the screen projection data 5, device 2 can display interface 613 while displaying interface 602, interface 603, interface 604, dock area 605 and navigation bar area 606. Interface 613 may include icons of one or more applications, and interface 613 may cover interface 602 and interface 603.

[0173] In some embodiments, device 2 may also highlight interface 613.

[0174] In some other embodiments, after receiving the projection data 5, the device 2 can replace the interface 603 with the interface 613.

[0175] Device 2 can detect user clicks on application icons or folders in several possible ways:

[0176] One possible implementation is that the display screen of device 2 is a touch screen, and the user's operation of clicking the application icon or folder can be a click operation of the user on the display screen of device 2. That is, device 2 can detect the user's operation of clicking the application icon or folder through the display screen.

[0177] One possible implementation is that device 2 is equipped with a remote control, and the user's action of clicking on the application icon or folder can be achieved by the user pressing the control button on the remote control. That is, device 2 can detect the user's action of clicking on the application icon or folder through the remote control.

[0178] In some embodiments, in addition to sending instruction information to device 1 in response to a user clicking an application icon or folder, device 2 can also send instruction information to device 1 in response to a user's voice command, which is used to instruct the user to launch an application or open a folder.

[0179] In this application, when device 2 detects that the user has enabled an application or opened a folder while displaying the desktop of device 1, it can send an instruction message to device 1 to instruct the user's operation. Device 1 can then generate and send new screen projection data to device 2 based on the instruction message, thereby updating the displayed interface of device 2 and enabling the user to interact with device 2, thus improving the user experience.

[0180] exist Figure 6 The GUI shown is an example of detecting user actions when multiple desktop interfaces are displayed simultaneously on device 2. The following text will introduce the example of detecting user actions when multiple application interfaces are displayed simultaneously on device 2.

[0181] Figure 7 A set of GUIs provided in embodiments of this application is shown.

[0182] like Figure 7 As shown in (a), device 2 can simultaneously display interface 701, interface 702, interface 703 and interface 704 when receiving screen projection data 1 sent by device 1. Interfaces 701, 702, 703 and 704 belong to application 1. The interface content corresponding to interface 701 is the interface content currently displayed by device 1.

[0183] When device 2 detects a user's action to return to the desktop, in response, device 2 can send instruction information 1 to device 1. Instruction information 1 instructs the user to return to the desktop. Upon receiving instruction information 1, device 1 can generate and send screen projection data 2 to device 2. Screen projection data 2 includes projection data for one or more interfaces of the desktop. Upon receiving screen projection data 2, device 2 can display... Figure 7 The GUI shown in (b) is shown in the image.

[0184] like Figure 7 As shown in (b), after receiving the screen projection data 2, device 2 can simultaneously display one or more interfaces of the desktop.

[0185] It should be understood that, regarding Figure 7 For a detailed description of (b) in this document, please refer to the document specifically for... Figure 4 For the sake of brevity, the description of (i) in the text will not be repeated here.

[0186] In this application, when device 2 detects the user's return to the desktop while displaying multiple interfaces of an application, it can send an instruction message to device 1 to instruct the user's operation. Device 1 can generate and send new screen projection data to device 2 based on the instruction message, thereby updating the displayed interface of device 2, enabling the user to interact with device 2 and improving the user experience.

[0187] It should be noted that, Figure 7 The example of returning to the desktop is an upward swipe, but this should not be construed as a specific limitation on the embodiments of this application. In other embodiments of this application, the operation of returning to the desktop can also be a user's voice command, a user's right swipe operation on the left edge of the display screen of device 2, a user's left swipe operation on the right edge of the display screen of device 2, etc.

[0188] In some embodiments, interfaces 701, 702, 703, and 704 may include one or more controls, each corresponding to one or more services. For example, interface 701 includes a playback control 705 for playing music. When device 2 detects a user clicking the playback control 705, it can send an instruction to device 1 in response, instructing the user to click the playback control 705. After receiving the instruction, device 1 can send service data to device 2, including audio data corresponding to the song. After receiving the service data, device 2 can play the song through its own speaker. It is understood that in these embodiments, since device 1 will have corresponding animation effects when playing the song (e.g., scrolling the lyrics of the song, displaying the playback progress, etc.), device 1 can also send the corresponding animation effect projection data to device 2, so that device 2 can display the corresponding animation effect on interface 701.

[0189] In this application, when device 2 detects a user's click operation on a control while displaying multiple interfaces of an application, it can send an instruction to device 1 to instruct the user's operation. Device 1 can then send corresponding business data to device 2 based on the instruction, enabling the user to execute business operations on device 1 through device 2, which helps improve the user experience.

[0190] The above description uses the example of device 1 having multiple interfaces on its desktop. However, the embodiments of this application are not limited to this. In other embodiments of this application, the desktop of device 1 can also be a single interface. The following description, in conjunction with... Figure 8 Please provide an explanation.

[0191] Figure 8 A set of GUIs provided in embodiments of this application is shown.

[0192] like Figure 8 As shown in (a), device 1 displays interface 801, which is the main desktop interface, and device 1 has one desktop interface. In response to a user's screen mirroring operation, device 1 can send screen mirroring data 1 to device 2. After receiving screen mirroring data 1 from device 1, device 2 can display as shown in [image missing]. Figure 8 The GUI shown in (b) or (c) is shown in the image.

[0193] like Figure 8 As shown in (b), after receiving the projection data 1, device 2 can display interface 802 on the left side of the screen, wherein the interface content of interface 802 is the same as that of interface 801.

[0194] It should be noted that the embodiments of this application only take the display of interface 802 on the left side of the display screen by device 2 as an example, but it should not be construed as a specific limitation on the embodiments of this application. In other embodiments of this application, device 2 may also display interface 802 in the middle area or on the right side of the display screen.

[0195] like Figure 8 As shown in (c), after receiving the screen projection data 1, device 2 can display interface 803 and dock area 804. Interface 803 includes icons of applications in the work area of ​​interface 801.

[0196] It should be noted that, in Figure 8 In the GUIs shown in (b) and (c), device 2 does not use black borders to fill the blank areas when displaying interface 802. These blank areas are used to display other interfaces, allowing device 2 to simultaneously display multiple interfaces in response to user actions. The following example illustrates how device 2 displays the interface after receiving projection data 1. Figure 8 The GUI shown in (c) will be used as an example for introduction.

[0197] Interface 803 includes icons for one or more applications and one or more folders, such as icon 805, icon 806, icon 807, and folder 808. Icon 805 corresponds to a calculator application, which includes one interface; icon 806 corresponds to a music application, which includes multiple interfaces; icon 807 corresponds to a game application, which supports landscape layout; and folder 808 includes icons for one or more applications.

[0198] When device 2 detects a user's click on icon 805, in response, it can send instruction information 1 to device 1. Instruction information 1 instructs the user on the click on icon 805. Device 1 receives instruction information 1, generates projection data 2 based on it, and then sends projection data 2 to device 2. Device 2 receives projection data 2 and updates the displayed interface accordingly, displaying, as shown below. Figure 8 The GUI shown in (d) is shown in the image.

[0199] like Figure 8 As shown in (d), after receiving the projection data 2, device 2 can simultaneously display interface 803 and interface 809, where interface 809 is the interface of the calculator application.

[0200] Understandable, Figure 8 (d) in this document is only an example of an application having only one interface, but should not be construed as a specific limitation corresponding to the embodiments of this application. In other embodiments, the number of interfaces included in the application is greater than 1, but the sum of the number of interfaces included in the application and the number of interfaces on the desktop is less than the number of interfaces displayed simultaneously by the device 2. The device 2 can still display multiple interfaces of the application and interfaces on the desktop simultaneously.

[0201] When device 2 detects a user's click on icon 806, in response, it can send instruction information 2 to device 1. Instruction information 2 instructs the user on the click on icon 806. Device 1 receives instruction information 2, generates projection data 3 based on it, and then sends projection data 3 to device 1. Device 2 receives projection data 3 and updates the displayed interface accordingly, displaying, as shown below. Figure 8 The GUI shown in (e) is shown in the image.

[0202] like Figure 8 As shown in (e), after receiving the screen projection data 3, device 2 can simultaneously display multiple interfaces of the music application, and no longer displays interface 803 and dock area 804.

[0203] When device 2 detects a user's click on icon 807, in response, it can send instruction information 3 to device 1. Instruction information 3 instructs the user on the click on icon 807. Device 1 receives instruction information 3, generates projection data 4 based on it, and then sends projection data 4 to device 2. Device 2 receives projection data 4 and updates the displayed interface accordingly, displaying, as shown below. Figure 8 The GUI shown in (f) is shown in the image.

[0204] like Figure 8As shown in (f), after receiving the screen projection data 4, device 2 can display interface 810 in full screen, which is the interface of the game application.

[0205] When device 2 detects a user clicking on folder 808, in response, it can send instruction information 4 to device 1. Instruction information 4 instructs the user to click on folder 808. Device 1 receives instruction information 4, generates projection data 5 based on it, and then sends projection data 5 to device 2. Device 2 receives projection data 5 and updates the displayed interface accordingly, displaying, as shown below. Figure 8 The GUI shown in (g) is shown in the image.

[0206] like Figure 8 As shown in (g), after receiving the screen projection data 5, device 2 can simultaneously display interface 803, dock area 804 and interface 811, which may include icons of one or more applications.

[0207] In this application, when device 2 detects that the user has enabled an application or opened a folder while displaying the desktop of device 1, it can send an instruction message to device 1 to instruct the user's operation. Device 1 can then generate and send new screen projection data to device 2 based on the instruction message, thereby updating the displayed interface of device 2 and enabling the user to interact with device 2, thus improving the user experience.

[0208] The above example uses device 2 receiving screen projection data sent by device 1, but the embodiments of this application are not limited to this. In other embodiments of this application, device 2 can receive screen projection data sent by multiple devices at the same time, so that device 2 can display the interfaces being displayed by the multiple devices at the same time.

[0209] Figure 9 A set of GUIs provided in embodiments of this application is shown.

[0210] like Figure 9 As shown, device 2 can receive screen projection data sent by device 1, device 3, device 4 and device 5, thereby simultaneously displaying interfaces 901, 902, 903 and 904. The interface content in interface 901 is the interface content currently displayed by device 1, the interface content in interface 902 is the interface content currently displayed by device 3, the interface content in interface 903 is the interface content currently displayed by device 4, and the interface content in interface 904 is the interface content currently displayed by device 5.

[0211] It is understandable that when device 2 receives screen projection data from more devices, device 2 can also display a navigation bar while displaying multiple interfaces simultaneously.

[0212] In some embodiments, when device 2 receives screen projection data from one device, device 2 can simultaneously display the interface content of multiple interfaces of that device. When device 2 receives screen projection data from multiple devices, device 2 can simultaneously display the interface content of the interfaces currently being displayed by those multiple devices.

[0213] For example, device 2 can first receive the screen projection data sent by device 1, and device 2 can display the content of multiple interfaces on device 1, that is, device 2 can display such as... Figure 4 The GUI is shown in (g) or (i). When device 2 receives screen projection data from devices 3, 4, and 5, device 2 can simultaneously display the interface content currently displayed by devices 1, 3, 4, and 5; that is, device 2 can display, as shown in (g) or (i). Figure 9 The GUI shown.

[0214] Similarly, when device 2 switches from receiving screen projection data from multiple devices to receiving screen projection data from one device, device 2 can switch from simultaneously displaying the interface content of the interfaces being displayed by the multiple devices to displaying the interface content of multiple interfaces from one device.

[0215] It should be noted that, Figure 9 Taking the example of device 2 receiving screen projection data from 4 devices, it should not be construed as a specific limitation on the embodiments of this application. In other embodiments of this application, device 2 may receive screen projection data from more or fewer devices.

[0216] Since device 2 displays interfaces 901, 902, 903, and 904 simultaneously, the user can interact with any of these interfaces.

[0217] For example, interface 901 includes icon 905, which corresponds to a music application. When device 2 detects that a user clicks icon 905, it can send an instruction to device 1 in response to this operation. After receiving the instruction, device 1 can generate new screen mirroring data and then send the new screen mirroring data to device 2. After receiving the new screen mirroring data, device 2 can simultaneously display multiple interfaces of the music application and will no longer display interfaces 901, 902, 903, and 904.

[0218] For example, interface 901 includes icon 905, which corresponds to a music application. When device 2 detects that a user clicks icon 905, it can send an instruction to device 1 in response to this operation. After receiving the instruction, device 1 can generate new screen mirroring data and then send the new screen mirroring data to device 2. After receiving the new screen mirroring data, device 2 can replace interface 901 with the main interface of the music application, and simultaneously display interfaces 902, 903, and 904.

[0219] For example, interface 901 includes icon 906, which corresponds to the calculator application. When device 2 detects that the user clicks icon 906, it can send an instruction to device 1 in response to this operation. After receiving the instruction, device 1 can generate new screen projection data and then send the new screen projection data to device 2. After receiving the new screen projection data, device 2 can replace interface 901 with the main interface of the calculator application, and simultaneously display interfaces 902, 903, and 904.

[0220] For example, interface 901 includes icon 907, which corresponds to a game application. When device 2 detects that a user clicks icon 907, it can send an instruction to device 1 in response to this operation. After receiving the instruction, device 1 can generate new screen mirroring data and then send the new screen mirroring data to device 2. After receiving the new screen mirroring data, device 2 can display the game application interface in full screen.

[0221] For example, interface 901 includes folder 908. When device 2 detects that the user clicks on folder 908, in response to this operation, it can send an instruction message to device 1. After receiving the instruction message, device 1 can generate new screen projection data and then send the new screen projection data to device 2. After receiving the new screen projection data, device 2 can replace interface 901 with the interface of folder 908, and simultaneously display interfaces 902, 903, and 904.

[0222] In this embodiment, device 2 can receive screen projection data sent by multiple devices, thereby simultaneously displaying the interface content being displayed by the multiple devices. This fully utilizes the display screen, avoids black borders, and helps improve the user experience.

[0223] The screen projection method provided by the embodiments of this application has been described above in conjunction with the GUI. The screen projection method provided by the embodiments of this application will be described below in conjunction with the method flowchart.

[0224] Figure 10This illustration shows a schematic flowchart of the screen projection method provided in an embodiment of this application. This method can be executed by a device or by a component of the device (e.g., a processor, chip system, etc.). The following description uses a device as the executing entity. Figure 8 As shown, the method includes:

[0225] S1001, the first device detects the user's screen mirroring operation on the first interface.

[0226] The first device can detect the user's screen mirroring operation on the first interface when the first interface is displayed.

[0227] In some embodiments, the first interface is an interface on the desktop.

[0228] For example, such as Figure 4 As shown in (a), device 1 displays interface 401.

[0229] In some embodiments, the first interface is an interface of an application.

[0230] For example, such as Figure 5 As shown in (a), device 1 displays interface 501.

[0231] S1002, the first device sends the first screen projection data to the second device.

[0232] Correspondingly, the second device receives the first screen projection data sent by the first device.

[0233] Specifically, upon detecting a user's screen mirroring operation on the first interface, the first device, in response to this operation, can generate and send first screen mirroring data to the second device based on the display parameters of the second device, the currently displayed first interface, and N interfaces associated with the first interface, where N ≥ 1 and is an integer. The display parameters of the second device include, but are not limited to, width, height, and dots per inch (DPI), where the width and height are the width and height of the surface created by the second device, which can be used to carry the screen mirroring data sent by the first device.

[0234] In some embodiments, the display parameters of the second device may be sent by the second device to the first device.

[0235] In some embodiments, when the first device is connected to the second device, it can obtain the model number of the second device, and then the first device can determine the display parameters of the second device based on the model number of the second device.

[0236] S1003, the second device simultaneously displays the content of the first interface and M interfaces out of N interfaces based on the first projection data.

[0237] Specifically, after receiving the first projection data, the second device can control the display device (e.g., the display screen) to simultaneously display the first interface and the interface content of M interfaces out of N interfaces, where N≥M≥1 and M is an integer.

[0238] Understandably, due to the size limitations of the second device's display screen, the second device may not be able to display the content of the first interface and the N interfaces simultaneously. Therefore, the second device can display the content of the first interface and M interfaces out of the N interfaces.

[0239] For example, such as Figure 4 As shown in (g), device 1 can send the projection data of interface 401, interface 402, interface 403 and interface 404 to device 2. Device 2 can simultaneously display interface 409, interface 410 and interface 411, wherein interface 409 is associated with interface 402, interface 410 is associated with interface 401, and interface 411 is associated with interface 403.

[0240] In some embodiments, the first interface and the M interfaces are interfaces on a desktop. The second device can display icons corresponding to the resident applications of the desktop in a first position, icons of applications in the working areas of the first interface and the M interfaces in a second position, and a navigation bar in a third position.

[0241] For example, such as Figure 4 As shown in (i), device 1 can send the projection data of interface 401, interface 402, interface 403 and interface 404 to device 2, and device 2 can simultaneously display interface 413, interface 414, interface 415, dock area 416 and navigation bar area 417.

[0242] It is understandable that when the second device can display all the interfaces of the first device's desktop at the same time, the second device may not display the navigation bar.

[0243] In this embodiment of the application, when the first device projects its screen to the second device, the projection data sent may include projection data for multiple interfaces, so that the second device can display multiple interfaces at the same time, making full use of the display area of ​​the screen and avoiding black border filling, which helps to improve the user experience.

[0244] In some embodiments, the method further includes:

[0245] S1004, the second device sends the first instruction information to the first device.

[0246] Correspondingly, the first device receives a first instruction message sent by the second device, which is used to instruct the user on a first operation on the second device.

[0247] Specifically, when the second device simultaneously displays the content of the first interface and the M interfaces, upon detecting the user's first operation, it can generate first instruction information in response to the user's first operation. This first instruction information can instruct the user on the first operation on the second device, and then the second device can send the first instruction information to the first device.

[0248] S1005, the first device sends the second screen projection data to the second device.

[0249] Correspondingly, the second device receives the second screen projection data sent by the first device.

[0250] Specifically, the first device can determine the user's first operation based on the first instruction information, then determine the second interface corresponding to the user's first operation based on the user's first operation, and then generate and send the second screen projection data to the second device based on the display parameters of the second device and the second interface.

[0251] S1006, the second device updates the interface content based on the second projection data.

[0252] After receiving the second projection data, the second device can control the display device to update the content of the currently displayed interface based on the second projection data.

[0253] In some embodiments, the second projection data further includes projection data of X interfaces associated with the second interface, where X ≥ 1 and X is an integer. The first interface includes an icon of a first application, and the user's first operation is to click the icon of the first application. The second interface may be the main interface of the first application. S1006, the second device updates the interface content according to the second projection data, including:

[0254] The second device controls the display device to simultaneously display the interface content of the second interface and Y interfaces out of X interfaces, and no longer displays the interface content of the first interface and M interfaces, where X≥Y≥1 and Y is an integer.

[0255] For example, such as Figure 6 As shown in (b) and (c), device 2 can send instruction information to device 1 to instruct the user to click icon 607. After receiving the instruction information, device 1 can generate and send new screen mirroring data to device 2. After receiving the new screen mirroring data, device 2 can simultaneously display multiple interfaces of the music application.

[0256] For example, such as Figure 8As shown in (c) and (e), device 2 can send instruction information to device 1 to instruct the user to click icon 806. After receiving the instruction information, device 1 can generate and send new screen mirroring data to device 2. After receiving the new screen mirroring data, device 2 can simultaneously display multiple interfaces of the music application.

[0257] In some embodiments, the first interface includes an icon of a first application, the user's first operation is clicking the icon of the first application, and the second interface is the main interface of the first application. S1006, the second device updates the interface content according to the second screen projection data, including:

[0258] The second device controls the display device to display the interface content included in the second interface, and no longer displays the interface content included in the first interface.

[0259] Since the second device can display the content of multiple interfaces of the first device simultaneously in various ways, there are also different ways to prevent the second device from displaying the content of the first interface:

[0260] One possible implementation is that the second device no longer displays the interface content included in the working area of ​​the first interface.

[0261] For example, such as Figure 6 As shown in (b) and (d), device 2 can display interface 611 and no longer display interface 603, that is, no longer display the interface content included in the working area of ​​interface 601.

[0262] One possible implementation is that the second device no longer displays all the interface content included in the first interface.

[0263] For example, such as Figure 4 As shown in (g), the device simultaneously displays interfaces 409, 410, and 411. Interface 409 is associated with interface 402, interface 410 is associated with interface 401, and interface 411 is associated with interface 403. When device 2 detects that the user clicks the calculator application icon in interface 410, device 2 can display the interface corresponding to that calculator application to replace interface 410, that is, it will no longer display all the interface content included in interface 401.

[0264] In some embodiments, the second device is in landscape mode, the first interface includes an icon of a first application, the first application supports landscape layout, the user's operation is to click the icon of the first application, the second interface is the interface of the first application, S1006, the second device updates the interface content according to the second projection data, including: the second device controls the display device to display the interface content included in the second interface in full screen.

[0265] For example, such as Figure 6 As shown in (b) and (e), device 2 can send instruction information to device 1 to instruct the user to click icon 609. After receiving the instruction information, device 1 can generate and send new projection data to device 2. After receiving the new projection data, device 2 can display interface 612 in full screen.

[0266] In some embodiments, the first interface is the interface of the first application, the user's first operation is the operation of returning to the desktop, the second interface is the desktop interface, S1006, the second device updates the interface content according to the second projection data, including: the second device controls the display device to simultaneously display the interface content included in the second interface and B interfaces out of V interfaces, and no longer displays the interface content included in the first interface and M interfaces, where V≥B≥0, and Y is an integer.

[0267] For example, such as Figure 7 As shown in (a) and (b), device 2 can send instruction information to device 1 to instruct the user to swipe up. After receiving the instruction information, device 1 can generate and send new screen projection data to device 2. After receiving the new screen projection data, device 2 can simultaneously display the content of multiple interfaces on the desktop of device 1.

[0268] In some embodiments, the first electronic device receives first instruction information and can also control the display device to display a second interface based on the first instruction information.

[0269] In some embodiments, the first interface includes a first control, the first control corresponding to a first business function, and the method further includes:

[0270] The second device responds to the user's click on the first control by sending a second instruction message to the first device.

[0271] The first device generates the first business data based on the second instruction information.

[0272] The first device sends the first service data to the second device.

[0273] For example, such as Figure 7 As shown in (a), device 2 can send instruction information to device 1 to instruct the user to click the play control 705. After receiving the instruction information, device 1 can generate corresponding audio data, and then send the audio data to device 2 so that device 2 can play the audio data.

[0274] In some embodiments, the method further includes:

[0275] The second device receives the third screen projection data sent by the third device.

[0276] The second device controls the display device to simultaneously display the interface content included in the first interface and the third interface based on the first projection data and the third projection data, and no longer displays the interface content included in the M interfaces, wherein the third interface is the interface currently displayed by the third device.

[0277] Specifically, the second device can simultaneously receive screen projection data from multiple devices. When the second device first receives screen projection data from the first device, the first device can simultaneously display the content of multiple interfaces of the first device. When the second device then receives screen projection data from the third device, the second device can simultaneously display the content of the interfaces currently displayed by the first and third devices, and can stop displaying the content of the M interfaces of the first device.

[0278] For example, device 2 can first receive the screen projection data sent by device 1, and device 2 can display the content of multiple interfaces on device 1, that is, device 2 can display such as... Figure 4 The GUI is shown in (g) or (i). When device 2 receives screen projection data from devices 3, 4, and 5, device 2 can simultaneously display the interface content currently displayed by devices 1, 3, 4, and 5; that is, device 2 can display, as shown in (g) or (i). Figure 9 The GUI shown.

[0279] The screen projection method provided by the embodiments of this application has been described in detail above. In the various embodiments of this application, unless otherwise specified or logically conflicting, the terminology and / or descriptions between the various embodiments are consistent and can be referenced by each other. The technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationships.

[0280] The foregoing primarily describes a screen projection method provided by the embodiments of this application from the perspective of the device. It is understood that, in order to achieve the above functions, the device includes corresponding hardware structures and / or software modules for executing each function. Those skilled in the art should readily recognize that, in conjunction with the algorithm steps of the examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and software. Whether a function is executed in hardware or software-driven hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0281] Given that each function is divided into functional modules (or units) corresponding to its specific functions, the following description will introduce the screen projection method provided in this application embodiment from the perspective of functional modules.

[0282] Figure 11A schematic flowchart of the screen mirroring method provided in an embodiment of this application is shown, such as... Figure 11 As shown, the method includes:

[0283] S1101, Second Launcher Creation Page.

[0284] The second launcher is the launcher for the second device, and the page created by the second launcher can be used to carry the screen projection data.

[0285] S1102, the second initiator sends the page parameters to the second collaborative capability module.

[0286] The second collaborative capability module is the collaborative capability module for the second device, and its page parameters include the page width and height. The second collaborative capability module includes... Figure 2 The collaborative configuration management module, connection module, etc. are shown.

[0287] S1103, the second collaborative capability module and the first collaborative capability module are initialized.

[0288] The first collaboration capability module is the collaboration capability module of the first device. The second collaboration capability module performs initialization processing with the first collaboration capability module, including sending page parameters to the first collaboration capability module. Furthermore, the second collaboration capability module can also send the DPI of the second device's display screen to the first collaboration capability module.

[0289] S1104, the first projection module and the second projection module establish a connection.

[0290] The first projection module is the projection module of the first device, and the second projection module is the projection module of the second device. The first projection module and the second projection module can establish a connection after initialization.

[0291] S1105, the first projection module creates a virtual display.

[0292] The first screen mirroring module can create a virtual display (VD).

[0293] S1106, the first screen mirroring module starts screen mirroring.

[0294] It should be noted that the execution order of steps S1105 and S1106 is not limited in this embodiment. For example, step S1105 may be executed before step S1106 is executed by the first projection module, or step S1105 may be executed after step S1106 is executed.

[0295] S1107, the first launcher creates the first activity.

[0296] S1108, the first launcher draws the layout.

[0297] After screen mirroring is initiated, the first launcher can render the currently displayed first interface and N interfaces associated with it onto the first activity based on the display parameters of the second device. This allows the second device to display the content of multiple interfaces simultaneously. The first activity can also be referred to as the first interface capability.

[0298] S1109, the first projection module sends the first projection data to the second projection module.

[0299] Correspondingly, the second projection module receives the first projection data sent by the first projection module, so that the second launcher can display the content of the first interface and M interfaces out of N interfaces.

[0300] S1110, the first launcher and the second launcher refresh synchronously.

[0301] A synchronous refresh mechanism can be established between the first launcher and the second launcher. When the interface displayed on the first device changes, the interface displayed on the second device will also change.

[0302] S1111, the second initiator sends the first instruction information to the first initiator.

[0303] Correspondingly, the first initiator receives the first instruction information sent by the second initiator, which is used to instruct the user on the first operation.

[0304] It should be understood that a detailed description of the first instruction information can be found above, and for the sake of brevity, it will not be repeated here.

[0305] This application provides a program product (also referred to as a computer program product) that, when run on a device, causes the device to execute the technical solutions described above. Its implementation principle and technical effects are similar to those of the related embodiments described above, and will not be repeated here.

[0306] This application provides a readable storage medium (also known as a computer-readable storage medium) containing instructions that, when the device is running, cause the device to execute the technical solution described in the above embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

[0307] This application provides a chip for executing instructions. When the chip is running, it executes the technical solutions described in the above embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

[0308] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0309] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0310] The above description is merely a specific implementation of the embodiments of this application, but the protection scope of the embodiments of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the embodiments of this application should be included within the protection scope of the embodiments of this application. Therefore, the protection scope of the embodiments of this application should be determined by the protection scope of the claims.

Claims

1. A screen projection method, characterized in that, The method includes: The user's screen mirroring operation on the first screen was detected; In response to the screen mirroring operation, first screen mirroring data is sent to the second device. The first screen mirroring data includes the screen mirroring data of the first interface and N interfaces associated with the first interface, where N≥1 and is an integer.

2. The method according to claim 1, characterized in that, The method further includes: Receive first indication information sent by the second device, the first indication information being used to instruct the user on a first operation on the second device; According to the first instruction information, the second screen projection data is sent to the second device, wherein the second screen projection data includes screen projection data of the second interface, and the second interface is associated with the first operation.

3. The method according to claim 2, characterized in that, The first interface includes an icon of a first application, and the first operation is the operation of clicking the icon of the first application. The second screen projection data also includes screen projection data of X interfaces associated with the second interface, where X ≥ 1 and is an integer.

4. The method according to claim 2 or 3, characterized in that, The method further includes: The control display device displays the second interface.

5. The method according to any one of claims 2 to 4, characterized in that, When the second device is in landscape mode and the second interface supports landscape layout, the second projection data includes projection data corresponding to the landscape layout of the second interface.

6. The method according to claim 1, characterized in that, The first interface includes a first control, the first control corresponding to a first business function, and the method further includes: Receive a second instruction message sent by the second device, the second instruction message being used to instruct the user to click the first control; Send the service data of the first service to the second device.

7. A screen projection method, characterized in that, The method includes: Receive first projection data sent by the first device, the first projection data includes projection data of the first interface and N interfaces associated with the first interface, where N≥1 and is an integer; The display device is controlled to simultaneously display the content of the first interface and M interfaces out of the N interfaces, where N≥M≥1 and M is an integer, based on the first projection data.

8. The method according to claim 7, characterized in that, The method further includes: In response to the user's first operation, the first instruction information is sent to the first device, the first instruction information being used to indicate the first operation; Receive second screen projection data sent by the first device, wherein the second screen projection data includes screen projection data of a second interface, and the second interface is associated with the first operation; Update the interface content based on the second screen projection data.

9. The method according to claim 8, characterized in that, The first interface includes an icon of a first application, the first operation is clicking the icon of the first application, the second interface is the interface of the first application, and the second projection data also includes projection data of X interfaces associated with the second interface, where X ≥ 1 and is an integer. Updating the interface content according to the second projection data includes: The display device is controlled to simultaneously display the content of the second interface and the Y interfaces among the X interfaces, and no longer display the content of the first interface and the M interfaces, where Y≥1 and is an integer.

10. The method according to claim 8, characterized in that, The first interface includes an icon of a first application, the first operation is clicking the icon of the first application, the second interface is the interface of the first application, and updating the interface content according to the second screen projection data includes: The display device is controlled to display the interface content included in the second interface to replace the interface content included in the first interface.

11. The method according to claim 8, characterized in that, The first interface includes an icon of a first application, the second interface supports a landscape layout, the first operation is clicking the icon of the first application, the second interface is the interface of the first application, and updating the interface content according to the second screen projection data includes: Control the display device to display the interface content included in the second interface in full screen.

12. The method according to claim 7, characterized in that, The first interface includes a first control, the first control corresponding to a first business function, and the method further includes: In response to the user clicking the first control, a second instruction message is sent to the first device, wherein the second instruction message is used to instruct the user to click the first control; Receive service data for the first service sent by the first device.

13. The method according to any one of claims 7 to 12, characterized in that, The method further includes: Receive third-party screen projection data sent by a third device; Based on the first projection data and the third projection data, the display device is controlled to simultaneously display the interface content included in the first interface and the third interface, and the interface content included in the M interfaces is no longer displayed, wherein the third interface is the interface currently displayed by the third device.

14. A device, characterized in that, It includes one or more processors; one or more memories; said one or more memories storing one or more programs, said one or more programs including instructions that, when executed by said one or more processors, cause the method of any one of claims 1 to 6 or the method of any one of claims 7 to 13 to be performed.

15. A readable storage medium, characterized in that, The readable storage medium stores instructions that, when executed on the device, cause the method of any one of claims 1 to 6 or any one of claims 7 to 13 to be performed.

16. A chip, characterized in that, The chip includes a processor and a communication interface, the communication interface being used to receive signals and transmit the signals to the processor, the processor processing the signals such that the method of any one of claims 1 to 6 or the method of any one of claims 7 to 13 is executed.

17. A program product, characterized in that, When the program product is run on the device, the device performs the method as described in any one of claims 1 to 6 or the method as described in any one of claims 7 to 13.

Images