Multi-device collaboration method and electronic device

Abstract

The present application discloses a multi-device collaboration method and an electronic device. In the method, a source device and a target device may determine, by means of the type of a first application, a manner of collaboratively displaying a first interface of the first application, for example, when the first application is of a first type, the first interface of the first application is collaboratively displayed in a screen mirroring manner; or when the first application is of a second type, the first interface of the first application is collaboratively displayed in a continuation screen manner. That is, for an interface that can support collaboration in a continuation manner, the collaboration in a continuation manner is selected, and for an interface that cannot support the collaboration in a continuation manner, collaboration in a screen mirroring manner is selected, thereby reducing encoding and decoding operations that need to be performed and data that needs to be transmitted when a source device and a target device collaboratively display an interface. In a multi-device or multi-service scenario, more devices or more services can be collaborated at the same time, thereby improving collaboration efficiency.

Description

A multi-device collaboration method and electronic device

[0001] Cross-reference to related applications

[0002] This application claims priority to Chinese Patent Application No. 202410808492.0, filed on June 20, 2024, entitled "A Multi-Device Collaboration Method and Electronic Device", the entire contents of which are incorporated herein by reference; and to Chinese Patent Application No. 202411173039.3, filed on August 26, 2024, entitled "A Multi-Device Collaboration Method and Electronic Device", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of electronic technology, and in particular to a multi-device collaboration method and electronic device. Background Technology

[0004] With the increasing number of personal distributed devices, the demand for multi-device collaboration is also growing stronger. Multi-device collaboration technology refers to the ability for different operating systems and devices (such as smartphones, tablets, computers, and televisions) to be compatible across operating systems and to transmit digital multimedia content (such as high-definition video, audio, and images) via communication networks. Multi-device collaboration technology enables the collaborative display of content on different devices, enriching users' multimedia experiences.

[0005] Currently, in terms of multi-device collaboration functionality, there is an interface sharing mode, where different devices collaborate by sharing the system interface or application interface. Multi-device collaboration solutions supporting interface sharing often use screen mirroring. However, screen mirroring collaboration requires significant encoding and decoding operations and the transmission of substantial data, placing high demands on system computing power and communication capabilities. Therefore, in multi-device or multi-service scenarios, it cannot achieve simultaneous collaboration for more devices or more services, resulting in poor collaboration efficiency. Summary of the Invention

[0006] This application provides a multi-device collaboration method and electronic device to improve the efficiency of multi-device collaboration.

[0007] In a first aspect, embodiments of this application provide a multi-device collaboration method applicable to a multi-device collaboration system, the multi-device collaboration system including a source device and a target device. The method includes: when the source device and the target device have established a collaborative connection, the source device and / or the target device receives a first trigger event; wherein the first trigger event is used to trigger the source device and the target device to collaboratively display a first interface, the first interface being the interface of a first application of the source device; in response to the first trigger event: when the type of the first application is a first type, the source device and the target device collaboratively display the first interface via screen projection; or, when the type of the first application is a second type, the source device and the target device collaboratively display the first interface via a sequential method.

[0008] In this embodiment, the source device and the target device can determine the method for collaboratively displaying the first interface of the first application based on the type of the first application. For example, if the first application is of type 1, the first interface of the first application is collaboratively displayed via screen projection; or, if the first application is of type 2, the first interface of the first application is collaboratively displayed via continuous screen display. In other words, continuous screen display is selected for interfaces that support continuous screen display, while screen projection is selected only for interfaces that do not support continuous screen display. This reduces the encoding and decoding operations and data transmission required for collaborative display between the source and target devices. In multi-device or multi-service scenarios, this enables more devices or more services to collaborate simultaneously, improving collaboration efficiency.

[0009] In one possible implementation, the first type is used to indicate that the first application is a system application, and the second type is used to indicate that the first application is a non-system application.

[0010] In this implementation, since the screen mirroring method has no restrictions on the interface to be displayed collaboratively, any frame of the interface from the source device can be mirrored to the target device. The continuation method, on the other hand, has restrictions on the interface to be displayed collaboratively. The interface of non-system applications (such as video applications and music applications that are not related to the system) from the source device can be continuated to the target device, while the interface of the system interface and system applications (such as system-related settings applications) from the source device cannot be continuated to the target device. Therefore, the source device and the target device can determine the appropriate collaborative display method according to the type of interface to be displayed collaboratively, making the collaborative display method more flexible and improving the collaborative efficiency.

[0011] In one possible implementation, the source device and the target device collaboratively display the first interface via screen projection. The method includes: when the communication method between the source device and the target device is short-range communication, the source device and the target device collaboratively display the first interface via same-source screen projection; or, when the communication method between the source device and the target device is non-short-range communication, the source device and the target device collaboratively display the first interface via different-source screen projection.

[0012] In this implementation, since same-source projection has high latency requirements while different-source projection has low latency requirements, when the source and target devices collaboratively display an interface via projection, they can choose between same-source or different-source projection based on their communication method. For example, for short-range communication, same-source projection is chosen to reduce synchronization latency and improve user experience; for non-short-range communication, different-source projection is chosen to improve interface lag and enhance user experience.

[0013] In one possible implementation, before the source device and the target device collaboratively display the first interface in a sequential manner, the method further includes: the target device determining whether the first application supports data migration; if the first application supports data migration and the target device does not have the first application installed, the target device installs and launches the first application.

[0014] In this implementation, when the source device and target device collaboratively display a first interface via a sequential method, and the first interface is the interface of a non-system application on the source device, both the source and target devices need to determine whether the non-system application on the source device supports data migration. If the non-system application supports data migration, the source device sends the data of the non-system application to the target device. The target device then installs and launches the non-system application based on the data to sequentially display the first interface, ensuring user data security and improving collaborative security. Furthermore, the target device can silently install the non-system application in the background without notifying the user, improving the user experience.

[0015] In one possible implementation, the method further includes: the target device detecting that the first display window has no focus; wherein the first display window is the window through which the target device displays the first interface; the target device performing a first operation; wherein the first operation includes one or more of the following: switching the size of the first display window from a first size to a second size, the first size being larger than the second size; switching the resolution of the interface in the first display window from a first resolution to a second resolution, the first resolution being larger than the second resolution; pausing the collaborative display of the interface in the first display window with the source device.

[0016] In this implementation, if the first display window has focus, it means that the user may not be paying attention to the first display window at present. In this case, the target device can perform operations such as reducing the size and resolution of the first display window and / or pausing the update of the interface in the first display window (i.e., synchronizing with the source device). This will not significantly reduce the user experience and will also reduce resource consumption.

[0017] In one possible implementation, after the target device performs the first operation, the method further includes: the target device detecting that there is focus on the first display window; the target device performing a second operation; wherein, when the first operation includes switching the size of the first display window from the first size to the second size, the second operation includes switching the size of the first display window from the second size to the first size; when the first operation includes switching the resolution of the interface in the first display window from the first resolution to the second resolution, the second operation includes switching the resolution of the interface in the first display window from the second resolution to the first resolution; when the first operation includes pausing the collaborative display of the interface in the first display window with the source device, the second operation includes resuming the collaborative display of the interface in the first display window with the source device.

[0018] In this implementation, if focus reappears on the first display window, it indicates that the user may be refocusing on the first display window. The target device can then perform operations such as restoring the size and resolution of the first display window and / or updating the interface in the first display window to ensure user experience.

[0019] In one possible implementation, the method further includes: the source device and / or the target device receiving a second trigger event; wherein the second trigger event is used to trigger the source device and the target device to stop collaboratively displaying the first interface and to trigger the target device to continue running the first application; in response to the second trigger event, the source device determines whether the first application supports data migration; when the first application supports data migration, the source device sends data required for continuing to run the first application on the target device to the target device; the target device receives the data from the source device and runs the first application according to the data.

[0020] In this implementation, after the source and target devices enter a collaborative state, they can switch from the collaborative state to a continuous state in response to corresponding trigger events. For example, if the source device's battery power is insufficient to support the collaborative display of a non-system application interface between the source and target devices, the source device can disconnect from the target device and migrate the data of the non-system application from the source device to the target device, allowing the target device to continue running the non-system application. This satisfies more user needs in various scenarios and improves the user experience.

[0021] In one possible implementation, the method further includes: the source device and / or the target device receiving a third trigger event, the third trigger event being used to trigger the source device and the target device to collaboratively display a second interface, the second interface being the interface of the first application; in response to the third trigger event: when the type of the first application is the first type, the source device and the target device collaboratively display the second interface via screen mirroring and synchronize the data of the first application; or, when the type of the first application is the second type, the source device and the target device collaboratively display the second interface via a sequential method and synchronize the data of the first application.

[0022] In this implementation, after the source device and the target device enter the connection state, the source device and the target device can respond to the corresponding trigger events. For example, if the power of the source device is sufficient to support the source device and the target device to collaboratively display the interface of a non-system application, the source device can re-establish a collaborative connection with the target device and collaboratively display the interface of the non-system application according to the collaborative connection. Furthermore, the source device and the target device synchronize the data of the non-system application, thereby meeting more user needs in various scenarios and improving the user experience.

[0023] Secondly, embodiments of this application also provide a multi-device collaboration method. This method is applicable to a source device or a component (e.g., unit / module, circuit, or chip) in a multi-device collaboration system. Taking the source device as an example, the method includes: when a collaborative connection has been established between the source device and the target device, the source device receives a first trigger event; wherein the first trigger event is used to trigger the source device and the target device to collaboratively display a first interface, the first interface being the interface of a first application of the source device; in response to the first trigger event: when the type of the first application is a first type, the source device collaboratively displays the first interface to the target device via screen projection; or, when the type of the first application is a second type, the source device collaboratively displays the first interface to the target device via a sequential method.

[0024] In one possible implementation, the first type is used to indicate that the first application is a system application, and the second type is used to indicate that the first application is a non-system application.

[0025] In one possible implementation, the source device collaboratively displays the first interface to the target device via screen projection. The method includes: when the communication method between the source device and the target device is short-range communication, the source device collaboratively displays the first interface to the target device via same-source screen projection; or, when the communication method between the source device and the target device is non-short-range communication, the source device collaboratively displays the first interface to the target device via different-source screen projection.

[0026] In one possible implementation, the method further includes: the source device receiving a second trigger event; wherein the second trigger event is used to trigger the source device and the target device to stop collaboratively displaying the first interface and to trigger the target device to continue running the first application; in response to the second trigger event, the source device determines whether the first application supports data migration; when the first application supports data migration, the source device sends the data required for the first application to continue running on the target device to the target device.

[0027] In one possible implementation, the method further includes: the source device receiving a third trigger event, the third trigger event being used to trigger the source device and the target device to collaboratively display a second interface, the second interface being the interface of the first application; in response to the third trigger event: when the type of the first application is the first type, the source device collaboratively displays the second interface to the target device via screen mirroring, and receives data from the first application on the target device; or, when the type of the first application is the second type, the source device collaboratively displays the second interface to the target device via a sequential display, and receives data from the first application on the target device.

[0028] Thirdly, embodiments of this application also provide a multi-device collaboration method. This method is applicable to a target device or a component (e.g., unit / module, circuit, or chip) in a multi-device collaboration system. Taking the target device as an example, the method includes: when a source device and a target device have established a collaborative connection, the target device receives a first trigger event; wherein the first trigger event is used to trigger the source device and the target device to collaboratively display a first interface, the first interface being the interface of a first application of the source device; in response to the first trigger event: when the type of the first application is a first type, the target device sends a first request to the source device, the first request being used to request the source device to collaboratively display the first interface on the target device via screen projection; or, when the type of the first application is a second type, the target device sends a second request to the source device, the second request being used to request the source device to collaboratively display the first interface on the target device via a continuation method.

[0029] In one possible implementation, the first type is used to indicate that the first application is a system application, and the second type is used to indicate that the first application is a non-system application.

[0030] In one possible implementation, when the communication method between the source device and the target device is short-range communication, the first request is used to request the source device to collaboratively display the first interface on the target device via same-source projection. When the communication method between the source device and the target device is non-short-range communication, the first request is used to request the source device to collaboratively display the first interface on the target device via different-source projection.

[0031] In one possible implementation, before the target device sends a second request to the source device, the method further includes: the target device determining whether the first application supports data migration; if the first application supports data migration and the target device does not have the first application installed, the target device installs and starts the first application.

[0032] In one possible implementation, the method further includes: the target device detecting that the first display window has no focus; wherein the first display window is the window through which the target device displays the first interface; the target device performing a first operation; wherein the first operation includes one or more of the following: switching the size of the first display window from a first size to a second size, the first size being larger than the second size; switching the resolution of the interface in the first display window from a first resolution to a second resolution, the first resolution being larger than the second resolution; pausing the collaborative display of the interface in the first display window with the source device.

[0033] In one possible implementation, after the target device performs the first operation, the method further includes: the target device detecting that there is focus on the first display window; the target device performing a second operation; wherein, when the first operation includes switching the size of the first display window from the first size to the second size, the second operation includes switching the size of the first display window from the second size to the first size; when the first operation includes switching the resolution of the interface in the first display window from the first resolution to the second resolution, the second operation includes switching the resolution of the interface in the first display window from the second resolution to the first resolution; when the first operation includes pausing the collaborative display of the interface in the first display window with the source device, the second operation includes resuming the collaborative display of the interface in the first display window with the source device.

[0034] In one possible implementation, the method further includes: the target device receiving a second trigger event; wherein the second trigger event is used to trigger the source device and the target device to stop collaboratively displaying the first interface and to trigger the target device to continue running the first application; in response to the second trigger event, the source device determines whether the first application supports data migration; when the first application supports data migration, the target device receives data from the source device required to continue running the first application on the target device, and runs the first application according to the data.

[0035] In one possible implementation, the method further includes: the target device receiving a third trigger event, the third trigger event being used to trigger the source device and the target device to collaboratively display a second interface, the second interface being the interface of the first application; in response to the third trigger event: when the type of the first application is the first type, the target device sends a third request and data of the first application to the source device, the third request being used to request the source device to collaboratively display the second interface to the target device via screen mirroring; or, when the type of the first application is the second type, the target device sends a fourth request and data of the first application to the source device, the third request being used to request the source device to collaboratively display the second interface to the target device via a continuation method.

[0036] Fourthly, embodiments of this application also provide an electronic device, the electronic device including a processor, a memory, and one or more programs; wherein the one or more programs are stored in the memory, and the one or more programs include instructions that, when executed by the processor, cause the electronic device to perform the method described in the first aspect or any possible design of the first aspect, or to perform the method described in the second aspect or any possible design of the second aspect, or to perform the method described in the third aspect or any possible design of the third aspect.

[0037] Fifthly, embodiments of this application also provide a computer-readable storage medium for storing a computer program that, when run on a computer, causes the computer to perform the method described in the first aspect or any possible design of the first aspect, or to perform the method described in the second aspect or any possible design of the second aspect, or to perform the method described in the third aspect or any possible design of the third aspect.

[0038] Sixthly, embodiments of this application also provide a computer program product, including a computer program that, when run on a computer, causes the computer to perform the method described in the first aspect or any possible design of the first aspect, or to perform the method described in the second aspect or any possible design of the second aspect, or to perform the method described in the third aspect or any possible design of the third aspect.

[0039] In a seventh aspect, embodiments of this application also provide a chip system, including a processor and an interface, wherein the processor is configured to call and execute instructions from the interface to cause the chip system to perform the method described in the first aspect or any possible design of the first aspect, or to perform the method described in the second aspect or any possible design of the second aspect, or to perform the method described in the third aspect or any possible design of the third aspect.

[0040] The beneficial effects of the second to seventh aspects and their possible designs can be referred to the description of the beneficial effects of the methods described in the first aspect and any possible design. Attached Figure Description

[0041] Figure 1 is a schematic diagram of a multi-device collaborative system provided in an embodiment of this application;

[0042] Figure 2 is a schematic diagram of the hardware structure of an electronic device 100 provided in an embodiment of this application;

[0043] Figure 3 is a schematic diagram of the software structure of an electronic device 100 provided in an embodiment of this application;

[0044] Figure 4 is a schematic diagram of multi-device collaboration provided in an embodiment of this application;

[0045] Figure 5 is a flowchart illustrating a multi-device collaboration method provided in an embodiment of this application;

[0046] Figure 6 is a schematic diagram of a source device and a target device establishing a cooperative connection according to an embodiment of this application;

[0047] Figure 7 is a schematic diagram of a source device and a target device collaboratively displaying an interface via screen projection, according to an embodiment of this application.

[0048] Figure 8 is a schematic diagram of a source device and a target device collaboratively displaying an interface in a hybrid manner according to an embodiment of this application;

[0049] Figure 9 is a schematic diagram of a source device and a target device collaboratively displaying via screen projection, according to an embodiment of this application.

[0050] Figure 10 is a schematic diagram of a source device and a target device collaboratively displaying their data through a projection method, a connection method, or a hybrid method, according to an embodiment of this application.

[0051] Figure 11 is a schematic diagram of another source device and target device collaboratively displayed through a projection method, a connection method, or a hybrid method according to an embodiment of this application;

[0052] Figure 12 is a schematic diagram illustrating how a first application in a source device is determined to support data migration, according to an embodiment of this application.

[0053] Figure 13 is a schematic diagram of another source device and target device collaboratively displaying through screen projection, connection or hybrid methods according to an embodiment of this application;

[0054] Figure 14 is a schematic diagram of another source device and target device co-displaying through a projection method, a connection method, or a hybrid method according to an embodiment of this application;

[0055] Figure 15 is a schematic diagram of another method of collaborative display between a source device and a target device, including a projection method, a connection method, or a hybrid method, provided in an embodiment of this application.

[0056] Figure 16 is a schematic diagram of the hardware structure of another electronic device provided in an embodiment of this application. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the embodiments of this application will be further described in detail below with reference to the accompanying drawings.

[0058] The technical solutions provided in the embodiments of this application can be applied to multi-device collaborative systems. For example, Figure 1 is a schematic diagram of a multi-device collaborative system provided in an embodiment of this application. As shown in Figure 1, the multi-device collaborative system may include a source device 10 and a target device 20. It should be understood that Figure 1 only illustrates a multi-device collaborative system for ease of understanding, but this should not constitute any limitation on this application. The multi-device collaborative system may also include a greater number of source devices and a greater number of target devices; the target devices interacting with different source devices may be the same target device or different target devices; the number of target devices interacting with different source devices may be the same or different. This application does not impose any limitations in this regard.

[0059] In this context, source device 10 refers to a device that actively shares the system's interface or application's interface; for example, source device 10 can be a mobile phone, tablet, laptop, etc. Target device 20 refers to a device that passively receives the interface shared by source device 10; for example, target device 20 can be a mobile phone, tablet, laptop, etc. For instance, when source device 10 and target device 20 share their interfaces via screen mirroring, source device 10 encodes the shared interface into H.264 format data and then sends the H.264 format data to target device 20. Target device 20 can then decode and display the H.264 format data.

[0060] Source device 10 and target device 20 can communicate with each other through a communication network. This communication network can be a local area network (LAN) or a wide area network (WAN) relayed through a relay device. When the communication network is a LAN, for example, it can be a short-range communication network such as a Wi-Fi hotspot network, Wi-Fi Direct network, Bluetooth network, Zigbee network, or Near Field Communication (NFC) network. When the communication network is a WAN, for example, it can be a 3G network, a 4G network, a 5G network, a future public land mobile network (PLMN), or the Internet.

[0061] The source device 10 and target device 20 can be electronic devices with receiving and display functions, such as mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) devices, virtual reality (VR) devices, laptops, drones, ultra-mobile personal computers (UMPCs), netbooks, personal digital assistants (PDAs), and smart TVs. This application embodiment does not impose special limitations on the specific forms of the source device 10 and target device 20. For example, the source device 10 can be a mobile phone, and the target device 20 can be a tablet computer. Furthermore, the target device 20 can be one of the aforementioned electronic devices with both receiving and display functions, or it can be an electronic device that only includes receiving functions, such as a set-top box. If the target device 20 is an electronic device that only includes receiving functions, then the target device 20 can be connected to an electronic device with display functions. For example, the target device 20 is a set-top box, which is connected to a television display.

[0062] For example, Figure 2 is a schematic diagram of the hardware structure of an electronic device 100 provided in an embodiment of this application.

[0063] As shown in Figure 2, the source device 10 or target device 20 in this embodiment can be an electronic device 100. The following detailed description uses electronic device 100 as an example. It is understood that the illustrated electronic device 100 is merely an example of the source device 10 or target device 20, and electronic device 100 may have more or fewer components than shown in the figure, may combine two or more components, or may have different component configurations. The various components shown in the figure can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing or application-specific integrated circuits.

[0064] In Figure 2, the electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180I, a touch sensor 180J, an ambient light sensor 180K, a bone conduction sensor 180L, etc.

[0065] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0066] Processor 110 may include one or more processing units, such as application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0067] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0068] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0069] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0070] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a limitation on the structure of the electronic device. In other embodiments of this application, the electronic device may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0071] The charging management module 140 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 receives charging input from the wired charger via a USB interface 130. In some wireless charging embodiments, the charging management module 140 receives wireless charging input via the wireless charging coil of the electronic device. While charging the battery 142, the charging management module 140 can also supply power to the electronic device via the power management module 141.

[0072] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be located in the same device.

[0073] The wireless communication function of electronic devices can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor, etc.

[0074] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0075] The mobile communication module 150 can provide solutions for wireless communication applications including 2G / 3G / 4G / 5G in electronic devices. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0076] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 170A, receiver 170B, etc.) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

[0077] The wireless communication module 160 can provide solutions for wireless communication applications in electronic devices, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0078] In some embodiments, antenna 1 of the electronic device is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling the electronic device to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies. The GNSS may include Global Positioning System (GPS), Global Navigation Satellite System (GLONASS), BeiDou Navigation Satellite System (BDS), Quasi-Zenith Satellite System (QZSS), and / or Satellite Based Augmentation Systems (SBAS).

[0079] Electronic devices implement display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0080] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD). The display panel can also be manufactured using organic light-emitting diodes (OLEDs), active-matrix organic light-emitting diodes (AMOLEDs), flexible light-emitting diodes (FLEDs), miniled, microled, micro-oled, quantum dot light-emitting diodes (QLEDs), etc. In some embodiments, the electronic device may include one or N displays 194, where N is a positive integer greater than 1.

[0081] Electronic devices can achieve shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0082] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and color. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0083] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device may include one or N cameras 193, where N is a positive integer greater than 1.

[0084] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when an electronic device is selecting a frequency, a DSP can perform a Fourier transform on the frequency energy.

[0085] Video codecs are used to compress or decompress digital video. Electronic devices can support one or more video codecs. This allows the electronic device to play or record video in various encoded formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0086] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0087] Internal memory 121 may include one or more random access memory (RAM) and one or more non-volatile memory (NVM).

[0088] Random access memory can include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM, for example, fifth generation DDR SDRAM is generally called DDR5 SDRAM), etc.

[0089] Non-volatile memory can include disk storage devices and flash memory.

[0090] Flash memory can be classified according to its operating principle, including NOR FLASH, NAND FLASH, 3D NAND FLASH, etc.; according to the level of the storage cell, including single-level cell (SLC), multi-level cell (MLC), triple-level cell (TLC), quad-level cell (QLC), etc.; and according to the storage specification, including universal flash storage (UFS) and embedded multimedia card (eMMC), etc.

[0091] The random access memory can be directly read and written by the processor 110. It can be used to store executable programs (such as machine instructions) of the operating system or other running programs, as well as user and application data.

[0092] Non-volatile memory can also store executable programs and user and application data, and can be pre-loaded into random access memory for direct reading and writing by the processor 110.

[0093] The external memory interface 120 can be used to connect to external non-volatile memory, thereby expanding the storage capacity of the electronic device. The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to perform data storage functions. For example, music, video, and other files can be stored in the external non-volatile memory.

[0094] Electronic devices can implement audio functions such as music playback and recording through audio modules 170, speakers 170A, receivers 170B, microphones 170C, headphone jacks 170D, and application processors.

[0095] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0096] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. The electronic device can receive button input and generate key signal inputs related to user settings and function control of the electronic device.

[0097] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can correspond to touch operations performed on different applications (such as taking photos, playing audio, etc.). Motor 191 can also correspond to different vibration feedback effects for touch operations performed on different areas of the display screen 194. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0098] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0099] The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 195 to achieve contact and separation with the electronic device. The electronic device can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 195 simultaneously. The multiple cards can be of the same or different types. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The electronic device interacts with the network through the SIM card to achieve functions such as calls and data communication. In some embodiments, the electronic device uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device and cannot be separated from it.

[0100] Furthermore, the software system of the aforementioned electronic device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. The following explanation uses a layered operating system as an example to illustrate the software system of the electronic device 100.

[0101] For example, Figure 3 is a schematic diagram of the software structure of an electronic device 100 provided in an embodiment of this application.

[0102] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the operating system is divided into four layers, from top to bottom: the application layer, the application framework layer, the runtime and system libraries, and the kernel layer.

[0103] The application layer can include a series of application packages.

[0104] As shown in Figure 3, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, SMS, and settings.

[0105] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0106] As shown in Figure 3, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0107] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0108] Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, phone books, etc.

[0109] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0110] The phone manager is used to provide communication functions for electronic device 100. For example, it manages call status (including connection and disconnection).

[0111] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0112] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of download completion or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0113] Runtime consists of core libraries and a virtual machine. Runtime is responsible for the scheduling and management of the operating system.

[0114] The core library consists of two parts: one part is the functionalities that the Java language needs to call, and the other part is the core library of the operating system.

[0115] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0116] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0117] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0118] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0119] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0120] A 2D graphics engine is a graphics engine for 2D drawing.

[0121] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0122] The above describes the electronic devices to which the embodiments of this application are applicable. The following describes the relevant technical solutions involved in the embodiments of this application.

[0123] Multi-device collaboration technology refers to the ability for different operating systems and devices (such as smartphones, tablets, computers, and televisions) to be compatible across operating systems and to transmit digital multimedia content (such as high-definition video, audio, and images) via communication networks. Multi-device collaboration technology enables the collaborative display of content on different devices, enriching users' multimedia experience.

[0124] From the perspective of multi-device collaboration, there exists an interface sharing mode, where different devices collaborate by sharing the system's interface or an application's interface. Currently, multi-device collaboration solutions that support interface sharing often use screen mirroring. For example, Figure 4 is a schematic diagram of multi-device collaboration provided in an embodiment of this application. As shown in Figure 4, in multi-device collaboration, the interface A displayed on screen A of device A (e.g., a mobile phone) is mirrored to the display window B of screen B of device B (e.g., a tablet computer). After device A mirrors the interface A displayed on screen A to device B, both device A and device B can control interface A. The control operations and results on device A can be synchronously displayed in the display window B of screen B of device B, and the control operations and results on device B can be synchronously displayed on screen A of device A.

[0125] However, since screen projection collaboration requires a lot of encoding and decoding operations and the transmission of a lot of data, that is, screen projection collaboration has high requirements for system computing power and communication capabilities. Therefore, in multi-device or multi-service scenarios, it is impossible to achieve simultaneous collaboration of more devices or more services, resulting in poor collaboration efficiency.

[0126] In view of this, embodiments of this application provide a multi-device collaboration method to solve the problem of poor collaboration efficiency.

[0127] In the embodiments of this application, "when," "if," and "if" all refer to the device taking corresponding actions under certain objective circumstances, and are not time-limited, nor do they require the device to perform a judgment action, nor do they imply any other limitations. Unless otherwise specified, "if" and "if" can be substituted, and "when" and "in the case of" can be substituted. "When" and "if" / "if" can be substituted.

[0128] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0129] In this application embodiment, the number of nouns, unless otherwise specified, refers to "singular nouns or plural nouns," that is, "one or more." "At least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, or B exists alone, where A and B can be singular or plural. The character " / " can indicate that the related objects before and after are in an "or" relationship. For example, A / B means: A or B. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c means: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0130] In this application, the ordinal numbers such as "first" and "second" are used to distinguish multiple objects, and are not used to limit the size, content, order, timing, priority, or importance of the multiple objects. For example, "first data" and "second data" refer to two different data, and do not indicate a difference in priority or importance between the two data. For a technical feature, the technical features within that technical feature are distinguished by "A," "B," "C," and "D," and there is no sequential or hierarchical order among the technical features described by "A," "B," "C," and "D." For example, operation A and operation B in this document are only used to distinguish different contents, and do not limit the sequential or hierarchical order, priority, or importance between operation A and operation B.

[0131] The solutions provided by the embodiments of this application are described in detail below with reference to the accompanying drawings. In the following description, the multi-device collaboration method provided by the embodiments of this application is used as an example applied to the multi-device collaboration system shown in Figure 1.

[0132] Please refer to Figure 5, which is a flowchart of a multi-device collaboration method provided in an embodiment of this application. The specific process of this multi-device collaboration method is described below.

[0133] S500 establishes a collaborative connection between the source device and the target device.

[0134] In this embodiment of the application, the establishment of a cooperative connection between the source device and the target device can be understood as the source device and the target device entering a cooperative state.

[0135] For example, Figure 6 is a schematic diagram of a source device and a target device establishing a cooperative connection according to an embodiment of this application.

[0136] As shown in Figure 6(1), the source device detects user-triggered operation A and, in response to operation A, starts the multi-device collaboration function. Operation A can be a user-triggered operation to start the multi-device collaboration function, such as a user-triggered click operation on the "Start" control 601.

[0137] As shown in Figure 6(2), after the source device starts the multi-device collaboration function, it searches for at least one candidate device through the short-range device discovery function and / or the non-short-range device discovery function.

[0138] In this context, a short-range device refers to a device that communicates with the source device over a short distance, while a non-short-range device (or long-range device) refers to a device that communicates with the source device over a long distance. Short-range communication can include Wi-Fi hotspot communication, Wi-Fi Direct communication, Bluetooth communication, Zigbee communication, or NFC communication, while non-short-range communication can be any other type of communication. This application does not limit this specific communication method. For example, after the source device activates the Bluetooth device discovery function, it can search for Bluetooth broadcast signals sent by other devices. Upon receiving a Bluetooth broadcast signal, it will consider the device that sent the signal as a candidate device. It can be understood that when the source device finds device a through both the short-range and non-short-range device discovery functions, the source device will only consider device a as one of the found candidate devices.

[0139] When the source device displays at least one candidate device found on the interface, the source device displays the identifier of each candidate device among the at least one candidate device, for example, the source device displays "Device a", "Device b", "Device c" and "Device d" on the interface.

[0140] The source device detects user-triggered operation B, and in response to operation B, the source device determines the target device from at least one candidate device. Operation B can be a user-triggered operation for determining the target device; for example, operation B can be a user-triggered click operation on the "Device d" control 602.

[0141] As shown in Figure 6(3), the source device can send a collaborative connection request to the target device. The target device receives the collaborative connection request from the source device and, in response to the collaborative connection request, displays a "Device e requests to establish a collaborative connection" pop-up window. The target device detects the user-triggered operation C and, in response to operation C, establishes a collaborative connection with the source device. Operation C can be a user-triggered operation used to determine the establishment of a collaborative connection; for example, operation C can be a user-triggered click operation on the "OK" control 603.

[0142] As shown in Figure 6(4), the system interface displayed on the screen of the source device is collaboratively displayed on the display window 1 of the screen of the target device. It can be understood that in Figure 6(4), the source device and the target device can achieve collaborative display through screen projection.

[0143] S501. When the source device and the target device have established a cooperative connection, the source device and / or the target device receive a first trigger event; wherein, the first trigger event is used to trigger the source device and the target device to collaboratively display a first interface, and the first interface is the interface of the first application of the source device.

[0144] In this embodiment of the application, the first triggering event may be user interaction, changes in network status, messages sent by other devices to the source device and / or the target device, etc., and this embodiment of the application does not limit it.

[0145] In some embodiments, the first triggering event may also be a "tap-to-pay" event, which may include, for example, the distance between the source device and the target device meeting the conditions for NFC communication between the source device and the target device. When a "tap-to-pay" event occurs, the NFC modules of the source device and the target device are brought close together, and the source device and the target device can transfer the data required for the source device and the target device to collaboratively display the first interface via NFC.

[0146] In some embodiments, the first triggering event may include one or more sub-triggering events. When the source device and the target device receive the first triggering event, it can be understood that the source device and the target device each receive different sub-triggering events of the first triggering event. The number of sub-triggering events received by the source device and the number of sub-triggering events received by the target device may be the same or different; this embodiment does not limit this. For example, the first triggering event may include sub-triggering event 1, sub-triggering event 2, and sub-triggering event 3. When the source device and the target device receive the first triggering event, it can be understood that the source device receives sub-triggering events 1 and 2, and the target device receives sub-triggering event 3.

[0147] The first application can be a system application or a non-system application. System applications can be understood as applications related to the system (or operating system) or applications with device management functions (i.e., device management applications), such as settings applications and file management applications. Non-system applications can be understood as applications related to the system or applications without device management functions (i.e., non-device management applications), such as video applications (e.g., Huawei Video, iQiyi, Bilibili, etc.) and music applications (e.g., Huawei Music, QQ Music, NetEase Cloud Music, etc.). The distinction between system and non-system applications can be determined through a preset whitelist, a configuration file, or automatically based on the application's functionality; this application does not limit this distinction.

[0148] S502, In response to the first triggering event: When the type of the first application is the first type, the source device and the target device collaboratively display the first interface through screen mirroring; or, when the type of the first application is the second type, the source device and the target device collaboratively display the first interface through a connection method.

[0149] To facilitate understanding of the solutions in the embodiments of this application, several collaborative display methods involved in the embodiments of this application will be introduced below.

[0150] (1) Screen mirroring method:

[0151] The source device and the target device collaborate to display their interfaces via screen mirroring. This can be understood as the interface displayed on the source device's screen (e.g., a screen visible to the user, or a screen invisible to the user) being mirrored onto a display window (e.g., a mirroring window) on the target device's screen.

[0152] When a source device and a target device collaboratively display an interface via screen mirroring, the source device can send the encoded data of the interface to the target device. The target device then decodes the encoded data into the interface and displays it on the mirrored window. Alternatively, the source device can send the Uniform Resource Locator (URL) corresponding to the multimedia resources (such as video and audio) in the interface to the target device. The target device can then directly display the interface on the mirrored window based on the URL corresponding to the multimedia resource. For example, it can retrieve the video from the server based on the URL corresponding to the video and directly display the video playback interface on the mirrored window.

[0153] Generally, there are no restrictions on the interfaces that can be displayed collaboratively via screen mirroring. When the source and target devices collaborate to display an interface via screen mirroring, any frame of the interface displayed on the source device's screen can be mirrored to the target device. In other words, when the source and target devices collaborate to display an interface via screen mirroring, the interface can be any frame of the interface displayed on the source device's screen.

[0154] For example, when the source device and the target device collaboratively display their interfaces via screen mirroring, the interface can be the source device's system interface, such as the desktop, lock screen, negative one screen, multitasking, control center, etc.; it can also be the interface of the source device's system applications, such as the settings application, file management application; or it can be the interface of the source device's non-system applications, such as video applications (e.g., Huawei Video, iQiyi, Bilibili), music applications (e.g., Huawei Music, QQ Music, NetEase Cloud Music), etc.

[0155] It is understood that for the specific descriptions of "system applications" and "non-system applications" involved in S502, please refer to the specific descriptions of the relevant technical features in S501, which will not be repeated here.

[0156] Furthermore, based on different screen mirroring needs, screen mirroring methods can be divided into same-source screen mirroring and different-source screen mirroring. These will be introduced below.

[0157] Same-source casting refers to mirroring the interface displayed on the source device's main screen (i.e., the screen visible to the user on the source device) to the target device. In other words, the target device's casting window mirrors the content of the source device's main screen. In same-source casting scenarios, the interface displayed in the target device's casting window is identical to the interface displayed on the source device's main screen. The user sees the content corresponding to the interface displayed in the target device's casting window from the source device's main screen, thus perceiving a noticeable delay. Therefore, same-source casting has high latency requirements. In other words, when the source and target devices collaboratively display their interfaces using same-source casting, it can be understood as the interface displayed on the source device's main screen being cast to the target device's casting window.

[0158] In some embodiments, the same-source screen mirroring method can also be referred to as mirroring, full-share screen mirroring, wireless display, etc.

[0159] Distributed screen mirroring refers to mirroring the interface displayed on the virtual screen of the source device (i.e., the screen invisible to the user on the source device) to the target device. In other words, the target device's mirrored window only mirrors the content of the source device's virtual screen, not the content of the source device's main screen. In distributed screen mirroring scenarios, the interface displayed in the target device's mirrored window differs from the interface displayed on the source device's main screen. The user cannot see the content corresponding to the interface displayed in the target device's mirrored window from the source device's main screen, thus the user will not perceive any latency in the target device's mirrored window. Therefore, distributed screen mirroring has low latency requirements. In essence, when the source and target devices collaboratively display their interfaces via distributed screen mirroring, it can be understood as the interface displayed on the source device's virtual screen being mirrored to the target device's mirrored window.

[0160] In some embodiments, heterogeneous projection methods may also be referred to as non-mirror projection, application projection, network projection, online projection, digital living network alliance (DLAN) projection, etc.

[0161] For example, Figure 7 is a schematic diagram of a source device and a target device collaboratively displaying an interface through screen projection, according to an embodiment of this application.

[0162] As shown in Figure 7(1), the source device and the target device collaboratively display their interfaces through a shared-source projection method. The interface displayed in the projection window of the target device is the same as the interface displayed on the main screen visible to the user on the source device. In other words, the content seen by the user on the main screen of the source device is consistent with the content seen by the user on the projection window of the target device. Therefore, the user's control operations and results on the main screen of the source device can be synchronously displayed in the projection window of the target device, and the user's control operations and results on the projection window of the target device can be synchronously displayed in the main screen of the source device.

[0163] As shown in Figure 7(2), the source device and the target device collaboratively display their interfaces through a heterogeneous projection method. The interface displayed in the projection window of the target device is the same as the interface displayed on the virtual screen that is invisible to the user on the source device. In other words, the content seen by the user on the main screen of the source device is inconsistent with the content seen by the user on the projection window of the target device. Therefore, the user's control operations and results on the main screen of the source device cannot be synchronously displayed in the projection window of the target device, and the user's control operations and results on the projection window of the target device cannot be synchronously displayed in the main screen of the source device.

[0164] (2) Connection method:

[0165] The source device and the target device coordinate to display the interface in a continuous manner. This can be understood as the interface displayed on the screen of the source device being continuously displayed in a display window (e.g., a continuous window) on the screen of the target device.

[0166] When the source device and the target device collaboratively display an interface via a connection method, the source device can send the data of the application corresponding to the interface to the target device. The target device can then launch the application on the target device based on the received application data (if the application is not installed on the target device, it needs to be installed first) to display the interface of the application on the target device.

[0167] In other words, the prerequisite for collaborative display via the connection method is that both the source and target devices have the same application installed. This allows the target device to launch the application and seamlessly connect the application's interface from the source device's screen to the target device. However, the target device often cannot install the source device's system applications (such as settings applications specifically for configuring the source device's system); it can only install non-system applications. Therefore, compared to the aforementioned screen mirroring method that has no restrictions on the collaborative display interface, the connection method restricts the collaborative display interface. When the source and target devices collaboratively display interfaces via the connection method, the interface of a non-system application displayed on the source device's screen can be seamlessly connected to the target device, but the system interface and system application interface displayed on the source device's screen cannot be seamlessly connected to the target device. That is, when the source and target devices collaboratively display interfaces via the connection method, the interface can be the interface of a non-system application displayed on the source device's screen.

[0168] For example, when the source device and the target device collaboratively display an interface through a connection method, the interface can only be the interface of a non-system application of the source device, such as the interface of a video application (e.g., Huawei Video, iQiyi, Bilibili), a music application (e.g., Huawei Music, QQ Music, NetEase Cloud Music), and so on.

[0169] Furthermore, based on different connection requirements, connection methods can be divided into synchronous connection methods and asynchronous connection methods. These will be introduced below.

[0170] Synchronous connection refers to the process of seamlessly connecting the interface displayed on the source device's screen to the target device, ensuring that the interface displayed in the target device's connection window is identical to that displayed on the source device's screen. In other words, in a synchronous connection scenario, after the target device launches the corresponding application, the interface of that application displayed in the target device's connection window is the same as the interface displayed on the source device's screen. Therefore, the content seen by the user on the source device's screen is consistent with the content seen by the user in the target device's connection window. User control operations and results on the source device's screen can be synchronously displayed in the target device's connection window, and vice versa.

[0171] Asynchronous connection refers to a situation where the interface displayed on the source device's screen is seamlessly connected to the target device's screen, resulting in a different interface displayed in the target device's connection window compared to the source device's screen. In other words, in an asynchronous connection scenario, after the target device launches the corresponding application, the interface displayed in the target device's connection window is not the same as the application's interface displayed on the source device's screen. For example, the target device's connection window might display the application's initial interface, while the source device's screen displays a different interface. Therefore, the content seen by the user on the source device's screen is inconsistent with the content seen in the target device's connection window. User controls and their results on the source device's screen cannot be synchronously displayed in the target device's connection window, and vice versa.

[0172] It's understandable that the source and target devices collaboratively display their interfaces via a connection method. Compared to screen mirroring, this method reduces the encoding / decoding operations and data transmission required by screen mirroring. Therefore, in multi-device or multi-service scenarios, it enables more devices or services to collaborate simultaneously, improving collaboration efficiency. However, the connection method has limitations on the interfaces that can be collaboratively displayed; it's only suitable for displaying non-system application interfaces from the source device's screen to the target device. In contrast, screen mirroring has no such limitations and can be used to mirror any frame of the interface from the source device's screen to the target device, making it more widely applicable.

[0173] (3) Mixing method:

[0174] The source device and the target device collaborate to display the interface in a hybrid manner. This can be understood as the interface displayed on the screen of the source device being mixed and displayed in a display window (e.g., a hybrid window) on the screen of the target device.

[0175] The "mixed mode" can be understood as a combination of screen mirroring and continuous display. "Mixed display" can be understood as a combination of screen mirroring and continuous display. For example, interface elements can be text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, videos, audio, etc. Screen mirroring involves projecting all interface elements from the source device's screen to the target device's screen mirroring window. "Continuous display" involves continuously displaying all interface elements from the source device's screen to the target device's continuous display window. The "mixed mode," which combines screen mirroring and continuous display, involves projecting some interface elements (e.g., screen mirroring elements) from the source device's screen to the target device's mixed window, while other interface elements (e.g., continuous display elements) are continuously displayed in the target device's mixed window.

[0176] In other words, the interface displayed on the source device's screen is mixed and displayed in the mixed window of the target device's screen. This can be understood as the screen-casting elements in the interface displayed on the source device's screen being cast and displayed in the mixed window of the target device's screen, and subsequent elements being displayed in the mixed window of the target device's screen.

[0177] When the source and target devices collaboratively display an interface in a hybrid manner, for a projected element within the interface, the source device can send the encoded data of the projected element to the target device. The target device can then decode the encoded data into the projected element and display it on the hybrid window. Alternatively, the source device can send the URL corresponding to the projected element to the target device, allowing the target device to retrieve the projected element from the server using the URL and directly display it on the hybrid window. For subsequent elements within the interface, the source device can send data from the application corresponding to that interface to the target device. The target device can then launch the application on its own device (if the application is not installed on the target device, it needs to be installed first) based on the received application data and display the corresponding element from the application's interface on the target device on the hybrid window.

[0178] To reduce the required encoding / decoding operations and data transmission, and to enable simultaneous collaboration among more devices or services in multi-device or multi-service scenarios, thereby improving collaboration efficiency, the interface elements can be categorized into projection elements (for screen mirroring) and successor elements (for continuous display) based on the required encoding / decoding operations and data transmission. For example, interface elements with fewer encoding / decoding operations and data transmissions are projection elements, while those with more are successor elements.

[0179] For example, Figure 8 is a schematic diagram of a source device and a target device collaboratively displaying an interface in a hybrid manner according to an embodiment of this application.

[0180] As shown in Figure 8, the screen of the source device displays the playback interface of a video application, which includes three interface elements: "Video," "Button," and "Text Box." Since the "Video" component involves more encoding / decoding operations and data transmission, while the "Button" and "Text Box" involve less, the "Video" component in the playback interface of the video application on the source device can be continuously displayed in the mixed window of the target device, while the "Button" and "Text Box" components in the playback interface of the video application on the source device can be projected onto the mixed window of the target device.

[0181] Since screen mirroring has no restrictions on the interface that can be displayed collaboratively, while connection methods do, the hybrid method, which combines both screen mirroring and connection methods, also restricts the interface that can be displayed collaboratively. When the source and target devices collaboratively display an interface using a hybrid method, the non-system application interface displayed on the source device's screen can be displayed on the target device, but neither the system interface nor the system application interface displayed on the source device's screen can be displayed on the target device. In other words, when the source and target devices collaboratively display an interface using a hybrid method, the interface can be the non-system application interface displayed on the source device's screen.

[0182] For example, when the source device and the target device collaboratively display an interface in a hybrid manner, the interface can only be the interface of a non-system application of the source device, such as the interface of a video application (e.g., Huawei Video, iQiyi, Bilibili), a music application (e.g., Huawei Music, QQ Music, NetEase Cloud Music), and so on.

[0183] It's understandable that source and target devices collaboratively display their interfaces through a hybrid method. Compared to screen mirroring, this reduces the encoding / decoding operations and data transmission required by screen mirroring. Therefore, in multi-device or multi-service scenarios, it enables more devices or services to collaborate simultaneously, improving collaboration efficiency. However, the hybrid method has limitations on the interfaces that can be collaboratively displayed; it's only suitable for mixing and displaying non-system application interfaces from the source device's screen onto the target device. In contrast, screen mirroring has no such limitations and can be used to mirror any frame of the interface from the source device's screen onto the target device, making it more widely applicable.

[0184] The above describes three methods for collaborative display between source and target devices: screen mirroring, continuous display, and hybrid display. In practical implementation, screen mirroring has no restrictions on the interface to be displayed; for example, any frame of the interface displayed on the source device's screen can be mirrored to the target device. Continuous display has restrictions on the interface; for example, the interface of a non-system application displayed on the source device's screen can be continuously displayed on the target device, but the interface of the system or system applications displayed on the source device's screen cannot be continuously displayed on the target device. Hybrid display also has restrictions; for example, the interface of a non-system application displayed on the source device's screen can be mixed and displayed on the target device, but the interface of the system or system applications displayed on the source device's screen cannot be mixed and displayed on the target device. Therefore, when source and target devices collaboratively display the first interface of a first application, they can determine which collaborative display methods to use based on the type of the first application (e.g., the first type is a system application, and the second type is a non-system application) to improve the efficiency of collaborative display. These will be discussed below.

[0185] For example, when the first application is of type 1, meaning it's a system application (such as a settings application), the source device and the target device can collaboratively display the first interface via screen mirroring. In other words, when the first interface is a system application interface, since screen mirroring is suitable for system application interfaces while connection and hybrid methods are not, the source device and the target device will only use screen mirroring to collaboratively display the first interface.

[0186] For example, when the first application is of type two, meaning the first interface is a non-system application (e.g., music applications such as Huawei Music, QQ Music, NetEase Cloud Music, etc., or video applications such as Huawei Video, iQiyi, Bilibili, etc.), the source device and the target device can determine to collaboratively display the first interface via screen mirroring, or they can determine to collaboratively display the first interface via a sequential method, or they can also collaboratively display the first interface via a hybrid method. In other words, when the first interface is a non-system application interface, since screen mirroring, sequential methods, and hybrid methods are all applicable to non-system application interfaces, the source device and the target device can use screen mirroring, sequential methods, or hybrid methods to collaboratively display the first interface.

[0187] Furthermore, since screen mirroring methods can be divided into same-source and different-source mirroring, same-source mirroring has higher latency requirements, while different-source mirroring has lower latency requirements. Therefore, when the source device and the target device collaboratively display the first interface via screen mirroring, the source and target devices can determine whether to use same-source or different-source mirroring to collaboratively display the first interface based on their communication method (e.g., short-range or non-short-range communication) to meet latency requirements. This will be explained below.

[0188] It is understood that for the specific descriptions of "short-range communication" and "non-short-range communication" involved in S502, please refer to the specific descriptions of the relevant technical features in S500, which will not be repeated here.

[0189] For example, when the communication method between the source device and the target device is short-range communication (such as Bluetooth or NFC), the source and target devices can determine that the screen projection method is the same-source projection method. In other words, because the same-source projection method has high latency requirements, when the source and target devices communicate at short distances, they can collaboratively display the first interface through the same-source projection method, thereby reducing synchronization latency and improving the user experience.

[0190] When the communication method between the source device and the target device is non-short-range communication (i.e., communication other than short-range communication), the source device and the target device can determine that the screen projection method is heterogeneous projection. In other words, because heterogeneous projection has low latency requirements, when the source device and the target device are not communicating over short distances, they can collaboratively display the first interface through heterogeneous projection, thereby improving interface lag and enhancing the user experience.

[0191] To facilitate understanding of the embodiments of this application, the following description, in conjunction with the accompanying drawings, illustrates how the source device and the target device collaboratively display the first interface through screen projection, connection, or a hybrid method.

[0192] In scenario one, the first application is a system application (i.e., the first interface is the interface of the system application), and the source device and the target device collaboratively display the first interface through screen mirroring.

[0193] Figure 9 is a schematic diagram of a source device and a target device collaboratively displaying via screen projection, according to an embodiment of this application.

[0194] As shown in Figure 6 above, when the source device and the target device establish a collaborative connection, the source device detects user-triggered operations A and B, and the target device detects user-triggered operation C. After the collaborative connection is established, the source device's screen displays its system interface, which includes application icons such as those for email, SMS, gallery, video, phone, settings, camera, and calendar applications. When the source and target devices collaboratively display the source device's system interface via screen mirroring, the source device can send encoded data of its system interface to the target device. The target device can then decode this encoded data and display the source device's system interface in display window 1.

[0195] As shown in Figure 9, after the source device and target device collaboratively display the source device's system interface via screen mirroring, the source device detects user-triggered operation D and, in response to operation D, launches the settings application. Operation D can be a user-triggered action to launch the settings application; for example, operation D could be a user-triggered click on the settings application's icon control 901. After the source device launches the settings application, the interface displayed on the source device's screen needs to change from the source device's system interface to the initial interface of the source device's settings application. The interface displayed in the target device's display window 1 also needs to change from the source device's system interface to the initial interface of the source device's settings application; that is, the source device and target device need to collaboratively display the initial interface of the source device's settings application.

[0196] Since the source device's settings application is a system application, and both the connection method and the hybrid method have limitations on the interface for collaborative display, they are only applicable to the non-system application interface of the source device, not to the system interface or system application interface of the source device. The screen projection method has limitations on the interface for collaborative display, applicable to any frame of the source device's interface. Therefore, in order to achieve collaborative display of the initial interface of the source device's settings application, the source device and the target device can choose to collaboratively display the initial interface of the source device's settings application through the screen projection method.

[0197] When the source device and target device collaboratively display the initial interface of the source device's settings application via screen mirroring, the source device can send encoded data of the initial interface of its settings application to the target device. The target device can then decode this encoded data and display the initial interface of the source device's settings application in display window 1.

[0198] In this embodiment, the size of display window 1 can be equal to the screen size of the source device. In some embodiments, "equal to" may refer to the size of display window 1 as seen by the user's naked eye, which is exactly equal to the screen size of the source device; for example, the screen size of the source device is 6.8 inches, the size of display window 1 is also 6.8 inches, and the aspect ratio of display window 1 is the same as that of the source device's screen, such as 19:9. In other embodiments, "equal to" may refer to the aspect ratio of display window 1 being equal to that of the source device's screen, but the actual sizes may differ; for example, the aspect ratio of the source device's screen is 19:9, the aspect ratio of display window 1 is also 19:9, but the size of the source device's screen is 6.8 inches, while the size of display window 1 is 5.8 inches or 7.8 inches, etc.

[0199] In some embodiments, the content of the interface in display window 1 may be exactly the same as the content of the interface currently displayed on the screen of the source device; that is, all the content of the interface currently displayed on the screen of the source device is correspondingly presented on the interface in display window 1. In other implementations, the content of the interface in display window 1 may be partially the same as and partially different from the content of the interface currently displayed on the screen of the source device. For example, one or more controls near the bottom edge of the screen (e.g., the return to desktop control, the view multitasking control, the return to previous level control) in the content of the interface currently displayed on the screen of the source device may not be displayed in display window 1 of the target device, while all other content may be displayed in display window 1 of the target device.

[0200] Therefore, in the above implementation, after the source device and target device enter the collaborative state, since the projection method is applicable to any frame of the source device's interface, while the continuity method and the hybrid method are not applicable to the source device's system interface and system application interface, but only to the source device's non-system application interface, in order to achieve collaborative display between the source device and the target device, as shown in Figures 6 and 9 above, the source device's system interface can be projected onto the target device's screen display window 1. If the next frame of the source device's system interface is the source device's system application interface, the source device's system application interface can continue to be projected onto the target device's screen display window 1. In other words, when the source device and target device are performing collaborative display, they can select an appropriate collaborative display method (e.g., projection method, continuity method, hybrid method) based on whether the collaborative display interface (e.g., system interface, system application interface, non-system application interface) is suitable for a collaborative display method (e.g., projection method, continuity method, hybrid method) to improve collaborative efficiency.

[0201] In the specific implementation process, as shown in Figure 9 above, when the source device and the target device collaboratively display the first interface via screen projection, the source device can send first data to the target device. The target device can receive the first data from the source device and display the first interface in the first display window based on the first data. The first data can be data required for projecting the first interface onto the target device, such as the encoded data of the first interface or the URL corresponding to the multimedia resources in the first interface. That is, the first data is the projection data corresponding to the first interface, and the first display window is the projection window corresponding to the first interface.

[0202] In one possible implementation, after the first interface displayed on the source device's screen is projected onto the first display window of the target device's screen, to reduce and conserve data transmission bandwidth for collaborative display between the source and target devices, and to lower power consumption, the target device can determine whether the user is currently focused on the first display window based on whether there is focus on it. It can be understood that if there is no focus on the first display window, it indicates that the user may not be currently focused on it. In this case, the target device can perform operations such as reducing the size and resolution of the first display window and / or pausing the update of the interface in the first display window (i.e., synchronizing with the source device), which will not significantly degrade the user experience and can reduce resource consumption. Furthermore, if focus reappears on the first display window, it indicates that the user may be refocusing on it. In this case, the target device can perform operations such as restoring the size and resolution of the first display window and / or resuming the update of the interface in the first display window to ensure a good user experience.

[0203] Specifically, when the target device detects that there is no focus on the first display window, the target device can perform a first operation on the first display window.

[0204] The first operation applied to the first display window includes one or more of the following: switching the size of the first display window from a first size to a second size, wherein the first size is larger than the second size; switching the resolution of the interface in the first display window from a first resolution to a second resolution, wherein the first resolution is larger than the second resolution; and pausing the collaborative display of the first interface (i.e., the interface in the first display window) with the source device.

[0205] It is understandable that after pausing the collaborative display of the first interface with the source device, the interface displayed in the first display window can be the last frame of the display interface obtained by the target device from the source device before pausing; or, the interface displayed in the first display window can be the interface after processing the aforementioned last frame of the display interface, such as the interface after blurring or grayscale processing of the aforementioned last frame of the display interface; or, the interface displayed in the first display window can be preset content, such as a preset image, preset controls, or preset colors, etc.

[0206] After the target device performs a first operation on the first display window, when the target device detects that there is focus on the first display window, the target device can perform a second operation on the first display window.

[0207] The first operation applied to the first display window includes switching the size of the first display window from a first size to a second size, and the second operation applied to the first display window may include switching the size of the first display window from the second size to the first size; the first operation applied to the first display window includes switching the resolution of the interface in the first display window from a first resolution to a second resolution, and the second operation applied to the first display window includes switching the resolution of the interface in the first display window from the second resolution to the first resolution; the first operation applied to the first display window includes pausing the collaborative display of the first interface with the source device, and the second operation applied to the first display window may include resuming the collaborative display of the first interface with the source device.

[0208] It is understood that the ratio between the first size and the second size, and the ratio between the first resolution and the second resolution, can be pre-configured, or can be defined by a standard, or can be agreed upon in advance by the source device and the target device. This application embodiment does not limit this.

[0209] It can be understood that "having focus" as mentioned above can refer to the first display window being in a state where it has focus. For example, when the target device's mouse pointer hovers over the first display window, the first display window has focus. Another example is when the first display window is the window on which the last user interaction on the target device was performed. Yet another example is when the first display window's display hierarchy (e.g., Z-order) is at the top level, the first display window has focus.

[0210] Similarly, the aforementioned "no focus" can indicate that the first display window is in a state where it has no focus. For example, if the target device's mouse pointer is not hovering over the first display window, the first display window has no focus. Another example is if the first display window is not the window on which the last user interaction on the target device was performed. Yet another example is if the first display window's display hierarchy (e.g., Z-order) is not at the top level, the first display window has no focus.

[0211] It is understandable that the first display window may have corresponding identification information to indicate whether the window has focus, and the target device can determine whether there is focus on the first display window based on the identification information.

[0212] Therefore, in the above implementation, the target device can perform a first operation or a second operation on the first display window based on whether there is focus on the first display window. For example, when the first display window changes from having focus to not having focus, the target device can reduce the size and resolution of the first display window and / or pause the updating of the interface in the first display window; when the first display window changes from not having focus to having focus, the target device can restore the size and resolution of the first display window and / or resume the updating of the interface in the first display window. Thus, the size, resolution, and / or displayed interface content of the first display window can be adjusted as needed, ensuring a good user experience while reducing resource consumption.

[0213] Scenario 2: The first application is a non-system application (i.e., the first interface is the interface of a non-system application). In other words, the source device and the target device can collaboratively display the first interface through a connection method, or through a screen mirroring method, or through a hybrid method.

[0214] For example, Figure 10 is a schematic diagram of a source device and a target device co-displaying through a projection method, a connection method, or a hybrid method according to an embodiment of this application.

[0215] As shown in Figure 6 above, the source device's screen displays the source device's system interface, and the target device's display window 1 projects the source device's system interface. As shown in Figure 10, after the source device and target device collaboratively display the source device's system interface via projection, the source device detects user-triggered operation E and, in response to operation E, launches the video application. Operation E can be a user-triggered operation to launch the video application, such as a user-triggered click on the video application's icon control 1001. After the source device launches the video application, the interface displayed on the source device's screen changes from the source device's system interface to the initial interface of the source device's video application. The interface displayed in the target device's display window 1 also needs to change from the source device's system interface to the initial interface of the source device's video application; that is, the source device and target device need to collaboratively display the initial interface of the source device's video application.

[0216] Since the video application on the source device is a non-system application, and the casting, connection, and hybrid methods are all applicable to the non-system applications on the source device, the source device and the target device can choose to collaboratively display the initial interface of the source device's video application through any one of the casting, connection, and hybrid methods.

[0217] Furthermore, screen mirroring requires the most encoding / decoding operations and data transmission, followed by hybrid methods, while continuous screen mirroring requires the fewest. Also, hybrid and continuous methods require migrating data from the video application on the source device to the target device, meaning the video application on the source device needs to support data migration. Screen mirroring, however, does not require this migration. Therefore, when choosing how to collaboratively display the initial interface of the source device's video application, the source and target devices can consider two aspects: firstly, minimizing the required encoding / decoding operations and data transmission to enable simultaneous collaboration across multiple devices or services in multi-device or multi-service scenarios, thus improving efficiency; secondly, determining whether the video application on the source device supports data migration.

[0218] When the source device's video application does not support data migration, in order to collaboratively display the initial interface of the source device's video application, the source device and the target device can choose to collaboratively display the initial interface of the source device's video application via screen mirroring.

[0219] For example, when a source device and a target device collaboratively display the initial interface of a video application on the source device via screen mirroring, the source device can send the encoded data of the initial interface of its video application to the target device. The target device can then decode this encoded data and display the initial interface of the source device's video application in display window 1.

[0220] When the source device's video application supports data migration, in order to minimize the encoding and decoding operations and the amount of data to be transmitted, the source and target devices can choose to collaboratively display the initial interface of the source device's video application in a sequential or hybrid manner.

[0221] For example, when the source device and the target device collaboratively display the initial interface of the source device's video application in a sequential manner, the source device can send the data of its video application to the target device. The target device can then launch the video application on its own device based on the received data (if the target device does not have the video application installed, it needs to install it first), and display the initial interface of the source device's video application in display window 1.

[0222] For example, when the source device and target device collaboratively display the initial interface of the source device's video application in a hybrid manner, the initial interface of the video application includes three interface elements: "video," "button," and "text box." The "video" element involves more encoding / decoding operations and data transmission, while the "button" and "text box" elements involve less. Therefore, the source device can send the data from its video application, as well as the encoded data from the "button" and "text box" elements in its initial interface, to the target device. The target device can then use the received video application data and the decoded data from the encoded data of the "button" and "text box" elements in the source device's initial interface to launch the video application on its own device (if the target device does not have the video application installed, it needs to install it first). The target device will then continue displaying the "video" element from the source device's initial video application interface in display window 1, and project the "button" and "text box" elements from the source device's initial video application interface onto display window 1.

[0223] In addition, in order to enable users to switch the collaborative display interface on the target device as needed and improve collaboration efficiency, the target device can also continue to collaboratively display the source device's system interface in display window 2 via screen projection.

[0224] Specifically, when the source device and target device collaboratively display the initial interface of the source device's video application via screen mirroring, the size of display window 1 can be equal to the size of the source device's screen. When the source device and target device collaboratively display the initial interface of the source device's video application via a sequential or hybrid method, the size of display window 1 may not be equal to the size of the source device's screen. The size of display window 2 can be equal to the size of the source device's screen. Figure 10 illustrates examples where the size of display window 1 and display window 2 are equal to the size of the source device's screen.

[0225] In some embodiments, "equal to" may refer to the size of display window 1 or display window 2 as seen by the user's naked eye, which is exactly equal to the size of the source device's screen; for example, the source device's screen size is 6.8 inches, the size of display window 1 or display window 2 is also 6.8 inches, and the aspect ratio of display window 1 or display window 2 is the same as the aspect ratio of the source device's screen, such as 19:9. In other embodiments, "equal to" may refer to the aspect ratio of display window 1 or display window 2 being equal to the aspect ratio of the source device's screen, but the actual sizes of the two may be different; for example, the aspect ratio of the source device's screen is 19:9, the aspect ratio of display window 1 and display window 2 is also 19:9, but the size of the source device's screen is 6.8 inches, while the size of display window 1 or display window 2 is 5.8 inches or 7.8 inches, and so on.

[0226] Similarly, "not equal to" can refer to the size of the display window 1 as seen by the user's naked eye, which is not equal to the size of the source device's screen at all; for example, the size of the source device's screen is 6.8 inches, and the size of the display window 1 is 5.8 inches or 7.8 inches, and the aspect ratio of the display window 1 is also different from that of the source device's screen, such as the aspect ratio of the source device's screen being 19:9, while the aspect ratio of the display window 1 is 2:3.

[0227] In some embodiments, the content of the interface in display window 1 may be exactly the same as the content of the interface currently displayed on the screen of the source device; that is, all the content of the interface currently displayed on the screen of the source device is correspondingly presented on the interface in display window 1. In other implementations, the content of the interface in display window 1 may be partially the same as and partially different from the content of the interface currently displayed on the screen of the source device. For example, one or more controls near the bottom edge of the screen (e.g., the return to desktop control, the view multitasking control, the return to previous level control) in the content of the interface currently displayed on the screen of the source device may not be displayed in display window 1 of the target device, while all other content may be displayed in display window 1 of the target device.

[0228] In some embodiments, the content of the interface in display window 2 may be exactly the same as the content of the interface displayed in the previous frame of the source device's screen; that is, all the content of the interface displayed in the previous frame of the source device's screen is correspondingly presented on the interface in display window 2. In other implementations, the content of the interface in display window 2 may be partially the same as and partially different from the content of the interface displayed in the previous frame of the source device's screen. For example, one or more controls near the bottom edge of the screen in the content of the interface displayed in the previous frame of the source device's screen (e.g., the return to desktop control, the view multitasking control, the return to previous level control) may not be displayed in display window 2 of the target device, while all other content may be displayed in display window 2 of the target device.

[0229] For example, Figure 11 is a schematic diagram of another source device and target device collaboratively displayed through screen projection, connection or hybrid methods according to an embodiment of this application.

[0230] As shown in Figure 10 above, the source device's screen displays the initial interface of the source device's video application. The target device's display window 1 projects, continues, or hybridizes the initial interface of the source device's video application, while the target device's display window 2 projects the source device's system interface. As shown in Figure 11, after the source device and target device collaboratively display the initial interface of the source device's video application and the system interface of the source device through projection, continuation, or hybrid methods, the source device detects user-triggered operation F and, in response to operation F, plays video a. Operation F can be a user-triggered operation for playing video a, such as a user-triggered click operation on the playback control 1101. After the source device plays the video, the screen display of the source device changes from the initial interface of the source device's video application to the playback interface of video a. The display window 1 of the target device also needs to change from the initial interface of the source device's video application to the playback interface of video a. The display window 2 of the target device also needs to change from the system interface of the source device to the initial interface of the source device's video application. That is, the source device and the target device need to coordinate to display the playback interface of video a and coordinate to display the initial interface of the source device's video application.

[0231] Referring to Figure 10 above, the source device and the target device can choose to collaboratively display the playback interface of video a through any one of the following methods: screen casting, connection, or hybrid method, and collaboratively display the initial interface of the source device's video application through any one of the following methods: screen casting, connection, or hybrid method. These details will not be elaborated here.

[0232] In this context, when the source device and target device collaboratively display the playback interface of video 'a' via screen mirroring, the size of display window 1 can be equal to the screen size of the source device; when the source device and target device collaboratively display the playback interface of video 'a' via a sequential or hybrid method, the size of display window 1 may not be equal to the screen size of the source device. Figure 11 illustrates an example where the size of display window 1 is not equal to the screen size of the source device. Similarly, when the source device and target device collaboratively display the initial interface of the source device's video application via screen mirroring, the size of display window 2 can be equal to the screen size of the source device; when the source device and target device collaboratively display the initial interface of the source device's video application via a sequential or hybrid method, the size of display window 2 may not be equal to the screen size of the source device. Figure 11 illustrates an example where the size of display window 2 is equal to the screen size of the source device. It is understood that the specific details of display windows 1 and 2 in Figure 11 can be referenced from the specific details of display windows 1 and 2 in Figure 10 above, and will not be repeated here.

[0233] Therefore, in the above implementation, after the source device and target device enter the collaborative state, since the projection method is applicable to any frame of the source device's interface, while the successive and mixed methods are not applicable to the source device's system interface and system application interface, but only to the source device's non-system application interface, in order to achieve collaborative display between the source device and the target device, as shown in Figures 6, 10, and 11 above, the source device's system interface can be projected onto the target device's screen display window 1. If the next frame of the source device's system interface is the initial interface of the source device's non-system application, the initial interface of the source device's non-system application can be projected, successive, or mixed onto the target device's screen display window 1. If the next frame of the source device's non-system application's initial interface is a non-initial interface of the source device's non-system application, the non-initial interface of the source device's non-system application can be projected, successive, or mixed onto the target device's screen display window 1.

[0234] Furthermore, the screen mirroring method requires the most encoding / decoding operations and data transmission, followed by the hybrid method, while the continuous screen mirroring method requires the fewest. Also, the hybrid and continuous methods require migrating data from non-system applications on the source device to the target device, while the screen mirroring method does not. Therefore, to achieve collaborative display between the source and target devices while minimizing the required encoding / decoding operations and data transmission, when the source device's non-system applications do not support data migration, the initial or non-initial interface of the source device's non-system applications can be mirrored to the target device's display window 1; when the source device's non-system applications support data migration, the initial or non-initial interface of the non-system applications can be continuously or mixedly displayed to the target device's display window 1.

[0235] Furthermore, to enable users to switch collaboratively displayed interfaces on the target device as needed and improve collaboration efficiency, the previous frame of the interface currently displayed on the source device's screen can be projected, displayed sequentially, or mixed with the previous frame of the interface displayed on the target device's screen in display window 2. For example, if the previous frame of the interface currently displayed on the source device's screen is a system interface, that system interface can also be projected and displayed in display window 2 of the target device's screen; if the previous frame of the interface currently displayed on the source device's screen is a non-system application interface, that non-system application interface can also be projected, displayed sequentially, or mixed with the previous frame of the interface displayed on the target device's screen in display window 2.

[0236] In other words, when the source and target devices perform collaborative display, they can select a suitable collaborative display method based on whether the interface being displayed (e.g., system interface, system application interface, non-system application interface) is suitable for a particular collaborative display method (e.g., screen mirroring, connection, hybrid method), the encoding and decoding operations required for the collaborative display method, the data to be transmitted, and whether the application corresponding to the interface on the source device supports data migration. This improves collaborative efficiency. Furthermore, the target device can collaborate with the source device through two display windows: one window (display window 1 above) displays the currently displayed interface on the source device's screen, and the other window (display window 2 above) displays the interface displayed in the previous frame on the source device's screen. This allows users to switch the collaborative display interface on the target device as needed, further improving collaborative efficiency.

[0237] In the specific implementation process, as shown in Figures 10 and 11 above, when the source device and the target device collaboratively display the first interface through screen projection, connection, or a hybrid method, the source device can send second data and third data to the target device. The target device receives the second data and third data from the source device, displays the first interface (i.e., the interface currently displayed on the source device's screen) in the first display window according to the second data, and displays the previous frame of the first interface (i.e., the interface displayed on the source device's screen before displaying the first interface) in the second display window according to the third data.

[0238] The second data may be the data required for projecting, continuing, or mixing the first interface on the target device.

[0239] For example, the encoded data of the first interface or the URL corresponding to the multimedia resources in the first interface, i.e., the second data is the casting data corresponding to the first interface, and the first display window is the casting window corresponding to the first interface. Another example is the data of the first application corresponding to the first interface, i.e., the second data is the continuation data corresponding to the first interface, and the first display window is the continuation window corresponding to the first interface. Yet another example is the data of the first application corresponding to the first interface, and the encoded data or corresponding URL of the interface elements (i.e., casting elements) that need to be cast in the first interface, i.e., the second data is the mixed data corresponding to the first interface, and the first display window is the mixed window corresponding to the first interface.

[0240] The third data is the data required for projecting, continuing, or mixing the previous frame of the first interface on the target device.

[0241] For example, the encoded data of the previous frame of the first interface or the URL corresponding to the multimedia resources in the previous frame of the first interface, i.e., the third data is the casting data corresponding to the previous frame of the first interface, and the second display window is the casting window corresponding to the previous frame of the first interface. Another example is the application data corresponding to the previous frame of the first interface, i.e., the third data is the continuation data corresponding to the previous frame of the first interface, and the second display window is the continuation window corresponding to the previous frame of the first interface. Yet another example is the application data corresponding to the previous frame of the first interface, and the encoded data or corresponding URL of the interface elements (i.e., casting elements) that need to be cast in the previous frame of the first interface, i.e., the third data is the casting continuation data corresponding to the previous frame of the first interface, and the second display window is the mixed window corresponding to the previous frame of the first interface.

[0242] In one possible implementation, before the source device sends the data of the first application corresponding to the first interface to the target device, in order to improve the data transmission security of collaborative display between the source and target devices, the source and target devices can determine whether the first application in the source device supports data migration. If the first application in the source device supports data migration, the source device will send the data of the first application in the source device to the target device, so that the target device can start the first application in the target device to continue displaying the first interface based on the received data of the first application, thereby ensuring user data security.

[0243] It is understandable that if the target device does not have the first application installed, the target device needs to install the first application first. When the target device installs the first application, it can be installed silently in the background without notifying the user, so as to improve the user experience.

[0244] Specifically, the source device can determine whether the first application in the source device supports data migration by determining whether the first application in the source device has the permission for data migration; alternatively, it can determine whether the first application in the source device supports data migration through user interaction. This embodiment of the application does not limit this approach. After determining whether the first application in the source device supports data migration, the source device can proactively send a data migration notification message to the target device, or it can receive a data migration request message from the target device and then send a data migration response message back to the target device. This embodiment of the application does not limit this approach either.

[0245] For example, consider a scenario where the source device receives a data migration request message from the target device and then sends a data migration response message back to the target device. Figure 12 is a schematic diagram illustrating how to determine if a first application in the source device supports data migration, according to an embodiment of this application.

[0246] As shown in Figure 6 above, the source device's screen displays the source device's system interface, and the target device's display window 1 projects the source device's system interface. As shown in Figure 12, after the source device and target device collaboratively display the source device's system interface via projection, the target device detects user-triggered operation G. In response to operation G, the target device sends a data migration request for the video application to the source device. This data migration request is used to request the migration of data from the video application in the source device. Operation G can be a user-triggered operation to launch the video application; for example, operation G can be a user-triggered click operation on the video application's icon control 1201.

[0247] After the source device receives a data migration request message from the target device's video application, a pop-up window titled "Device e requests video application migration" appears on the source device's screen. The source device detects an operation H triggered by the user. In response to operation H, the source device determines that the video application in the source device supports data migration and sends a data migration response message to the target device. This response message indicates that the video application in the source device supports data migration. Operation H can be a user-triggered operation to determine that the video application in the source device supports data migration; for example, operation H could be a user-triggered click on the "OK" control 1202.

[0248] In one possible implementation, after the first interface currently displayed on the source device's screen is projected, continued, or mixed into a first display window on the target device's screen, and the previous frame of the first interface previously displayed on the source device's screen is projected, continued, or mixed into a second display window on the target device's screen, in order to reduce and save data transmission bandwidth for collaborative display between the source and target devices and reduce power consumption, the target device can determine whether the user is currently paying attention to the second display window based on whether there is focus on it. It can be understood that if there is focus on the second display window, it means the user may not be currently paying attention to it. In this case, the target device can perform operations such as reducing the size and resolution of the second display window and / or pausing the update of the interface in the second display window (i.e., synchronizing with the source device), which will not significantly degrade the user experience and can reduce resource consumption. Furthermore, if focus reappears on the second display window, it means the user may be re-engaging with it. In this case, the target device can perform operations such as restoring the size and resolution of the second display window and / or restoring the update of the interface in the second display window to ensure a good user experience.

[0249] Specifically, when the target device detects that there is no focus on the second display window, the target device can perform a first operation on the second display window.

[0250] The first operation applied to the second display window includes one or more of the following: switching the size of the second display window from a first size to a second size, wherein the first size is larger than the second size; switching the resolution of the interface in the second display window from a first resolution to a second resolution, wherein the first resolution is larger than the second resolution; pausing the previous frame of the interface that is co-displayed with the source device.

[0251] It is understandable that after pausing the display of the first interface in collaboration with the source device, the interface displayed in the second display window can be the last frame of the display interface obtained by the target device from the source device before pausing; or, the interface displayed in the second display window can be the interface after processing the aforementioned last frame of the display interface, such as the interface after blurring or grayscale processing of the aforementioned last frame of the display interface; or, the interface displayed in the second display window can be preset content, such as a preset image, a preset control, or a preset color, etc.

[0252] After the target device performs the first operation on the second display window, when the target device detects that there is focus on the second display window, the target device can perform the second operation on the second display window.

[0253] The first operation applied to the second display window includes switching the size of the second display window from a first size to a second size, and the second operation applied to the second display window may include switching the size of the second display window from the second size to the first size; the first operation applied to the second display window includes switching the resolution of the interface in the second display window from a first resolution to a second resolution, and the second operation applied to the second display window includes switching the resolution of the interface in the second display window from the second resolution to the first resolution; the first operation applied to the second display window includes pausing the previous frame of the interface co-displaying with the source device, and the second operation applied to the second display window may include resuming the previous frame of the interface co-displaying with the source device.

[0254] It is understood that the specific details of the first and second operations applied to the second display window can be found in the first and second operations applied to the first display window described above, and will not be repeated here.

[0255] For example, Figure 13 is a schematic diagram of another source device and target device co-displaying through screen projection, connection or hybrid methods according to an embodiment of this application.

[0256] As shown in Figure 10 above, the source device's screen displays the initial interface of the source device's video application. The target device's display window 1 projects, continues, or hybridizes the initial interface of the source device's video application. The target device's display window 2 projects the source device's system interface. As shown in Figure 13, after the source device and target device collaboratively display the initial interface of the source device's video application and the system interface of the source device through projection, continuation, or hybrid methods, and after the target device detects that the mouse pointer is not hovering over display window 2 (i.e., there is no focus on display window 2), the target device can shrink the size of display window 2 from size 1 to size 2, reduce the resolution of the interface in display window 2 from resolution 1 to resolution 2, and pause the collaborative display of the source device's system interface with the source device through projection.

[0257] For example, Figure 14 is a schematic diagram of another source device and target device co-displaying through screen projection, connection or hybrid methods according to an embodiment of this application.

[0258] As shown in Figure 13 above, when the target device detects that the mouse pointer is not hovering over display window 2, it can shrink the size of display window 2 from size 1 to size 2, reduce the resolution of the interface in display window 2 from resolution 1 to resolution 2, and pause the collaborative display of the source device's system interface with the source device via screen mirroring. As shown in Figure 14, when the target device detects that the mouse pointer is hovering over display window 2, i.e., there is focus on display window 2, the target device can restore the size of display window 2 from size 2 to size 2, restore the resolution of the interface in display window 2 from resolution 2 to resolution 1, and resume the collaborative display of the source device's system interface with the source device via screen mirroring.

[0259] Therefore, in the above implementation, the target device can perform either the first operation or the second operation based on whether there is focus on the second display window. For example, when the second display window changes from having focus to not having focus, the target device can reduce the size and resolution of the second display window and / or pause the updating of the interface in the second display window; when the second display window changes from not having focus to having focus, the target device can restore the size and resolution of the second display window and / or resume the updating of the interface in the second display window. Thus, the size, resolution, and / or displayed interface content of the second display window can be adjusted as needed, ensuring a good user experience while reducing resource consumption.

[0260] In one possible implementation, after the first interface currently displayed on the source device's screen is projected, continued, or mixed into a second display window on the target device's screen, and after the previous frame of the first interface displayed on the source device's screen before the first interface is projected, continued, or mixed into a second display window on the target device's screen, the source device and the target device can respond to corresponding trigger events to switch from a collaborative state to a continuous state, and vice versa. This is to meet more user scenario requirements.

[0261] Specifically, when switching from the collaborative state to the continuous state, the source device and / or the target device can receive a second trigger event. This second trigger event can be used to trigger the source device and the target device to stop collaboratively displaying the first interface and to trigger the target device to continue running the first application.

[0262] The second triggering event can be user interaction, changes in network status, messages sent by other devices to the source or target device, etc., and this application embodiment does not limit this. For example, the second triggering event could be that the source device has low battery and cannot continue to cooperate with the target device for display, or it could be that the distance between the target devices is too far and they cannot continue short-range communication with the source device, etc.

[0263] The second triggering event may include one or more sub-triggering events. When the source device and the target device receive the second triggering event, it can be understood that the source device and the target device each receive different sub-triggering events of the second triggering event. The number of sub-triggering events received by the source device and the number of sub-triggering events received by the target device may be the same or different; this embodiment does not limit this. For example, the second triggering event may include sub-triggering event 1, sub-triggering event 2, and sub-triggering event 3. When the source device and the target device receive the second triggering event, it can be understood that the source device receives sub-triggering events 1 and 2, and the target device receives sub-triggering event 3.

[0264] In response to the second triggering event, the source device determines whether the first application in the source device supports data migration. If the first application in the source device supports data migration, the source device can send fourth data to the target device. The target device can receive the fourth data from the source device and run the non-system application based on the fourth data. The fourth data is used to provide the necessary data for continuing to run the non-system application on the target device.

[0265] For example, Figure 15 is a schematic diagram of another method of collaborative display between a source device and a target device, including a projection method, a connection method, or a hybrid method, provided in an embodiment of this application.

[0266] As shown in Figure 11 above, the source device's screen displays the playback interface of video a, the target device's display window 1 projects, continues, or mixes the playback interface of video a, and the target device's display window 2 projects the initial interface of the source device's video application.

[0267] As shown in Figure 15, the source device and the target device collaboratively display the initial interface of the source device's video application through a projection, continuation, or hybrid method. After the initial interface of the source device's video application is displayed collaboratively through projection, continuation, or hybrid methods, the target device detects user-triggered operation I. In response to operation I, the target device stops collaboratively displaying the playback interface of video a and the initial interface of the source device's video application, and resumes running the source device's video application. Operation I can be a user-triggered operation to stop collaborative display and resume operation; for example, operation I can be a user-triggered click operation on the "Stop Collaborative Display and Resume Operation" control 1501. After the target device stops collaboratively displaying the playback interface of video a and the initial interface of the source device's video application, and resumes running the source device's video application, the target device resumes running the playback interface of video a in display window 1.

[0268] It is understood that when the target device continues to run the video application of the source device, the target device can send a data migration request to the source device and receive a data migration response from the source device. The data migration request is used to request the source device to migrate the data of the video application in the source device to the target device, and the data migration response is used to indicate the data of the video application in the source device, so that the target device can continue to run the video application in the source device according to the data of the video application in the source device.

[0269] When switching from a connected state to a collaborative state, the source device and / or the target device can receive a third trigger event. This third trigger event can be used to trigger the source device and the target device to collaboratively display a second interface, which can be the interface of the first application.

[0270] The third triggering event can be user interaction, changes in network status, messages sent by other devices to the source or target device, etc., and this application embodiment does not limit this. In some embodiments, the third triggering event can also be a "tap-to-pay" event, which may include, for example, the distance between the source and target devices meeting the conditions for NFC communication between the source and target devices. When a "tap-to-pay" event occurs, the NFC modules of the source and target devices are brought close together, and the source and target devices can transfer the data required for the source and target devices to collaboratively display the second interface via NFC.

[0271] The third triggering event may include one or more sub-triggering events. When the source device and the target device receive the third triggering event, it can be understood that the source device and the target device each receive different sub-triggering events of the third triggering event. The number of sub-triggering events received by the source device and the number of sub-triggering events received by the target device may be the same or different; this application embodiment does not limit this. For example, the third triggering event may include sub-triggering event 1, sub-triggering event 2, and sub-triggering event 3. When the source device and the target device receive the third triggering event, it can be understood that the source device receives sub-triggering events 1 and 2, and the target device receives sub-triggering event 3. This application embodiment does not limit this.

[0272] In response to the third triggering event, the source device and the target device can collaboratively display the second interface and synchronize the data of the first application. The specific steps for collaboratively displaying the second interface are the same as those for collaboratively displaying the first interface described above, and will not be repeated here.

[0273] Therefore, in the above implementation, after the source and target devices enter a collaborative state, they can switch from the collaborative state to a continuous state in response to corresponding trigger events. For example, if the source device's battery level is insufficient to support the collaborative display of a non-system application's interface with the target device, the source device can disconnect from the target device and migrate the data of that non-system application from the source device to the target device, allowing the target device to continue running the non-system application. Furthermore, after the source and target devices enter a continuous state, they can respond to corresponding trigger events. For example, if the source device's battery level is sufficient to support the collaborative display of a non-system application's interface with the target device, the source device can re-establish a collaborative connection with the target device, collaboratively display the non-system application's interface based on this connection, and synchronize the non-system application's data between the source and target devices. This satisfies more user needs across various scenarios and improves the user experience.

[0274] Based on the above embodiments and the same concept, this application also provides an electronic device for implementing the multi-device collaboration method provided in this application.

[0275] As shown in Figure 16, the electronic device 1600 may include: a memory 1601, one or more processors 1602, and one or more computer programs (not shown in the figure). These devices can be coupled via one or more communication buses 1603. Optionally, when the electronic device 1600 is used to implement the multi-device collaboration method provided in the embodiments of this application, the electronic device 1600 may further include a display screen 1604.

[0276] The memory 1601 stores one or more computer programs (code), each including computer instructions; one or more processors 1602 call the computer instructions stored in the memory 1601, causing the electronic device 1600 to execute the multi-device collaboration method provided in this embodiment. The display screen 1604 is used to display images, videos, application interfaces, and other related user interfaces.

[0277] In a specific implementation, memory 1601 may include high-speed random access memory and may also include non-volatile memory, such as one or more disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Memory 1601 may store an operating system (hereinafter referred to as the system), such as embedded operating systems like Android, iOS, Windows, HarmonyOS, or Linux. Memory 1601 can be used to store implementation programs of the embodiments of this application. Memory 1601 may also store network communication programs, which can be used to communicate with one or more additional devices, one or more user devices, or one or more electronic devices. One or more processors 1602 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of programs in the scheme of this application.

[0278] It should be noted that Figure 16 is only one implementation of the electronic device 1600 provided in this application embodiment. In actual applications, the electronic device 1600 may include more or fewer components, which is not limited here.

[0279] Based on the above embodiments and the same concept, this application also provides a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to execute the multi-device collaboration method provided in the above embodiments.

[0280] Based on the above embodiments and the same concept, this application also provides a computer program product, which includes a computer program or instructions. When the computer program or instructions are run on a computer, the computer executes the multi-device collaboration method provided in the above embodiments.

[0281] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0282] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0283] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0284] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0285] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A multi-device collaborative method, characterized in that, Applied to a multi-device collaborative system, the multi-device collaborative system including a source device and a target device, the method includes: When the source device and the target device have established a cooperative connection, the source device and / or the target device receive a first trigger event; wherein, the first trigger event is used to trigger the source device and the target device to collaboratively display a first interface, the first interface being the interface of a first application of the source device; In response to the first triggering event: When the first application is of type one, the source device and the target device collaboratively display the first interface via screen mirroring; or... When the first application is of type two, the source device and the target device collaboratively display the first interface in a sequential manner.

2. The method as described in claim 1, characterized in that, The first type is used to indicate that the first application is a system application, and the second type is used to indicate that the first application is a non-system application.

3. The method as described in claim 1 or 2, characterized in that, The source device and the target device collaboratively display the first interface via screen projection, the method comprising: When the communication method between the source device and the target device is short-range communication, the source device and the target device collaboratively display the first interface through a co-source projection method; or... When the communication method between the source device and the target device is non-short-distance communication, the source device and the target device collaboratively display the first interface through a heterogeneous projection method.

4. The method according to any one of claims 1-3, characterized in that, Before the source device and the target device collaboratively display the first interface in a sequential manner, the method further includes: The target device determines whether the first application supports data migration; When the first application supports data migration and the target device does not have the first application installed, the target device installs and launches the first application.

5. The method according to any one of claims 1-4, characterized in that, The method further includes: The target device detects that the first display window does not have focus; wherein, the first display window is the window through which the target device displays the first interface; The target device performs a first operation; wherein the first operation includes one or more of the following: The size of the first display window is switched from a first size to a second size, where the first size is larger than the second size; Switch the resolution of the interface in the first display window from a first resolution to a second resolution, where the first resolution is greater than the second resolution; Pause the collaborative display of the interface in the first display window with the source device.

6. The method as described in claim 5, characterized in that, After the target device performs the first operation, the method further includes: The target device detects that there is focus on the first display window; The target device performs a second operation; Wherein, when the first operation includes switching the size of the first display window from the first size to the second size, the second operation includes switching the size of the first display window from the second size to the first size; When the first operation includes switching the resolution of the interface in the first display window from the first resolution to the second resolution, the second operation includes switching the resolution of the interface in the first display window from the second resolution to the first resolution; When the first operation includes pausing the collaborative display of the interface in the first display window with the source device, the second operation includes resuming the collaborative display of the interface in the first display window with the source device.

7. The method according to any one of claims 1-6, characterized in that, The method further includes: The source device and / or the target device receive a second trigger event; wherein the second trigger event is used to trigger the source device and the target device to stop collaboratively displaying the first interface and to trigger the target device to continue running the first application; In response to the second triggering event, the source device determines whether the first application supports data migration; When the first application supports data migration, the source device sends the data required for the first application to continue running on the target device to the target device. The target device receives the data from the source device and runs the first application based on the data.

8. The method as described in claim 7, characterized in that, The method further includes: The source device and / or the target device receive a third trigger event, which is used to trigger the source device and the target device to collaboratively display a second interface, the second interface being the interface of the first application; In response to the third triggering event: When the first application is of the first type, the source device and the target device collaboratively display the second interface via screen mirroring and synchronize the data of the first application; or, When the first application is of the second type, the source device and the target device collaboratively display the second interface in a sequential manner and synchronize the data of the first application.

9. A multi-device collaborative method, characterized in that, The method includes: When a collaborative connection has been established between the source device and the target device, the source device receives a first trigger event; wherein, the first trigger event is used to trigger the source device and the target device to collaboratively display a first interface, the first interface being the interface of a first application of the source device; In response to the first triggering event: When the first application is of type one, the source device displays the first interface to the target device via screen mirroring; or... When the first application is of type two, the source device will display the first interface to the target device in a sequential manner.

10. The method as described in claim 9, characterized in that, The first type is used to indicate that the first application is a system application, and the second type is used to indicate that the first application is a non-system application.

11. The method as described in claim 9 or 10, characterized in that, The source device displays the first interface to the target device via screen mirroring, and the method includes: When the communication method between the source device and the target device is short-range communication, the source device displays the first interface on the target device via a co-source projection method; or... When the communication method between the source device and the target device is non-short-distance communication, the source device will collaboratively display the first interface on the target device through a heterogeneous projection method.

12. The method as described in any one of claims 9-11, characterized in that, The method further includes: The source device receives a second trigger event; wherein the second trigger event is used to trigger the source device and the target device to stop collaboratively displaying the first interface and to trigger the target device to continue running the first application; In response to the second triggering event, the source device determines whether the first application supports data migration; When the first application supports data migration, the source device sends the data required for the continued operation of the first application on the target device to the target device.

13. The method as described in claim 12, characterized in that, The method further includes: The source device receives a third trigger event, which is used to trigger the source device and the target device to collaboratively display a second interface, the second interface being the interface of the first application; In response to the third triggering event: When the first application is of the first type, the source device displays the second interface to the target device via screen mirroring and receives data from the first application on the target device; or, When the first application is of the second type, the source device displays the second interface to the target device in a sequential manner and receives data from the first application from the target device.

14. A multi-device collaborative method, characterized in that, The method includes: When a collaborative connection has been established between the source device and the target device, the target device receives a first trigger event; wherein, the first trigger event is used to trigger the source device and the target device to collaboratively display a first interface, the first interface being the interface of a first application of the source device; In response to the first triggering event: When the first application is of type 1, the target device sends a first request to the source device, the first request being used to request the source device to collaboratively display the first interface on the target device via screen mirroring; or... When the first application is of type two, the target device sends a second request to the source device. The second request is used to request the source device to collaboratively display the first interface on the target device in a sequential manner.

15. The method as described in claim 14, characterized in that, The first type is used to indicate that the first application is a system application, and the second type is used to indicate that the first application is a non-system application.

16. The method as described in claim 14 or 15, characterized in that, When the communication method between the source device and the target device is short-range communication, the first request is used to request the source device to collaboratively display the first interface on the target device via a same-source projection method. When the communication method between the source device and the target device is non-short-distance communication, the first request is used to request the source device to collaboratively display the first interface on the target device through a heterogeneous projection method.

17. The method according to any one of claims 14-16, characterized in that, Before the target device sends the second request to the source device, the method further includes: The target device determines whether the first application supports data migration; When the first application supports data migration and the target device does not have the first application installed, the target device installs and launches the first application.

18. The method as described in any one of claims 16-17, characterized in that, The method further includes: The target device detects that the first display window does not have focus; wherein, the first display window is the window through which the target device displays the first interface; The target device performs a first operation; wherein the first operation includes one or more of the following: The size of the first display window is switched from a first size to a second size, where the first size is larger than the second size; Switch the resolution of the interface in the first display window from a first resolution to a second resolution, where the first resolution is greater than the second resolution; Pause the collaborative display of the interface in the first display window with the source device.

19. The method as described in claim 18, characterized in that, After the target device performs the first operation, the method further includes: The target device detects that there is focus on the first display window; The target device performs a second operation; Wherein, when the first operation includes switching the size of the first display window from the first size to the second size, the second operation includes switching the size of the first display window from the second size to the first size; When the first operation includes switching the resolution of the interface in the first display window from the first resolution to the second resolution, the second operation includes switching the resolution of the interface in the first display window from the second resolution to the first resolution; When the first operation includes pausing the collaborative display of the interface in the first display window with the source device, the second operation includes resuming the collaborative display of the interface in the first display window with the source device.

20. The method according to any one of claims 14-19, characterized in that, The method further includes: The target device receives a second trigger event; wherein the second trigger event is used to trigger the source device and the target device to stop collaboratively displaying the first interface and to trigger the target device to continue running the first application; In response to the second triggering event, the source device determines whether the first application supports data migration; When the first application supports data migration, the target device receives the data required by the source device to continue running the first application on the target device, and runs the first application according to the data.

21. The method as described in claim 20, characterized in that, The method further includes: The target device receives a third trigger event, which triggers the source device and the target device to collaboratively display a second interface, the second interface being the interface of the first application; In response to the third triggering event: When the first application is of the first type, the target device sends a third request and the data of the first application to the source device. The third request is used to request the source device to collaboratively display the second interface on the target device via screen mirroring; or... When the type of the first application is the second type, the target device sends a fourth request and the data of the first application to the source device. The third request is used to request the source device to collaboratively display the second interface on the target device through a sequential method.

22. An electronic device, characterized in that, The electronic device includes: one or more processors and memory; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the electronic device to perform the method as described in any one of claims 1-8, or the method as described in any one of claims 9-13, or the method as described in any one of claims 14-21.

23. A chip system, characterized in that, The chip system is applied to an electronic device, the chip system including one or more processors, the processors being configured to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1-8, or the method as described in any one of claims 9-13, or the method as described in any one of claims 14-21.

24. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on an electronic device, the electronic device performs the method as described in any one of claims 1-8, or the method as described in any one of claims 9-13, or the method as described in any one of claims 14-21.

25. A computer program product, characterized in that, It includes computer-readable instructions that, when executed by one or more processors, implement the method as claimed in any one of claims 1-8, or implement the method as claimed in any one of claims 9-13, or implement the method as claimed in any one of claims 14-21.

Images