Wired screen mirroring method, electronic device and storage medium

Abstract

The present application relates to the technical field of terminals. Disclosed are a wired screen mirroring method, an electronic device and a storage medium. By using the method, the electronic device can implement a DP screen mirroring function even if a DP screen mirroring path of a software system of the electronic device fails. Specifically, the electronic device may actively monitor for a DP plug-in event, and acquires, when a DP plug-in event is detected, a display identifier carried in the DP plug-in event, wherein the display identifier is an identifier of a virtual display screen corresponding to a display device. Furthermore, the electronic device may render, to the virtual display screen on the basis of the display identifier, content to be mirrored, and mirrors the content on the virtual display screen to the display device, so as to implement DP screen mirroring.

Description

Wired screen mirroring methods, electronic devices and storage media

[0001] This application claims priority to Chinese Patent Application No. 202411491829.6, filed with the State Intellectual Property Office of China on October 23, 2024, entitled "Wired Projection Method, Electronic Device and Storage Medium", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of terminal technology, and in particular to a wired screen projection method, electronic device, and storage medium. Background Technology

[0003] With the development of multi-device collaboration technology, wired screen mirroring technology, with its high stability and low latency, has been widely used in daily life and work. Wired screen mirroring technology is a technology that transmits the content of one electronic device to another display device through a physical connection (such as a cable). For example, a DisplayPort (DP) cable can be used to establish a physical connection between device A (such as a computer, mobile phone, or tablet) and a large-screen device B (such as a TV or projector). Through wired screen mirroring technology, the content of device A can be projected onto the large-screen device B for presentations or collaboration.

[0004] DisplayPort (DP) projection is a common wired projection technology used to enable screen sharing between devices based on the DP protocol. However, electronic devices cannot use DP projection in some situations. For example, software system upgrades can cause the original DP projection path to fail, rendering the DP projection function unusable. Summary of the Invention

[0005] This application provides a wired screen projection method, electronic device, and storage medium, which enables the electronic device to perform DP screen projection when the DP projection path of the software system fails.

[0006] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0007] Firstly, this application provides a wired screen projection method applied to electronic devices. This method is a wired screen projection method based on the DisplayPort (DP) protocol, i.e., the DP screen projection method. Devices that project using the DP protocol all support the DP interface (i.e., the physical interface used to implement DP protocol transmission). The electronic device can actively listen for the presence of a display device connected to it via the DP interface. If a display device is connected to the electronic device via the DP interface, the electronic device can obtain the identifier of the virtual display screen corresponding to that display device, which can be recorded as a display identifier (i.e., displayid). This display device is the external display device, and the virtual display screen corresponding to this display device is used to write and process the content that needs to be displayed on the physical screen of the display device. It should be understood that the content in the virtual display screen is the content in the electronic device, for example, the content displayed on the screen of the electronic device. Furthermore, the electronic device can project the content in the virtual display screen corresponding to the display device onto the display device based on the display identifier.

[0008] In the above solution, the electronic device does not rely on the software system's native DP projection path. Instead, it adopts a new method to achieve DP projection. Specifically, the electronic device actively listens to obtain the displayid of the virtual display screen corresponding to the external display device and uses this displayid to perform DP projection. In this way, even if the software system's native DP projection path fails, the DP projection function can still be restored to normal use, avoiding or reducing restrictions on the electronic device's functionality and improving the user experience.

[0009] In one possible implementation of the first aspect, when an external display device is plugged into the electronic device via a DP cable and DP interface, a DP insertion event is generated. This DP insertion event triggers subsequent DP projection processing. The electronic device can listen for DP insertion events to detect whether a display device is connected to it via the DP interface. Upon detecting the DP insertion event, the electronic device obtains the display identifier carried in the DP insertion event. Based on this display identifier, the electronic device can project the content of the virtual display screen corresponding to the external display device onto the external display device, thereby achieving DP projection.

[0010] In the above solution, by listening to events, it is possible to accurately and efficiently determine whether a device supporting the DP protocol has been inserted into the electronic device, and to conveniently obtain the displayid of the virtual display screen corresponding to the external display device. Overall, this improves the accuracy and efficiency of DP projection.

[0011] In one possible implementation of the first aspect, the electronic device includes a DP driver, a display management service, and a multi-device collaborative management service. The multi-device collaborative management service is a self-developed service program proposed in this application. After a DP insertion event occurs, it is reported from the underlying layer to the display management service (DMS). If a display device is connected to the electronic device via a DP interface, the DP driver will report the DP insertion event layer by layer until it reaches the display management service. Due to reasons such as the failure of the native DP projection path in the software system, the DP insertion event cannot be processed after it is reported to the display management service. The electronic device can listen to the DP insertion event in the display management service through the multi-device collaborative management service to determine whether there is a display device that needs to perform DP projection.

[0012] In the above solution, by setting up a self-developed multi-terminal collaborative management service in electronic devices and actively listening to DP insertion events in the display management service, it is possible to accurately and timely determine whether there are display devices that need to be connected for DP projection, thereby ensuring that subsequent DP projection processing can proceed normally. This effectively solves the problem of not being able to detect DP projection needs due to DP projection path failure or other reasons.

[0013] In one possible implementation of the first aspect, a preset function is registered in the display management service by the multi-device collaboration management service to listen for DP insertion events in the display management service. For example, the multi-device collaboration management service can be run when the electronic device is powered on to register the preset function. More specifically, after a DP insertion event is reported to the display management service, the display management service can call the preset function to pass the display identifier of the virtual display screen corresponding to the external display device to the multi-device collaboration management service. The multi-device collaboration management service can then listen for or become aware that a DP insertion event has occurred.

[0014] In one possible implementation of the first aspect, upon detecting the DP insertion event, the multi-device collaborative management service can conveniently and effectively interact with the display management service through a preset function to obtain the display identifier (displayid) of the virtual display screen corresponding to the external display device. Specifically, upon detecting the DP insertion event, the display management service can call the preset function to pass the displayid to the multi-device collaborative management service. The multi-device collaborative management service can then obtain the displayid passed by the display management service through the call to the preset function, and perform subsequent processing of DP projection based on the obtained displayid.

[0015] In the above solution, the multi-terminal collaborative management service can actively monitor the display by registering preset functions in the display management service, thus improving the convenience of monitoring. Furthermore, it conveniently obtains the displayid without relying on the software system's native DP projection path, ensuring successful DP projection.

[0016] In one possible implementation of the first aspect, the preset function is the OnDisplayConnected function.

[0017] In the above solution, the multi-terminal collaborative management service can conveniently obtain the displayid from the display management service (DMS) based on the OnDisplayConnected function, which facilitates successful DP projection and avoids dependence on the software system's native DP projection path, making it highly applicable.

[0018] In one possible implementation of the first aspect, the multi-terminal collaborative management service, in addition to listening to DP insertion events and obtaining the displayid, also has the function of performing DP projection processing based on the displayid. Specifically, after obtaining the display identifier displayid, the electronic device can use the multi-terminal collaborative management service to project the content in the virtual display screen onto the display device based on the displayid, thereby realizing DP projection.

[0019] The above solution proposes a multi-functional multi-terminal collaborative management service that integrates multiple functions such as sensing DP insertion events, obtaining displayid, and initiating DP projection processing. Through this multi-terminal collaborative management service, the normal use of DP projection functionality can be achieved conveniently and efficiently without relying on the software system's native DP projection path.

[0020] In one possible implementation of the first aspect, the virtual display screen includes a structure for processing display content; projecting the content in the virtual display screen onto the display device based on the display identifier includes: creating a structure corresponding to the display device based on the display identifier, and writing the content to be projected into the structure; projecting the content written into the structure onto the display device. For example, an electronic device can instruct a multi-device collaborative management service (DMS) to create a structure corresponding to the display device and write the content to be displayed on the display device into the structure.

[0021] It should be understood that a virtual display screen is a virtual display object corresponding to a physical screen, used to generate content that can be displayed on the physical screen. It includes a structure for processing the display content, or in other words, for implementing DP projection. Through a series of processing steps within the structure, the content that can ultimately be displayed on the physical screen based on the DP protocol is obtained. Therefore, in the process of projecting content from the virtual display screen to the display device based on this display identifier, the electronic device can create a structure corresponding to the external display device and write the content to be projected into this structure. The content written into this structure is the content that needs to be projected to the external display device for display. The electronic device can project the content written into this structure to the display device; that is, the electronic device can accurately and effectively display the content to be projected on the external display device based on this structure, thus achieving DP projection.

[0022] In one possible implementation of the first aspect, the virtual display screen includes structures created not only in the DMS (Distributed Management System) of the application framework layer but also in the layer compositor of the local service layer. After the content to be projected is written to the structure created by the DMS, the content can be sent to the structure created by the layer compositor in the local service layer. The content is then written into the structure created by the layer compositor, and layer compositing is performed. The result of the layer compositing is then presented on the display device. It should be understood that structures in multiple layers of the software system collectively constitute the virtual display screen. For example, the virtual display screen may include structures created by the DMS of the application framework layer for the display device and structures created by the layer compositor of the local service layer for the display device. Through the joint action and collaborative processing of structures in multiple layers, the content is ultimately presented on the display device.

[0023] In one possible implementation of the first aspect, the screen projection mode corresponding to the electronic device is a same-source projection mode, which means that after projection, the electronic device and the display device display the same content. In the same-source projection mode (also called the same-source projection mode), the screen of the electronic device and the screen of the external display device need to display the same content. Therefore, in the same-source projection mode, when the electronic device uses the displayid of the virtual display screen corresponding to the external display device, it can create only the second structure corresponding to the display device. It should be understood that before DP projection, the screen of the electronic device itself also displays content. Therefore, the electronic device creates a corresponding first structure for its own screen. This first structure can be written with the content that will be displayed on the screen of the electronic device after projection, i.e., the content to be displayed on the screen of the electronic device. This content can be recorded as the first content, which also belongs to the content to be projected. The electronic device can then control the second structure to mirror the first content written in the first structure, so that both the first and second structures contain the same content, so that the screen of the electronic device and the screen of the external display device can subsequently display the same content.

[0024] In the above solution, under the same-source projection mode, it is only necessary to mirror the content written in the first structure corresponding to the screen of the electronic device itself for the second structure corresponding to the external display device. This can easily achieve DP projection under the same-source projection mode, which not only successfully achieves DP projection, but also allows for adaptive processing based on the corresponding projection mode, making DP projection more accurate and efficient.

[0025] In one possible implementation of the first aspect, under the same-source projection mode (i.e., same-source projection mode), after writing the content to be displayed to the first structure corresponding to the screen of the electronic device and the second structure corresponding to the external display device respectively, the electronic device can perform layer synthesis based on the first content through a layer synthesizer to obtain a first image, and transmit the first image to the screen of the electronic device and the screen of the display device respectively for display, thereby realizing DP projection.

[0026] In the above solution, under the same source projection mode, since the electronic device and the external display device need to display the same content, the layer compositer in the electronic device (such as Surfaceflinger) only needs to composite one image when compositing layers to achieve DP projection. This saves the resource consumption caused by the necessary layer compositing process and can achieve DP projection conveniently and efficiently.

[0027] In one possible implementation of the first aspect, the screen projection mode corresponding to the electronic device is a heterogeneous projection mode, which refers to a mode where, after projection, the electronic device and the display device display different content. In heterogeneous projection mode (also called heterogeneous projection mode), the electronic device and the external display device need to display different content. Specifically, the content to be displayed on the electronic device after projection in heterogeneous projection mode is denoted as the third content, which can also be called the content to be displayed on the screen of the electronic device. The content to be displayed on the external display device after projection in heterogeneous projection mode is denoted as the second content, which is also the content to be projected in this implementation. In other words, after projection in heterogeneous projection mode, some application content in the application layer can be displayed on the screen of the electronic device itself, and some application content can be displayed on the screen of the external display device, so that the electronic device and the display device display different content.

[0028] Because the electronic device and the external display device need to display different content, the electronic device cannot directly mirror the content in the structure corresponding to the external display device. Therefore, in addition to creating a second structure corresponding to the display device, the electronic device also creates a third structure and writes the second content into the third structure. The electronic device can control the second structure to mirror the second content written in the third structure. This allows different content to be written into the structures corresponding to the electronic device and the external display device, enabling differentiated display of content on the electronic device and the external display device, thus achieving DP projection in a heterogeneous projection mode.

[0029] The above solution successfully achieves DP projection and can adapt to different projection modes, enabling differentiated display in heterogeneous projection modes. This makes DP projection more accurate and efficient. Furthermore, it does not require all DP projection devices to display the same content, making DP projection more flexible and capable of expanding the screen, thus offering strong practicality.

[0030] Furthermore, in implementing DP projection in heterogeneous projection mode, the upper-layer application layer is unaware of the process. That is, the application layer still only interacts with the user and is responsible for transmitting the content to be displayed (or to be displayed) to the lower layers. The content to be displayed is broken down by the lower layer of the application layer in the electronic device, such as the application framework layer, and written into different structures. For example, content to be displayed on the screen of the electronic device is written into the first structure, and content that does not need to be displayed on the screen of the electronic device itself but needs to be displayed on the screen of an external display device is written into the third structure. Subsequently, the content written in the third structure is mirrored by the second structure corresponding to the display device, thus realizing DP projection in heterogeneous projection mode. The entire DP projection process in heterogeneous projection mode is unaware of the upper-layer application, making it more convenient and efficient.

[0031] In one possible implementation of the first aspect, in the heterogeneous projection mode, the third content is written into a first structure corresponding to the electronic device. The second content is written into a second structure corresponding to the external display device. Then, when the layer compositer performs layer compositing, the electronic device can use the layer compositer to perform layer compositing based on the second content written in the second structure to obtain a second image, and transmit the second image to the display device for display. The electronic device can also use the layer compositer to perform layer compositing based on the third content written in the first structure to obtain a third image, and transmit the third image to the screen of the electronic device for display.

[0032] In the above solution, in the heterogeneous projection mode, since the screen of the electronic device and the screen of the external display device need to display different content, the layer compositer in the electronic device (such as Surfaceflinger) can composite the corresponding images for the screen of the electronic device and the screen of the external display device respectively when compositing the layers, and transmit each composite image to its corresponding screen for display. This can realize the DP projection in the heterogeneous projection mode and ensure the accuracy of the DP projection in the heterogeneous projection mode.

[0033] In one possible implementation of the first aspect, the electronic device can detect the screen projection mode corresponding to the electronic device through a multi-terminal collaborative management service; the screen projection mode is either a same-source screen projection mode or a different-source screen projection mode. Furthermore, the electronic device can create a structure corresponding to the display device based on the screen projection mode.

[0034] In the above solution, the functionality of the multi-terminal collaborative management service has been further enriched and improved. It now includes a screen projection mode detection function, allowing electronic devices to accurately detect the projection mode and subsequently perform DP projection processing accordingly, thus improving the accuracy of DP projection. Furthermore, the entire DP projection process is primarily implemented by a newly developed multi-terminal collaborative management service, which is highly convenient and significantly improves the efficiency of DP projection.

[0035] Secondly, this application provides an electronic device comprising at least a display screen, a memory, and one or more processors. The display screen is used to display content, and the memory is used to store computer instructions. When the one or more processors execute the computer instructions, the electronic device performs the method described in any of the first aspects above.

[0036] Thirdly, this application provides a chip system applied to an electronic device, the chip system including one or more processors, the processors being used to invoke computer instructions to cause the electronic device to perform any of the methods described in the first aspect above.

[0037] Fourthly, this application provides a computer storage medium including computer instructions that, when executed on an electronic device, cause the electronic device to perform any of the methods described in the first aspect above.

[0038] Fifthly, this application provides a computer program product that, when run on a computer, causes the computer to perform any of the methods described in the first aspect above. Attached Figure Description

[0039] Figure 1A is a schematic diagram of a DP projection scenario provided in an embodiment of this application;

[0040] Figure 1B is a schematic diagram of the DP projection scenario provided in the embodiment of this application;

[0041] Figure 1C is a schematic diagram of DP screen mirroring failure provided in an embodiment of this application;

[0042] Figure 2 is a hardware structure block diagram of an electronic device provided in an embodiment of this application;

[0043] Figure 3 is a software structure block diagram of an electronic device provided in an embodiment of this application;

[0044] Figure 4 is a schematic flowchart of the wired screen projection method provided in the embodiment of this application;

[0045] Figure 5 is a schematic diagram of the wired screen projection method provided in the embodiment of this application;

[0046] Figure 6 is a schematic diagram of the wired screen projection method provided in the embodiment of this application (II).

[0047] Figure 7 is a schematic diagram of the wired screen projection method provided in the embodiment of this application.

[0048] Figure 8 is a schematic diagram of the wired screen projection method provided in the embodiments of this application.

[0049] Figure 9 is a schematic flowchart of the wired screen projection method provided in the embodiment of this application. Detailed Implementation

[0050] The technical solutions of the embodiments of this application are described below with reference to the accompanying drawings. In the description of the embodiments of this application, the terminology used in the following embodiments is for the purpose of describing specific embodiments only and is not intended to limit the application. As used in the specification and appended claims of this application, the singular expressions "a," "the," "the," "the," and "this" are intended to also include expressions such as "one or more," unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, "at least one" and "one or more" refer to one or more (including two). The term "and / or" is used to describe the relationship between related objects, indicating that three relationships can exist; for example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. In the description of the embodiments, unless otherwise stated, "multiple" means two or more.

[0051] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized. The term "connection" includes direct connections and indirect connections, unless otherwise stated. The terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first," "second," "third," etc., may explicitly or implicitly include one or more of that feature.

[0052] In the embodiments of this application, the words "exemplary," "for example," or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the words "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a specific manner.

[0053] Before introducing the embodiments of this application, the relevant terms involved in this application will be briefly introduced below:

[0054] 1. DisplayPort projection.

[0055] DP (DisplayPort) projection refers to a wired projection technology based on the DisplayPort protocol (hereinafter referred to as DP protocol). That is, after a wired connection (also known as a physical connection) is established between the source device and the target device via a DP cable, the digital signal transmission capability of the DP protocol is used to transmit the content from the source device, such as the screen content of the source device, to the screen of the target device for display. It should be understood that in heterogeneous projection mode, the screen content of the source device may not be transmitted to the screen of the target device. Instead, content accessed on the source device but not displayed on the source device's screen can be transmitted to the screen of the target device for display via DP projection. The heterogeneous projection mode will be described in more detail below.

[0056] The DP protocol is a digital video transmission protocol developed by VESA (Video Electronics Standards Association) to provide high-quality video and audio transmission.

[0057] DP cable, short for DisplayPort cable, is a digital signal transmission standard cable used to transmit digital signals between the DP interfaces of two devices. The DP interface, or DisplayPort interface, is the physical interface that implements the DP protocol for data transmission.

[0058] It should be understood that both the source and target devices support the DisplayPort (DP) interface. One end of the DP cable is connected to the DP interface of the source device, and the other end is connected to the DP interface of the target device, thus establishing a wired connection between the source and target devices. Therefore, it can also be understood that the target device is plugged into the source device via the DP interface, thereby establishing a wired connection between the two, which facilitates subsequent DP screen projection.

[0059] 2. Source equipment and target equipment.

[0060] The source device is the content sharing end, used to share content to the target device for display via DisplayPort (DP). The target device is the content receiving end, generally a display device connected to the source device via a DP cable, referred to below as the display device or external display device.

[0061] 3. Same-source projection mode and different-source projection mode.

[0062] Based on whether the content displayed on the source device and the target device is the same after screen mirroring, DP screen mirroring modes can be divided into same-source screen mirroring mode (also known as same-source screen mirroring mode) and different-source screen mirroring mode (also known as different-source screen mirroring mode).

[0063] Same-source casting mode refers to a mode where, after casting, the screens of the source device and the target device display the same content. Different-source casting mode refers to a mode where, after casting, the screens of the source device and the target device display different content.

[0064] Please refer to Figure 1A for an illustration of a DP (Distributed Display) projection scenario, taking the same-source projection mode from a mobile phone to a monitor as an example. Mobile phone 101 is the source device, and monitor 102 is the target device. Mobile phone 101 and monitor 102 establish a wired connection via DP cable 103. Based on this DP cable 103, mobile phone 101 can transmit the content displayed on its screen to the screen of monitor 102 for display. As shown in Figure 1A, after DP projection, the screen of monitor 102 can display the same content as the screen of mobile phone 101. Please refer to Figure 1B for an illustration of a DP projection scenario, taking the different-source projection mode from a mobile phone to a monitor as an example. After mobile phone 101 establishes a wired connection with monitor 102 via DP cable 103, when projecting using the different-source projection mode, the content displayed on the screen of mobile phone 101 is different from the content displayed on the screen of monitor 102.

[0065] The solution presented in this application is primarily applied to DisplayPort (DP) projection scenarios, aiming to resolve the issue of electronic devices being unable to perform DP projection. For example, when the native DP projection path of an electronic device's software system fails, the solution presented in this application enables the electronic device to perform DP projection normally. The software system can be Android, an open-source mobile operating system based on the Linux kernel.

[0066] It should be understood that in some cases, the software system of electronic devices, such as the Android system, has a native DP (Display Targeting) projection path, through which DP projection can be achieved. Here, the native DP projection path refers to the DP projection path built into the software system. However, in some situations, such as when the software system undergoes a version upgrade, the native DP projection path may become invalid, rendering the DP projection function unusable and significantly impacting the user experience.

[0067] Taking the same-source screen mirroring mode as an example, this illustrates the scenario where DP screen mirroring fails. Before the phone 101 and monitor 102 establish a wired connection via DP cable 103, the content displayed on the screen of phone 101 is shown in Figure 1A, while the screen of monitor 102 displays a blank desktop. That is, before DP screen mirroring, the screens of phone 101 and monitor 102 display different content. After the phone 101 and monitor 102 establish a wired connection via DP cable 103, if DP screen mirroring is successful, as shown in Figure 1A, the screen of monitor 102 will display the same or nearly the same content as the screen of phone 101. If DP screen mirroring fails due to the failure of the software system's native DP screen mirroring path, as shown in Figure 1C, monitor 102 will still display a blank desktop. In other words, in the case of DP screen mirroring failure, it is impossible to help the user transmit the content displayed on phone 101 to monitor 102 for large-screen display.

[0068] To address the aforementioned issues, this application provides a wired screen mirroring method for use in electronic devices. Even when the native DP screen mirroring path of the software system is invalid (i.e., the native DP screen mirroring function of the software system is limited), the electronic device can still achieve DP screen mirroring using the wired screen mirroring method proposed in this application, restoring the DP screen mirroring function to normal use and improving the user experience. Specifically, the electronic device no longer relies on the native DP screen mirroring path of the software system, but actively obtains the display identifier (referred to as "displayid") of the virtual display screen corresponding to the external display device connected via the DP interface through active listening. For example, it actively listens to whether a display device is connected to the electronic device via the DP interface; if so, it actively obtains the displayid of the virtual display screen corresponding to that display device. Furthermore, the electronic device can use the DP screen mirroring function normally based on the obtained displayid. It should be understood that if no display device is detected connected to the electronic device via the DP interface, listening can continue. Here, the external display device connected via the DP interface refers to a display device connected to the electronic device via the DP interface, or a display device inserted into the electronic device via the DP interface. The virtual display screen corresponding to the display device is a virtual display object corresponding to the physical screen, used to project the content in the virtual display screen onto the display device.

[0069] It should be understood that when any display device supporting the DP protocol is connected to an electronic device via a DP interface supported by the electronic device, if DP projection is to be achieved, the electronic device needs to create a corresponding virtual display screen at the software level to draw the content to be projected onto the virtual display screen. This virtual display screen then projects the content from the electronic device onto the display device connected to the electronic device via the DP interface. Therefore, after actively listening for the display ID, the electronic device can use this display ID to draw the content to be displayed on its display device onto the corresponding virtual display screen, and then project the content from the virtual display screen onto the display device connected to the electronic device via the DP interface using the DP protocol.

[0070] The content to be displayed on the display device, also known as the content to be projected, can include the screen content of the electronic device, or content accessed within the electronic device but not displayed on its screen. For example, in heterogeneous projection mode, the screen of the electronic device can display the content of application A, while the content of application B accessed within the electronic device can be displayed on the display device via DP projection instead of its own screen. Therefore, the content of application B is the content to be projected that is not displayed on the screen of the electronic device but belongs to the electronic device.

[0071] It should be understood that the aforementioned electronic device is the source device, and the external display device is the target device. In the various embodiments of this application, the external display device may include devices with display functions and supporting the DP protocol, such as mobile phones, laptops, desktop computers (PCs), televisions, projectors, or monitors. It should be noted that the display device does not only have display functions; any electronic device with display functions and supporting the DP protocol falls within the protection scope of the display devices in the embodiments of this application, and will not be elaborated further here.

[0072] For example, the aforementioned electronic devices can be mobile phones, tablets, desktops, laptops, handheld computers, notebook computers, ultra-mobile personal computers (UMPCs), netbooks, as well as cellular phones, personal digital assistants (PDAs), augmented reality (AR) / virtual reality (VR) devices, and other electronic devices that support DP protocol screen projection. This application does not impose special limitations on the specific form of the electronic device. The following mainly uses a mobile phone as an example to illustrate the solution of this application.

[0073] Referring to Figure 2, the mobile phone may include a processor 210, an external memory interface 220, an internal memory 221, a universal serial bus (USB) interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, a headphone jack 270D, a sensor 280, buttons 290, a motor 291, an indicator 292, a camera 293, a display screen 294, and a subscriber identification module (SIM) card interface 295, etc.

[0074] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the mobile phone. In other embodiments of this application, the mobile phone may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0075] Processor 210 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. The different processing units may be independent devices or integrated into one or more processors.

[0076] In some embodiments, the mobile phone can use the processor 210 to perform wired screen mirroring, enabling the normal use of the phone's DP screen mirroring function. For example, even if the native DP screen mirroring path of the software system fails, the mobile phone can still perform DP screen mirroring.

[0077] The wireless communication function of a mobile phone can be implemented through antenna 1, antenna 2, mobile communication module 250, wireless communication module 260, modem processor, and baseband processor.

[0078] The mobile phone implements its display function through a GPU, a display screen 294, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 294 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. The processor 210 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0079] Display screen 294 is used to display images, videos, text, etc. In some embodiments, the mobile phone can display a system interface, such as a desktop, gallery, or third-party application interface, through display screen 294. It should be understood that display screen 294 is a physical display screen. There can be one or more display screens 294. For example, if the mobile phone is a foldable phone, it can have multiple display screens 294 to achieve the screen folding function. Therefore, display screen 294 can be a foldable screen (also known as a foldable display). It should be noted that, in addition to mobile phones, the screen of a display device plugged into a mobile phone via a DP interface can also be a candybar screen or a foldable screen, and there is no limitation on this.

[0080] Mobile phones can achieve shooting functions through ISP, camera 293, video codec, GPU, display 294 and application processor.

[0081] Mobile phones can perform audio functions, such as music playback and recording, through an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, a headphone jack 270D, and an application processor.

[0082] Buttons 290 may include a power button, volume buttons, etc. Buttons 290 can be mechanical buttons or touch buttons. The mobile phone can receive button input and generate key signal inputs related to user settings and function control. Motor 291 can generate vibration alerts. Motor 291 can be used for incoming call vibration alerts or for touch vibration feedback. Indicator 292 can be an indicator light, used to indicate charging status, battery level changes, messages, missed calls, notifications, etc. SIM card interface 295 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 295 to achieve contact and separation with the mobile phone.

[0083] Mobile phone software systems can adopt layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses a layered architecture software system as an example to illustrate the software structure of a mobile phone. It should be noted that this application embodiment is not limited to the software structure of a mobile phone.

[0084] It should be understood that mobile phones can support the DP interface. The DP interface can be a dedicated interface or an extension of the USB interface. That is, the USB interface can be extended to support the DP protocol, and the USB interface can act as the DP interface.

[0085] Please refer to Figure 3, which illustrates the software structure block diagram of the mobile phone provided in this application embodiment, taking the Android system as an example.

[0086] As can be understood, a layered architecture divides software into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. As shown in Figure 3, a software system may include an application layer (APP layer), an application framework layer (Framework layer), a native service layer (Native layer), a hardware abstraction layer (HAL layer), and a kernel layer (also known as the driver layer).

[0087] The application layer can include a series of application packages, such as Application 1 to Application N. Application 1 to Application N can include system applications such as a gallery (not shown in the figure), and can also include third-party applications, such as third-party video applications (not shown in the figure), third-party instant messaging applications (not shown in the figure), etc.

[0088] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0089] In this embodiment of the application, the application framework layer may include a multi-terminal collaborative management service and a display management service (DisplayManagerService, which may be abbreviated as DMS).

[0090] The multi-device collaboration management service is a self-developed service program proposed in this application embodiment, used to realize multi-device collaborative processing. For example, the multi-device coordination service can realize DP screen casting without relying on the native DP screen casting path of the Android system. Taking screen casting from a mobile phone to a display device as an example, after the mobile phone and the display device establish a wired connection through a DP connection cable, even if the native DP screen casting path of the Android system fails, the mobile phone can still use the multi-device collaboration management service to cast content to other display devices, thereby enabling multi-device collaboration with other display devices, such as multi-device collaborative office work.

[0091] It should be understood that, specifically, the multi-device collaboration management service relies on the native DP projection path of the Android system to implement a series of DP projection functions. For example, the multi-device collaboration management service has a series of functions such as actively listening for the existence of external display devices inserted via the DP interface, actively obtaining the display identifier (i.e., displayid) of the virtual display screen corresponding to the display device, and performing DP projection based on the displayid. These functions of the multi-device collaboration management service can complete DP projection even when the native DP projection path of the Android system fails.

[0092] In some embodiments, the multi-terminal collaborative management service has an active listening function, which can actively listen for the presence of an external display device inserted or connected via the DP interface. When an external display device is detected, the service actively obtains the displayid of the virtual display screen corresponding to the external display device and uses the displayid to enable the normal use of the DP projection function.

[0093] It should be noted that the multi-terminal collaborative management service is not limited to DP screen projection; it can also have other functions, such as other functions for enabling multi-terminal collaborative processing.

[0094] Display Management System (DMS) is the system service responsible for display management. It manages the system's display devices and handles display-related events and requests.

[0095] The native service layer, also known as the native C / C++ library layer, sits between the application framework layer and the hardware abstraction layer. It mainly consists of a series of system libraries and runtime environments written in C / C++. These libraries provide access to the underlying hardware, as well as key functions such as high-performance computing and graphics rendering.

[0096] The native service layer and the application framework layer communicate via JNI (Java Native Interface). JNI is a programming framework that allows Java code to interact with applications or libraries written in other programming languages ​​(such as C, C++, or assembly). In a software system, JNI enables Java layer code to call functions of the native service layer, thereby providing support for underlying hardware and high-performance computing.

[0097] The local service layer may include a layer compositor, also known as a Surfaceflinger. The layer compositor is used to manage, compositor, and render the system's graphical interface, ensuring that the application's interface and the system's interface are displayed correctly and efficiently on the screen.

[0098] For example, in some embodiments of this application, a layer compositer can be used to composite elements or content to be displayed on the screen, thereby displaying them on a mobile phone screen and an external display device.

[0099] The Hardware Abstraction Layer (HAL) is an interface layer located between the local service layer and the kernel layer, providing a virtual hardware platform for the operating system.

[0100] In this embodiment, the hardware abstraction layer may include a hardware compositor (HWC). The hardware compositor is primarily responsible for the composition and display of windows (layers).

[0101] The kernel layer is the layer between hardware and software. It can include a DisplayPort driver and a Display driver. The DisplayPort driver is the driver required to implement the DisplayPort interface. The Display driver controls the content displayed on the screen in the hardware layer.

[0102] For example, in some embodiments of this application, the display driver is used to control the content displayed on the first screen of the mobile phone, and to control the second screen of the display device connected to the mobile phone to display the content transmitted from the mobile phone.

[0103] It should be noted that Figure 3 is only used to illustrate the modules in the mobile phone's software system, and does not limit the mobile phone's software system to having or only having these modules.

[0104] The wired screen mirroring method provided in this application can be executed on a mobile phone with the above-described hardware and software structure. The wired screen mirroring method provided in this application is described below in conjunction with the above-described hardware and software structure.

[0105] As shown in Figure 4, in some embodiments, the wired screen projection method may specifically include steps S41 to S43:

[0106] S41: The mobile phone actively listens for DP insertion events.

[0107] Among them, the DP insertion event refers to the event in which an external display device is connected to an electronic device via the DP interface.

[0108] It should be understood that when a display device (such as a monitor) establishes a wired connection with a mobile phone via a DP cable, a DP insertion event will be generated, and this display device is the external display device. Therefore, the mobile phone can actively listen for DP insertion events. For example, the mobile phone can create a listener object for DP insertion events, which will listen for whether a DP insertion event is generated. After detecting that a DP insertion event has been generated, the mobile phone can execute step S42.

[0109] In some embodiments, after a display device establishes a wired connection with a mobile phone via a DP cable, the DP insertion event of the display device is reported layer by layer by the underlying software system. After being reported to the application framework layer, the mobile phone can listen for the DP insertion event. It should be understood that the location where the mobile phone listens for DP insertion events can be set at the application framework layer. For example, a listener object can be set at the application framework layer to listen for the existence of DP insertion events in the Display Management Service (DMS) of the application framework layer, and the DP insertion event reported to the application framework layer carries the displayid.

[0110] For example, the initially generated DP insertion event does not carry a displayid. During the process of reporting the generated DP insertion event layer by layer, the displayid corresponding to the virtual display screen of the display device can be obtained, the obtained displayid can be written into the DP insertion event, and the DP insertion event carrying the displayid can be reported to the application framework layer.

[0111] Referring to Figure 5, when a display device, based on a hardware interface such as a DisplayPort (DP) interface, is plugged into a mobile phone, a DP insertion event is triggered. Further, this DP insertion event is reported sequentially through the kernel layer, hardware abstraction layer, and local service layer to the Display Management Service (DMS) in the application framework layer. As shown in Figure 5, the initially triggered DP insertion event does not carry a displayid. The displayid is obtained by the layer compositor in the local service layer after receiving the DP insertion event reported by the hardware abstraction layer's HWC. This displayid is then reported when the layer compositor reports the DP insertion event to the DMS in the application framework layer. Therefore, the DP insertion event reported by the layer compositor to the DMS carries the displayid. In other words, the DMS can obtain the DP insertion event carrying the displayid.

[0112] S42: Get the displayid from the DP insert event.

[0113] It should be understood that the DP insertion event carries the displayid of the virtual display screen corresponding to the external display device. The mobile phone can then obtain this displayid from the DP insertion event after listening to it. The external display device is the display device connected to the mobile phone.

[0114] In some embodiments, if a DP insertion event is reported to the Display Management Service (DMS), the mobile phone can actively obtain the displayid from the DMS. It should be understood that in this embodiment, the mobile phone actively obtains the displayid of the virtual display screen corresponding to the connected display device from the DMS through active listening, without requiring the DMS to actively report the displayid. This avoids the problem of DP projection failure due to the DMS's inability to report the displayid, greatly improving the success rate of DP projection.

[0115] S43: The phone performs DP screen mirroring based on displayid.

[0116] Specifically, after obtaining the displayid of the virtual display screen corresponding to the display device, the mobile phone can trigger the relevant processing of DP projection based on the displayid, thereby displaying the content on the mobile phone on the screen of an external display device (such as a second screen) through DP projection.

[0117] In some embodiments, the virtual display screen corresponding to the display device may include a structure for processing display content or for implementing DP projection. Through a series of processes within the structure, content to be displayed on the screen based on the DP protocol (i.e., content to be displayed on the physical screen of the display device) can be obtained. The virtual display screen corresponding to the display device is also a type of virtual display screen corresponding to the screen of the display device. After obtaining the displayid, the mobile phone can create a structure corresponding to the display device based on the displayid, and display the content to be projected from the mobile phone on the second screen of the external display device based on this structure, thereby achieving DP projection.

[0118] In some embodiments, the mobile phone can instruct the DMS to create a structure corresponding to the external display device based on the displayid. It should be understood that after the DP insertion event is reported to the DMS, the DMS will not actively create the structure corresponding to the display device. Instead, the mobile phone needs to actively listen for and obtain the displayid, and then instruct the DMS to create the corresponding structure based on the displayid.

[0119] In some embodiments, the mobile phone is equipped with a multi-device collaboration management service. The mobile phone can perform steps S41 to S43 through this multi-device collaboration management service. Specifically, referring to Figure 5, the mobile phone can listen for DP insertion events through the multi-device collaboration management service to obtain the displayid. After obtaining the displayid passed by the DMS to the multi-device collaboration management service, the multi-device collaboration management service can instruct the DMS to create a structure corresponding to the display device based on the displayid.

[0120] Furthermore, the mobile phone can display its content on the screen of an external display device based on this structure, thereby achieving DP projection. For example, referring to Figure 5, the mobile phone can write the content to be projected into the structure created by DMS corresponding to the external display device. This content to be projected is the content on the mobile phone that needs to be displayed on the external display device, such as the content in the mobile phone's application layer that needs to be displayed on the display device. After writing the content to be projected into the structure, the mobile phone can transmit the content to be projected layer by layer until it is transmitted to the second screen of the hardware display device for display through the display driver.

[0121] It should be noted that during the process of transmitting the content written in the structure created in DMS layer by layer downward, each layer will process the content accordingly. For example, the layer compositor in the local service layer will perform layer compositing processing on the content until the content that can be directly sent for display is generated, and finally it will be displayed on the second screen of the display device.

[0122] Figure 5 only provides a simplified illustration of the hierarchical reporting of DP insertion events. In actual processing, the method of reporting DP insertion events at each level differs. The following text describes the wired projection method by detailing the hierarchical reporting process of DP insertion events.

[0123] After the display device triggers the DP insertion event based on the hardware interface, as shown in Figure 6, the specific steps may include:

[0124] Step 1: The DP driver reports the DP insertion event.

[0125] Specifically, the kernel-level DP driver can report the DP insertion event of the display device to the upper-layer (i.e., hardware abstraction layer) hardware synthesizer (HWC) through the uevent mechanism. The uevent mechanism is an event notification mechanism used to send notifications about kernel and device state changes to user space.

[0126] Step 2: The Hardware Composer (HWC) notifies the Layer Composer to create a structure corresponding to the display device.

[0127] It should be understood that, since the content transmitted by the upper application layer needs to be transmitted down layer by layer to the lower physical screen, multiple layers of the software system need to create corresponding structures for the mobile phone and display device screen respectively, so as to represent the physical screen of the mobile phone and display device from the software level, in order to realize the transmission of content down layer by layer.

[0128] For example, in both the local service layer and the application framework layer of the software system, structures need to be created for the mobile phone and the external display device respectively. The structures created for the display device in multiple layers such as the local service layer and the application framework layer constitute the virtual display screen. That is, in this embodiment of the application, the virtual display screen may include the structures created for the display device in the local service layer and the application framework layer, thereby displaying content on the physical screen of the display device, such as a second screen, based on these structures. Therefore, in step ② above, HWC can notify the Surfaceflinger layer compositor of the local service layer to create structures. Here, the structure logically represents the target device for screen projection—that is, the external display device, used to receive and process content from the mobile phone.

[0129] After the layer compositor creates the structure corresponding to the display device, step ③ can be executed. For ease of explanation, the relevant processing of the layer compositor creating the structure corresponding to the display device is not shown in the figure, but this does not mean that this processing does not exist. In addition, since the phone's first screen already displays content before DP projection, and the first screen of the phone can display content based on the structure created for the phone, the structure corresponding to the phone is created before DP projection, not after the DP insertion event is detected. Therefore, it is not shown in Figures 5 and 6. Figures 5 and 6 are used to focus on describing the relevant processing after the DP insertion event occurs.

[0130] It's important to note that the hardware compositor (HWC) instructs the layer compositor to create a structure corresponding to the display device, essentially to notify the layer compositor that a DP insertion event has occurred. The only difference lies in the method used by different layers of the software system to report DP insertion events. For example, the DP driver uses the uevent mechanism to report DP insertion events, while HWC reports them by instructing the layer compositor to create a structure corresponding to the display device. Furthermore, in the following text, the layer compositor reports DP insertion events by calling the onHotplug function.

[0131] Step 3: The layer compositor calls the display management service DMS hot-plug (onHotplug) function to report the DP insertion event carrying the displayid.

[0132] Specifically, before reporting the Display Insertion (DP) event to the application framework layer, the Surfaceflinger layer compositor can obtain the displayid and, via JNI, call the onHotplug function of the Display Management Service (DMS) in the application framework layer to report this displayid. It should be understood that the layer compositor reporting the displayid is essentially to inform the DMS that a DP insertion event has occurred; therefore, it is equivalent to the layer compositor reporting a DP insertion event carrying the displayid to the DMS.

[0133] Furthermore, after the onHotplug function in DMS is called, DMS can obtain information related to the DP insertion event and initialize some data or resources related to the creation of the structure in DMS, in order to prepare for the subsequent creation of the structure corresponding to the display device in DMS.

[0134] Step 4: The multi-terminal collaborative management service listens for DP insertion events to obtain the displayid.

[0135] Specifically, the multi-terminal collaborative management service in the application framework layer can listen for DP insertion events in DMS and obtain the displayid passed by DMS.

[0136] Step 5: The multi-device collaborative management service instructs DMS to create a structure corresponding to the display device based on the displayid.

[0137] Step 6: The application layer writes the content to be projected into the structure created by DMS.

[0138] Step 7: DMS transmits the content to be projected layer by layer downwards.

[0139] Steps ⑤ to ⑦ describe the processing of displaying the content to be projected on the second screen after obtaining the displayid. It should be understood that in addition to the second screen of the display device displaying the corresponding content, the DP projection process also involves the display processing of the phone's first screen. For example, the phone writes the content to be displayed on the phone to a structure created for the phone, and transmits it layer by layer down until it is displayed on the first screen. The display processing of the phone's first screen is not illustrated in Figures 5 and 6, but will be described in detail in Figures 7 and 8 below.

[0140] In some embodiments, the processing for displaying the content to be projected on the second screen will differ depending on the different projection modes, such as same-source projection mode and different-source projection mode. Specifically, for the current target projection mode (same-source projection mode or different-source projection mode), processing logic matching the target projection mode can be used to display the content to be projected on the second screen.

[0141] In some embodiments, after obtaining the displayid, the mobile phone can analyze the current screen casting mode corresponding to the obtained displayid and create a corresponding structure based on the determined target screen casting mode (same-source screen casting mode or different-source screen casting mode). For example, the mobile phone can detect the screen casting mode corresponding to the mobile phone through a multi-device collaboration management service. The structures created for same-source screen casting mode and different-source screen casting mode are different.

[0142] Specifically, in the same-source projection mode, the content displayed on the phone is the same as the content displayed on the external display device. Therefore, the phone only needs to create a structure corresponding to the display device, which can be denoted as Structure 2. The structure created for the phone (which can also be understood as the structure created for the phone's first screen) can be denoted as Structure 1. The phone can control Structure 2 to mirror the content in the phone's corresponding Structure 1. The content in Structure 1 is the content that needs to be displayed on the phone's first screen, which can be denoted as Content 1. By mirroring Content 1 in Structure 1, the same content can be written into the structures (Structure 1 and Structure 2) corresponding to the screens of both the phone and the display device. The phone can transmit Content 1, written in Structure 2 through mirroring, to the second screen of the display device for display. In addition, the phone can also transmit the content written in Structure 1 to the phone's first screen for display. In this way, the first screen of the phone and the second screen of the display device can display the same content.

[0143] In heterogeneous projection mode, the content displayed on the phone differs from the content displayed on the external display device. Therefore, the display device's structure 2 cannot directly mirror the content in the phone's corresponding structure 1. Consequently, the phone needs to create two structures for the display device: structure 2 and structure 3. Structure 3 contains the content to be projected onto the external display device (referred to as content 2), while structure 1 contains the content to be displayed on the phone's first screen (referred to as content 3). The display device's structure 2 can then mirror content 2 written in structure 3. This allows different content to be written in structure 2 and structure 1. The phone can then display content 2 (obtained through mirroring from structure 2) on the second screen of the display device and content 3 (written from structure 1) on its first screen, achieving differentiated display between the phone and the display device.

[0144] To better understand the similarities and differences between the two different screen mirroring modes, a more detailed description is provided below with reference to Figures 7 and 8. Figure 7 illustrates the DP screen mirroring process in the same-source screen mirroring mode. Figure 8 illustrates the DP screen mirroring process in the different-source screen mirroring mode.

[0145] As shown in Figures 7 and 8, regardless of whether it's a same-source or different-source screen mirroring mode, the phone will report the DP insertion event layer by layer until it reaches the DMS. The multi-device collaboration management service can listen for the DP insertion event to obtain the displayid. That is, the multi-device collaboration management service can obtain the displayid transmitted by the DMS. In addition, both Figures 7 and 8 show the creation of a corresponding structure 1 for the phone. For example, when the phone initializes its display, the corresponding structure 1 can be created for the phone.

[0146] After obtaining the displayid from the multi-device collaborative management service, there are differences in the handling of same-source and different-source projection modes. These differences mainly manifest as variations in structure creation and / or layer compositing. The following sections will describe these differences in more detail.

[0147] Please refer to Figure 7. After obtaining the displayid, the Multi-Device Collaboration Management Service detects the screen mirroring mode. If it is a same-origin screen mirroring mode, it instructs DMS to create structure 2 for the display device based on the displayid. The application layer can transmit content 1 to the application framework layer. The application framework layer can write content 1 to structure 1 created for the first screen of the phone. The phone can control structure 2 in DMS to mirror the content 1 in structure 1. Furthermore, DMS can transmit content 1 to the layer compositor.

[0148] Since the content in Structure 1 and Structure 2 is identical—meaning the content displayed by the phone and the display device is the same—the layer compositor only needs to composite one layer to generate one image, denoted as Image 1. The layer compositor can then transmit Image 1 layer by layer down to the display driver. Furthermore, the display driver can transmit Image 1 to the first screen using the transmission protocol used by the phone. Additionally, the display driver can transmit Image 1 to the second screen via the DP protocol.

[0149] Please refer to Figure 8. The Multi-Device Collaboration Management Service detects the projection mode. If it is a heterogeneous projection mode, it instructs DMS to create Structure 2 and Structure 3 for the display device based on the displayid. The application layer also transmits content to the application framework layer. Unlike the same-source projection mode, the Multi-Device Collaboration Management Service can control the splitting of the content transmitted from the application layer into two parts: one part is the content to be displayed on the first screen of the phone (such as the content of application 1 and application 2), which can be denoted as Content 3; the other part is the content to be displayed on the second screen of the display device (such as the content of application N), which can be denoted as Content 2, and belongs to the content to be projected. The Multi-Device Collaboration Management Service can control the writing of Content 3 to Structure 1 in DMS and control the writing of Content 2 to Structure 3. Furthermore, DMS can control Structure 2 to mirror Content 2 written in Structure 3. Finally, DMS can transmit Content 3 and Content 2 to the layer compositor.

[0150] Because the contents of structure 1 and structure 2 are inconsistent, the image compositer needs to generate two layers: image 3 based on content 3 and image 2 based on content 2. The image compositer can then transmit image 3 and image 2 layer by layer down to the display driver. Furthermore, the display driver can transmit image 3 to the first screen using the transmission protocol used by the mobile phone. Additionally, the display driver can transmit image 2 to the second screen using the DP protocol.

[0151] As shown in Figure 9, in some embodiments, the wired screen projection method is described in more detail by combining the multi-terminal collaborative management service of the mobile phone, which may specifically include steps S91 to S95:

[0152] S91: The mobile phone runs a multi-terminal collaborative management service, and based on this service, a listener object for DP insertion events is created, and the default function of DMS is overridden.

[0153] The preset functions are used to implement data transmission between the multi-terminal collaborative management service and DMS. When called by DMS, these preset functions can pass specified data (such as displayid) from DMS to the multi-terminal collaborative management service. For example, the preset function could be the onDisplayConnected function, which is used to implement data transmission between services. The preset functions can also be other functions, as long as they can be called to pass data from DMS to the multi-terminal collaborative management service.

[0154] S92: The mobile phone's multi-device collaborative management service continuously listens for DP insertion events.

[0155] S93: The mobile phone's multi-device collaborative management service can determine whether a DP insertion event has occurred. If yes, then execute S94; otherwise, continue to execute step S92 and continue listening.

[0156] The monitoring result includes whether a DP insertion event has occurred or not. If a DP insertion event is detected, step S94 can be executed. If no DP insertion event is detected, monitoring can continue.

[0157] S94: DMS calls the preset function to pass the displayid in the DP insertion event to the multi-terminal collaborative management service.

[0158] It should be understood that, due to factors such as the failure of the native DP path in the software system, DMS will not or cannot proactively report the displayid in the DP insertion event to the multi-terminal collaborative management service after obtaining the DP insertion event.

[0159] In this embodiment, the multi-terminal collaborative management service overrides a preset function in the DMS. It should be understood that overriding the preset function is equivalent to registering the preset function in the display management service through the multi-terminal collaborative management service. After receiving a DP insertion event in the DMS, the DMS can call the preset function, such as the onDisplayConnected function, to pass the displayid from the DP insertion event to the multi-terminal collaborative management service. In this embodiment, the above processing solves the problem of the software system's native DP path failing and being unable to report the displayid, thus preparing the data for successful DP projection in the future.

[0160] S95: Multi-terminal collaborative management service performs DP screen projection based on displayid.

[0161] As mentioned above, after obtaining the displayid reported by DMS, the mobile phone's multi-device collaborative management service can instruct DMS to create a structure corresponding to the display device based on the displayid, and then display the mobile phone's content on the second screen of the external display device based on this structure, thereby realizing DP projection. A description of how to implement DP projection based on the structure can be found above and will not be repeated here.

[0162] This application also provides a chip system applied to an electronic device. The chip system includes one or more processors, which are used to invoke computer instructions to cause the electronic device to perform the functions or steps described in the above method embodiments.

[0163] This application also provides a computer-readable storage medium including computer instructions that, when executed on the electronic device, cause the electronic device to perform the various functions or steps described in the method embodiments.

[0164] This application also provides a computer program product that, when run on a computer, causes the computer to perform the functions or steps described in the above method embodiments.

[0165] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0166] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0167] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0168] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0169] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, in essence, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0170] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A wired screen projection method, characterized in that, Applied to electronic devices, the method includes: Listen for DP insertion events; the DP insertion event is used to characterize an event in which a display device connects to the electronic device via a DP interface; Upon detecting the DP insertion event, the display identifier carried in the DP insertion event is obtained; the display identifier is the identifier of the virtual display screen corresponding to the display device. Based on the display identifier, the content to be projected is drawn onto the virtual display screen, and the content in the virtual display screen is projected onto the display device.

2. The method according to claim 1, characterized in that, The electronic device includes DP driver, display management service and multi-terminal collaborative management service; The listening for DP insertion events includes: After the display device is connected to the electronic device via the DP interface, the DP driver reports a DP insertion event to the display management service. The multi-terminal collaborative management service listens for the DP insertion event in the display management service.

3. The method of claim 2, wherein, The step of listening to the DP insertion event in the display management service through the multi-terminal collaborative management service includes: The multi-terminal collaborative management service registers a preset function in the display management service to listen for the DP insertion event in the display management service.

4. The method according to claim 3, characterized in that, The step of obtaining the display identifier carried in the DP insertion event includes: The display identifier is obtained by the display management service through the multi-terminal collaborative management service, which is passed by the display management service by calling the preset function.

5. The method according to any one of claims 1-4, characterized in that, The preset function is the OnDisplayConnected function.

6. The method according to any one of claims 1-5, characterized in that, The virtual display screen includes a structure for processing display content; the step of drawing the content to be projected onto the virtual display screen based on the display identifier, and projecting the content in the virtual display screen onto the display device, includes: Based on the display identifier, a structure corresponding to the display device is created, and the content to be projected is written into the structure; The contents of the structure are projected onto the display device.

7. The method of claim 6, wherein, The screen projection mode corresponding to the electronic device is the same-source screen projection mode; The step of creating the structure corresponding to the display device and writing the content to be projected into the structure includes: Create a second structure corresponding to the display device; Control the second structure to mirror the first content in the first structure; Wherein, the first structure is a structure corresponding to the electronic device, and the first content is the content to be projected; the first content is the same as the content displayed on the screen of the electronic device.

8. The method according to claim 6, characterized in that, The screen projection mode corresponding to the electronic device is the heterogeneous screen projection mode; The step of creating the structure corresponding to the display device and writing the content to be projected into the structure includes: Create a second structure corresponding to the display device and create a third structure; Write the second content into the third structure; the second content is the content to be projected. The second structure is controlled to mirror the second content in the third structure; the second content is different from the third content; the third content is the content to be displayed on the screen of the electronic device.

9. The method according to any one of claims 6-8, characterized in that, Creating the structure corresponding to the display device includes: The screen projection mode of the electronic device is detected by the multi-terminal collaborative management service; the screen projection mode is either a same-source screen projection mode or a different-source screen projection mode. Create the structure corresponding to the display device according to the screen projection mode.

10. An electronic device, characterized in that, The electronic device includes at least: a display screen, a memory, and one or more processors; the display screen is used to display content, the memory is used to store computer instructions, and when the one or more processors execute the computer instructions, the electronic device performs the method as described in any one of claims 1-9.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1-9.

12. A computer program product, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1-9.

Images