Screen projection display method and electronic device

Abstract

The application discloses a screen projection display method and electronic equipment. In the call in the screen projection scene, the user's privacy can be protected, and the user can normally use other screen projection applications on the target equipment. The method comprises the following steps: a source end equipment projects a first display interface of a first application into a first window of a target equipment for display, and the source end equipment synchronously displays the display content in the first window; the source end equipment projects a second display interface of a second application into a second window of the target equipment for display, and the source end equipment can continue to synchronously display the display content in the first window; in this case, if the source end equipment detects a call event, whether an object is close to the source end equipment can be detected; if an object is close to the source end equipment, the source end equipment itself can stop displaying a call interface corresponding to the call event, and triggers the target equipment to display a preset shielding picture in the first window, and the shielding picture is used for shielding part or all of the call interface.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a screen projection display method and electronic device. Background Technology

[0002] Currently, mobile phones and other electronic devices can enable anti-accidental touch functions during calls to prevent accidental touches when the user's ear is close to the screen. For example, after the phone detects that the user is in a call with a contact, it can use a proximity sensor to detect if there is an object near the earpiece. If an object is detected near the earpiece, the phone can automatically turn off the screen (or turn it off) to prevent accidental touches when the user's ear is close to the screen, and also to prevent the user's privacy from being leaked during the call.

[0003] In some screen mirroring scenarios, a source device such as a mobile phone (or source device) can project multiple applications into a multi-window format onto a target device such as a PC (or sink device). For example, Figure 1 As shown, the mobile phone can project the display interface of the calling app onto window 101 of the PC, and the mobile phone can project the display interface of the video app onto window 102 of the PC. At this time, the content displayed on the mobile phone can be synchronized with the content displayed in window 101, that is, the mobile phone also displays the display interface of the calling app.

[0004] So, if a user calls contact A on their phone, the phone will trigger the aforementioned anti-accidental touch function to detect if there is an object near the earpiece. If an object is detected near the earpiece, then... Figure 2 As shown, the phone can stop drawing the display interface on the phone screen and enter a screen-off state. Because the phone stops drawing the display interface on the phone screen, it cannot send the display interfaces of various applications to the PC. At this time, windows 101 and 102 on the PC cannot display the display interfaces of the relevant applications normally, causing users to be unable to use the relevant functions in windows 101 and 102 normally, thus reducing the user experience in screen projection scenarios. Summary of the Invention

[0005] This application provides a screen mirroring method and electronic device. When making a call in a screen mirroring scenario, it can protect user privacy without affecting the user's normal use of other screen mirroring applications on the target device, thereby improving the user experience.

[0006] In a first aspect, this application provides a screen projection method, comprising: a source device can project a first display interface of a first application onto a first window of a target device, and the source device synchronously displays the content displayed in the first window; subsequently, the source device can project a second display interface of a second application onto a second window of the target device, at which time the source device can continue to synchronously display the content displayed in the first window; in this scenario, if the source device detects a call event, it can detect whether an object is approaching the source device; if an object is approaching the source device, the source device itself can stop displaying the call interface corresponding to the current call event and trigger the target device to display a preset obscuring image in the first window, the obscuring image being used to obscure part or all of the call interface.

[0007] In other words, when multiple applications are projected as multi-windows onto the target device from the source device, the content displayed on the source device is synchronized with the content displayed in a specific window (e.g., the first window) on the target device. Therefore, when the source device detects an object approaching during a call, not only can the source device stop displaying the call interface, but the target device can also obscure the call interface in the first window by displaying an obscuring image. This prevents the target device from leaking user-related privacy information during a call, while applications unrelated to the call can still be displayed normally on the target device, providing relevant functions and improving the user experience.

[0008] The aforementioned call interface can be the interface displayed by the call application when dialing, or the interface displayed by the call application during a call.

[0009] In some embodiments, in the above scenario, the source device can stop displaying the call interface corresponding to the current call event by entering a screen-off state.

[0010] In other embodiments, when the source device detects an object approaching it, the first windows of both the source and target devices may not simultaneously display the call interface. For example, the call application can be switched to the background by the user. During the call, the first windows of both the source and target devices can simultaneously display the interface of a third application. In this case, the source device can stop displaying the interface of the third application (e.g., turn off the screen) and trigger the target device to use an occlusion image to cover the interface of the third application in the first window.

[0011] In one possible implementation, the source device triggers the target device to display a preset obscuring image in a first window, including: the source device sending a notification event to the target device, the notification event including an identifier of the first window. Upon receiving the notification event, the target device can draw the aforementioned obscuring image in the first window according to the identifier of the first window in the notification event, with the obscuring image positioned above the currently displayed call interface.

[0012] Alternatively, the source device can stop sending the display content corresponding to the first window to the target device, causing the target device to stop displaying the current call interface in the first window.

[0013] In one possible implementation, if an object approaches the source device, the method further includes: the source device can continue to send a second display interface to the target device, triggering the target device to continue displaying the second display interface in the second window. In this way, in multi-application screen mirroring scenarios, when the source device detects a call event, it can stop displaying the call interface in windows related to the call event on both its own device and the target device, avoiding accidental touches and privacy leaks during calls. However, the source device can continue to display windows unrelated to the call event normally on the target device, ensuring that the call event does not interfere with the user's use of other mirrored applications on the target device, thereby improving the user experience in both screen mirroring and call scenarios.

[0014] In one possible implementation, the source device projects the first display interface of the first application onto a first window of the target device, including: the source device drawing the first display interface of the first application in a first virtual screen; and then, the source device outputting the first display interface in the first virtual screen to the displays of both the target device and the source device; wherein, the source device projects the second display interface of the second application onto a second window of the target device, including: the source device drawing the second display interface of the second application in a second virtual screen; and then, the source device outputting the second display interface in the second virtual screen to the target device.

[0015] In one possible implementation, the source device projects a first display interface of a first application onto a first window of the target device, including: the source device drawing the first display interface of the first application within a first area of ​​a first virtual screen; and then, the source device outputs the first display interface in the first area to the displays of both the target device and the source device; wherein, the source device projects a second display interface of a second application onto a second window of the target device, including: the source device drawing the second display interface of the second application within a second area of ​​a first virtual screen; and then, the source device outputs the second display interface in the second area to the target device.

[0016] In one possible implementation, the occluding image may include a notification message to inform the user that a call is in progress. This notification message could be text, an image, an animation, etc. Furthermore, parameters such as the size, position, or transparency of the occluding image within the first window can be set by those skilled in the art based on practical experience.

[0017] In one possible implementation, after the source device triggers the target device to display a preset obscuring image in the first window, the method further includes: responding to an operation input by the user into the second window, the source device receiving operation information sent by the target device; the source device updating the second display interface to a third display interface based on the operation information; and the source device sending the third display interface to the target device, triggering the target device to display the third display interface in the second window. In other words, while the target device uses an obscuring image to cover the call interface in the first window, it can still normally provide the user with relevant application functions in the second window, and the screen mirroring process of the second application in the second window will not be affected.

[0018] In one possible implementation, after the source device triggers the target device to display a preset occlusion image in the first window, the method further includes: when the source device detects that the call event has ended or no object is near the source device, the first windows of both the source device and the target device synchronously display a fourth display interface, which may be the same as or different from the first display interface. For example, when the call event ends, the source device can return to the first display interface before receiving the call event, and at this time, the first window of the target device also synchronously displays the first display interface. As another example, if no object is detected near the source device during a call, the source device can display the current call interface, and at this time, the first window of the target device also synchronously displays the call interface.

[0019] In one possible implementation, when the source device detects a call event, it detects whether an object is approaching the source device, including: when the source device detects a call event, it can use a proximity sensor to detect the distance between the obstacle and the source device; when the distance between the obstacle and the source device is less than a preset value, the source device can determine that an object is approaching the source device; when the distance between the obstacle and the source device is greater than or equal to the preset value, the source device can determine that no object is approaching the source device.

[0020] In one possible implementation, the aforementioned call events include either answering a contact's call or dialing a contact's call.

[0021] Secondly, this application provides a screen projection method, comprising: a source device displaying a first display interface of a first application and sending the first display interface to a target device; the target device receiving the first display interface and displaying the first display interface in a first window; while displaying the first display interface, the source device may also send a second display interface of a second application to the target device; after receiving the second display interface, the target device may display the second display interface in a second window; at this time, the source device projects multiple applications to multiple windows of the target device for display; subsequently, when the source device detects a call event, it may detect whether an object is approaching the source device; if an object is approaching the source device, the source device stops displaying the call interface corresponding to the call event (e.g., enters a screen-off state), and the source device may send a notification event to the target device, the notification event including an identifier of the first window displayed synchronously with the source device; then, in response to the above notification event, the target device may display a preset obscuring image in the first window, the obscuring image being used to obscure part or all of the call interface, to prevent the leakage of user privacy during the call. However, windows unrelated to the call event can be displayed normally on the target device, ensuring that the call event does not interfere with the user's use of other screen-cast applications on the target device, thereby improving the user experience in both screen-casting and call scenarios.

[0022] In one possible implementation, after the target device receives the second display interface and displays it in the second window, the method further includes: the target device receiving a first operation input by the user into the second window; the target device sending first operation information corresponding to the first operation to the source device, causing the source device to update the second display interface to a third display interface based on the first operation information, and then sending the third display interface to the target device; subsequently, the target device displays the third display interface in the second window. In other words, while the target device uses an obscuring image to cover the call interface in the first window, it can still normally provide the user with relevant application functions in the second window, and the screen mirroring process of the second application in the second window will not be affected.

[0023] In one possible implementation, after the target device receives the first display interface and displays it in the first window, the method further includes: the target device receiving a second operation input by the user into the first window, such as opening a second application; then, the target device can send second operation information corresponding to the second operation to the source device; at this time, when the source device displays the first display interface, it sends the second display interface of the second application to the target device, including: when the source device displays the first display interface, it can run the second application in the background according to the second operation information; then, the source device sends the second display interface of the second application running to the target device.

[0024] In one possible implementation, after the target device displays a preset occlusion image in the first window, the method further includes: when the source device detects that the call event has ended or no object is near the source device, the source device displays a fourth display interface; the source device sends the fourth display interface and a recovery event to the target device, the recovery event including an identifier of the first window; in response to the recovery event, the target device stops displaying the occlusion image in the first window and displays the fourth display interface.

[0025] In one possible implementation, the source device can enter a screen-off state when it stops displaying the call interface corresponding to the call event.

[0026] Thirdly, this application provides an electronic device, which is a source device, comprising: a memory, a display screen, and one or more processors; the memory, the display screen, and the processors are coupled. The memory stores computer program code, which includes computer instructions; when the electronic device is running, the processor executes one or more computer instructions stored in the memory to cause the electronic device to perform the screen projection display method as described in any of the first or second aspects above.

[0027] Fourthly, this application provides an electronic device, which is a target device comprising: a memory, a display screen, and one or more processors; the memory, the display screen, and the processors are coupled. The memory stores computer program code, which includes computer instructions; when the electronic device is running, the processor executes one or more computer instructions stored in the memory to cause the electronic device to perform the screen projection display method as described in any one of the first or second aspects above.

[0028] Fifthly, this application provides a screen projection display system, including the aforementioned source device and destination device, wherein the source device and destination device can interact to execute the screen projection display method as described in any one of the first or second aspects above.

[0029] Sixthly, this application provides a computer-readable storage medium including computer instructions. When the computer instructions are executed on an electronic device, the electronic device causes the electronic device to perform the screen projection display method as described in any one of the first or second aspects above.

[0030] In a seventh aspect, this application provides a computer program product that, when run on an electronic device, causes the electronic device to perform the screen projection display method as described in any one of the first or second aspects above.

[0031] Understandably, the electronic devices, computer-readable storage media, and computer program products provided in the above aspects are all applied to the corresponding methods provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods provided above, and will not be repeated here. Attached Figure Description

[0032] Figure 1 Application scenarios of screen projection display methods in existing technologies Figure 1 ;

[0033] Figure 2 Application scenarios of screen projection display methods in existing technologies Figure 2 ;

[0034] Figure 3 This application provides an application scenario for a screen projection display method. Figure 1 ;

[0035] Figure 4 This application provides an application scenario for a screen projection display method. Figure 2 ;

[0036] Figure 5 Application scenarios of screen projection display methods in existing technologies Figure 3 ;

[0037] Figure 6 This application provides a schematic diagram of the structure of an electronic device. Figure 1 ;

[0038] Figure 7 This application provides a schematic diagram of the architecture of an operating system in an electronic device.

[0039] Figure 8 This application provides a schematic diagram of the principle of a screen projection display method. Figure 1 ;

[0040] Figure 9 This application provides a schematic diagram of the principle of a screen projection display method. Figure 2 ;

[0041] Figure 10 This application provides a schematic diagram of the principle of a screen projection display method. Figure 3 ;

[0042] Figure 11 This application provides a schematic diagram of the principle of a screen projection display method. Figure 4 ;

[0043] Figure 12 This application provides a schematic diagram of the principle of a screen projection display method. Figure 5 ;

[0044] Figure 13 This application provides an interactive flowchart of a screen projection display method according to an embodiment of the present application;

[0045] Figure 14 This application provides an application scenario for a screen projection display method. Figure 3 ;

[0046] Figure 15 This application provides an application scenario for a screen projection display method. Figure 4 ;

[0047] Figure 16 This application provides an application scenario for a screen projection display method. Figure 5 ;

[0048] Figure 17 This application provides an application scenario for a screen projection display method. Figure 6 ;

[0049] Figure 18 This application provides an application scenario for a screen projection display method. Figure 7 ;

[0050] Figure 19 This application provides an application scenario for a screen projection display method. Figure 8 ;

[0051] Figure 20 This application provides a schematic diagram of the structure of an electronic device. Figure 2 ;

[0052] Figure 21 This application provides a schematic diagram of the structure of an electronic device. Figure 3 . Detailed Implementation

[0053] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this embodiment, unless otherwise stated, "a plurality of" means two or more.

[0054] Currently, multi-device collaboration is a common practice for both work and entertainment. When multiple devices work together, the source device (or source end) can establish a connection with the target device (or sink end). For example, the source device can establish a Wi-Fi connection, Bluetooth connection, or P2P (peer-to-peer) connection with the target device. Furthermore, the source device can project images, documents, audio, video, or applications from its device to the target device for display or playback via the MiraCast protocol or the DLNA (Digital Living Network Alliance) protocol, allowing users to utilize the relevant functions provided by the source device on the target device.

[0055] For example, taking a mobile phone as the source device and a PC as the target device, such as... Figure 3 As shown, an electronic tag 201 can be set on the PC. The electronic tag 201 can also be called an NFC (near field communication) tag or NFC patch. The electronic tag 201 generally contains a coil, and the PC's device information can be pre-written into the coil of the electronic tag 201 at the factory. For example, this device information may include one or more of the following: the PC's name, Bluetooth MAC (media access control) address, or IP address.

[0056] When a user needs to project data such as applications and documents from their mobile phone onto a PC, they can enable the phone's NFC function and bring the phone close to or touch the electronic tag 201 on the PC. In this way, when the phone and electronic tag 201 are close to each other, the phone can read the PC's device information from the electronic tag 201 by emitting near-field signals. Then, the phone can establish a wireless communication connection with the PC based on the PC's device information. For example, this wireless communication connection can be a Bluetooth connection, a Wi-Fi connection, or a Wi-Fi P2P connection, etc., and this application embodiment does not impose any limitations on this.

[0057] Of course, in addition to the above-mentioned "tap to connect" method, the mobile phone can also connect to the PC by searching for nearby devices or by dragging gestures. Alternatively, the mobile phone can connect to the PC through other communication technologies such as UWB (Ultra Wide Band). This application embodiment does not impose any restrictions on this.

[0058] After the mobile phone and PC establish a wireless communication connection, it still works as before. Figure 3As shown, the mobile phone can transmit its current display interface 202 to the PC in real time via an established wireless communication connection. For example, the mobile phone can transmit the current display interface 202 to the PC in real time as a video stream. The PC can display the display interface 202 through window 203, which may include a control bar 204, including buttons for maximizing, minimizing, and closing. Taking the display interface 202 as the mobile phone's desktop as an example, after the PC displays the mobile phone's desktop in window 203, the user can use the various functions provided by the mobile phone in window 203.

[0059] For example, a user can use the PC's keyboard or mouse to type "open video app" in window 203. The PC can then send the user's input to the phone, triggering the phone to respond and project the video app onto the PC. For instance, the phone can run the video app in the background, such as... Figure 4 As shown, the mobile phone can send the display data generated when running a video app (i.e., the video app's display interface 205) to the PC as a video stream. The PC can create a new window 206 and display the video app's display interface 205 in window 206. At this time, the PC can not only display the desktop application running in the foreground on the mobile phone through window 203, but also display the video app running in the background on the mobile phone through window 206. In this way, the mobile phone (i.e., the source device) can project multiple applications to the PC (i.e., the target device) in a multi-window format.

[0060] Still Figure 4 As shown, the content displayed in window 203 on the PC is synchronized with the content displayed on the mobile phone. Therefore, as... Figure 5 As shown, if a user calls contact Sam using the phone's calling app, the phone can display call interface 401. Simultaneously, the phone can send call interface 401 to the PC, which then updates it in window 203. Furthermore, when the phone detects that the user has called contact Sam, or after the user answers a call with contact Sam, it can use a proximity sensor to detect if there is an object near the earpiece. If an object is detected near the earpiece, the phone stops drawing call interface 401 and enters a screen-off state. At this time, the phone also stops sending display data to the PC, preventing the PC from displaying the relevant application interfaces in windows 203 and 206, and the user cannot use the relevant functions in windows 203 and 206 normally on the PC.

[0061] Alternatively, after the phone stops drawing the aforementioned call interface 401, it can instruct the PC to display preset overlays, animations, text, or images in windows 203 and 206 to indicate to the user that the phone is in a call. Similarly, the user will not be able to use the relevant functions in windows 203 and 206 normally on the PC at this time.

[0062] To address this, this application provides a screen projection method. Taking a mobile phone as the source device, when the mobile phone detects a call, if it is projecting multiple application windows onto the target device, and the mobile phone detects an object near the earpiece, it can control the target device to continue displaying the windows other than the call interface normally. For example, in... Figure 5 In the scenario shown, the mobile phone can control the PC to stop displaying the interface in window 203 when the screen is off, but continue to display the video app's interface in window 206. In this way, when multiple applications from the mobile phone are projected onto the target device, applications unrelated to the current call event can be displayed normally on the target device to provide relevant functions to the user, without affecting the user's use of applications unrelated to the current call event on the target device, thereby improving the user experience in screen projection and call scenarios.

[0063] Let's take a mobile phone as the source device in the above screen mirroring scenario as an example. Figure 6 A schematic diagram of the mobile phone structure is shown.

[0064] like Figure 1 As shown, a mobile phone may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, etc.

[0065] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the mobile phone. In other embodiments of this application, the mobile phone may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0066] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0067] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0068] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0069] The wireless communication function of a mobile phone can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor.

[0070] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the mobile phone can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0071] The mobile communication module 150 can provide solutions for wireless communication applications in mobile phones, including 2G / 3G / 4G / 5G. The mobile communication module 150 may include at least one filter, switch, power amplifier, low-noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0072] The wireless communication module 160 can provide solutions for wireless communication applications in mobile phones, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0073] In some embodiments, antenna 1 of the mobile phone is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling the mobile phone to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS may include Global Positioning System (GPS), Global Navigation Satellite System (GLONASS), BeiDou Navigation Satellite System (BDS), Quasi-Zenith Satellite System (QZSS), and / or Satellite Based Augmentation Systems (SBAS).

[0074] The mobile phone implements its display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0075] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the mobile phone may include one or N displays 194, where N is a positive integer greater than 1.

[0076] Taking OLED displays as an example, an OLED display can contain multiple OLED pixel units arranged in an array. For example... Figure 2 As shown, each OLED pixel unit includes a cathode 201, an anode 202, and an electron transport layer 203, a hole transport layer 204, and a light-emitting layer 205 located between the cathode 201 and the anode 202. The cathode 201 can be a metal electrode, and the anode 202 can be an ITO (indium tin oxide) transparent electrode.

[0077] After a driving voltage V is input to the cathode 201 and anode 202, electrons are transported from the cathode 201 to the electron transport layer 203 under the action of the driving voltage V, and holes are injected from the anode 202 into the hole transport layer 204. When these two electrons meet in the light-emitting layer 205, excitons are generated, exciting the light-emitting molecules in the light-emitting layer 205, which then generate a light source after radiation. When the driving voltage V is different, the corresponding OLED pixel units can be excited to display different colors and brightness. In this way, each OLED pixel unit in the OLED display can display the corresponding image under different driving voltages.

[0078] The organic materials in the electron transport layer 203, hole transport layer 204, and light-emitting layer 205 gradually age during use. The image retention phenomenon in OLED displays is actually caused by a fixed OLED pixel unit displaying the same static image for a long time. This causes the organic materials in this pixel unit to wear down more than those in other locations, resulting in a faster decline in luminous efficiency and leaving an image retention on the OLED display.

[0079] Mobile phones can achieve shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0080] The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0081] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the mobile phone may include one or N cameras 193, where N is a positive integer greater than 1.

[0082] Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when a mobile phone is selecting a frequency, the DSP performs Fourier transforms on the frequency energy.

[0083] Video codecs are used to compress or decompress digital video. A mobile phone can support one or more video codecs. This allows the phone to play or record videos in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0084] The external storage interface 120 can be used to connect an external storage card, such as a Micro SD card, to expand the phone's storage capacity. The external storage card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external storage card.

[0085] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various mobile phone functions and data processing by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during mobile phone use (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0086] Mobile phones can perform audio functions, such as music playback and recording, through components like the audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0087] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0088] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. Mobile phones can use the speaker 170A to listen to music or make hands-free calls.

[0089] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When answering a phone call or voice message, the receiver 170B can be brought close to the user's ear to hear the voice.

[0090] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. A mobile phone can have at least one microphone 170C. In some embodiments, a mobile phone can have two microphones 170C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, a mobile phone can have three, four, or more microphones 170C, enabling sound signal collection, noise reduction, sound source identification, and directional recording, among other functions.

[0091] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0092] The sensor module 180 may include pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, bone conduction sensors, etc.

[0093] Of course, the mobile phone may also include a charging management module, a power management module, a battery, buttons, indicators, and one or more SIM card interfaces, etc., and this application embodiment does not impose any restrictions on this.

[0094] The software system of the aforementioned mobile phone can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment uses a layered architecture. Taking the system as an example, we will illustrate the software structure of the mobile phone.

[0095] Figure 7 This is a software structure block diagram of a mobile phone according to an embodiment of this application.

[0096] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, [the following is omitted as the text is incomplete and likely refers to a specific implementation or feature]. The system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0097] 1. Application Layer

[0098] The application layer can include a series of application packages.

[0099] like Figure 7 As shown, apps such as call, memo, browser, contacts, camera, gallery, calendar, map, Bluetooth, music, video, and SMS can be installed in the application layer.

[0100] 2. Application Framework Layer

[0101] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0102] In the embodiments of this application, such as Figure 7 As shown, the application framework layer may include ActivityManagerService (AMS) and WindowManagerService (WMS).

[0103] AMS (Activity Management System) can be used to manage the lifecycle of an application. Applications typically run in the operating system as Activities. For each Activity, there is a corresponding application record (ActivityRecord) in the Activity Manager, which records the state of the application's Activities. The Activity Manager can use this ActivityRecord as an identifier to schedule the application's Activity processes.

[0104] WMS can be used to manage graphical user interface (GUI) resources used on mobile phone screens, including window creation and destruction, window display and hiding, window layout, focus management, and input method and wallpaper management.

[0105] In this embodiment of the application, taking the mobile phone running desktop (also known as desktop application, launcher, etc.) as an example, after the mobile phone starts running the desktop, as follows: Figure 8 As shown, AMS can create a corresponding application stack for the desktop, such as stack 1. Stack 1 can include one or more Activities that need to be executed on the desktop. When an Activity in stack 1 is executed, WMS can draw the corresponding display interface in real time in the phone's default display module (e.g., display 0). Then, WMS can output the display interface drawn in display 0 to the phone's monitor for display, thus presenting the desktop to the user on the phone screen. Here, the display module can also be called a virtual screen or virtual display, etc. The phone can use the display module as a canvas that occupies a certain amount of storage space. For example, display 0 can occupy storage space 1, area 1 in display 0 corresponds to a storage address in storage space 1, and area 2 in display 0 corresponds to another storage address in storage space 1, that is, the display module can have a location attribute.

[0106] If the phone has its screen mirroring function enabled and has established a connection with the target device (such as a PC), it will still work as before. Figure 8 As shown, WMS can also output the display interface drawn in display 0 to the PC, so that the PC can create a corresponding window to display the mobile phone desktop, thereby realizing the screen projection function in multi-device collaborative scenarios.

[0107] In this screen mirroring scenario, if the PC detects that the user has opened an application (such as a video app) on the phone's home screen, the PC can send a corresponding screen mirroring command to the phone, instructing the phone to project the video app onto the PC's display. At this point, if... Figure 9 As shown, AMS can create a corresponding Stack 2 for the video app. Stack 2 can include one or more Activities that the video app needs to execute. At this time, since the user has not opened the video app on the phone, the phone's AMS can run the video app in the background, without displaying the video app's interface.

[0108] Still Figure 9 As shown, after AMS creates stack 2, it needs to execute the Activities in stack 2 and stack 1 in parallel. At this time, WMS can draw the display interface of the video app and the desktop in real time in the phone's default display module (e.g., display 0). In this case, area 1 of display 0 includes the desktop display interface, and area 2 of display 0 includes the video app display interface. After WMS sends the display data from display 0 to the PC, the PC can display the desktop display interface from area 1 in window 1 and the video app display interface from area 2 in window 2. In this way, multiple applications on the phone can be projected onto the PC in a multi-window format.

[0109] Or, such as Figure 10 As shown, after AMS creates stack 2, WMS can also create a corresponding display module (e.g., display 1) for stack 2. Then, WMS can draw the video app's display interface in real time in display 1. Simultaneously, WMS can continue to draw the desktop display interface in real time in display 0. Furthermore, WMS can send the display data from display 0 and display 1 to the PC. The PC can then display the desktop interface from display 0 in window 1 and the video app's display interface from display 1 in window 2, thus enabling the projection of multiple applications from the phone onto the PC.

[0110] Regardless of the method used to project multiple apps from a phone to a PC, the display interface within a specific window on the PC is synchronized with the phone's display interface. Subsequently, the phone may detect call events reported by the calling app, which may include answering or making calls to contacts. At this point, if... Figure 11 As shown, in addition to creating a corresponding stack 3 for the calling app, the phone's AMS can also call WMS to draw the display interface of the calling app in area 1 of display 0, where the desktop was originally drawn, and output the display interface of the calling app to the phone and PC in real time.

[0111] Furthermore, after detecting a call event reported by the calling app, the phone can also use a proximity sensor located near the earpiece to detect if an object is approaching the earpiece. If an object is detected approaching the earpiece, it indicates that someone is likely holding their ear close to the earpiece while answering a call. The phone can then query the area in the display module used to draw the display interface of the calling app.

[0112] For example, each application's runtime stack corresponds to an identifier, such as a task ID. The phone can obtain stack 3 corresponding to the currently calling app from the AMS (Application Management System). Then, the phone can query the WMS (Web Management System) based on the task ID of stack 3 to find the display interface of the calling app, which is drawn in area 1 of display 0. Furthermore, as... Figure 12 As shown, the phone can instruct WMS to stop rendering the relevant display interface in area 1, making area 1 a black screen. Simultaneously, WMS can continue rendering the display interface of the running video app in display 0 (or display 1), meaning the display interface of non-call apps can be rendered normally. In this way, no content is output to the phone and PC in area 1 of display 0, allowing the phone and PC to stop displaying the call interface, preventing users from accidentally touching the phone screen while answering a call, and preventing user privacy from being leaked through the phone and PC. Meanwhile, the display interface of the video app in display 0 (or display 1) can be output normally to the PC, allowing applications unrelated to the current call to be displayed normally to provide relevant functions to the user, thereby improving the user experience in screen mirroring and call scenarios.

[0113] The specific display process of multi-application screen mirroring between mobile phones (source devices) and PCs (target devices) will be described in detail in subsequent embodiments, so it will not be repeated here.

[0114] In addition, the application framework layer may also include power management services, content services, view systems, resource management services, notification management services, etc., and this application embodiment does not impose any restrictions on this.

[0115] 3. Android runtime and system libraries

[0116] The Android runtime consists of core libraries and a virtual machine. The Android runtime is responsible for scheduling and managing the Android system.

[0117] The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

[0118] The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

[0119] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0120] The Surface Manager manages the display subsystem and provides fusion of 2D and 3D layers for multiple applications. The Media Library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG. The 3D Graphics Processing Library implements 3D graphics drawing, image rendering, compositing, and layer processing. The 2D Graphics Engine is the drawing engine for 2D graphics.

[0121] 4. Kernel layer

[0122] The kernel layer is the layer between hardware and software. The kernel layer includes at least display drivers, camera drivers, audio drivers, sensor drivers, etc., but this application embodiment does not impose any limitations on this.

[0123] The following example, using a mobile phone as the source device and a PC as the target device in a screen mirroring scenario, will be used to illustrate in detail a screen mirroring display method provided by the embodiments of this application, in conjunction with the accompanying drawings.

[0124] like Figure 13 As shown, a screen projection method provided in this application embodiment may include the following steps S1201-S1209:

[0125] S1201, The mobile phone establishes a connection with the PC.

[0126] In some embodiments, the mobile phone and PC can establish a wired connection. For example, the mobile phone and PC can establish a wired connection via a data cable.

[0127] In other embodiments, the mobile phone and PC can establish a wireless connection.

[0128] For example, a user can tap their mobile phone against an electronic tag set on a PC. The phone then reads the device information stored in the tag, such as the PC's IP address and Bluetooth MAC address. The phone can then establish a connection with the PC using the NFC protocol based on this device information.

[0129] For example, both the mobile phone and the PC have Bluetooth and Wi-Fi enabled. The PC can broadcast Bluetooth signals to discover nearby devices, such as displaying a list of discovered devices, which may include the identifier of the mobile phone discovered by the PC. Furthermore, during the device discovery process, the PC can exchange connection information, such as IP addresses, with the discovered devices. Then, after the PC receives a user's selection of the mobile phone's identifier from the displayed device list, the PC can establish a connection with the mobile phone using the Wi-Fi protocol based on the mobile phone's IP address.

[0130] For example, if both the mobile phone and the PC are connected to the cellular network, and both are logged into the same account (such as a Huawei account), then the mobile phone and the PC can establish a connection based on the cellular network using that Huawei account.

[0131] After a successful connection is established between the mobile phone and the PC, they can be used together. To improve the efficiency of collaborative use, users can use an input device, such as a PC mouse, to control both the PC and the mobile phone. Of course, users can also use a PC mouse to control the PC, and users can use touch to control the mobile phone; this application embodiment does not impose any limitations on these methods.

[0132] S1202, The mobile phone projects the first display interface of the first application onto the first window of the PC, and the mobile phone simultaneously displays the first display interface in the first window.

[0133] After a mobile phone establishes a connection with a PC, in some embodiments, the mobile phone can automatically project the display interface of the first application (i.e., the first display interface) onto the PC for display. For example, as Figure 14As shown, after the mobile phone establishes a connection with the PC, the mobile phone is running its desktop. At this time, the mobile phone's WMS can draw the desktop display interface 1301 in real time in the default display 0. The mobile phone's WMS can output the display content in display 0 to the mobile phone's monitor in real time, so that the mobile phone can display the display interface 1301 through the monitor.

[0134] Simultaneously, after the mobile phone and PC establish a connection, the mobile phone's WMS can also output the display content in display 0 to the PC in real time. For example, the mobile phone's WMS can encode the display content in display 0 in the form of a video stream, and then send the encoded display content to the PC. After receiving the display content, the PC can decode it to obtain the decoded display content in display 0. Then, the PC can create a first window 1302 and display the display content in display 0 (i.e., display interface 1301) in the first window 1302. In addition to the display interface 1301, the first window 1302 can also include components such as a control bar, status bar, or toolbar. In other words, the PC can display the display interface of the application projected from the mobile phone in the form of a window.

[0135] It should be noted that the specific content of the display interface 1301 on the mobile phone and the display interface 1301 displayed in the first window 1302 on the PC can be the same, but the shape, size, position, layout, resolution or DPI (Dots Per Inch) and other display parameters of the two can be different. This application embodiment does not impose any restrictions on this.

[0136] After the mobile phone projects its running desktop display interface 1301 onto the first window 1302 of the PC, the content displayed on the mobile phone and the first window 1302 on the PC are synchronized. If the mobile phone detects that the user has entered an operation into the display interface 1301 (such as clicking an application icon, opening the control center, etc.), the mobile phone can respond to the operation and update the display content in display 0 according to the above method, thereby synchronously updating the display content in the first window 1302 on both the mobile phone and the PC.

[0137] S1203: The mobile phone projects the second display interface of the second application onto the second window of the PC, while the mobile phone continues to display the first display interface in the first window synchronously.

[0138] In some embodiments, a mobile phone can project multiple applications on the phone to a PC in a multi-window format, allowing users to enjoy the services provided by multiple applications on the mobile phone on the PC at the same time.

[0139] Continuing with the example of projecting the mobile phone's desktop display 1301 onto the PC's first window 1302, if a user wants to open a video app on their mobile phone on the PC, they can input the command to open the video app into the PC's first window 1302. For example... Figure 15 As shown, if the PC detects that the user double-clicks the video app icon 1400 in the first window 1302 with the PC mouse, the PC can send the corresponding operation information to the mobile phone in the screen casting command 1. This screen casting command 1 instructs the mobile phone to project the video app onto the PC for display. For example, the operation information may include the video app's identifier, or it may include the coordinates of the mouse position when the double-click occurs. After receiving the screen casting command 1, the mobile phone can determine whether to send the video app's display interface to the PC for display based on the video app's identifier or the mouse position coordinates in the screen casting command 1.

[0140] Furthermore, as Figure 15 As shown, the phone can start running a video app in the background. At this time, the phone's WMS needs to render both the desktop display interface 1301 running in the foreground and the video app display interface 1401 running in the background in real time. For example, the phone's WMS can divide display 0 into different areas, each area corresponding to a different application stack, i.e., each area corresponds to a different application's task ID. For instance, the phone's WMS can render the desktop display interface 1301 in area 1 of display 0, and the video app display interface 1401 in area 2 of display 0, where the desktop's task ID can be the first task ID, and the video app's task ID can be the second task ID. Furthermore, after the phone's WMS outputs the content displayed in display 0 to the PC in real time, if the PC extracts the two display interfaces in area 1 and area 2, the PC can display the display interface 1301 in area 1 in the first window 1302. Additionally, the PC can create a second window 1402 for the content displayed in area 2 (i.e., the video app's display interface 1401), displaying the display interface 1401 in area 2 in the second window 1402. In this way, the phone's desktop and the video app can be projected onto the PC through two windows.

[0141] In the above implementation, the mobile phone's WMS records the correspondence between different applications, different areas in display 0, and different windows on the PC. For example, there is a correspondence between the first task ID on the desktop, area 1 in display 0, and the first window 1302 on the PC. Similarly, there is a correspondence between the second task ID of the video app, area 2 in display 0, and the second window 1402 on the PC.

[0142] In other embodiments, after the mobile phone determines that it needs to send the video app's display interface to the PC, the phone's WMS can create a new display module, such as display 1. Then, the phone's WMS can continue drawing the desktop display interface 1301 in display 0, and the phone's WMS can draw the video app's display interface 1401 in display 1. Subsequently, the phone's WMS can output the display content in display 0 and display 1 to the PC in real time. After receiving the display content in display 0 (i.e., display interface 1301), the PC can update the display content in display 0 in the first window 1302. After receiving the display content in display 1 (i.e., display interface 1401), the PC can create a second window 1402 corresponding to display 1 and update the display content in display 1 in the second window 1402. Thus, as... Figure 15 As shown, the desktop and video apps on the phone can also be projected to the PC through two windows.

[0143] In the above implementation, the mobile phone's WMS records the correspondence between different applications, different display modules, and different windows on the PC. For example, there is a correspondence between the first task ID and display 0 on the desktop and the first window 1302 on the PC. Similarly, there is a correspondence between the second task ID and display 1 of the video app and the second window 1402 on the PC.

[0144] Subsequently, if the PC detects that the user has input an operation (such as a return operation, a pause operation, etc.) into the display interface 1401 of the second window 1402, the PC can send the identifier of the second window 1402 and the corresponding operation information to the mobile phone in the screen casting command 2. After receiving the screen casting command 2, the mobile phone can update the display content in display 1 (or area 2 in display 0) corresponding to the second window 1402 in response to the user's input operation according to the above correspondence, thereby synchronously updating the display content in the second window 1402 on the PC.

[0145] In other embodiments, besides triggering the PC to display multiple applications from the mobile phone in a multi-window manner by operating the first window 1302 on the PC, the user can also input a preset gesture on the mobile phone to trigger the mobile phone to project multiple applications to the PC in a multi-window manner. For example, taking the mobile phone's desktop display interface 1301 as an example, if it is detected that the user drags the WeChat APP icon to a preset area of ​​the display interface 1301, or drags the WeChat APP icon out of the boundary of the display interface 1301, the mobile phone can run the WeChat APP in the background according to the above method, and send the drawn WeChat APP display interface to the PC, so that the PC can create a corresponding window to display the WeChat APP display interface.

[0146] Furthermore, the above embodiments illustrate the use of two windows on the PC to display the interfaces of two applications on the mobile phone. It is understood that if the user wishes to project more applications (e.g., 3 or 4 applications) from the mobile phone to the PC, the mobile phone and PC can continue to project multiple applications from the mobile phone onto the PC in a multi-window manner as described above. This application embodiment does not impose any restrictions on this.

[0147] S1204: The mobile phone detected a call event reported by the call app.

[0148] When a mobile phone projects multiple applications into a multi-window display on a PC, the screen on the phone remains synchronized with the screen in a single window on the PC. (The sentence is incomplete and requires further context.) Figure 15 As shown, the content displayed in the first window 1302 on the PC is synchronized with the content displayed on the mobile phone. For example, both the first window 1302 and the mobile phone display the desktop display interface 1301.

[0149] In this scenario, the mobile phone may detect a call event reported by the calling app. For example, this call event could be a user answering a call from a contact, or a user dialing a contact's number. After detecting such a call event, the phone's calling app can report it to the phone's PhoneManager. At this point, the PhoneManager can, on the one hand, transmit the call event to the phone's WMS, triggering the phone to continue executing step S1205 below; on the other hand, the PhoneManager can invoke the proximity sensor to execute step S1206 below.

[0150] S1205. In response to the above call event, the third display interface of the call APP is displayed in the first window of the PC and the mobile phone.

[0151] Taking the call event detected by the mobile phone as an example, such as the event of a user answering a call from a contact, for instance... Figure 16As shown, if the phone receives a call from contact Sam while displaying the desktop interface 1301, the phone can switch the foreground application from the desktop to the calling app. At this time, the phone's WMS can draw the calling app's display interface 1501 in the area (e.g., area 1) where the display interface 1301 was originally drawn in display 0. Furthermore, the phone's WMS can continue to draw the video app's display interface 1401 in the relevant display module according to the method described in step S1503.

[0152] Furthermore, such as Figure 16 As shown, after the mobile phone's display receives the call app's display interface 1501 from display 0, it can display the display interface 1501. After the PC receives the call app's display interface 1501 from display 0, it can display the call app's display interface 1501 in the first window 1302 that originally displayed the display interface 1301. That is, at this time, the first window 1302 corresponds to the call app's display interface 1501. At the same time, the PC can receive the video app's display interface 1401 from the relevant display module and continue to display the display interface 1401 in the second window 1402.

[0153] In other words, the PC's first window 1302 can dynamically display the mobile phone's display interface, while the PC's second window 1402 can continue to display the display interface of the projected application.

[0154] Subsequently, if the user clicks the answer button 1502 on the phone's display interface 1501, the calling app can generate a call event and report it to the phone's WMS via the call manager. At this time, the phone's WMS can continue to project the phone's real-time display interface onto the PC's first window 1302 using the method described above, and project the already projected video app's display interface onto the PC's second window 1402.

[0155] S1206. Mobile phone detection: Detect if there is an object near the earpiece.

[0156] In step S1206, if it is detected that the user clicks the answer button 1502 on the display interface 1501 of the mobile phone, the call APP reports the call event to the call manager. The call manager can also call the proximity sensor located near the earpiece of the mobile phone to detect whether there is an object approaching the earpiece. Of course, the mobile phone can also call the proximity sensor to detect whether there is an object approaching the earpiece before detecting the above-mentioned call event. This application embodiment does not impose any restrictions on this.

[0157] For example, the aforementioned proximity sensor can be a Hall sensor, photoelectric sensor, or other sensor capable of detecting distance. The proximity sensor can be placed near the phone's earpiece. The phone can call the Sensor.TYPE_PROXIMITY interface to invoke the proximity sensor to detect the distance between the phone and obstacles. When the detected distance between the phone and the obstacle is less than a preset value (e.g., 5cm), it indicates that the user is close to the phone's earpiece and is using it to answer the call. At this point, the phone can continue to execute step S1207, allowing the PC to normally display the display interface of applications unrelated to the current call event in the screen-casting scenario, while stopping the display of the interface where the call event is located.

[0158] Correspondingly, if the distance between the phone and the obstacle is greater than or equal to the aforementioned preset value, it indicates that the user is not currently using the phone's earpiece to answer the call. In this case, the phone can continue to display the call app's interface. Furthermore, the phone can continue to project the call app's interface onto the PC's first window 1302 using the method described above.

[0159] S1207. If an object is detected near the earpiece of the phone, the phone enters a screen-off state and instructs the PC to stop displaying the call app's interface in the first window.

[0160] In step S1207, after the mobile phone detects an object approaching the earpiece, in order to prevent accidental touches when the user answers a call and to prevent leakage of the user's privacy in the display interface during the call, the mobile phone's WMS can stop drawing the display interface of the call app in area 1 of display 0.

[0161] For example, the phone's WMS can first query the specific display module or a specific area within the display module corresponding to the currently calling app. For instance, since the WMS establishes a correspondence between the calling app, area 1 in display 0, and the first window 1302 on the PC, after the phone's WMS receives the call event, it can determine that the calling app's display interface is being drawn in area 1 of display 0 based on this correspondence. Furthermore, the phone's WMS can call the `stop render()` function to stop drawing in area 1 of display 0. For example, if the phone detects an object approaching the earpiece while drawing the calling app's interface in area 1 of display 0, the phone can stop drawing the calling interface. Similarly, if the phone is displaying the interface of another application during a call, meaning the phone is drawing the interface of another application in area 1 of display 0, and in this case, if an object is detected approaching the earpiece, the phone can also stop drawing the display interface in area 1.

[0162] Furthermore, such as Figure 17 As shown, after the mobile phone's display can no longer obtain display content from area 1 of display 0, it can enter a screen-off (or black screen) state. Furthermore, the mobile phone can send a notification event to the PC to instruct the PC to stop displaying the interface in the first window 1302. After receiving the notification event, since the PC also cannot obtain display content from area 1 of display 0, the PC can simultaneously enter a black screen state in the first window 1302, just like the mobile phone. In this way, the PC will not display information related to the call interface in the first window 1302, preventing the leakage of user privacy. The PC can display the control bar in the first window 1302 normally, or it can stop displaying the control bar in the first window 1302; this embodiment does not impose any restrictions on this.

[0163] At the same time, still as Figure 17 As shown, the mobile phone's WMS can continue to draw the video app's display interface 1401 in area 2 (or display 1) of display 0. Then, after the PC obtains the video app's display interface 1401 from area 2 (or display 1) of display 0, it can continue to display the video app's display interface 1401 in the corresponding second window 1402.

[0164] In this way, in multi-application screen mirroring scenarios, when the phone (i.e., the source device) detects a call event, the phone can choose not to display windows related to the call event (such as the first window 1302 mentioned above) on both the phone itself and the PC (i.e., the target device), thus avoiding accidental touches and privacy leaks during calls. However, the phone can continue to display windows unrelated to the call event normally on the PC, ensuring that the call event does not interfere with the user's use of other mirrored applications on the PC, thereby improving the user experience in both screen mirroring and call scenarios.

[0165] In other embodiments, when an object is detected approaching the phone's earpiece, the phone's WMS can continue to send the display interfaces in areas 1 and 2 of display 0 to the PC in real time. Simultaneously, the phone can send a notification event to the PC, which may include an identifier for the first window 1302 to notify the PC to stop displaying the display interface in the first window 1302. Upon receiving the notification event, the PC, as follows... Figure 18 As shown, a preset obscuring image 1701 can be displayed in the first window 1302 according to the identifier of the first window 1302 in the notification event. The obscuring image 1701 can be used to obscure all or part of the interface currently displayed in the first window 1302. The identifier of the first window 1302 can be the task ID of the call APP running in the first window 1302.

[0166] For example, after the PC obtains the corresponding display content (such as the call interface) in real time from area 1 of display 0, it can display the call interface in the first window 1302. When the PC receives the above-mentioned notification event, it can draw an obscuring image 1701 on top of the call interface according to the identifier of the first window 1302 in the notification event. In this way, the obscuring image 1701 can not only obscure the display interface of the underlying call APP to avoid accidental touches and privacy leaks during the call, but also include text, images or animations to prompt the user that a call is currently in progress. At this time, the mobile phone can still continue to draw the display interface 1401 of the video APP in area 2 of display 0 (or display 1) according to the above method, and project the display interface 1401 of the video APP onto the second window 1402 of the PC for display.

[0167] S1208. In response to the user's input to the second window, the PC updates the display interface in the second window.

[0168] like Figure 17 or Figure 18 As shown, when the phone detects a call event, although the phone cannot display the relevant interface of the call app in the first window 1302 of the PC for the user to operate, the video app that the phone was originally projecting in the second window 1401 of the PC will not be affected.

[0169] In this scenario, the user can normally perform related operations on the video app's display interface 1401 within the PC's second window 1401. For example, if the PC detects that the user inputs an operation (such as a back operation or a pause operation) into the display interface 1401 in the second window 1402, the PC can send the identifier of the second window 1402 and the received specific operation information to the mobile phone in the screen casting command 3. After receiving the screen casting command 3, the mobile phone can respond to the user's input operation and update the display content in display 1 (or area 2 in display 0) corresponding to the second window 1402, thereby synchronously updating the display content in the second window 1402 on the PC.

[0170] S1209. When the above-mentioned call event ends, or when no object is detected near the mobile phone's earpiece, the mobile phone and the PC's first window simultaneously display the fourth display interface.

[0171] In step S1206 above, after acquiring a call event, the mobile phone continuously detects whether any object is approaching the earpiece. If an object is detected approaching the earpiece for a period of time during the call and then moves away from the earpiece, it indicates that the user has moved away from the earpiece during the call and may subsequently need to operate the phone screen. Alternatively, if a call end event is detected after a period of time following a call event reported by the call app, it indicates that the user has ended the call and may subsequently need to operate the phone screen.

[0172] In step S1209, if the end of the current call event is detected, or if no object is detected near the phone's earpiece, the phone can restore the display interface displayed before the current call event. At the same time, the first window 1302 of the PC can also restore the display interface displayed before the current call event (i.e., the fourth display interface, which may be the same as or different from the first display interface) in sync with the phone.

[0173] For example, after the call ends, the phone's AMS can restart the desktop in the foreground, and then the phone's WMS can redraw the desktop display interface 1301 in area 1 of display 0. Thus, as... Figure 15 As shown, after the mobile phone's display obtains the content displayed in area 1 of display 0, it can display the desktop display interface 1301 again on the mobile phone screen. After the PC obtains the content displayed in area 1 of display 0, it can display the desktop display interface 1301 again in the first window 1302.

[0174] For example, if no object is detected near the earpiece during a call, the phone's AMS can continue running the call app in the foreground. Then, the phone's WMS can redraw the call app's call interface 1801 in area 1 of display 0. Thus, as... Figure 19 As shown, after the mobile phone's display obtains the content displayed in area 1 of display 0, it can display the call interface 1801 of the calling app on the mobile phone screen. After the PC obtains the content displayed in area 1 of display 0, it can display the call interface 1801 of the calling app in the first window 1302.

[0175] For example, when the call ends or no object is detected near the phone's earpiece, the phone can send a recovery event to the PC. Similar to the notification event mentioned above, this recovery event can include the identifier of the first window 1302. Then, the PC can respond to this recovery event, removing the obstructing image 1701 displayed in the first window 1302, thereby restoring the display interface displayed before the call to the first window 1302.

[0176] While the mobile phone's WMS is drawing the aforementioned display interface 1301 (or the aforementioned call interface) in area 1 of display 0, it can also continue to draw the video app's display interface 1401 in area 2 of display 0 (or display 1). Furthermore, after the PC obtains the display content in area 2 of display 0 (or display 1), it can display the desktop display interface 1401 in the second window 1402.

[0177] In this way, in a multi-application screen mirroring scenario, when the call ends or no object is detected near the phone's earpiece, the phone (i.e., the source device) can restore the display content in the windows related to the call event on both the phone and the PC (i.e., the target device), allowing the user to continue using the multiple applications projected from the phone to the PC on the PC.

[0178] Furthermore, the above embodiments use mobile phones as the source device and PCs as the target device in the screen projection scenario as examples. It is understood that the source device in the screen projection scenario can also be a tablet computer, a watch, or other electronic device with call function, and the target device can also be a TV or a tablet computer or other electronic device with display function. This application embodiment does not impose any restrictions on this.

[0179] It should be noted that the above embodiments illustrate a specific method for implementing distributed shooting functionality among various functional modules using the Android system as an example. It is understood that the same method can also be implemented in other operating systems (such as HarmonyOS). As long as the functions implemented by each device and functional module are similar to those in the embodiments of this application, they fall within the scope of the claims of this application and their equivalents.

[0180] like Figure 20 As shown in the illustration, this application discloses an electronic device, which can be the aforementioned source device (e.g., a mobile phone). Specifically, the electronic device may include: a touchscreen 2001, which includes a touch sensor 2006 and a display screen 2007; one or more processors 2002; a memory 2003; a communication module 2008; one or more application programs (not shown); and one or more computer programs 2004. These devices can be connected via one or more communication buses 2005. The one or more computer programs 2004 are stored in the memory 2003 and configured to be executed by the one or more processors 2002. The one or more computer programs 2004 include instructions that can be used to perform the relevant steps executed by the source device in the above embodiments.

[0181] like Figure 21As shown in the illustration, this application discloses an electronic device, which can be the aforementioned target device (e.g., a PC). Specifically, the electronic device may include: a display screen 2101; one or more processors 2102; a memory 2103; a communication module 2108; one or more application programs (not shown); and one or more computer programs 2104. These devices can be connected via one or more communication buses 2105. The one or more computer programs 2104 are stored in the memory 2103 and configured to be executed by the one or more processors 2102. The one or more computer programs 2104 include instructions that can be used to perform the relevant steps executed by the target device in the above embodiment.

[0182] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0183] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0184] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0185] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0186] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0187] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A screen projection display method, characterized in that, include: The source device projects the first display interface of the first application onto the first window of the target device, and the source device synchronously displays the content displayed in the first window; The source device projects the second display interface of the second application onto the second window of the target device, while the source device continues to synchronously display the content displayed in the first window; When the source device detects a call event, it checks whether an object is approaching the source device. If an object approaches the source device, the source device stops displaying the call interface corresponding to the call event in the first display interface, and triggers the target device to display a preset occlusion image in the first window. The occlusion image is used to occlude part or all of the call interface in the first window. After the source device triggers the target device to display a preset occlusion image in the first window, the method further includes: In response to an operation input by the user into the second window, the source device receives operation information sent by the target device; The source device updates the second display interface to the third display interface according to the operation information; The source device sends the third display interface to the target device, triggering the target device to display the third display interface in the second window.

2. The method according to claim 1, characterized in that, The source device triggers the target device to display a preset occlusion image in the first window, including: The source device sends a notification event to the target device, the notification event including the identifier of the first window, triggering the target device to respond to the notification event and draw the occlusion image in the first window above the call interface.

3. The method according to claim 1, characterized in that, If an object approaches the source device, the method further includes: The source device continues to send the second display interface to the target device, triggering the target device to continue displaying the second display interface in the second window.

4. The method according to claim 1, characterized in that, The source device projects the first display interface of the first application onto the first window of the target device for display, including: The source device draws the first display interface of the first application in the first virtual screen; The source device outputs the first display interface in the first virtual screen to the display of the target device and the display of the source device; Wherein, the source device projects the second display interface of the second application onto the second window of the target device for display, including: The source device draws the second display interface of the second application in the second virtual screen; The source device outputs the second display interface in the second virtual screen to the target device.

5. The method according to claim 1, characterized in that, The source device projects the first display interface of the first application onto the first window of the target device for display, including: The source device draws the first display interface of the first application within a first area of ​​the first virtual screen; The source device outputs the first display interface in the first area to the display of the target device and the display of the source device; Wherein, the source device projects the second display interface of the second application onto the second window of the target device for display, including: The source device draws the second display interface of the second application in the second area of ​​the first virtual screen; The source device outputs the second display interface in the second area to the target device.

6. The method according to any one of claims 1-5, characterized in that, The obscured image includes a notification message that indicates the user is on a call.

7. The method according to any one of claims 1-6, characterized in that, After the source device triggers the target device to display a preset occlusion image in the first window, the method further includes: When the source device detects that the call event has ended or no object is near the source device, the source device and the target device simultaneously display a fourth display interface in the first window. The fourth display interface may be the same as or different from the first display interface.

8. The method according to any one of claims 1-7, characterized in that, When the source device detects a call event, it detects whether an object is approaching the source device, including: When the source device detects a call event, the source device uses a proximity sensor to detect the distance between the obstacle and the source device; When the distance between the obstacle and the source device is less than a preset value, the source device determines that an object is approaching the source device; When the distance between the obstacle and the source device is greater than or equal to a preset value, the source device determines that no object is approaching the source device.

9. The method according to any one of claims 1-8, characterized in that, The call events include: the event of answering a call from a contact, or the event of dialing a call from a contact.

10. A screen projection display method, characterized in that, include: The source device displays the first display interface of the first application and sends the first display interface to the target device. The target device receives the first display interface and displays the first display interface in a first window; When the source device displays the first display interface, it sends the second display interface of the second application to the target device. The target device receives the second display interface and displays the second display interface in a second window; When the source device detects a call event, it checks whether an object is approaching the source device. If an object approaches the source device, the source device stops displaying the call interface corresponding to the call event in the first display interface, and the source device sends a notification event to the target device, the notification event including the identifier of the first window; In response to the notification event, the target device displays a preset occlusion image in the first window, the occlusion image being used to obscure part or all of the call interface in the first window; After the target device displays the preset occlusion image in the first window, the method further includes: The target device receives a first operation input by the user into the second window; The target device sends the first operation information corresponding to the first operation to the source device; The source device updates the second display interface to the third display interface according to the first operation information, and sends the third display interface to the target device; The target device displays the third display interface in the second window.

11. The method according to claim 10, characterized in that, After the target device receives the first display interface and displays the first display interface in the first window, the method further includes: The target device receives a second operation input by the user into the first window; The target device sends the second operation information corresponding to the second operation to the source device; Wherein, when the source device displays the first display interface, it sends a second display interface of the second application to the target device, including: When the source device displays the first display interface, it runs the second application in the background according to the second operation information; The source device sends the second display interface of the second application to the target device.

12. The method according to any one of claims 10-11, characterized in that, After the target device displays the preset occlusion image in the first window, the method further includes: When the source device detects that the call event has ended or no object is near the source device, the source device displays a fourth display interface; The source device sends the fourth display interface and the recovery event to the target device, wherein the recovery event includes the identifier of the first window; In response to the recovery event, the target device stops displaying the obstructing image in the first window and displays the fourth display interface.

13. The method according to any one of claims 10-12, characterized in that, When the source device stops displaying the call interface corresponding to the call event, the source device enters a screen-off state.

14. An electronic device, characterized in that, The electronic device is a source device, and the source device includes: Display screen; One or more processors; Memory; Communication module; The memory stores one or more computer programs, each including instructions that, when executed by the source device, cause the source device to perform the screen projection method as described in any one of claims 1-9.

15. An electronic device, characterized in that, The electronic device is the target device, and the target device includes: Display screen; One or more processors; Memory; Communication module; The memory stores one or more computer programs, each including instructions that, when executed by the target device, cause the target device to perform the screen projection method as described in any one of claims 10-13.

16. A screen projection display system, characterized in that, The system includes the electronic device as claimed in claim 14, and the electronic device as claimed in claim 15.

17. A computer-readable storage medium storing instructions, characterized in that, When the instruction is executed on the electronic device, the electronic device performs the screen projection display method as described in any one of claims 1-9 or 10-13.

18. A computer program product containing instructions, characterized in that, When the computer program product is run on an electronic device, it causes the electronic device to perform the screen projection display method as described in any one of claims 1-9 or 10-13.

Images