Screen casting control methods and display devices

Abstract

This application provides a screen casting control method and a display device. When the display device establishes a device identification interaction link with an electronic device and receives a screen casting management request from the electronic device, it actively launches the target application for screen casting management. Based on the determined interconnection protocol corresponding to the electronic device, it completes pairing and connection with the electronic device, obtains the application list sent by the electronic device, and displays the identifiers of each application in the application list on the target application. Subsequently, in response to the user's selection operation of the application identifier, the display device sends the corresponding application launch command to the electronic device, allowing the user to directly select the application to be cast on the large screen. After receiving the video stream generated by the virtual display interface of the application corresponding to the launch command sent by the electronic device, the display device displays the screen corresponding to the video stream. This realizes the display device's active control and process management of the screen casting process, improves the interactive flexibility of the screen casting operation, and optimizes the user experience.

Description

Technical Field

[0001] This application relates to the field of display technology, and in particular to a screen projection control method and display device. Background Technology

[0002] With the development of technology, users tend to wirelessly project the screen content or media streams of electronic devices such as mobile phones, tablets, and laptops to display devices such as TVs and projectors to obtain large-screen display effects and a high-quality audio-visual experience.

[0003] In traditional screen mirroring technology architecture, the screen mirroring process is led by the electronic device, requiring the user to manually initiate the search and pairing operation on the electronic device side; while the display device only acts as a passive receiver, passively waiting for connection requests from the terminal, as well as the screen content, media streams, and other screen mirroring data to be mirrored. The function of the display device is to decode the screen mirroring data and complete the display.

[0004] However, this passive technical architecture has technical drawbacks, such as a cumbersome device pairing process that requires users to manually complete multiple steps, resulting in a poor user experience. Summary of the Invention

[0005] This application provides a screen projection control method and display device, which can solve the passive limitation of traditional screen projection technology architecture dominated by electronic devices.

[0006] In a first aspect, a screen casting control method is provided, applied to a display device, comprising: upon establishing a device identification interaction link with a first electronic device and receiving a first request for screen casting management sent by the first electronic device, launching a target application for screen casting management, and determining a first interconnection protocol corresponding to the first electronic device based on the device information of the first electronic device; the first request carrying the device information of the first electronic device; pairing and connecting with the first electronic device based on the first interconnection protocol; receiving a first application list sent by the first electronic device, and displaying the identifiers of applications in the first application list on a first page of the target application; the first application list includes a first application, which is an application installed on the first electronic device; in response to a user's selection operation of the first application identifier, sending a first instruction to the first electronic device, so that the first electronic device, while displaying a second application on the foreground page, simultaneously launches the first application in the background, records and generates a video stream of the virtual display interface of the first application; receiving the video stream sent by the first electronic device, and displaying the screen corresponding to the video stream on a second page of the target application.

[0007] In screen mirroring scenarios, establishing an identification and interaction link between the display device and the electronic device means that both parties complete the mutual identification of the device's identity.

[0008] It should be noted that the interconnection protocols corresponding to different device information may differ. The display device can determine the first interconnection protocol corresponding to the first electronic device based on the device information of the first electronic device, and complete the pairing connection with the first electronic device based on the determined first interconnection protocol, thereby establishing a stable communication link.

[0009] The screen casting control method provided in this application uses the display device as the control center. When the display device establishes a device identification interaction link with the electronic device and receives a screen casting management request from the electronic device, it actively launches the target application for screen casting management. Based on the determined interconnection protocol of the electronic device, it completes pairing and connection with the electronic device, obtains the application list sent by the electronic device, and displays the identifiers of each application in the application list on the target application. Subsequently, in response to the user's selection of the application identifier, the display device sends the corresponding application launch command to the electronic device, allowing the user to directly select the application to be cast on the large screen. After receiving the video stream generated by the virtual display interface of the application corresponding to the launch command sent by the electronic device, the display device displays the screen corresponding to the video stream. This achieves active control and process management of the screen casting process by the display device, improves the interactive flexibility of screen casting control, and optimizes the user experience.

[0010] In one possible implementation, after receiving the first application list sent by the first electronic device, the method further includes: storing the first application list in a preset format; and displaying the identifiers of the applications in the first application list on the first page of the target application, which means: displaying the identifiers of the applications in the first application list in a preset display format on the first page of the target application.

[0011] For example, the display device stores the target application lists (first application list, second application list) sent by each electronic device into the target database in a preset format.

[0012] The screen projection control method provided in this application adopts a unified preset format to standardize the data structure of the application list, avoiding problems such as format confusion and missing fields when the display device parses the application information due to differences in the description methods of application information from different electronic device manufacturers, and shortening the reading and loading time of the application list.

[0013] This provides standardized data support for presenting various applications in a preset display format based on this data structure, ensuring that application information from different manufacturers' devices can be uniformly formatted and categorized on the display device. It also provides an efficient data foundation for users to quickly search and switch between screen-casting applications.

[0014] In one possible implementation, the method further includes: when establishing a device identification interaction link with the second electronic device and receiving a second request for screen projection management sent by the second electronic device, determining the second interconnection protocol corresponding to the second electronic device based on the device information of the second electronic device; the second request carrying the device information of the second electronic device; pairing and connecting with the second electronic device based on the second interconnection protocol; obtaining a second application list sent by the second electronic device and storing the second application list in a preset format; the second application list includes at least one application installed on the second electronic device.

[0015] The screen casting control method provided in this application takes the display device as the control center, which can integrate electronic devices from multiple manufacturers and with different protocols, and present a unified application list display page on the display device, thereby realizing application-level screen casting and improving the user experience.

[0016] In one possible implementation, the method further includes: receiving a device switching operation, the device switching operation being used to instruct the first electronic device to be switched to a second electronic device; in response to the device switching operation, displaying the identifier of the application in the second application list on a third page of the target application in a preset display format.

[0017] The screen projection control method provided in this application does not rely on operation on the electronic device. Users can switch the projection source on the display device side, making the display device the control center for multi-device screen projection scenarios and improving the operational autonomy in such scenarios.

[0018] In one possible implementation, the device information of the first electronic device includes the manufacturer information of the first electronic device; based on the device information of the first electronic device, determining the first interconnection protocol corresponding to the first electronic device includes: determining the first interconnection protocol corresponding to the first electronic device based on the mapping relationship between the manufacturer information of the first electronic device and a preset protocol library.

[0019] The display device has a pre-stored or can obtain a preset protocol library from the server. Given that the interconnection protocols corresponding to different device information are different, the display device can determine the first interconnection protocol corresponding to the first electronic device based on the mapping relationship between the device information of the first electronic device and the preset protocol library.

[0020] The preset protocol library stores the mapping data between device information and corresponding interconnection protocols for each electronic device. For example, it may include a one-to-one correspondence between device brand, manufacturer, model, and other identifying information and the compatible interconnection protocols. During the use of the display device, a protocol library version verification request can be sent to the server at a preset period to achieve regular updates to the preset protocol library.

[0021] The device information of the second electronic device includes the manufacturer information of the second electronic device, and the manufacturer information may be different from that of the first electronic device.

[0022] In one possible implementation, the method further includes: after the display device is powered on, sending a query command to the cloud platform so that the cloud platform can query the inclusion status of the target application in the app store; receiving the inclusion status returned by the cloud platform; and establishing a device identification interaction link with the first electronic device, wherein if the inclusion status is "already listed", then a device identification interaction link is established with the first electronic device.

[0023] The screen casting control method provided in this application embodiment effectively reduces the control function of display devices that do not support multiple screen casting devices by determining in advance the inclusion status of the target application in the application store in the cloud platform and the installation status of the target application on the local machine, and also avoids the interruption of the screen casting process due to application missing.

[0024] In one possible implementation, after the second page of the target application displays the video stream, the method further includes: in response to the user's control operation on the target application's display interface, sending a second instruction to the first electronic device, so that the first electronic device performs a corresponding control action on the first application in the background according to the type of control operation, and records and generates a video stream of the virtual display interface of the first application, wherein the control operation includes at least one of a click operation, a confirm operation, and a move operation.

[0025] After the video stream is displayed on the second page of the target application, the method further includes: in response to the user's return operation on the target application's display interface, sending a third instruction to the first electronic device, so that the first electronic device returns to the previous page of the first application in the background or exits the first application in the background, and records and generates a video stream of the virtual display interface of the first application; when receiving the fourth instruction from the first electronic device to exit the first application, displaying the identifier of the application in the first application list on the target application's display page.

[0026] The screen mirroring control method provided in this application seamlessly maps operations on the display device to actions on the electronic device. When a user performs control or return operations on the display device, the electronic device can automatically complete the corresponding actions in the background and transmit the video stream, eliminating the need for manual operation of the electronic device. Specifically, the logic for the return operation is further divided into two strategies based on whether the current application page is the homepage: returning to the previous level or exiting the application, preventing the screen mirroring process from being interrupted due to accidental operations. Simultaneously, after the electronic device exits an application, it triggers a synchronized return to the application list on the display device, improving the smoothness of multi-application screen mirroring.

[0027] Secondly, a screen casting control method is provided, applied to an electronic device, comprising: establishing a device identification interaction link with a display device; responding to a user's first operation, sending a request to the display device to wake up the display device for screen casting management; after pairing and connecting with the display device, determining a target application list and sending the target application list to the display device, the target application list including at least one application installed on the electronic device; after receiving an instruction from the display device that a first application has been selected, while displaying a second application on the foreground page, simultaneously launching the first application in the background, recording and generating a video stream of the virtual display interface of the first application; and sending the video stream to the display device.

[0028] In one possible implementation, determining the target application list includes: obtaining a list of installed applications; comparing the list of installed applications with an application whitelist to obtain the target application list.

[0029] In one possible implementation, recording and generating a video stream of the virtual display interface of the first application includes: capturing video frames of the virtual display interface of the first application; and encapsulating the video frames into a video stream.

[0030] In one possible implementation, establishing a device identification interaction link with the display device includes: receiving a first device discovery broadcast packet sent by the display device, the first device discovery broadcast packet carrying device information of the display device and device information of devices that have established a cooperative relationship with the display device; if the electronic device is a device that has established a cooperative relationship with the display device, generating a first response message and sending the first response message to the display device; after establishing a device identification interaction link with the display device, generating a reminder message, the reminder message being used to determine whether the user should wake up the target application for screen casting management on the display device.

[0031] In one possible implementation, establishing a device identification interaction link with the display device includes: sending a second device discovery broadcast packet, the second device discovery broadcast packet carrying device information of the electronic device; after receiving a second response message returned by the display device, generating a reminder message, the reminder message being used to determine whether the user should wake up the target application used for screen casting management on the display device, the second response message carrying device information of the display device.

[0032] Thirdly, a display device is provided, including a display, a memory, a controller, and a computer program stored in the memory and executable on the controller, wherein the controller executes the computer program to implement any of the screen projection control methods of the first aspect.

[0033] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a display device, causes the display device to perform any of the screen projection control methods described in the first aspect.

[0034] Fifthly, a computer program product is provided, comprising: a computer program that, when run by a display device, causes the display device to execute any of the screen projection control methods described in the first aspect.

[0035] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0036] Figure 1 This diagram illustrates an application scenario of screen mirroring between a display device and an electronic device. Figure 2 The diagram illustrates an operational scenario between a display device and a control device according to some embodiments of this application; Figure 3 A configuration block diagram of a display device provided in an embodiment of this application is shown; Figure 4 A system block diagram of a display device provided in an embodiment of this application is shown; Figure 5 This illustration shows a schematic diagram of the framework for screen projection control between a display device and an electronic device according to an embodiment of this application; Figure 6 A flowchart of a screen projection control method provided in an embodiment of this application is shown; Figure 7 This illustration shows a pairing connection diagram between a display device and a first electronic device according to an embodiment of this application; Figure 8 The illustration shows a schematic diagram of a target application provided in some embodiments of this application; Figure 9 The flowcharts of some embodiments of the screen projection control method provided in this application are shown; Figure 10 The illustration shows a schematic diagram of a target application provided in some embodiments of this application; Figure 11 This application provides schematic diagrams illustrating screen projection switching scenarios for electronic devices according to some embodiments. Figure 12 The diagram illustrates the interaction of screen projection control methods provided in some embodiments of this application; Figure 13 The diagram illustrates the interaction of screen projection control methods provided in some embodiments of this application; Figure 14The diagram illustrates the interaction of screen projection control methods provided in some embodiments of this application; Figure 15 A schematic diagram of a screen projection control device provided in some embodiments of this application is shown. Detailed Implementation

[0037] Before providing a detailed explanation of the screen projection control method and display device provided in the embodiments of this application, the application scenarios and related technologies of the screen projection control method and display device will be explained first.

[0038] With the continuous development of technology, wireless screen projection technology has been widely used in various scenarios. This technology can project the screen content or media streams of electronic devices such as mobile phones, tablets, and laptops to display devices such as TVs and projectors for presentation and playback, thereby providing users with large-screen display effects and a high-quality audio-visual experience.

[0039] In traditional screen mirroring scenarios, the display device typically acts as a passive receiver, waiting for the electronic device to initiate a connection request. After receiving screen content, media streams, and other mirrored data pushed by the electronic device, the display device only decodes the mirrored data and displays the image. This passive technical architecture has technical flaws, resulting in a poor user experience.

[0040] The screen mirroring field suffers from fragmented protocols: different brands of electronic and display devices have varying levels of support for protocols such as Miracast, AirPlay, and the Digital Living Network Alliance (DLNA), and there is a lack of unified interoperability standards between these protocols. Furthermore, display devices, as passive receivers, only support the manufacturer's pre-built protocol types and lack proactive adaptation capabilities, leading to frequent screen mirroring failures between cross-brand devices. Users must manually find compatible solutions on the electronic device being mirrored.

[0041] In real-world screen mirroring scenarios, users often own multiple electronic devices, each with its own independent connection status and content. Each device needs to establish a separate connection with the display device. Users often struggle to intuitively grasp the screen mirroring information for all their electronic devices, resulting in a poor user experience during the mirroring process. Figure 1 This diagram illustrates an application scenario of screen mirroring between a display device and an electronic device. Figure 1 As shown, display devices (such as TV 200) and electronic devices (such as first mobile phone 110, second mobile phone 120, tablet 130, etc.) can achieve screen projection through traditional Wi-Fi LAN, Wi-Fi Direct (i.e. Wi-Fi P2P peer-to-peer) and other wireless communication methods.

[0042] Among them, the first mobile phone 110, the second mobile phone 120, and the tablet 130 support different screen projection protocols. Some devices can be adapted to the Miracast and DLNA protocols, while some devices are only compatible with AirPlay, resulting in inconsistent screen projection compatibility between different electronic devices and display devices 200.

[0043] Existing screen mirroring technologies can only achieve two basic functions: one is screen mirroring, which projects the entire screen content of the electronic device (including notification pop-ups, privacy information, etc.) to the large screen simultaneously, and the screens of the electronic device and the display device are strongly bound together, so the electronic device cannot perform other operations independently; the other is media streaming, which only supports the transmission of media content such as audio, video, and images. The display device can directly obtain the corresponding network resources through the Uniform Resource Locator (URL) and complete the decoding and playback, but it cannot realize in-application interactive control of the screen-mirrored content.

[0044] In any of the above screen mirroring modes, the display device always plays only a passive display role, and all operation commands must be completed on the electronic device. The screen mirroring connection is initiated by the electronic device. The user must first locate the target content on the electronic device, then retrieve the screen mirroring function layer by layer and select the corresponding display device. The operation steps are redundant and complex. At the same time, the display device cannot actively discover nearby electronic devices with screen mirroring capabilities, nor can it achieve reverse control of the electronic device through remote control or other control devices. Operations such as adjusting video progress and switching playback content must all be completed by returning to the electronic device.

[0045] In screen mirroring scenarios where multiple electronic devices coexist, switching the mirroring source is often involved. If it is necessary to switch the electronic device to be mirrored, the new electronic device needs to repeat the entire pairing process; from accessing the same network and scanning and identifying the display device to completing protocol matching and connection confirmation, each step is indispensable. This process affects the smoothness of the screen mirroring process and also reduces the user experience.

[0046] In view of this, this application provides a screen casting control method. When a display device establishes a device identification interaction link with an electronic device and receives a screen casting management request from the electronic device, it actively launches the target application for screen casting management and determines the corresponding interconnection protocol of the electronic device. Based on the interconnection protocol, it completes pairing and connection with the electronic device, obtains the application list sent by the electronic device, and displays the identifiers of each application in the application list in the target application. Subsequently, in response to the user's selection operation of the application identifier, the display device sends the corresponding application launch command to the electronic device, supporting the user to directly select the application to be cast on the large screen. After receiving the video stream generated by the virtual display interface of the application corresponding to the launch command sent by the electronic device, the display device displays the screen corresponding to the video stream. This realizes the display device's active control and process management of the screen casting process, improves the interactive flexibility of the screen casting operation, and optimizes the user experience.

[0047] The display device provided in this application can have various implementation forms, such as a television, a smart television, a laser projection device, a monitor, an electronic bulletin board, an electronic table, an in-vehicle device, etc.

[0048] Figure 2 The diagram illustrates an operational scenario between a display device and a control device according to some embodiments of this application, such as... Figure 2 As shown, the user can operate the display device 200 through the control device 500 or the smart device 300.

[0049] The control device 500 can be a remote control. Communication between the remote control and the display device includes infrared protocol communication, Bluetooth protocol communication, and other short-range communication methods, controlling the display device 200 wirelessly or via wired means. Users can control the display device 200 by inputting user commands through buttons on the remote control, voice input, control panel input, etc.

[0050] In some embodiments, the remote control may also have a pointing control function, supporting operations such as pointing and selecting similar to a mouse, making the large-screen operation after multiple devices are projected more intuitive and smooth, and improving the user experience.

[0051] In some embodiments, a smart device 300 (such as a mobile terminal, tablet computer, computer, laptop computer, etc.) may also be used to control the display device 200. For example, an application running on the smart device 300 may be used to control the display device 200.

[0052] In some embodiments, the display device 200 may receive instructions not through the control device 500 or the smart device 300 described above, but through touch or gestures.

[0053] In some embodiments, the display device 200 can also be controlled in ways other than the control device 500 and the smart device 300. For example, it can be controlled by directly receiving the user's voice commands through a module configured inside the display device 200 for acquiring voice commands, or it can be controlled by receiving the user's voice commands through a voice control device set outside the display device 200.

[0054] Display device 200 can also communicate with the cloud platform, supporting communication connections via local area network (LAN), wireless local area network (WLAN), and other networks. The cloud platform, relying on its distributed cluster server resources, can provide display device 200 with various content distribution and interactive services. For example, the cloud platform can use its distributed cluster server resources to provide display device with services such as application search, deployment, and content interaction from an application store.

[0055] In some embodiments, the display device 200 can also communicate with the server 400, allowing the display device 200 to communicate via a local area network (LAN), wireless local area network (WLAN), and other networks. The server 400 can provide various content and interactive features to the display device 200. The server 400 can be a single cluster or multiple clusters, and a cluster can include one or more types of servers.

[0056] Figure 3 A configuration block diagram of a display device according to an exemplary embodiment is shown, such as Figure 3 As shown, the display device 200 includes at least one of the following: a tuner 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0057] In some embodiments, the controller includes a processor, a video processor, an audio processor, a graphics processor, RAM, ROM, and a first interface to an nth interface for input / output.

[0058] The display 260 includes a display screen assembly for presenting images, a driving assembly for driving image display, a component for receiving image signals from the controller output, and a user control UI interface for displaying video content, image content, menu control interface, and user control UI interface.

[0059] The display 260 can be an LCD display, an OLED display, or a projection display, and can also be a projection device and a projection screen.

[0060] The communicator 220 is a component used to communicate with external devices or servers according to various communication protocol types. For example, the communicator may include at least one of the following: a Wi-Fi module, a Bluetooth module, a wired Ethernet module, other network communication protocol chips or near-field communication protocol chips, and an infrared receiver. The display device 200 can establish data transmission and reception with electronic devices, servers 400, etc., through the communicator 220.

[0061] The user interface can be used to receive control signals from the control device 500 (such as an infrared remote control).

[0062] Detector 230 is used to acquire signals from the external environment or interactions with the external environment. For example, detector 230 may include an image acquisition device, such as a camera, which can be used to acquire external environmental scenes, user attributes, or user interaction gestures.

[0063] The external device interface 240 may include, but is not limited to, one or more of the following: High Definition Multimedia Interface (HDMI), analog or high-definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, etc. It may also be a composite input / output interface formed by multiple interfaces mentioned above.

[0064] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0065] The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving a user command to select a UI object to display on the monitor 260, the controller 250 can execute operations related to the object selected by the user command.

[0066] In some embodiments, the controller includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), random access memory (RAM), read-only memory (ROM), a first to an nth interface for input / output, a communication bus, etc.

[0067] Users can input commands through a graphical user interface (GUI) displayed on the monitor 260, and the user input interface receives the user input commands through the GUI. Alternatively, users can input commands by entering specific sounds or gestures, and the user input interface receives the user input commands by recognizing the sounds or gestures through sensors.

[0068] A user interface (GUI) is the medium through which an application or operating system interacts and exchanges information with the user. It converts information from its internal form to a form that the user can accept. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be an icon, window, control, or other interface element displayed on the screen of an electronic device. Controls can include visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.

[0069] Figure 4 A system block diagram of a display device provided in an embodiment of this application is shown, such as... Figure 4 As shown, the system in the display device is divided into four layers, from top to bottom: the Applications layer (referred to as the "application layer"), the Application Framework layer (referred to as the "framework layer"), the Android runtime and system library layer (referred to as the "system runtime library layer"), and the kernel layer.

[0070] The application layer runs at least one application, which can be a built-in Windows program, system settings program, or clock program, or an application developed by a third-party developer. In practice, the application packages in the application layer are not limited to the examples above.

[0071] The application layer may include applications for screen casting management (such as target applications). These applications integrate core capabilities such as multi-protocol adaptation, device discovery and pairing, screen casting source switching, media stream control, and real-time control of screen casting content, providing users with a one-stop cross-device screen casting management service.

[0072] The framework layer provides application programming interfaces (APIs) and a programming framework for applications. The application framework layer includes predefined functions. It acts as a central processing unit, determining the actions taken by applications within the application layer. Through the API, applications can access system resources and obtain system services during execution.

[0073] like Figure 4 As shown, the application framework layer in this embodiment includes managers, content providers, etc., wherein the managers include at least one of the following modules: ActivityManager, which interacts with all activities running in the system; LocationManager, which provides access to system location services for system services or applications; PackageManager, which retrieves various information related to application packages currently installed on the device; NotificationManager, which controls the display and clearing of notification messages; and WindowManager, which manages icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0074] In some embodiments, the Activity Manager manages the lifecycle of individual applications and common navigation and back functions, such as controlling application exit, opening, and back actions. The Window Manager manages all window programs, such as obtaining the screen size, determining if a status bar is present, locking the screen, capturing the screen, and controlling display window changes (e.g., shrinking the display window, shaking the display, distorting the display, etc.).

[0075] In some embodiments, the system runtime library layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system runs the C / C++ libraries contained in the system runtime library layer to implement the functions that the framework layer needs to perform.

[0076] In some embodiments, the kernel layer is a layer between hardware and software. For example... Figure 4 As shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0077] Figure 5 This illustration shows a schematic diagram of the framework for screen projection control between a display device and an electronic device according to an embodiment of this application.Figure 5 As shown, the electronic device 100 and the display device 200 communicate with each other, specifically involving the core components of each device and their interaction relationships.

[0078] Electronic device 100 refers to the device that serves as the source of the screen projection signal, i.e., the screen projection device, which can be a smartphone, tablet computer, laptop computer, desktop computer, game console, camera, etc. Electronic device 100 includes a device discovery module 101, an application list service module 102, an application-level screen projection engine module 103, an instruction receiving and execution module 104, and a smart wake-up module 105.

[0079] The device discovery module 101 establishes a communication link with the display device 200 based on the interconnection protocol corresponding to the electronic device 100, in order to discover display devices that support screen projection and complete the connection and pairing process between devices.

[0080] Interconnection protocols are screen mirroring communication rules defined by electronic device manufacturers. Their function is to discover, authenticate, and establish links for display devices. These protocols can have proprietary attributes, and interconnection protocols from different manufacturers may be incompatible with each other. For example, electronic devices from manufacturer X may support interconnection protocol a, while electronic devices from manufacturer Y may support interconnection protocol b. The protocol identifier of the interconnection protocol can be used to distinguish the electronic devices from the corresponding manufacturers.

[0081] The device discovery module 101 discovers display devices that support screen projection control. This means that the device discovery module 101 can be used to establish a device identification interaction link between electronic devices and display devices 200. Specifically, this includes the following two methods: First, the electronic device 100 receives a device discovery broadcast packet sent by the display device 200 under the same communication protocol (such as Wi-Fi or Wi-Fi P2P) and establishes a device identification interaction link by returning a response message; Second, after the electronic device 100 is enabled (such as when a mobile phone screen is on), it periodically sends a device discovery broadcast packet. When the display device 200 under the same communication protocol receives the broadcast message, it will return a response message to establish a device identification interaction link.

[0082] The application list service module 102 is used to obtain all the applications installed in the electronic device 100, compare them with the application whitelist, filter and generate the application list corresponding to the electronic device 100 (such as the first application list corresponding to the first electronic device, the second application list corresponding to the second electronic device), and send the application list to the display device.

[0083] The application whitelist can be preset by the electronic device 100 or obtained by the electronic device 100 from the server. All applications in the application whitelist have screen casting functionality. Therefore, this application list is a list of applications that support screen casting, including core information such as the application's name, icon, and package name, and may even contain information about the electronic device 100.

[0084] The application-level screen mirroring engine module 103 is used to receive application launch commands (such as the first command to launch the first application) sent by the display device 200. It can launch the application corresponding to the command in the background (hereinafter referred to as the first application for simplicity) while other applications are displayed on the foreground page of the electronic device. This module records the video frames of the first application in the virtual display interface and generates a video stream of the virtual display interface of the application. That is to say, after receiving the application launch command, this module extracts and projects only the video stream of the real-time virtual display interface of the first application to the display device 200 in an application-level screen mirroring mode (such as based on the Android MediaProjection interface or the manufacturer's private API), without involving the system interface or other application content of the electronic device 100.

[0085] Virtual display interfaces are a type of logical layer interface that runs in the background of electronic devices. They lack a visual interactive medium and therefore do not directly provide viewing or operation functions to users. Instead, they serve as an invisible interactive framework for command transmission and data interaction between devices, applications, or systems, playing a core role in command transmission and data interaction. Because of their silent background operation, the operation of this interface will not interfere with other operations of the electronic device when users project applications from their electronic devices to external display devices.

[0086] The application-level projection engine module 103 is used to extract and project the video stream of the real-time virtual display interface of the application corresponding to the instruction to the display device 200 in an application-level projection mode after receiving the application launch instruction sent by the display device 200, without involving the system interface or other application content of the electronic device 100.

[0087] For example, the module receives a first instruction from the display device 200 to launch the first application, which enables the first application to be launched in the background while other applications (not the first application, such as system applications) are displayed on the foreground page of the electronic device, and to record the video frames of the first application on the virtual display interface to generate a video stream of the virtual display interface of the application.

[0088] After receiving the application launch command, the application-level screen mirroring engine module 103 can also perform the following operations: if the first application is not running, it first launches the first application in the background, and then records and generates a video stream of the virtual display interface of the application; if the first application is already running, it directly records and generates a video stream of the virtual display interface of the application. Subsequently, the module transmits the video stream to the display device 200 through the interconnection protocol for the display device 200 to decode and display the image.

[0089] Application-level screen mirroring modes can be implemented based on the Android MediaProjection interface and proprietary APIs of electronic device manufacturers. Among them, Android MediaProjection is a set of system-level APIs introduced by the Android system. Its core function is to allow the capture of the device screen (including virtual display interfaces) after obtaining user authorization, while providing standardized capabilities such as screenshots, screen recording, and screen mirroring.

[0090] The video stream mentioned above is the content displayed by the first application and does not include irrelevant information such as the desktop interface, system status bar, notification bar, and pop-ups of other applications of the electronic device 100, to ensure that the screen-cast content is clean and privacy is secure.

[0091] It should also be noted that the first application in electronic device 100 runs in the background and does not need to be loaded and displayed in the foreground. Therefore, in the scenario of screen mirroring between display device 200 and electronic device 100, it will not interfere with the user's operation of other functions of electronic device 100.

[0092] The instruction receiving and execution module 104 is used to receive control instructions sent by the display device 200 and perform control operations on the corresponding application, such as click operations, confirm operations, move operations, and return operations (exit operations). After receiving the control instructions, this module performs click, selection, and movement controls on the application. In the screen projection application scenario, it enables the display device 200 to control the electronic device 100.

[0093] The intelligent wake-up module 105 is used to wake up the screen projection management service in the display device 200 after the electronic device 100 and the display device 200 establish a device identification interaction link. It can send a request to wake up the screen projection management service of the display device 200 (such as a first request or a second request) to the display device 200 so that the display device 200 can start the application used for screen projection management.

[0094] It should be noted that each module in the aforementioned electronic device can be carried out and its corresponding function can be implemented by the electronic device's built-in application or system service.

[0095] It should also be noted that in practical application scenarios, the number of electronic devices 100 that can establish a screen mirroring connection with the display device 200 is N (N is a positive integer). Taking a home scenario as an example, when the display device is a television, electronic devices such as mobile phone A, mobile phone B, mobile phone C, tablet computer D, and game console E can all establish a screen mirroring connection with the television.

[0096] Display device 200 refers to a device that receives screen projection data, such as a television, smart TV, laser projector, monitor, electronic whiteboard, electronic desktop, in-vehicle device, etc. Display device 200 includes a device discovery and management module 201, an application list rendering module 202, a video stream receiving and decoding module 203, a command forwarding module 204, and an intelligent notification and launch module 205.

[0097] It should be noted that the display device 200 supports the installation of applications for screen mirroring management. For ease of description, this application will be referred to as the target application below.

[0098] The device discovery and management module 201 is used to periodically send device discovery broadcast packets, which carry device information of the display device and device information of electronic devices that have established cooperative relationships with the display device; and after receiving the response message returned by the electronic device, it establishes a device identification interaction link with the electronic device.

[0099] In some embodiments, the device discovery and management module 201 is further configured to receive a device discovery broadcast packet sent by the electronic device 100, the device discovery broadcast packet carrying the device information of the electronic device; match the device information of the electronic device with a pre-stored protocol library; and return a response message to the electronic device 100 after a successful match, thereby establishing a device identification interaction link with the electronic device.

[0100] The device discovery and management module 201 can also uniformly store and manage the device information of the electronic device 100.

[0101] The application list rendering module 202 is used to receive the application list (such as the first application list corresponding to the first electronic device) sent by the electronic device 100, and store the application list in a preset format, for example, store it in a target database in a preset format.

[0102] When a target application needs to display application list content, this module can read the stored application list data. Since the data format is consistent, it can be rendered onto the corresponding page of the target application according to the preset display format. This process is applicable to application list data pushed by all electronic devices, and has unified storage and rendering adaptation capabilities.

[0103] In addition, this module also supports statistical recording of the usage frequency of each application during the screen mirroring process. When the target application's page is rendered, the application can be displayed differently based on the above statistical records. For example, the applications with the highest usage frequency of M (M is a positive integer), or applications with a usage frequency of more than N times (N is a positive integer), can be displayed in a dedicated area corresponding to frequently used applications on the page, thereby improving the user experience.

[0104] The video stream receiving and decoding module 203 is used to receive the video stream sent by the electronic device 100, and call the player to decode, play, and display the video. It should be noted that this video stream is dedicated screen-casting data recorded by the electronic device 100 from the virtual display interface of the screen-casting application, and screen-casting transmission can be completed based on the linkage protocol of each electronic device 100.

[0105] The instruction forwarding module 204 is used to receive various operation instructions from users, including operations initiated by remote controls with pointing control, button functions, and voice input functions, as well as direct voice commands and touch screen commands. This module converts these operations into corresponding instructions and sends the instructions to the electronic device 100, thereby driving the electronic device 100 to perform the corresponding response operation, realizing the control of the electronic device 100 by the display device 200 in the screen projection scenario.

[0106] The intelligent notification and pull-up module 205 is used to receive device discovery broadcast or response messages sent by the electronic device 100, and establish a device identification interaction link with the electronic device 100 accordingly.

[0107] The intelligent notification and launch module 205 also has the ability to detect the status of the target application, which is used to analyze whether the display device 200 has installed the target application: if the display device 200 has installed the target application, when the device identification interaction link is established with the electronic device 100 and a request for wake-up screen projection management is received from the electronic device, the target application will be launched directly; if the display device 200 has not installed the target application, it will automatically jump to the application store, guide the user to complete the download and installation of the target application, and then launch the target application.

[0108] To facilitate a further understanding of the technical solutions in some embodiments of this application, the technical solutions of the screen projection control method and how these solutions solve the aforementioned technical problems are described in detail below with reference to specific embodiments and accompanying drawings. The embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of this application.

[0109] Based on the hardware structure and software architecture of the display device provided in the above embodiments, the screen projection control method provided in this application will be specifically described below with reference to the accompanying drawings. For ease of explanation, the examples all use a television as the display device.

[0110] Figure 6 The flowchart shown is a screen projection control method provided in an embodiment of this application, which is applied to the above. Figure 3 , Figure 4 The display device shown, such as Figure 6 As shown, the screen projection control method includes the following steps S610~S640: S610. When the display device establishes a device identification interaction link with the first electronic device and receives a first request for screen projection management sent by the first electronic device, it starts the target application for screen projection management.

[0111] The first request carries device information of the first electronic device.

[0112] In a screen mirroring scenario, establishing an identification and interaction link between the display device and the electronic device means that both parties complete the mutual identification of the device identities. The first electronic device is any electronic device that can establish an identification and interaction link with the display device in a screen mirroring application scenario.

[0113] The establishment of an identification interaction link between a display device and an electronic device can be achieved in two ways. The following explanation uses the establishment of a device identification interaction link between the display device and a first electronic device as an example.

[0114] Method 1 1. The display device sends a first device discovery broadcast packet, which carries the display device's own device information and the device information of electronic devices that have established a cooperative relationship with the display device.

[0115] 2. After receiving the first device discovery broadcast packet, if the first electronic device determines that it belongs to a device that has established a cooperative relationship with the display device, it generates a first response message and sends it to the display device; the response message carries the device information of the first electronic device.

[0116] 3. The display device receives the first response message returned by the first electronic device and completes the establishment of the device identification interaction link with the first electronic device.

[0117] 4. After the link is established, the first electronic device generates a reminder message to determine whether the user should wake up the target application used for screen mirroring management on the display device.

[0118] Method 2 1. The first electronic device sends a second device discovery broadcast packet. This broadcast packet carries the device information of the first electronic device and may also include an identifier indicating whether the first electronic device supports cross-device screen mirroring control. The sending of this second device discovery broadcast packet can be triggered by actions such as turning on the screen or unlocking the first electronic device.

[0119] 2. After receiving the second device discovery broadcast packet, if the display device determines that the first electronic device belongs to a device with which it has already established a cooperative relationship, it generates a second response message and sends it to the first electronic device; this response message may carry the device information of the display device. If it determines that the first electronic device does not belong to a device with which it has already established a cooperative relationship (indicating that the display device does not support the screen mirroring management of the first electronic device), it does not generate a response message.

[0120] 3. After receiving the second response message returned by the display device, the first electronic device generates a reminder message to determine whether the user should wake up the target application used for screen casting management on the display device.

[0121] When the aforementioned electronic device prompts the user whether it supports the screen mirroring management service on the display device, it may use various presentation methods such as pop-up prompts and voice interaction. This application embodiment does not make specific limitations on this.

[0122] After establishing a device identification interaction link with the first electronic device, the display device can receive a first request sent by the first electronic device. This first request instructs the display device to enable screen mirroring management, that is, instructs the display device to launch the target application for screen mirroring management. In some embodiments, the first electronic device responds to a user's first operation by sending a request (i.e., the first request) to the display device to wake it up and enable screen mirroring management. The first operation is a confirmation operation performed by the electronic device in response to a user's inquiry regarding whether screen mirroring control on the display device is supported, based on the establishment of the device identification interaction link with the display device.

[0123] It should be noted that the target application is an additional function service of the display device and usually does not have the feature of automatic startup. The application will only be triggered to start when the display device and the first electronic device complete the device identification interaction link and the first electronic device receives the first request for screen casting management sent by the first electronic device. This ensures that the screen casting management function is accurately triggered only when the link establishment and user needs are met, thereby reducing the target application's occupation of the display device's system resources.

[0124] In some embodiments, before the display device launches the target application for screen mirroring management, the method further includes: detecting whether the target application is installed locally; if the target application is installed, launching the target application directly; if the target application is not installed, generating a boot instruction to instruct the installation of the target application; and launching the target application after the target application is installed.

[0125] For example, if the target application is not installed, the user is redirected to the app store and guided to download and install the target application. Once the target application is installed, it is launched directly.

[0126] S620. The display device determines the first interconnection protocol corresponding to the first electronic device based on the device information of the first electronic device.

[0127] Among them, the device information of the first electronic device refers to the device parameters that can be used for identification, such as device name, device identifier, device model, manufacturer information, device brand and other parameters that reflect the device's identity information; and network connection parameters such as device MAC address, device IP address, and the name of the local area network accessed.

[0128] The display device has a pre-stored or can obtain a preset protocol library from the server. Given that the interconnection protocols corresponding to different device information are different, the display device can determine the first interconnection protocol corresponding to the first electronic device based on the mapping relationship between the device information of the first electronic device and the preset protocol library.

[0129] The preset protocol library stores the association mapping data between the device information of each electronic device and the corresponding interconnection protocol. For example, it may include the one-to-one correspondence between the device brand, manufacturer, model and other identification information and the adapted interconnection protocol.

[0130] During the use of the display device, a protocol library version verification request can be sent to the server according to a preset cycle to achieve regular updates of the preset protocol library.

[0131] For example, when the device information of the first electronic device includes the manufacturer information of the first electronic device, the first interconnection protocol corresponding to the first electronic device is determined based on the mapping relationship between the manufacturer information of the first electronic device and the preset protocol library.

[0132] It should be noted that the above protocol matching method is also applicable to other electronic devices, such as second electronic devices, that need to establish a screen mirroring connection with the display device.

[0133] S630, the display device is paired and connected to the first electronic device based on the first interconnection protocol.

[0134] Based on the mutual discovery and establishment of device identification interaction links between the aforementioned devices, the display device can pair and connect with the first electronic device according to the established first interconnection protocol, thereby establishing a stable communication link. This communication link, relying on the first interconnection protocol, can achieve stable transmission of screen-projected data; wherein, screen-projected data specifically refers to the video stream corresponding to application-level screen projection.

[0135] For example, the specific method of pairing and connecting the display device with each electronic device can be through interactive methods such as QR code pairing or PIN code verification pairing. Figure 7 This illustration shows a pairing connection diagram between a display device and a first electronic device according to an embodiment of this application. Figure 7 As shown in (a), the interactive interface of TV 710 displays connection code 711. After the user enters this connection code into the mobile phone to be paired with TV 710, the interface of TV 710 switches to the following... Figure 7 The status shown in (b) is "Connecting to mobile phone 720"; once the connection process is complete, the TV and the mobile phone will be successfully paired and connected.

[0136] Specifically, once the display device determines the interconnection protocol corresponding to the electronic device and completes the pairing connection based on that link, it can realize the function of transmitting the screen projection data of the electronic device to the display device.

[0137] S640, The display device receives a first application list sent by the first electronic device, and displays the identifiers of the applications in the first application list on the first page of the target application.

[0138] The first application list includes the first application, which is an application that has been installed on the first electronic device.

[0139] To improve the accuracy of screen mirroring interaction between the first electronic device and the display device, the first application typically refers to an application already installed on the first electronic device that supports screen mirroring. For example, the electronic device obtains a list of installed applications; it then compares this list with an application whitelist to obtain a target application list. The application whitelist is data preset by the electronic device or obtained from a server, and all applications in the whitelist have screen mirroring functionality. Therefore, the target application list is a list of applications that support screen mirroring, including core information such as the application's name, icon, and package name, and may even contain information about the electronic device.

[0140] It should be noted that the display device receives a list of target applications sent by an electronic device (such as a first application list sent by a first electronic device or a second application list sent by a second electronic device), and the list of target applications includes at least one application that has been installed on the electronic device.

[0141] The display device presents the identifiers of applications in the first application list on the first page of the target application, such as the application icon, name, and other information. Figure 8 The illustration shows schematic diagrams of target applications displayed according to some embodiments of this application. For example... Figure 8As shown, the display device's interface displays the first page of the target application, and this page presents the icons and names of each application in the first application list; it may also present the first electronic device identifier (such as name, identifier, etc.) and associated controls such as exiting the target application.

[0142] In some embodiments, a user can trigger switching between electronic devices by clicking the icon of the first electronic device. A user can also trigger exiting the target application by clicking its icon.

[0143] S650, in response to the user's selection operation of the first application identifier, the display device sends a first instruction to the first electronic device, so that while the first electronic device displays the second application on the foreground page, it starts the first application in the background, records and generates a video stream of the virtual display interface of the first application.

[0144] The selection operation is the user's selection of a first application from the first application list, used to instruct the application to be cast to the screen at the application level. Therefore, the function of the first instruction is to trigger the first electronic device to launch the first application and determine the video stream to be cast to the application.

[0145] After receiving the instruction from the display device that the first application has been selected, the first electronic device displays the second application on the foreground page while simultaneously launching the first application in the background, recording and generating a video stream of the virtual display interface of the first application, and sending the video stream to the display device.

[0146] The second application can be either a system application built into the first electronic device, such as the device homepage or system settings, or a third-party application installed on the first electronic device other than the first application.

[0147] It should be noted that for each electronic device, the carrier of the screen projection data is a video stream. This video stream is generated by first capturing video frames from the application's virtual display interface, then encapsulating and processing the captured video frames to obtain a video stream that can be used for screen projection. Generating a projection video stream based on the video frames of the application's virtual display interface can completely capture the application's display content, including dynamic interactive elements and screen rendering details.

[0148] The video stream generation method does not depend on the application's underlying source code or specific interfaces. It only processes video frames of the virtual display interface and can be adapted to various applications with different development architectures. Whether it is a video playback application, an office document application, or a game entertainment application, it can stably output a video stream that can be cast to the screen, and has strong technical versatility.

[0149] Taking an electronic device from manufacturer A as an example, if both the electronic device and the display device are pre-installed with and support the same interconnection protocol, then all devices from manufacturer A can achieve borderless screen mirroring without needing to consider protocol compatibility. This screen mirroring method based on a proprietary interconnection protocol establishes an independent screen mirroring transmission link for communication between the electronic device and the display device.

[0150] S660, The display device receives the video stream sent by the first electronic device and displays the screen corresponding to the video stream on the second page of the target application.

[0151] The display device decodes and renders the received video stream sequentially to achieve application-level screen mirroring functionality.

[0152] The screen casting control method provided in this application uses a display device as the control center. When the display device establishes a device identification interaction link with the electronic device and receives a screen casting management request from the electronic device, it actively launches the target application for screen casting management. Based on the determined interconnection protocol corresponding to the electronic device, it completes pairing and connection with the electronic device, obtains the application list sent by the electronic device, and displays the identifiers of each application in the application list on the target application. Subsequently, in response to the user's selection operation of the application identifier, the display device sends the corresponding application launch command to the electronic device, allowing the user to directly select the application to be cast on the large screen. After receiving the video stream generated by the virtual display interface of the application corresponding to the launch command sent by the electronic device, the display device displays the screen corresponding to the video stream. This realizes the display device's active control and process management of the screen casting process, improves the interactive flexibility of screen casting management, and optimizes the user experience.

[0153] Based on the screen projection control scheme described above, during the screen projection interaction between the display device and the first electronic device, after receiving the first application list sent by the first electronic device in step S640, the first application list can also be stored, and the stored application list can be displayed. This will be explained in more detail below; other content can be found in the previous description and will not be repeated here.

[0154] Figure 9 The flowcharts of screen projection control methods provided in some embodiments of this application are shown, such as... Figure 9 As shown, Figure 6 Step 640 includes the following steps S641~S642: S641, The display device stores the first application list in a preset format.

[0155] For example, the display device stores the target application lists (first application list, second application list) sent by each electronic device into the target database in a preset format.

[0156] The embodiments of this application adopt a unified preset format to standardize the data structure of the application list, avoid problems such as format confusion and missing fields when the display device parses the application information due to differences in the description methods of application information from different electronic device manufacturers, and shorten the reading and loading time of the application list.

[0157] This provides standardized data support for presenting various applications in a preset display format based on this data structure, ensuring that application information from different manufacturers' devices can be uniformly formatted and categorized on the display device. It also provides an efficient data foundation for users to quickly search and switch between screen-casting applications.

[0158] S642. The display device displays the first page of the target application in a preset display format, showing the identifiers of the applications in the first application list.

[0159] On the first page of the target application, the display device shows the icons of each application in the first application list. For a specific display effect, please refer to [reference needed]. Figure 8 .

[0160] In some embodiments, to further optimize the user experience, the display device also supports statistical analysis and recording of the usage frequency of each application during screen mirroring from the electronic devices (first electronic device, second electronic device). Based on this statistical data, the display device can differentiate the display of the application when rendering the first page of the target application.

[0161] Figure 10 The diagram illustrates a target application display provided by some embodiments of this application.

[0162] like Figure 10 As shown in (a), when the target application displays the first application list corresponding to the first electronic device 1020, the TV will display the top 7 most frequently used applications (application C, application X, application R, application A, application U, application F and application I) in the frequently used application area 1011 of the first page 1010 of the target application, thereby shortening the operation path for users to find frequently used screen casting applications and improving the user experience.

[0163] like Figure 10 As shown in (b), when the target application displays the second application list corresponding to the second electronic device 1040, the TV will display the top 7 most frequently used applications (application X, application B, application U, application E, application K, application I, and application J) in the frequently used application display area 1031 of the third page 1030 of the target application.

[0164] As can be seen, the application lists of different electronic devices are presented in a consistent format in the target application, while the specific content of the list is determined by the statistical results of the application usage frequency of each electronic device during the screen mirroring process.

[0165] The screen casting control method provided in this application embodiment unifies the display format of each application list in the target application on the display device, and achieves differentiated presentation based on usage frequency. This significantly shortens the operation path for users to find frequently used screen casting applications, reduces the user's learning cost, and improves the efficiency of screen casting operations.

[0166] Based on the screen projection control scheme introduced above, during the process of establishing screen projection interaction between the display device and the first electronic device, screen projection interaction can also be established in parallel with other electronic devices. The following section uses the second electronic device as an example to further explain this parallel interaction process.

[0167] First, the display device establishes a device identification interaction link with the second electronic device.

[0168] The display device sends a first device discovery broadcast packet, which carries device information of the display device itself, as well as device information of electronic devices that have established a cooperative relationship with the display device.

[0169] After receiving the first device discovery broadcast packet, if the second electronic device determines that it belongs to a device that has established a cooperative relationship with the display device, it generates a first response message and sends it to the display device; the response message carries the device information of the second electronic device.

[0170] Secondly, after the display device establishes a device identification interaction link with the second electronic device, it receives a second request (carrying device information of the second electronic device) sent by the second electronic device for screen projection management; subsequently, based on the device information of the second electronic device, it determines the second interconnection protocol corresponding to the second electronic device.

[0171] It should be noted that when the display device establishes a device identification interaction link with the second electronic device and receives the second request for screen casting management sent by the second electronic device, since the target application for screen casting management is already running, there is no need to trigger the application wake-up process again. The subsequent protocol matching and pairing connection operations can be directly performed based on the already running target application.

[0172] It should also be noted that the device information of the second electronic device may include its manufacturer information, and this manufacturer information may be different from that of the first electronic device; for example, the manufacturer information of the first electronic device is J, and the manufacturer information of the second electronic device is K.

[0173] Then, the display device completes pairing and connection with the second electronic device based on the second interconnection protocol.

[0174] The implementation process of this step can be referred to in the detailed description of the above-mentioned display device completing the pairing connection with the first electronic device based on the first interconnection protocol, which will not be repeated here.

[0175] Finally, the display device obtains the second application list sent by the second electronic device and stores the application list in a preset format; the second application list includes at least one application that has been installed in the second electronic device.

[0176] For example, the display device also stores the second application list sent by the second electronic device in a preset format in the target database.

[0177] It should be noted that after the display device and the second electronic device have completed pairing and connection, a pop-up window on the display device's screen will notify the user that the second electronic device has completed the screen mirroring connection; for example... Figure 10 As shown in (c), when the display device and the first electronic device 1020 have been connected for screen mirroring, such as when the application identifier in the first application list is displayed on the target application's page, if the display device is then paired and connected with the second electronic device, a pop-up window 1050 can be displayed on the display page of the display device, prompting the user that the second electronic device (e.g., the identifier is YYYY) has been connected and can be viewed by clicking the menu, that is, the screen mirroring source can be directly switched to the second electronic device.

[0178] The screen casting control method provided in this application takes the display device as the control center, and can integrate electronic devices from multiple manufacturers and with different protocols to present a unified application list display page on the display device, thereby realizing application-level screen casting and improving the user experience.

[0179] As described in the previous section, the screen mirroring control scheme mainly details the establishment of screen mirroring connections and data interaction processes between display devices and electronic devices. Building upon this, when a display device establishes screen mirroring connections with multiple electronic devices simultaneously, it can further enable flexible switching control of specific electronic devices being mirrored. This will be explained in more detail below; other related content can be found in the previous section and will not be repeated here.

[0180] The display device receives a device switching operation, which instructs the current projection source to be switched from the first electronic device to the second electronic device. In response to the device switching operation, the display device presents the application identifiers in the second application list in a preset display format on the third page of the target application, so that the user can quickly select the application to be projected.

[0181] Figure 11 The following are schematic diagrams illustrating screen projection switching scenarios of electronic devices provided in some embodiments of this application, such as... Figure 11As shown in (a), after the TV establishes a screen mirroring connection with the first electronic device (e.g., identified by XXXXX), the interface will display the first electronic device identifier control 1110. The user can select this control 1110 to trigger the "Device Selection" page. On this page, the user can either directly select the second electronic device (e.g., identified by YYYY) that has already been paired, or click the "Add Device" control to create a new screen mirroring connection with other electronic devices.

[0182] If the user selects the second electronic device identifier (identifier YYYY), the screen casting source will switch from the first electronic device (identifier XXXXX) to the second electronic device (identifier YYYY), and the TV will simultaneously display the third page 1120 of the target application, and the second electronic device identifier control 1130 will be displayed on this page.

[0183] The screen projection control method provided in this application does not rely on operation on the electronic device. Users can switch the projection source on the display device side, making the display device the control center for multi-device screen projection scenarios, improving the operational autonomy in multi-device screen projection scenarios and enhancing the user experience.

[0184] Based on the screen projection control scheme introduced above, before the display device and the first electronic device establish a device identification interaction link in step S610, it is also possible to perform actions such as whether the display device supports and installs the target application. The following will provide supplementary explanations on this matter.

[0185] Figure 12 The following diagram illustrates the interaction of screen projection control methods provided in some embodiments of this application, such as... Figure 12 As shown, whether the display device supports and the target application can be determined by the smart notification and launch module within the display device. For example... Figure 12 As shown, it includes the following steps: S1201. After the display device is powered on, the intelligent notification and pull-up module automatically starts with the system. The intelligent notification and pull-up module provides basic support for subsequent functions such as screen projection device recognition and screen projection request response.

[0186] S1202, The display device sends a query command to the cloud platform.

[0187] S1203. After receiving the query command, the cloud platform queries the inclusion status of the target application in the app store.

[0188] The app's listing status in the app store includes listed, under review, pending listing, and removed.

[0189] S1204, The cloud platform will send the collected status information to the display device.

[0190] S1205. The display device receives the recording status. If the recording status is "listed", a device identification interaction link can be established with the first electronic device.

[0191] The fact that the target application is listed in the app store indicates that the display device has the ability to install the target application. The target application can be downloaded, installed and updated through the app store. The successful deployment of the target application can provide the display device with support for core management functions such as identification, pairing, connection, switching and control of multiple screen projection devices, thereby ensuring the smooth operation of the multi-device screen projection interaction process.

[0192] It should be noted that, in some embodiments, after step S1205, it can be further determined whether the display device has the target application installed; this clarifies whether, when the target application needs to be launched, the installed application should be called directly, or the application should be downloaded and installed from the app store first.

[0193] It should also be noted that if the target application is not listed in the app store, it means that the display device does not have the ability to install the target application. In this case, there is no need to determine whether the display device has the target application installed.

[0194] The screen casting control method provided in this application embodiment effectively reduces the control function of display devices that do not support multiple screen casting devices by determining in advance the inclusion status of the target application in the application store in the cloud platform and the installation status of the target application on the local machine, and also avoids the interruption of the screen casting process due to application missing.

[0195] As explained above, user operations on the target application include selecting the application and switching the projection device. The following sections will explain other operations such as control and return operations; please refer to the previous sections for further details, which will not be repeated here.

[0196] Figure 13 The following diagram illustrates the interaction of screen projection control methods provided in some embodiments of this application, such as... Figure 13 As shown, the display device includes a target application. Taking the interaction with the first electronic device as an example, after the page in the target application displays the screen corresponding to the video stream, the following steps are also included: S1301. In response to a user's control operation on the target application display interface, the display device sends a second instruction to the first electronic device. The control operation includes at least one of a click operation, a confirm operation, and a move operation.

[0197] S1302. After receiving the second instruction, the first electronic device performs the corresponding control action on the first application in the background according to the type of control operation.

[0198] S1303, The first electronic device records and generates a video stream of the virtual display interface of the first application.

[0199] In some application scenarios, after the video stream is displayed on the page of the target application, the following steps are also included: S1304. In response to the user's return operation on the target application display interface, the display device sends a third instruction to the first electronic device.

[0200] S1305. After receiving the third instruction, the first electronic device returns to the previous page of the first application in the background.

[0201] It should be noted that, in the background of the first electronic device, this operation follows the following strategy: if the first application is currently displaying a page that is not its homepage, then return to the previous page of the first application; if the first application is currently displaying a page that is its homepage, then exit the first application.

[0202] S1306, The first electronic device records and generates a video stream of the virtual display interface of the first application.

[0203] It should be noted that if the third instruction is used to indicate exiting the first application, then the following is executed: S1307 The first electronic device sends a fourth instruction to the display device.

[0204] S1308. After receiving the fourth instruction, the display device exits the first application on the display page of the target application and displays the identifier of the application in the first application list.

[0205] The screen mirroring control method provided in this application seamlessly maps operations on the display device to actions on the electronic device. When a user performs control or return operations on the display device, the electronic device can automatically complete the corresponding actions in the background and transmit the video stream, eliminating the need for manual operation of the electronic device. Specifically, the logic for the return operation is further divided into two strategies based on whether the current application page is the homepage: returning to the previous level or exiting the application, preventing the screen mirroring process from being interrupted due to accidental operations. Simultaneously, after the electronic device exits an application, it triggers a synchronized return to the application list on the display device, improving the smoothness of multi-application screen mirroring.

[0206] Figure 14 The following diagram illustrates the interaction of screen projection control methods provided in some embodiments of this application, such as... Figure 14 As shown, the display device includes a smart notification and pull-up module and a target application. The screen casting control method includes the following steps: S1401. The display device establishes a device identification interaction link with the first electronic device and the second electronic device.

[0207] This embodiment will be described in the next step using the interaction with the first electronic device as an example.

[0208] S1402, The first electronic device generates a reminder message.

[0209] S1403. On the first electronic device, the user identifies the target application used for screen mirroring management in the wake-up display device.

[0210] S1404, The first electronic device sends a first request to the display device to wake up the display device and start the screen mirroring management.

[0211] S1405, Display device starts target application.

[0212] S1406. The display device determines the first interconnection protocol corresponding to the first electronic device.

[0213] S1407, The display device is paired and connected with the first electronic device.

[0214] S1408. After the first electronic device and the display device complete the pairing connection, determine its first application list.

[0215] S1409, The first electronic device sends a first application list to the display device.

[0216] S1410, The display device displays the identifiers of applications in the first application list.

[0217] S1411. In response to the user's selection operation of the first application identifier, the display device sends a first instruction to the first electronic device.

[0218] S1412. While the first electronic device displays the second application on the front-end page, it launches the first application in the background.

[0219] S1413, The first electronic device records and generates a video stream of the virtual display interface of the first application.

[0220] S1414, The first electronic device sends a video stream to the display device.

[0221] S1415. After receiving the video stream sent by the first electronic device, the display device presents the screen corresponding to the video stream on the second page of the target application.

[0222] For details on the implementation process of steps S1401 to S1415, please refer to the detailed description of steps S610 to S650 above, which will not be repeated here.

[0223] It should be understood that the sequence numbers of the processes in the above embodiments do not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention. The various embodiments described herein can be independent solutions or combinations based on internal logic, and all such solutions fall within the protection scope of this application.

[0224] It should also be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0225] Based on the same concept, as an implementation of the above method, this application provides a screen projection control device. This device embodiment corresponds to the aforementioned method embodiment. For ease of reading, this device embodiment will not repeat the details of the aforementioned method embodiment one by one, but it should be clear that the device in this embodiment can implement all the contents of the aforementioned method embodiment.

[0226] Figure 15 A schematic diagram of a screen projection control device provided in some embodiments of this application is shown. The screen projection control device 1500 includes a device identification unit 1510, a device pairing unit 1520, a display unit 1530, and an execution unit 1540. The functions of each unit are detailed below.

[0227] The device identification unit 1510 is used to display the target application for screen projection management when the device establishes a device identification interaction link with the first electronic device and receives a first request for screen projection management sent by the first electronic device. The first request carries the device information of the first electronic device.

[0228] The device pairing unit 1520 is used to display device information based on the first electronic device, determine the first interconnection protocol corresponding to the first electronic device, and pair and connect with the first electronic device based on the first interconnection protocol.

[0229] Display unit 1530 is used to display the first application list sent by the first electronic device, and to present the identifiers of the applications in the first application list on the first page of the target application; the first application list includes the first application, which is an application installed in the first electronic device.

[0230] The execution unit 1540 is used to send a first instruction to the first electronic device in response to the user's selection operation of the first application identifier, so that the first electronic device can start the first application in the background, record and generate a video stream of the virtual display interface of the first application while displaying the second application on the foreground page.

[0231] The display unit 1530 is also used to display the video stream received by the display device from the first electronic device and to display the screen corresponding to the video stream on the second page of the target application.

[0232] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the screen projection control method provided in the above embodiments.

[0233] This application also provides a chip, which includes a processor and a memory. The memory stores a computer program, which, when executed by the processor, implements the screen projection control method provided in the above embodiments.

[0234] This application also provides a computer program product, which includes a computer program. When the computer program is run by an electronic device, the electronic device implements the screen projection control method provided in the above embodiments.

[0235] It should be understood that the processor mentioned in the embodiments of this application may be a central processing unit, a graphics processing unit, an image signal processor (ISP), and / or a neural network processing unit (NPU), or other general-purpose processors. A general-purpose processor may be a microprocessor or any conventional processor.

[0236] It should also be understood that the memory mentioned in the embodiments of this application may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory.

[0237] In the embodiments provided in this application, the division of various frameworks, modules, or units is merely a logical functional division. In actual implementation, there may be other division methods. For example, multiple frameworks, modules, or units may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the functional modules in the various embodiments of this application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0238] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0239] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0240] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A screen projection control method, characterized in that, Applied to display devices, including: When a device identification interaction link is established with the first electronic device and a first request for screen projection management is received from the first electronic device, the target application for screen projection management is launched, and the first interconnection protocol corresponding to the first electronic device is determined based on the device information of the first electronic device; the first request carries the device information of the first electronic device. Based on the first interconnection protocol, it is paired and connected with the first electronic device; The system receives a first application list sent by the first electronic device and displays the identifiers of the applications in the first application list on a first page of the target application; the first application list includes a first application, which is an application that has been installed on the first electronic device. In response to the user's selection of the first application identifier, a first instruction is sent to the first electronic device, so that while the first electronic device displays the second application on the foreground page, it simultaneously launches the first application in the background, records and generates a video stream of the virtual display interface of the first application. The system receives the video stream sent by the first electronic device and displays the corresponding screen of the video stream on a second page of the target application.

2. The screen projection control method according to claim 1, characterized in that, After receiving the first application list sent by the first electronic device, the method further includes: The first application list is stored in a preset format; The identifier of the application in the first application list displayed on the first page of the target application is: On the first page of the target application, the identifiers of the applications in the first application list are presented in a preset display format.

3. The screen projection control method according to claim 1, characterized in that, Also includes: When a device identification interaction link is established with the second electronic device and a second request for screen projection management is received from the second electronic device, the second interconnection protocol corresponding to the second electronic device is determined based on the device information of the second electronic device. The second request carries device information of the second electronic device; Based on the second interconnection protocol, it is paired and connected with the second electronic device; Obtain the second application list sent by the second electronic device, and store the second application list in a preset format; The second application list includes at least one application that is already installed on the second electronic device.

4. The screen projection control method according to claim 3, characterized in that, Also includes: Receive device switching operation, the device switching operation being used to instruct the first electronic device to be switched to the second electronic device; In response to the device switching operation, the identifiers of the applications in the second application list are displayed on the third page of the target application in a preset display format.

5. The screen projection control method according to claim 3, characterized in that, The device information of the second electronic device includes the manufacturer information of the second electronic device, which is different from the manufacturer information of the first device.

6. The screen projection control method according to any one of claims 1 to 5, characterized in that, The establishment of a device identification interaction link with the first electronic device includes: Send a first device discovery broadcast packet, the first device discovery broadcast packet carrying device information of the display device, and device information of electronic devices that have established a cooperative relationship with the display device; Receive a first response message returned by the first electronic device, the first response message carrying device information of the first electronic device; Based on the first response message, a device identification interaction link is established with the first electronic device.

7. The screen projection control method according to any one of claims 1 to 5, characterized in that, The method further includes: After the display device is powered on, a query command is sent to the cloud platform so that the cloud platform can query the inclusion status of the target application in the app store. Receive the inclusion status returned by the cloud platform; The establishment of a device identification interaction link with the first electronic device is as follows: if the inclusion status is "listed", then a device identification interaction link is established with the first electronic device.

8. The screen projection control method according to claim 1, characterized in that, After the second page of the target application displays the screen corresponding to the video stream, the process further includes: In response to the user's return operation on the target application's display interface, a third instruction is sent to the first electronic device to cause the first electronic device to return to the previous page of the first application in the background or to exit the first application in the background, and to record and generate a video stream of the virtual display interface of the first application. Upon receiving a fourth instruction from the first electronic device to exit the first application, the identifier of the application in the first application list is displayed on the display page of the target application.

9. A screen projection control method, characterized in that, Applied to electronic devices, including: Establish a device identification and interaction link with the display device; In response to the user's first operation, a request is sent to the display device to wake up the display device for screen mirroring management; After pairing and connecting with the display device, a target application list is determined and sent to the display device. The target application list includes at least one application that has been installed on the electronic device. After receiving the instruction from the display device that the first application has been selected, while displaying the second application on the foreground page, the first application is started in the background, and a video stream of the virtual display interface of the first application is recorded and generated. The video stream is sent to the display device.

10. A display device, characterized in that, The device includes a display, a memory, a controller, and a computer program stored in the memory and executable on the controller, wherein the controller executes the computer program to implement the screen projection control method as described in any one of claims 1 to 8.

Images