Display device and processing method for display device

Abstract

The present application provides a display device and a processing method for a display device, the display device comprising: a display configured to display a user interface and / or screen projection data; the communicator is configured to communicate with external equipment and / or be in projection screen connection with terminal equipment according to a communication protocol; a memory configured to store computer instructions and / or data associated with the display device; the at least one processor is connected with the display, the communicator and the memory, and is configured to execute the computer instructions to enable the display equipment to obtain the preparation state of the first application and start the second application in response to the screen projection instruction; if the preparation state is a suspension supporting state, controlling the display to display a second application above the first application in a suspension manner; and if the preparation state is a state of not supporting suspension, controlling the display to display the second application in a full-screen manner. When the two applications are displayed in the same window at the same time, the two applications are independent of each other, the expansibility is higher, the operability is better, and the maintenance is more convenient.

Description

Display device and processing method for display device

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent applications filed on December 14, 2022, with application number 202211611751.8; filed on December 14, 2022, with application number 202211611843.6; filed on December 14, 2022, with application number 202211610603.4; and filed on December 14, 2022, with application number 202211610610.4, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the technical field of display devices, and in particular to a display device and a processing method for the display device. Background Art

[0004] When using a display device to play applications such as videos, usually only one application is played and displayed on the current interface. However, in some cases, users may need to play two or more applications on the current interface, that is, to play applications on the same screen.

[0005] For example, when a user wants to watch two applications at the same time, the lower application can be played in full screen, and the upper application can be suspended above the lower application. To achieve the purpose of playing two or more applications on the same screen, on some display devices, the application interface can be suspended through the native interface.

[0006] However, in the process of realizing the same-screen display of applications through native interfaces, if the native interface is used to realize the suspension method, when the upper-layer application and the lower-layer application display different application interfaces respectively, the application interface cannot be controlled by the remote control or other buttons, and the upper-layer application and the lower-layer application cannot communicate. Therefore, the use of native interfaces leads to greater interactivity and limitations of the application. Moreover, the solution of realizing the suspension of the application interface through this method relies on the native interface. When the application version changes, the underlying code needs to be modified. For example, under normal circumstances, a system upgrade may be required. Therefore, the application maintenance cost is also high.

[0007] Summary of the Invention

[0008] According to an embodiment of the present application, a display device includes: a display, configured to display a user interface and / or projection data; a communicator, configured to communicate with an external device and / or be connected to a terminal device for projection according to a communication protocol; a memory, configured to store computer instructions and / or data associated with the display device; at least one processor, connected to the display, the communicator and the memory, and configured to execute the computer instructions to enable the display device to: obtain a ready state of a first application and start a second application in response to a projection instruction; wherein the first application is an application displayed in the current user interface, and the second application is a projection application; the ready state is used to indicate whether the first application supports floating display; if the ready state is a state that supports floating, the display is controlled to display the second application in a floating manner above the first application; if the ready state is a state that does not support floating, the display is controlled to display the second application in full screen.

[0009] According to an embodiment of the present application, a processing method for a display device includes: in response to a screen projection instruction, obtaining the ready state of a first application, and starting a second application; wherein, the first application is an application displayed in the current user interface, and the second application is a screen projection application; the ready state is used to indicate whether the first application supports floating display; if the ready state is a state that supports floating, controlling the display to float and display the second application above the first application; if the ready state is a state that does not support floating, controlling the display to display the second application in full screen. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] FIG1 is a schematic diagram of an operation scenario between a display device and a control device according to some embodiments of the present application;

[0011] FIG2 is a block diagram of a hardware configuration of the display device in FIG1 according to some embodiments of the present application;

[0012] FIG3 is a block diagram of a hardware configuration of the control device in FIG1 according to some embodiments of the present application;

[0013] FIG4 is a schematic diagram of software configuration in the display device of FIG1 according to some embodiments of the present application;

[0014] FIG5 is a schematic diagram of an icon control interface of a display device application according to some embodiments of the present application;

[0015] FIG6 is a flowchart of a processing method for a display device according to some embodiments of the present application;

[0016] FIG7 is a schematic diagram of a process of establishing a first application and a second application according to some embodiments of the present application;

[0017] FIG8 is a schematic diagram of an operation prompt interface displayed above a first application according to some embodiments of the present application;

[0018] FIG9 is a schematic diagram of a process of floating a second application above a first application according to some embodiments of the present application;

[0019] FIG10 is a schematic diagram illustrating an effect of floating a second application above a first application according to some embodiments of the present application;

[0020] FIG11 is a schematic diagram of a focus instruction flow according to some embodiments of the present application;

[0021] FIG12 is a schematic diagram of a process of exiting a second application according to a disconnect instruction according to some embodiments of the present application;

[0022] FIG13 is a flowchart illustrating a method of analyzing a disconnection reason according to a disconnection instruction according to some embodiments of the present application;

[0023] FIG14 is a schematic diagram of a display device displaying a prompt message according to some embodiments of the present application;

[0024] FIG15 is a schematic diagram of a process for playing media data according to a screen switching instruction according to some embodiments of the present application;

[0025] FIG16 is a schematic diagram of a second application displayed in a horizontal state according to some embodiments of the present application;

[0026] FIG17 is a schematic diagram of a second application displayed in a portrait state according to some embodiments of the present application;

[0027] FIG18 is a flowchart of another processing method for a display device according to some embodiments of the present application;

[0028] FIG19 is a schematic diagram showing the effect of an initial user interface in which applications are displayed on the same screen according to some embodiments of the present application;

[0029] FIG20 is a schematic diagram showing the effect of an operation prompt page according to some embodiments of the present application;

[0030] FIG21 is a schematic diagram showing the effect of another operation prompt page according to some embodiments of the present application;

[0031] FIG22 is a schematic diagram of a process of establishing data communication between a first application and a second application according to some embodiments of the present application;

[0032] FIG23 is an example diagram of the effect of displaying applications on the same screen according to some embodiments of the present application;

[0033] FIG24 is a schematic diagram showing the effect of applications being displayed on the same screen after switching according to some embodiments of the present application;

[0034] FIG25 is a schematic diagram showing the effect of shutting down a lower-layer application according to some embodiments of the present application;

[0035] FIG26 is a schematic diagram showing the effect of a shutdown reason prompt message according to some embodiments of the present application;

[0036] FIG27 is a schematic structural diagram of a display device according to some embodiments of the present application;

[0037] FIG28 is a rear view of a display device according to some embodiments of the present application;

[0038] FIG29 is a flowchart of a third processing method for a display device according to some embodiments of the present application;

[0039] FIG30 is a schematic diagram showing the effect of a display in a horizontal state according to some embodiments of the present application;

[0040] FIG31 is a schematic diagram showing the effect of a display in a portrait mode according to some embodiments of the present application;

[0041] FIG32 is a schematic diagram showing the effect of starting a first application in a landscape mode according to some embodiments of the present application;

[0042] FIG33 is a schematic diagram showing the effect of applications being displayed on the same screen in landscape mode according to some embodiments of the present application;

[0043] FIG34 is a schematic diagram illustrating an effect of displaying a second application in full screen in a landscape mode according to some embodiments of the present application;

[0044] FIG35 is a schematic diagram showing the effect of an operation prompt page in a horizontal screen state according to some embodiments of the present application;

[0045] FIG36 is a schematic diagram showing the effect of another operation prompt page in the horizontal screen state according to some embodiments of the present application;

[0046] FIG37 is a schematic diagram showing the effect of an operation prompt page in a vertical screen state according to some embodiments of the present application;

[0047] FIG38 is a schematic diagram of a process of establishing data communication between a first application and a second application according to some embodiments of the present application;

[0048] FIG39 is a schematic diagram illustrating the effect of switching to a portrait mode and displaying applications on the same screen according to some embodiments of the present application;

[0049] FIG40 is a schematic diagram illustrating an effect of switching to a portrait screen state in which the first application does not support the portrait screen orientation according to some embodiments of the present application;

[0050] FIG41 is a schematic structural diagram of a display device according to some embodiments of the present application;

[0051] FIG42 is a first schematic diagram of a terminal device displaying a screen projection settings page according to an embodiment of the present application;

[0052] FIG43 is a second schematic diagram of a terminal device displaying a screen projection settings page according to an embodiment of the present application;

[0053] FIG44 is a schematic diagram of a display device displaying a screen projection request page according to an embodiment of the present application;

[0054] FIG45 is a schematic diagram of the display interface between the display device and the terminal device after the screen projection connection is successful according to an embodiment of the present application;

[0055] FIG46 is a schematic diagram of a terminal device displaying a first projection page according to an embodiment of the present application;

[0056] FIG47 is a schematic diagram of a terminal device displaying a screen recording settings page according to an embodiment of the present application;

[0057] FIG48 is a schematic diagram showing a principle of dynamically adjusting image quality parameters of a display device according to an embodiment of the present application;

[0058] FIG49 is a flowchart of a fourth processing method for a display device according to an embodiment of the present application;

[0059] Figure 50 is a second flowchart of the fourth processing method for a display device according to an embodiment of the present application. DETAILED DESCRIPTION

[0060] To make the objectives, technical solutions, and advantages of some embodiments of the present application more clear, the following will clearly and completely describe the technical solutions of some embodiments of the present application in conjunction with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present application, not all of the embodiments.

[0061] It should be noted that the brief descriptions of terms in some embodiments of the present application are only for the purpose of facilitating understanding of the embodiments described below, and are not intended to limit the embodiments of some embodiments of the present application. Unless otherwise specified, these terms should be understood according to their ordinary and usual meanings.

[0062] The terms "comprise," "include," and "have," and any variations thereof, are intended to cover but not exclude inclusion; for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0063] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functionality associated with that element.

[0064] The display device provided in the embodiments of the present application can have various implementation forms, for example, it can be a television, a smart TV, a laser projection device, a monitor, an electronic bulletin board, an electronic table, etc. Figures 1 and 2 illustrate a specific embodiment of the display device of the present application.

[0065] Figure 1 is a schematic diagram of an operation scenario between a display device and a control device according to some embodiments of the present application. As shown in Figure 1 , a user can operate a display device 200 through a terminal device 300 and a control device 100 .

[0066] In some embodiments, the control device 100 may be a remote control. Communication between the remote control and the display device may include infrared protocol communication, Bluetooth protocol communication, or other short-range communication methods, and the display device 200 may be controlled wirelessly or wired. The user may control the display device 200 by inputting user commands through buttons on the remote control, voice input, control panel input, etc. Alternatively, the control device 100 may be a mouse, and the mouse and the display device may be connected via a wired or wireless connection.

[0067] In some embodiments, the terminal device 300 can install software applications on the display device 200 and connect and communicate via a network communication protocol, enabling one-to-one control operations and data communication. Audio and video content displayed on the terminal device 300 can also be transmitted to the display device 200 for synchronized display. The terminal device 300 can be a mobile terminal, tablet computer, computer, laptop computer, tablet computer, smart screen, etc.

[0068] In some embodiments, the display device may not use the above-mentioned terminal device or control device to receive instructions, but may receive user control through touch or gestures.

[0069] In some embodiments, the display device 200 can also be controlled in a manner other than the control device 100 and the terminal device 300. For example, the user's voice command control can be directly received through a module for obtaining voice commands configured inside the display device 200, or the user's voice command control can be received through a voice control device set outside the display device 200.

[0070] As also shown in FIG1 , the display device 200 also communicates data with the server 400 via various communication methods. The display device 200 may be connected to a local area network (LAN), a wireless local area network (WLAN), or other networks. The server 400 may provide various content and interactions to the display device 200.

[0071] In addition to providing a broadcast reception television function, the display device 200 may also provide an Internet TV function, including but not limited to Internet TV, smart TV, Internet Protocol Television (IPTV), and the like.

[0072] FIG2 is a block diagram of a hardware configuration of the display device 200 in FIG1 according to some embodiments of the present application. The display device 200 includes at least one of a tuner / demodulator 210, a communicator 220, a detector 230, an external device interface 240, at least one processor 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0073] In some embodiments, detector 230 is used to collect signals from the external environment or external interactions. For example, detector 230 may include a light receiver, such as a sensor for collecting ambient light intensity; or an image collector, such as a camera, for collecting external environmental scenes, user attributes, or user interaction gestures; or a sound collector, such as a microphone, for receiving external sounds.

[0074] In some embodiments, display 260 includes a display screen component for presenting images, a driver component for driving image display, and a component for receiving image signals output by at least one processor to display video content, image content, a menu control interface, and a user control UI interface. Display 260 can be a liquid crystal display, an OLED display, or a projection display, and can also be a projection device and projection screen.

[0075] In some embodiments, the communicator 220 is a component used to communicate with an external device or server 400 using various communication protocols. For example, the communicator 220 may include at least one of a Wi-Fi module, a Bluetooth module, a wired Ethernet module, an RF (radio frequency) module, or other network communication protocol chip or a near-field communication protocol chip, as well as an infrared receiver. The display device 200 can establish communication with the control device 100 or server 400 through the communicator 220 to send and receive control signals and data signals.

[0076] In some embodiments, at least one processor 250 controls the operation of the display device and responds to user operations via various software control programs stored in memory. The at least one processor 250 controls the overall operation of the display device 200. For example, in response to receiving a user command to select a UI object for display on the display 260, the at least one processor 250 may perform operations related to the object selected by the user command.

[0077] In some embodiments, at least one processor 250 includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), a random access memory (RAM), a read-only memory (ROM), a first interface to an nth interface for input / output, a communication bus, etc.

[0078] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0079] In some embodiments, the external device interface 240 may include, but is not limited to, any one or more of the following: a high-definition multimedia interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input / output interface formed by multiple of the above interfaces.

[0080] In some embodiments, the tuner-demodulator 210 receives broadcast television signals via wired or wireless reception, and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

[0081] In some embodiments, at least one processor 250 and the tuner / demodulator 210 may be located in different separate devices, that is, the tuner / demodulator 210 may also be located in an external device of the main device where the at least one processor 250 is located, such as an external set-top box.

[0082] In some embodiments, the user interface is an interface that can be used to receive control input.

[0083] FIG3 is a block diagram of the hardware configuration of the control device in FIG1 according to some embodiments of the present application. As shown in FIG3 , the control device 100 includes at least one processor 110 , a communication interface 130 , a user input / output interface 140 , a memory 190 , and a power supply 180 .

[0084] The control device 100 is configured to control the display device 200 , and can receive user input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200 .

[0085] The control device 100 further includes a RAM 120, a ROM 121 and a communication bus. At least one processor 110 is used to control the operation and functionality of the control device 100, as well as communication and coordination between internal components and external and internal data processing functions.

[0086] Under the control of at least one processor 110, the communication interface 130 communicates control signals and data signals with the display device 200. The communication interface 130 may include at least one of a WiFi chip 131, a Bluetooth module 132, an NFC module 133, or other near field communication modules.

[0087] The user input / output interface 140 includes at least one of a microphone 141 , a touch panel 142 , a sensor 143 , a button 144 and other input interfaces.

[0088] In some embodiments, the control device 100 includes at least one of a communication interface 130 and a user input / output interface 140. The control device 100 is configured with the communication interface 130, such as a WiFi, Bluetooth, or NFC module, to encode user input commands via the WiFi protocol, Bluetooth protocol, or NFC protocol and transmit them to the display device 200.

[0089] The memory 190 is used to store various operating programs, data, and applications for driving and controlling the control device 100 under the control of at least one processor. The memory 190 can store various control signal instructions input by the user.

[0090] The power supply 180 is used to provide operating power support for various components of the control device 100 under the control of at least one processor.

[0091] Figure 4 is a schematic diagram of the software configuration in the display device in Figure 1 according to some embodiments of the present application. In some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (referred to as "application layer"), the application framework layer (referred to as "framework layer"), the system library layer (referred to as "system runtime library layer"), and the kernel layer.

[0092] In some embodiments, at least one application is running in the application layer. These applications may be window programs, system settings programs, clock programs, camera applications, etc. that come with the operating system; or they may be applications developed by third-party developers.

[0093] The framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes predefined functions and acts as a processing center, determining the actions taken by applications in the application layer.

[0094] As shown in FIG4 , in some embodiments of the present application, the application framework layer includes managers, content providers, and a view system, etc.

[0095] In some embodiments, the activity manager is used to manage the lifecycle of each application and general navigation back functionality.

[0096] In some embodiments, a window manager is used to manage all window programs.

[0097] In some embodiments, the system runtime layer provides support for the upper layer, namely the framework layer. When the framework layer is accessed, the operating system will run the C / C++ library contained in the system runtime layer to implement the functions to be implemented by the framework layer.

[0098] In some embodiments, the kernel layer is a layer between hardware and software. As shown in FIG4 , the kernel layer includes at least one of the following drivers: an audio driver, a display driver, a Bluetooth driver, a camera driver, a Wi-Fi driver, a USB driver, an HDMI driver, and sensor drivers (such as a fingerprint sensor, a temperature sensor, a touch sensor, a pressure sensor, etc.).

[0099] In some embodiments, the core layer further includes a power driver module for performing power management.

[0100] As shown in Figure 5, the application layer of the display device 200 may include at least one application that can display corresponding icon controls in the display, such as: live TV application icon controls, video on demand application icon controls, media center application icon controls, application center icon controls, game application icon controls, etc.

[0101] Based on the display device 200, media data such as videos can be played. When using the display device 200 to play videos and other applications, usually only one application is played and displayed on the current interface. However, in some cases, the user may need to play two or more applications on the current interface, that is, to play applications on the same screen.

[0102] In some embodiments, when a user wants to watch two applications simultaneously, the lower application can be played in full screen, and the upper application can be suspended above the lower application. To achieve the purpose of playing two or more applications on the same screen, in some display devices, the application interface can be suspended through the native interface.

[0103] Take the scenario where the user uses a display device to assist in fitness as an example. When the user is exercising through the display device, he wants to display both the camera preview screen, that is, the user's own fitness action screen, and the projection screen or fitness video screen in the same window, that is, he wants to achieve the purpose of learning and practicing in the same window. For this scenario, the projection screen SDK (Software Development Kit) can be integrated into the fitness application in the display device. After the display device receives the projection screen instruction, the lower-level fitness application displays the user's fitness preview screen, and the upper-level application can be displayed floating above the lower-level application, thus achieving the purpose of displaying two applications in the same window. In some embodiments of the present application, the fitness application can be understood as a related application that supports users to display fitness pictures, which can be combined with the camera of the display device 200. The user stands in front of the display device 200, the camera captures the user's movements, and displays the user's fitness pictures through the fitness application.

[0104] However, integrating the SDKs of multiple applications into the same application will lead to problems such as poor interactivity and scalability during the maintenance of applications that integrate multiple SDKs. That is, when the projection application transmits data or connections to the fitness application, the fitness application needs to start playing its own interface first after receiving it. When the projection application needs to be switched between full screen or non-full screen, the code and design of the related applications may need to be rewritten, and the maintenance and upgrade costs are high. For example, if a third-party application wants to achieve same-window playback, the third-party application also needs to be integrated, and the scalability is also poor, which will affect the operating efficiency of the display device and the user experience.

[0105] In addition, in the process of realizing the same-screen display of applications through the native interface mentioned above, when the upper-layer application and the lower-layer application display different application interfaces respectively, the application interface cannot be controlled by the remote control or other buttons, and the upper-layer application and the lower-layer application cannot communicate. Therefore, the use of the native interface method leads to greater interactivity and limitations of the application. Moreover, the solution of realizing the suspension of the application interface through this method relies on the native interface. When the application version changes, the underlying code needs to be modified. For example, under normal circumstances, a system upgrade may be required. Therefore, the application maintenance cost is also high.

[0106] In order to solve the problems of large interactivity, limitations and high maintenance costs of applications during the same-screen display of applications, some embodiments of the present application provide a processing method for a display device. In some embodiments, the display device 200 includes a display 260, which is configured to display a user interface and / or projection data; a communicator 220, which is configured to communicate with an external device and / or connect to a terminal device for projection according to a communication protocol; a memory, which is configured to store computer instructions and / or data associated with the display device; at least one processor 250, which is connected to the display 260, the communicator 220 and the memory, and is configured to execute the computer instructions to enable the display device 200 to: respond to the projection instruction, obtain the preparation state of the first application, and start the second application; wherein the first application is the application displayed in the current user interface, and the second application is the projection application; the preparation state is used to indicate whether the first application supports floating display; if the preparation state is a state that supports floating, the display is controlled to display the second application in a floating manner above the first application; if the preparation state is a state that does not support floating, the display is controlled to display the second application in full screen. This application aims to establish interaction between upper and lower applications by determining the readiness state of the lower application to determine whether it should be displayed floating above the lower application. Because the lower application is suspended by the upper application and cannot capture the focus, the upper application captures the focus command and sends it to the lower application, which then sends it to other applications. Other applications can then perform corresponding operations based on the focus command.

[0107] To facilitate understanding of the technical solutions in some embodiments of the present application, the following describes each step in detail in conjunction with some specific embodiments and drawings. FIG6 is a flow chart of a processing method for a display device according to some embodiments of the present application. As shown in FIG6 , a processing method for a display device may include the following steps S11-S13, specifically including:

[0108] S11: In response to the screen projection instruction, obtain the preparation status of the first application and start the second application.

[0109] In some embodiments, the screen casting command can be issued through a terminal device such as a mobile phone, or can be issued by a remote control associated with the display device 200, or can be issued in other forms, which are not limited in this application. It is understood that a screen casting connection can be established through the screen casting command, so that the screen casting application can be displayed through the screen casting command.

[0110] In some embodiments, the user can issue a screen projection instruction through the terminal device 300, and send the screen projection instruction to the display device 200 through the WiFi network. The display device 200 then completes the transmission protocol configuration according to the screen projection instruction, thereby establishing a transmission channel for screen projection data with the terminal device 300. For example, for system applications or direct screen projection at the system level, the terminal device 300 can use Miracast projection, Airplay projection or WiDi projection protocol to establish a transmission channel for screen projection data with the display device 200. The display device 200 receives the screen projection data of the terminal device 300 through the transmission channel, that is, it can display the user interface of the application in the terminal device 300 on the display 260.

[0111] In some embodiments, the display device 200 can also achieve a screen projection connection with the terminal device 300 through different connection methods. For example, when the display device 200 and the terminal device 300 are connected to the same wireless local area network, a screen projection connection can be established based on the WiFi network. For another example, when both the display device 200 and the terminal device 300 are provided with NFC (Near Field Communication) components, a screen projection connection relationship can be established through the NFC component. Obviously, other wired or wireless connection methods can also be used between the display device 200 and the terminal device 300 to establish a screen projection connection relationship, such as RF radio frequency connection, infrared connection, cellular network, etc.

[0112] In order to establish the interaction between the upper-layer application and the lower-layer application, in some embodiments, the display device 200 can be implemented in the following manner. Figure 7 is a flow chart of establishing the first application and the second application according to some embodiments of the present application. As shown in Figure 7, the display device 200 traverses all deployed applications and filters out the second application that supports the floating function from all applications. It can be understood that the second application can be one or more. When there is one second application, two applications can be displayed at the same time in the same window. When there are multiple second applications, multiple applications are displayed at the same time in the same window. After the second application that supports the floating function is filtered out, the client services in the first application and the second application are bound, thereby establishing a data intercommunication relationship between the first application and the second application.

[0113] It should be noted that when establishing a data intercommunication relationship between the first application and the second application, you can also bind it to the client service of other applications after the first application starts the camera. In this way, after binding, the first application can send messages to other applications such as the second application, such as sending whether the first application supports the preparation status of the floating application. After receiving the message sent by the first application, other applications can use this as a basis to execute subsequent processes.

[0114] In some embodiments, the first application is an application displayed in the current user interface, such as a lower-layer application, and the second application is a screen projection application, such as an upper-layer application that is displayed in a floating state. The ready state is used to indicate whether the first application supports floating display. The ready state in the first application may include multiple states such as preparing, ready, and abnormal preparation, which are not limited in this application. After the second application is started, the first application can send its ready state to the second application so that the second application can determine whether to display it in a floating state above the first application.

[0115] In some embodiments, metadata information can be used to traverse all applications on the current machine to determine whether there is a second application that supports suspension. If a second application that supports suspension is found, the preset client services can be bound in turn, so that the first application and the second application complete the interaction.

[0116] In some embodiments, the status of the first application can be represented by different identifiers. The identifier of the first application and the content it represents may include: IDLE-preparing, PREPARE-preparation completed, FOCUS-preparation completed, FINISH-interface exit, ERROR-abnormal. It should be noted that the above identifiers are only for schematic illustration and do not constitute a limitation on the content of the identifier. After obtaining the preparation status of the first application, based on the interactive function between the first application and the second application, the first application can send the preparation status to the second application for the second application to determine whether it is suspended. After step S11 is completed, the following step S12 can be executed.

[0117] S12: If the ready state is a state that supports floating, the display device controls the display to display the second application in a floating manner above the first application.

[0118] In some embodiments, after receiving the preparation status sent by the first application, the second application can determine whether it supports the floating state based on the specific situation of the preparation status. When the preparation status of the first application is PREPARE preparation completed or FOCUS preparation completed, it can represent that it supports floating. At this time, the display device 200 can control the display 260 to display the second application in a floating state above the first application.

[0119] It should be noted that the identification of the ready state may be different for different applications. In some embodiments, the display device 200 may also set a corresponding identification to characterize the ready state of the first application according to the type of the first application. For example, for certain self-developed screen projection applications that rely on wireless network communication technology, they belong to background service applications. Since the background service applications themselves are configured with background operation and suspension functions, the PREPARED state represents a ready state that supports suspension. For non-background service applications, for example, some offline screen projections that rely on the startup interface can only be used on a certain page. The application can control its function switch through FOCUS to notify the current application that it can support the suspension interface display.

[0120] In some embodiments, before the second application is displayed floating above the first application, the display device 200 can control the display 260 to display an operation prompt interface above the first application to provide operation prompts for the user to operate the screen projection and guide the user on how to implement the screen projection operation.

[0121] Figure 8 is a schematic diagram of the operation prompt interface displayed above the first application according to some embodiments of the present application. As shown in Figure 8, the user screen captured by the camera can be displayed in interface 71, and the name of the projection device, the network name, the current projection progress status, operation prompts, etc. will be displayed in interface 72, that is, the operation prompt interface. In this way, the user can operate according to the operation prompts on the one hand, and on the other hand, the display device 200 can feedback the current status of the projection connection on the display 260.

[0122] FIG9 is a schematic diagram of a process for floating a second application above a first application according to some embodiments of the present application. As shown in FIG9 , in some embodiments, floating the second application above the first application can be performed in the following manner: first, the display device 200 controls the display 260 to display the first application in full screen; then, the theme of the second application is set to transparent; and finally, the display 260 is controlled to display the second application with a transparent theme over the first application.

[0123] FIG10 is a schematic diagram of the effect of floating a second application above a first application according to some embodiments of the present application. As shown in FIG10 , taking the first application as a fitness application as an example, when the second application is displayed floating on the first application, the fitness application will first be displayed in full screen on the interface 91. After receiving the screen projection instruction, the display device 200 will determine the current readiness state of the fitness application. If the readiness state of the first application is a prepared state such as PREPARED or FOCUS, it means that the first application supports the floating state. That is to say, in the embodiment of the present application, when performing the floating display operation, the second application displayed in the floating state will determine the readiness state of the first application in the lower layer. Only when the lower application supports floating display will the second application be displayed in the floating state above the first application in the lower layer.

[0124] In some embodiments, when a second application is displayed floating above a first application, the theme of the second application can be set to transparent. In this way, the second application can be displayed in the same window as the first application, and the second application can be displayed floating above the first application without completely blocking the first application, thus realizing the function of floating the second application above the first application. It should be noted that during the screen projection process, one application can be displayed floating above the first application, or multiple applications can be displayed floating, which is not specifically limited in this application.

[0125] Still taking the first application as a fitness application as an example, the interface 91 in the fitness application displays the camera preview screen, that is, the user's fitness screen. When the second application is displayed in a floating manner above the first application, the position of the floating display can be pre-set. For example, it can be displayed in the upper right corner or in the upper left corner. Taking the floating display of the second application in the upper right corner as an example, assuming that the second application is a projected fitness video screen, its top right corner interface 92 is the fitness video screen, and the floating interface outside the screen is transparent. In this way, only the fitness video screen partially blocks the first application, the purpose of which is to learn and practice at the same time. The user can still see the other part of the fitness application interface, so that it will not affect the user's viewing of his own fitness screen, thereby achieving the purpose of displaying two applications in the same window at the same time.

[0126] In some embodiments, combined with the operation prompt interface of Figure 8, when the first application supports the floating display of the second application, the floating display operation of the second application can be performed. At the same time, after the second application is displayed in a floating manner above the first application, the underlying fitness application can be notified that the floating operation has been started by sending a message, etc., and a floating receipt is generated, and the floating receipt is sent to the first application. In the embodiment of the present application, the floating receipt can be understood as a receipt information that the second application has successfully achieved the state of floating display. In this way, after receiving the floating receipt, the first application can exit the operation prompt interface according to the floating receipt. At this time, the operation prompt interface in Figure 8 disappears, and only the second application is displayed in a floating manner above the fitness application.

[0127] The floating receipt sent by the second application to the first application can be status information, for example, it can be a connected CONNECT state or a disconnected DISCONNECT state. If the upper-layer second application sends a connected CONNECT state to the lower-layer first application, it means that the floating display of the first application and the second application has been successful, and the operation prompt interface in Figure 8 disappears. On the contrary, if the second application sends a disconnected DISCONNECT state to the first application, it means that the floating display of the two applications has not been successful, and the operation prompt interface in Figure 8 can be retained to allow the user to perform the floating display operation again based on it. After step S102 is completed, the following step S3 can be executed.

[0128] S13: If the preparation state is a state that does not support suspension, the display device controls the display to display the second application in full screen.

[0129] In some embodiments, after the second application receives the ready status sent by the first application, it determines whether it supports the floating state through the ready status. If the ready status is a state that does not support floating, the display device 200 can control the display 260 to display the second application in full screen.

[0130] The state that does not support suspension may be the FINISH interface exit or ERROR exception state, that is, the first application may have an abnormal situation. For example, the camera of the display device 200 may have an abnormality, or the first application interface may have an abnormality. In this scenario, after receiving the screen projection instruction, the display device 200 can control the display 260 to display the second application in full screen, that is, to display the screen projection interface in full screen, so that the user can complete the fitness exercise based on the second application.

[0131] It should be noted that in the embodiment of the present application, when two applications are displayed simultaneously in the same window, that is, when two application screens are displayed on the same screen, since the two applications are independent of each other and the SDKs of the two applications are not integrated into the same application, the first application and the second application do not rely on native interfaces, and the application is more scalable and operable, and maintenance is easier and reduces maintenance costs. In addition, since interactive communication is established between the two applications, while the two applications are displayed in superposition, interaction between the two applications is also achieved. The two applications can communicate with each other and perform related operations as needed, thereby solving the problems of application interactivity, large limitations, and high maintenance costs during the display of applications on the same screen.

[0132] It can be seen from the above technical solution that the above embodiment provides a processing method for a display device that responds to a screen projection instruction, obtains the ready state of the first application, and starts the second application; if the ready state is a state that supports floating, controls the display to display the second application in a floating state above the first application; if the ready state is a state that does not support floating, controls the display to display the second application in full screen. When the embodiment of the present application displays two applications at the same time in the same window, since the two applications are independent of each other and are not integrated into the same application, the first application and the second application do not rely on the native interface, the application is more expandable and more operable, and maintenance is easier and reduces maintenance costs. In addition, since interactive communication can be established between the two applications, while the two applications are displayed in superposition, interaction between the two applications can also be achieved. The two applications can communicate with each other and perform related operations as needed, thereby solving the problems of application interactivity, large limitations, and high maintenance costs during the same-screen display of applications.

[0133] In some embodiments, after a second application is displayed floating above a first application, the first application below it cannot receive a focus command because the first application is obscured by the second application. The focus command may be a command issued by a remote control associated with the display device 200. Consequently, the remote control cannot control the first application, and the first application cannot receive the focus command issued by the remote control.

[0134] In order to ensure that the first application in the lower layer can also obtain the focus instruction, the display device 200 can also perform the following operations. Figure 11 is a schematic diagram of the focus instruction flow according to some embodiments of the present application. As shown in Figure 11, in some embodiments, the display device 200 can control the second application to capture the focus instruction in the user interface, and then the second application can send the captured focus instruction to the first application. After receiving the focus instruction, the first application can parse the focus instruction and generate a parsing result. At the same time, the parsing result of the focus instruction is sent to other applications so that other applications perform related operations based on the parsing result, wherein other applications are applications in the display device other than the first application and the second application.

[0135] After the focus command is issued through the remote control, the second application can capture the focus command because it is displayed above the first application in a floating manner, while the first application below cannot capture the focus command, and the remote control cannot control the first application below.

[0136] In order to enable the first application to also obtain the focus instruction, in some embodiments, after the second application captures the focus instruction, it can send the focus instruction to the first application based on the data intercommunication relationship established between the first application and the second application. After the first application receives the focus instruction, it can parse the focus instruction and generate a parsing result. The focus instruction can be to adjust the contrast of the current screen. When the second application captures the focus instruction, it can send an instruction to adjust the contrast of the screen to the first application. After the first application receives the instruction, it can notify the instruction to other applications except the first application and the second application. In this way, all applications obtain the focus instruction issued by the remote control and perform corresponding operations according to the specific content of the instruction.

[0137] FIG12 is a schematic diagram of a process for exiting a second application according to a disconnection instruction in accordance with some embodiments of the present application. As shown in FIG12 , in some embodiments, the display device 200 may further execute the following process, including: in response to a screen projection disconnection event, the display device 200 generates a disconnection instruction. The screen projection disconnection event may be a disconnection instruction sent by a remote control, or an operation to disconnect the screen projection issued by a terminal device such as a mobile phone, which is not specifically limited in this application.

[0138] After the display device 200 generates a disconnect instruction, the second application at the upper layer can capture the disconnect instruction and send the disconnect instruction to the first application through the data intercommunication relationship established with the first application. After the first application receives the disconnect instruction sent by the second application, it can send the disconnect instruction to other applications other than the first application and the second application. In this way, since the second application itself is a screen projection application, after receiving the disconnect instruction, it can perform the screen projection operation by itself or through other applications. At the same time, the display device 200 can control the display 260 to exit the second application according to the disconnect instruction. In this way, after capturing the focus instruction through the upper-layer application interface, it can be notified to the underlying application through cross-process communication. The underlying application is exemplified as the first application. After receiving the focus instruction, the underlying application sends the focus instruction to other applications through the underlying application itself. On the one hand, it realizes the mutual communication of multiple applications and the purpose of displaying multiple applications at the same time in the same window. On the other hand, it solves the problem that the first application cannot obtain the focus instruction.

[0139] After the focus command is issued, the second application at the top captures it and forwards it to the first application at the bottom. This forwards or executes the event or task in the focus command through the first application at the bottom, notifying other applications, such as middleware applications, to perform related operations. This allows two or more applications to be displayed simultaneously, while also ensuring that all applications receive the focus command, ensuring smooth execution of commands on display device 200.

[0140] In order to let the user know the reason why the second application disconnected from the screen projection, in some embodiments, the disconnection instruction can also carry the disconnection reason. Figure 13 is a schematic diagram of the process of parsing the disconnection reason based on the disconnection instruction according to some embodiments of the present application. As shown in Figure 13, the user can be prompted with the reason for disconnecting the screen projection in the following ways.

[0141] When the display device 200 receives the disconnect instruction, it can first parse the disconnect reason of the disconnect instruction from the disconnect instruction. The screen projection application may be issued by a terminal device such as a mobile phone. When the mobile phone wants to disconnect the screen projection connection with the display device 200, it can notify the second application to disconnect the screen projection connection by sending a message, and at the same time carry the disconnection reason, such as active disconnection by the mobile phone, disconnection due to network reasons, disconnection due to multiple screen projection conflicts, etc. In some embodiments, after receiving the above-mentioned disconnection reason, the second application can send a disconnect message to the underlying first application. In this way, the first application can send the disconnection message to other applications besides the first application and the second application, and the display device 200 can generate a prompt message based on the disconnection instruction and the disconnection reason.

[0142] Figure 14 is a schematic diagram of a display device displaying a prompt message according to some embodiments of the present application. As shown in Figure 14, after the display device 200 generates a prompt message based on the disconnection instruction and the disconnection reason, it can control the display 260 to display the prompt message when exiting the second application such as the projection application. In this way, the user can intuitively understand the reason why the second application disconnects the projection. It is understandable that after the prompt message is displayed on the display 260, the second application will no longer be displayed floating above the first application. In other words, when the mobile phone issues a disconnection instruction, the interface of the second application will be destroyed and the interface of the second application will no longer be displayed.

[0143] In some embodiments, the display device 200 can also detect the running status of the first application. If the running status of the first application is abnormal, it detects the projection status of the second application; if the second application is in the projection state, it exits the second application; if the second application is in the non-projection state, it controls the display 260 to display the second application in full screen.

[0144] The first application may run abnormally for some reasons, for example, the first application configuration is abnormal, the process is abnormal, etc. After the display device 200 detects that the running state of the first application is abnormal, it can adaptively set the state of the second application. In some embodiments, after detection, if the running state of the first application is abnormal, when the second application is in the projection state, because the first application may no longer support the projection state when an abnormality occurs, the first application can send an unprepared state to the second application, and then the second application can be exited at this time to ensure the consistency of the cooperation between the first application and the second application. If the second application is in a non-projection state, then at this time you don’t need to pay attention to whether the first application supports projection. In order to ensure that the user can see the interface, the display device 200 can directly control the display 260 to display the second application in full screen. In this way, although the first application has an abnormality, other applications that have not performed projection can still be displayed normally.

[0145] In order to support the user to switch the projection display effect, in some embodiments, the display device 200 can also perform corresponding operations according to the screen switching instructions input by the user, such as displaying the media data in the second application in horizontal or vertical screen according to the screen switching instructions.

[0146] Figure 15 is a flow chart of playing media data according to a screen switching instruction in accordance with some embodiments of the present application. As shown in Figure 15, in some embodiments, the media data in the second application can be displayed in a horizontal or vertical screen according to the screen switching instruction in the following manner. In response to the user's screen switching instruction to play media data for the second application, the display device 200 obtains the playback window size of the second application. After obtaining the current playback window size, the display device 200 can modify the playback window size according to the screen switching instruction, and play the media data in the second application according to the modified playback window size. It can be understood that the playback window size can include the width and height of the second application, i.e., the projection application interface. When the width of the interface is greater than the height, the second application is played in a horizontal screen; when the width is less than the height, the second application is played in a vertical screen. The process of playing media data according to the screen switching instruction is also the process of switching the horizontal and vertical screen display effects.

[0147] The screen switching instruction can be issued by the user through the horizontal and vertical screen switching of the mobile phone, or it can be issued by switching the menu key of the remote control, which is not limited in this application. It is understandable that the display device 200 can receive the screen switching instruction in the following manner and execute the corresponding process according to the received screen switching instruction.

[0148] When a screen switching instruction is issued through the mobile phone or the remote control menu key, the second application first obtains the screen switching instruction, and then the display device 200 obtains the current playback window size of the second application.

[0149] After obtaining the current size of the playback window, the display device 200 can modify the size of the playback window according to the screen switching instruction. The screen switching instruction can be to switch the playback mode of the current second application from landscape to portrait. Then, through calculation, the display device 200 can calculate the aspect ratio corresponding to portrait, that is, the modified size of the playback window. In this way, the media data in the second application can be played according to the modified size of the playback window.

[0150] In some embodiments, when the mobile phone switches between landscape and portrait or the remote control menu key is pressed, the display device 200 can modify the display size of the screen mirroring application interface according to the width and height of the video stream. At the same time, certain display rules can be set. For example, FIG. 16 is a schematic diagram of the second application displayed in landscape mode according to some embodiments of the present application. If the screen-mirrored second application is in landscape display mode, then as shown in FIG. 16, the second application can be floatingly displayed in the upper right corner of the current window. FIG. 17 is a schematic diagram of the second application displayed in portrait mode according to some embodiments of the present application. If the screen-mirrored second application is in portrait display mode, then as shown in FIG. 17, the second application can be floatingly displayed on the right side of the current window.

[0151] In some embodiments, the process of displaying the media data in the second application in landscape or portrait according to the screen switching instruction can be implemented in the following manner. If the current video stream is in landscape, it is default to be floating in the upper right corner, with a fixed width, and the height is scaled according to the original ratio. The size of the screen mirroring application interface is dynamically set through a specific field such as the setLayoutParams field. If the current video stream is in portrait, it is default to be floating on the right side, with a fixed height, and the width is scaled according to the ratio of the video stream on the mobile phone side. The size of the screen mirroring application interface is dynamically set through the setLayoutParams. The remote control can also follow this logic. The specific calculation methods for the width and height of the screen mirroring application interface can be as follows:

[0152] Denote the width of the video stream as VideoWidth and the height of the video stream as VideoHeight. In some embodiments, if VideoWidth≥VideoHeight, the width of the parent layout is a fixed width, denoted as X, and the height of the parent layout is X*VideoHeight / VideoWidth; if VideoWidth<VideoHeight, the height of the parent layout is a fixed height, denoted as Y, and the width of the parent layout is Y*VideoWidth / VideoHeight. Here, the parent layout is the width and height of the outer screen mirroring application interface.

[0153] In some embodiments, the method for calculating the width and height when switching between portrait and landscape modes on the mobile device or by pressing the menu key on the remote control may also be as follows: Obtain the width and height of the current parent layout, denoted as ViewWidth and ViewHeight respectively. If VideoWidth ≥ VideoHeight, the height of the parent layout is a fixed height, denoted as Y, and the width of the parent layout is Y * VideoHeight / VideoWidth. If VideoWidth < VideoHeight, the height of the parent layout is a fixed height, denoted as Y, and the width of the parent layout is Y * VideoWidth / VideoHeight. It should be noted that the above calculation process is only for illustrative purposes and does not represent the actual calculation process. There may be other calculation methods or calculation steps, etc., which are not limited in this application, and the above calculation method does not constitute a limitation to this application either.

[0154] As can be seen from the above technical solutions, in addition to implementing the simultaneous display of two or more applications in the same window, solving the problems of large interactivity and limitations, and high maintenance costs during the co-screen display of applications, the processing method for a display device provided in the above embodiments can also switch the portrait and landscape display effects of the second application according to the screen switching instruction. In this way, while meeting the user experience, the utilization rate of the display 260 of the display device 200 can be increased.

[0155] Based on the above processing method for a display device, a display device 200 according to some embodiments of the present application includes a display 260 configured to display a user interface and / or screen mirroring data; a communicator 220 configured to communicate with an external device according to a communication protocol and / or establish a screen mirroring connection with a terminal device; a memory configured to store computer instructions and / or data associated with the display device; and at least one processor 250 connected to the display 260, the communicator 220, and the memory, and configured to execute the computer instructions to cause the display device 200 to: in response to a screen mirroring instruction, obtain the preparation status of a first application, and start a second application; wherein, the first application is the application displayed in the current user interface, and the second application is the screen mirroring application; the preparation status is used to indicate whether the first application supports floating display; if the preparation status is a state of supporting floating, control the display to floatingly display the second application above the first application; if the preparation status is a state of not supporting floating, control the display to full-screen display the second application.

[0156] It can be seen from the above technical solution that the display device 200 provided in the above embodiment responds to the screen projection instruction, obtains the ready state of the first application, and starts the second application; if the ready state is a state that supports floating, controls the display 260 to display the second application in a floating state above the first application; if the ready state is a state that does not support floating, controls the display 260 to display the second application in full screen. When the embodiment of the present application displays two applications at the same time in the same window, since the two applications are independent of each other and are not integrated into the same application, the first application and the second application do not rely on the native interface, the application is more expandable and more operable, and maintenance is easier and reduces maintenance costs. In addition, since interactive communication is established between the two applications, while the two applications are displayed in superposition, interaction between the two applications is also realized. The two applications can communicate with each other and perform related operations as needed, thereby solving the problems of application interactivity, large limitations, and high maintenance costs during the same-screen display of applications.

[0157] In addition, the display device 200 can also establish a screen projection connection with other terminal devices, so that the display device 200 plays and displays the screen content in other terminal devices.

[0158] In the related art, the method of setting a floating window based on the native interface to display applications on the same screen cannot enable data communication between the various display windows. Then, when the display device 200 is displaying on the same screen, the user cannot control the application interface displayed by the display device 200 through the control device 100 such as a remote control, which leads to low response efficiency of the display device 200 during the application display on the same screen, reducing the user experience. Integrating the SDKs of various applications into the same application can realize data communication between multiple applications. However, the method of integrating the SDKs of multiple applications into the same application will cause the applications that integrate multiple SDKs to have problems such as complex interactivity and poor scalability during the maintenance process, which will also affect the operating efficiency of the display device 200.

[0159] Based on the above application scenarios, in order to improve the user experience and alleviate the problem of low response efficiency of the display device 200 during the same-screen display of applications, some embodiments of the present application provide another processing method for the display device. As shown in Figure 18, the method specifically includes the following steps:

[0160] S181: Receive a switching instruction for switching an application display position.

[0161] Display device 200 can receive various control commands input by the user, including some control commands for switching the display position of an application. In some embodiments, the switching command can be issued by terminal device 300, such as a mobile phone, or by a remote control associated with display device 200, or by a touch-sensitive component external to display device 200. This application does not impose any specific limitations.

[0162] Obviously, when the user inputs the switching command, the display device 200 is already in the state of displaying the application on the same screen. Since the floating display is a special display state, when the display device 200 displays the application in a floating state, the display device 200 needs to wait until the application enters the floating state and is ready before the display device 200 can display the application's user interface in the floating window.

[0163] Therefore, in some embodiments, the display device 200 receives a startup instruction for displaying an application screen, and in response to the startup instruction, obtains the preparation status of the first application and starts the second application. As shown in Figure 19, if the preparation status is ready, the display device 200 controls the display 260 to display the second application in a floating layer above the first application, so that the display device 200 can display the user interfaces of multiple applications on the same screen. If the preparation status is not ready, the display device 200 controls the display 260 to display the second application in full screen, so that the user can view the user interface of the second application on a large screen.

[0164] For example, the user interface of a first application is being displayed on display device 200. At this time, the user sends a startup instruction to display a second application screen to display device 200. Display device 200 detects the readiness status of the first application and detects that the first application is not yet ready. If the first application is not yet ready, it means that the current state does not support the simultaneous display of the user interfaces of the first and second applications on display device 200. Then, display device 200 controls display 260 to display the user interface of the second application in full screen for the user to view.

[0165] Furthermore, in order to facilitate the display device 200 to obtain the status of the first application, in some embodiments, the display device 200 also receives a status message of the first application, and parses an identifier contained in the status message to determine the readiness status of the current application.

[0166] For applications that can only be launched from the startup interface, the display device 200 requires the user to perform corresponding operations on the startup interface before the second application is displayed in a floating manner. Therefore, in some embodiments, if the second application is not a background service application, the display device 200 controls the display 260 to display an operation prompt interface when the second application is launched. After the display 260 displays the operation prompt interface, the display device 200 also monitors the startup status of the second application. If the second application is launched, the display device 260 is controlled to close the operation prompt interface.

[0167] Furthermore, when the display device 200 displays the operation prompt interface, the user can also input an operation instruction to the display device 200 to change the corresponding operation prompt interface to select the startup mode of the application. That is, in some embodiments, the display device 200 receives an operation instruction input by the user to change the operation interface, and in response to the operation instruction, changes the corresponding operation prompt interface to switch the startup mode.

[0168] For example: Taking the screen projection application as an example, after the user starts the screen projection function in the display device 200, the display device 200 displays the operation prompt page of the screen projection connection as shown in Figure 20. After the user clicks the designated button on the remote control, the display device 200 responds to the user's operation and switches to the operation prompt page as shown in Figure 21 on the display 260. The user can perform corresponding operations on the screen projection device side synchronously according to the operation prompt page to establish a screen projection relationship with the display device 200. In addition, after the screen projection device establishes a screen projection connection relationship with the display device 200, the display device 200 closes the operation prompt page.

[0169] It can be understood that the first application and the second application in the embodiment of the present application are applications that establish a data intercommunication relationship with each other and both support floating display.

[0170] Therefore, in order to establish a data intercommunication relationship between the first application and the second application, in some embodiments, the display device 200 also traverses the applications in the display device 200 and filters out applications that support the floating function as the first application or the second application. Then, the client services in the first application and the second application are bound to establish a data intercommunication relationship between the first application and the second application. That is, as shown in Figure 22, the display device 200 traverses all applications that it has deployed and filters out applications that support the floating function from all applications. After filtering out the applications that support the floating function, the client services between the applications are bound, so that the data intercommunication relationship between the first application and the second application can be established, so that the first application and the second application can communicate data.

[0171] For example: the first application is a background service application. After the second application is started, the display device 200 can traverse the current whole machine application through metadata information to determine whether there is an application that supports suspension. If there is an application that supports suspension, the agreed client services are bound in sequence to complete the data intercommunication between the applications that support suspension in the display device 200. The status information of the first application is received, and the preparation state of the first application is parsed as PREPARE. As shown in Figure 19, the display device 200 displays the user interface of the second application in a floating window on the upper layer of the first application.

[0172] Obviously, when the display device 200 displays the first application in full screen, the user interface of the first application can be scaled according to the preset ratio based on the size of the display 260. However, when the display device 200 displays the second application in a floating window, the size of the floating window needs to be planned according to the screen state of the second application before the appropriate screen size can be displayed on the display 260.

[0173] Therefore, in order to facilitate the planning of the size of the floating window, in some embodiments, the display device 200 obtains the media data of the second application and parses the screen size in the media data to obtain the screen state of the second application screen. The screen state includes a landscape state and a portrait state. If the screen state is a landscape state, the width of the floating window is determined to be a default width, and the aspect ratio of the screen size is obtained, and the window height is calculated according to the default width and aspect ratio; if the screen state is a portrait state, the window height is determined to be a default height, and the aspect ratio of the screen size is obtained, and the window width is calculated according to the default height and aspect ratio.

[0174] For example: Taking the second application as a screen projection application, obtain the media data of the screen projection application, that is, the video stream of the screen projection application. Record the width of the video stream as VideoWidth, and the height of the video stream as VideoHeight. When the display device 200 plans a floating window on the upper layer of the first application, obtain the width and height of the current parent layout, record the width of the parent layout as ViewWidth, and record the height of the parent layout as ViewHeight. And calculate the width and height of the floating window according to the following formula:

[0175] If VideoWidth ≥ VideoHeight, determine that the width of the parent layout is a fixed width, denoted as X. Then the height of the parent layout is X * VideoHeight / VideoWidth, and the width and height of the floating window are the same as those of the parent layout. If VideoWidth < VideoHeight, determine that the height of the parent layout is a fixed height, denoted as Y. Then the width of the parent layout is Y * VideoWidth / VideoHeight, and the width and height of the floating window are the same as those of the parent layout. The display device 200 can set the floating window according to the planned width and height dimensions to display the user picture of the screen mirroring application through the floating window.

[0176] S182: In response to the switching instruction, switch the levels of the first application and the second application to control the display device to full-screen display the user interface of the second application.

[0177] After the display device 200 displays the second application through the planned floating window, the user interface of the second application is displayed on top of the first application. Moreover, since the first application and the second application establish a data interconnection relationship with each other and both support floating display. Therefore, after the display device 200 receives the switching instruction, it can exchange the levels of the first application and the second application to make the user interface of the second application full-screen display and the user interface of the first application floating display.

[0178] For example: Take the fitness mirror application of the display device 200 as the first application and the screen mirroring application as the second application. As shown in FIG. 23, the display 260 full-screen displays the picture of the fitness mirror application and floatingly displays the picture of the screen mirroring application. That is, the fitness mirror application is the lower-layer application in FIG. 23, and the screen mirroring application is the upper-layer application in FIG. 23. At this time, if the user wants to enlarge the picture in the screen mirroring application, a switching instruction is sent to the display device 200 through the remote control. After the display device 200 receives the switching instruction, it exchanges the levels of the fitness mirror application and the screen mirroring application, switches the screen mirroring application to the lower-layer application, and switches the fitness mirror application to the upper-layer application.

[0179] In addition, to facilitate the first application and the second application to execute the operations corresponding to the switching instruction, in some embodiments, the display device 200 also monitors the focus event of the first application, encapsulates the event for switching the screen position as a switching instruction. Then controls the first application to execute the switching instruction and forwards the switching instruction to the second application so that the second application executes the switching instruction. That is to say, since the first application and the second application have a data interconnection relationship, the display device 200 can capture the focus event only through the lower-layer application. After capturing the focus event for switching the screen position, the lower-layer application can notify the upper-layer application to exchange the levels and display positions.

[0180] Furthermore, in some embodiments, after detecting a focus event, the display device 200 further parses the focus event. Based on the parsing results, the focus event is packaged as a corresponding control instruction. For example, an event for rotating a horizontal or vertical screen image is packaged as a rotation instruction. The display device 200 can parse the application targeted by the control instruction, execute the control instruction through the first application, and / or control the first application to forward the control instruction to a second application, so that the application corresponding to the control instruction performs the corresponding operation.

[0181] For example: Take the first application as a fitness mirror application and the second application as a screen projection application as an example. First, the user starts the fitness mirror application in the display device 200. The fitness mirror application starts a Window, which is designated as W1, sets the type to X, captures the focus event, and binds the client service for communication. At this time, the fitness mirror application is the lower-layer application of the display device 200. Then, after the display device 200 receives the screen projection message, the screen projection application also starts a window, which is designated as W2, sets the type to Y, does not capture the focus event, and Y>X. At this time, as shown in Figure 23, the screen projection application is the upper-layer application of the display device 200. The display device 200 captures the focus event through the lower-layer W1. When W1 captures the focus event, it notifies W2 to switch between horizontal and vertical screens or interchange positions, and adjusts the UI synchronously.

[0182] S183: Read the screen size of the first application, and plan a floating window on the upper layer of the second application according to the screen size.

[0183] After the display device 200 switches the levels of the first application and the second application in response to the switching instruction, since the user interfaces of the first application and the second application may have different screen ratios, it is also necessary to re-plan the floating window according to the screen size of the first application so as to display the first application through a floating window of appropriate size.

[0184] To facilitate planning the floating window size, in some embodiments, the display device 200 further obtains the window size of the first application. If the window width of the first application is greater than or equal to the window height, the window height is obtained, and a floating window is planned on the upper layer of the second application based on the window height and the screen size. If the window width of the first application is less than the window height, the window width is obtained, and a floating window is planned on the upper layer of the second application based on the window width and the screen size, so that the user interface of the first application is displayed in the floating window at an appropriate size.

[0185] For example: Obtain the media data of the first application, that is, the video stream in the first application. Denote the width of the video stream as VideoWidth and the height of the video stream as VideoHeight. When planning a floating window on the upper layer of the second application, obtain the width and height of the current parent layout. Denote the width of the parent layout as ViewWidth and the height of the parent layout as ViewHeight. Calculate the width and height of the floating window according to the following formula:

[0186] If ViewWidth ≥ ViewHeight and VideoWidth ≥ VideoHeight, determine that the height of the parent layout is a fixed height, denoted as Y. Then the width of the parent layout is Y * VideoHeight / VideoWidth, the width of the floating window is Y * VideoHeight / VideoWidth, and the height of the floating window is Y * VideoHeight * VideoHeight / VideoWidth / VideoWidth; if ViewWidth ≥ ViewHeight and VideoWidth < VideoHeight, determine that the height of the parent layout is a fixed height, denoted as Y. Then the height of the parent layout is Y * VideoWidth / VideoHeight, and the width and height of the floating window are the same as those of the parent layout.

[0187] If ViewWidth < ViewHeight and VideoWidth ≥ VideoHeight, determine that the width of the parent layout is a fixed width, denoted as X. Then the height of the parent layout is X * VideoHeight / VideoWidth, and the width and height of the floating window are the same as those of the parent layout; if ViewWidth < ViewHeight and VideoWidth < VideoHeight, determine that the width of the parent layout is a fixed width, denoted as X. Then the height of the parent layout is X * VideoWidth / VideoHeight, the width of the SurfaceView is X * VideoWidth * VideoWidth / VideoHeight / VideoHeight, and the height of the floating window is X * VideoWidth / VideoHeight. After planning the floating window of the first application, the display device 200 sets the floating window according to the planned size to display the user interface of the first application through the floating window.

[0188] Before switching the levels, the display device 200 detects focus events through the first application in the lower layer. Similarly, in order to facilitate the interactive process within the display device 200, after switching the levels of the first application and the second application, the first application is changed to the upper layer application. At this time, the display device 200 still monitors the focus events of the first application and encapsulates the focus events as control instructions corresponding to the events. The first application is controlled to execute the control instructions, or the control instructions are forwarded to the second application so that the second application executes the control instructions. In other words, the display device 200 always keeps capturing focus events through the first application.

[0189] Therefore, in some embodiments, the display device 200 further detects the hierarchical position of the first application and marks the window of the first application as the target window according to the hierarchical position. It then monitors the focus events of the target window and encapsulates the focus events into control instructions corresponding to the events. In this way, regardless of how the hierarchies of the first and second applications switch, the display device 200 can always capture focus events through the first application and then forward the focus events through the first application, thereby simplifying the interaction process within the display device 200.

[0190] S184: Control the display to display the user interface of the first application via a floating window.

[0191] After the display device 200 plans the size of the floating window, it displays the user interface of the first application in a floating manner on top of the second application through the floating window.

[0192] In some embodiments, after the display device 200 plans the size of the floating window, it displays the floating window in a preset display area, such as the upper right corner, upper left corner, etc. of the second application, and then reasonably displays the user interfaces of the first application and the second application on the display 260 at the same time.

[0193] For example: Taking screen projection as an example, the first application is a fitness mirror application, and the second application is a screen projection application. As shown in Figure 23, the display 260 displays the screen of the fitness mirror application in full screen, and displays the screen of the screen projection application in a floating manner. At this time, if the user wants to enlarge the screen in the screen projection application, he sends a switching instruction to the display device 200 through the remote control. After receiving the switching instruction, the display device 200 switches the hierarchy of the fitness mirror application and the screen projection application, and re-plans the floating window of the fitness mirror application. After the floating window of the fitness mirror application is planned, the display device 200 controls the display 260 to display the user interface shown in Figure 24.

[0194] Furthermore, as can be seen from the above embodiments, the display device 200 can capture focus events through the underlying application, allowing the user to input a rotation command to the display device 200 to change the horizontal or vertical orientation of the screen in the floating window. However, in some scenarios, the application's user interface may also automatically switch between horizontal and vertical orientations based on the device's placement.

[0195] Therefore, in some embodiments, the display device 200 also obtains the media data of the target application, which is the first application or the second application. The screen size in the media data is then parsed to obtain the screen state of the target application screen, which includes the horizontal screen state and the vertical screen state. If the screen state of the target application screen changes, a rotation instruction for the horizontal and vertical screen is generated so that the target application executes the rotation instruction. That is, the display device 200 can monitor whether the horizontal and vertical screen states of the application screen have changed based on the media data of the application, so that when the horizontal and vertical screen states of the application screen change, the horizontal and vertical screen states of the screens displayed in each display window of the display 260 are synchronously adjusted.

[0196] Obviously, when rotating the screen from horizontal to vertical, the display device 200 also needs to re-plan the window size. Therefore, in some embodiments, if the screen state of the target application screen changes, the display device 200 obtains the window size of the target application. If the window width of the target application is greater than or equal to the window height, the window height is obtained, and a floating window is planned on the upper layer of the second application according to the window height and screen size. If the window width of the target application is less than the window height, the window width is obtained, and a floating window is planned on the upper layer of the second application according to the window width and screen size.

[0197] For example: Taking the second application as a projection application, the display device 200 obtains the media data of the second application, that is, the video stream of the second application. The width of the video stream is recorded as VideoWidth, and the height of the video stream is recorded as VideoHeight. The placement state of the projection device has changed, so the state of the screen it displays has also changed. When the screen state of the projection application is detected to have changed, the width and height of the current parent layout are obtained, the width of the parent layout is recorded as ViewWidth, and the height of the parent layout is recorded as ViewHeight. Calculate the width and height of the window according to the following formula:

[0198] If ViewWidth ≥ ViewHeight and VideoWidth ≥ VideoHeight, determine that the height of the parent layout is a fixed height, denoted as Y. Then the width of the parent layout is Y * VideoHeight / VideoWidth, the width of the floating window is Y * VideoHeight / VideoWidth, and the height of the floating window is Y * VideoHeight * VideoHeight / VideoWidth / VideoWidth. If ViewWidth ≥ ViewHeight and VideoWidth < VideoHeight, determine that the height of the parent layout is a fixed height, denoted as Y. Then the height of the parent layout is Y * VideoWidth / VideoHeight, and the width and height of the window are the same as those of the parent layout.

[0199] If ViewWidth < ViewHeight and VideoWidth ≥ VideoHeight, determine that the width of the parent layout is a fixed width, denoted as X. Then the height of the parent layout is X * VideoHeight / VideoWidth, and the width and height of the floating window are the same as those of the parent layout. If ViewWidth < ViewHeight and VideoWidth < VideoHeight, determine that the width of the parent layout is a fixed width, denoted as X. Then the height of the parent layout is X * VideoWidth / VideoHeight, the width of the SurfaceView is X * VideoWidth * VideoWidth / VideoHeight / VideoHeight, and the height of the window is X * VideoWidth / VideoHeight. After re-planning the floating window of the second application, the display device 200 sets the floating window according to the planned size to display the user interface of the second application through the floating window.

[0200] It should be noted that the calculation processes involved in the above embodiments are only for illustrative purposes and do not represent the actual calculation processes. There may be other calculation methods or calculation steps, etc. This application does not make any limitations in this regard, and the above calculation methods do not constitute limitations to this application.

[0201] In addition, during the process of displaying applications on the same screen, the user can also close any application at any time to close its corresponding display window. That is, in some embodiments, the display device 200 receives a close instruction and sends the close instruction to the first application. The close instruction is executed by the first application, and / or the first application is controlled to forward the close instruction to the second application to control the second application to execute the close instruction. Since the display device 200 switches the level, the first application is changed to the upper layer application. At this time, the display device 200 captures the focus event through the upper first application, and encapsulates the event for closing the application in the focus event into a close instruction, and sends it to the display device 200. If the close instruction is a close event of the first application, the close instruction is executed by the first application; if the close instruction is a close instruction of the second application, the close instruction is forwarded to the second application through the first application, so that the second application executes the close instruction.

[0202] Obviously, after an application is closed, its user interface will no longer be displayed on display 260. Therefore, if the user closes the second application, display 260 will only display the user interface of the first application. However, since the first application is in a floating display state, if only the second application is closed, display 260 will still display the first application in a floating window, resulting in a large amount of display area on display 260 being wasted, reducing the user experience.

[0203] Therefore, in some embodiments, the display device 200 further detects the closing status of the first application and the second application. If the second application is closed and the first application is not closed, the display 260 is controlled to display the user interface of the first application in full screen.

[0204] For example, let's say the first application is a fitness mirror application, and the second application is a screen projection application. When the display device 200 displays the user interface shown in FIG24 , the user closes the screen projection application on the display device 200, and the display device 200 destroys the display window of the screen projection application. Simultaneously, as shown in FIG25 , the display device 200 also displays the user interface of the fitness mirror application in full screen.

[0205] It is understood that the close instruction is not limited to user input, and the application may be closed due to other reasons, such as network disconnection, conflict disconnection, etc. Therefore, in some embodiments, after the display device 200 detects an abnormal connection between the application and the display device 200, it automatically generates a close instruction to cause the display device 200 to close the application.

[0206] In the case of non-user-initiated shutdown, to facilitate the user's understanding of the reason for the application's shutdown, in some embodiments, the shutdown instruction also includes the reason for the application's shutdown. When encapsulating the shutdown instruction, the display device 200 also encapsulates the shutdown reason into the shutdown instruction. In this way, when executing the shutdown instruction, the display device 200 can also analyze the shutdown reason and control the display 260 to display a corresponding prompt message based on the shutdown reason.

[0207] For example, taking screen projection as an example, the first application is a fitness mirror application, and the second application is a screen projection application. When the display device 200 displays the user interface shown in Figure 23, it receives a shutdown instruction from the screen projection application due to network reasons. The display device 200 parses the shutdown reason from the shutdown instruction and notifies other applications of the display device 200 of the shutdown reason and shutdown instruction through the underlying screen projection application, so that the display device 200 can display a prompt message as shown in Figure 26 on the display 260 based on the shutdown reason.

[0208] Based on the above-mentioned another processing method for a display device, a display device 200 in some embodiments of the present application, as shown in FIG27 , includes: a display 260 and at least one processor 250. The display 260 is configured to display a user interface of a first application and to display a user interface of a second application on top of the first application; as shown in FIG6 , the at least one processor 250 is configured to execute computer instructions stored in a memory of the display device 200 so that the display device 200:

[0209] S181: receiving a switching instruction for switching an application display position;

[0210] S182: In response to the switching instruction, switching the levels of the first application and the second application to control the display to display the user interface of the second application in full screen;

[0211] The first application and the second application are applications that establish a data intercommunication relationship with each other and both support floating display;

[0212] S183: Reading the screen size of the first application, and planning a floating window on the upper layer of the second application according to the screen size;

[0213] S184: Control the display to display the user interface of the first application via a floating window.

[0214] It can be seen from the above technical solutions that the display device of some embodiments of the present application and another processing method for the display device can switch the hierarchy of the first application and the second application in response to a switching instruction for switching the display position of the application, so as to control the display 260 to display the user interface of the second application in full screen. The first application and the second application are applications that establish a data intercommunication relationship with each other and both support floating display. The method reads the screen size of the first application and plans a floating window on the upper layer of the second application according to the screen size. Then control the display 260 to display the user interface of the first application through the floating window, so as to quickly respond to the switching instruction input by the user when the display device displays the applications on the same screen, thereby improving the user experience.

[0215] In some embodiments, as shown in Figure 28, the display device 200 includes at least one processor 250, a display 260, a terminal interface 278 extending from the gap on the back panel, and a rotating component 280 connected to the back panel. The rotating component 280 can rotate the display screen. From the perspective of viewing from the front of the display device, the rotating component 280 can rotate the display screen to a vertical screen state, that is, the vertical side length of the screen is greater than the horizontal side length, and can also rotate the screen to a horizontal screen state, that is, the horizontal side length of the screen is greater than the vertical side length.

[0216] In some embodiments, the rotation assembly 280 may include a drive motor, a rotation shaft, and other components. The drive motor may be connected to at least one processor 250 and output a rotation angle under the control of the at least one processor 250. One end of the rotation shaft is connected to the power output shaft of the drive motor, and the other end is connected to the display 260, so that the display 260 can be fixedly mounted on a wall or a bracket via the rotation assembly 280.

[0217] Rotating assembly 280 may also include other components, such as transmission components and detection components. The transmission components can adjust the rotational speed and torque output by rotating assembly 280 through a specific transmission ratio, and may employ a gear transmission mechanism. The detection components may include sensors disposed on the rotating shaft, such as angle sensors and attitude sensors. These sensors can detect parameters such as the rotation angle of rotating assembly 280 and transmit these parameters to at least one processor 250, enabling the at least one processor 250 to determine or adjust the status of display device 200 based on the detected parameters. In practical applications, rotating assembly 280 may include, but is not limited to, one or more of the aforementioned components.

[0218] Under the related art, the display device 200 displays applications on the same screen by setting a floating window based on the native interface, which is highly dependent on the native interface and cannot enable data communication between the various display windows. Then, if the rotating component 280 of the display device 200 rotates, the user interface of the display device 200 cannot be dynamically adjusted according to the rotation event of the rotating component 280. Integrating the SDKs of multiple applications into the same application can realize data communication between multiple applications, and then the display device 200 can adjust the display screen direction of each application according to the rotation event of the rotating component 280. However, the way of integrating the SDKs of multiple applications into the same application will cause the application that integrates multiple SDKs to have problems such as complex interactivity and poor scalability during the maintenance process, resulting in the display device 200 responding slowly when rotating the user screen of each application, reducing the user experience.

[0219] Based on the above application scenarios, in order to improve the user experience and alleviate the problem of slow response speed of the display device 200 when rotating the user screen of each application, some embodiments of the present application provide a third processing method for the display device. As shown in Figure 29, the method specifically includes the following steps:

[0220] S291: In response to a rotation event of the rotation component, convert the display screen direction of the first application, and read the window size and screen size of the second application.

[0221] The display device 200 can monitor rotation events of the rotating assembly 280 so that when the rotating assembly 280 rotates, the display device 200 can adjust the display orientation of the display 260 accordingly. Therefore, in some embodiments, the display device 200 monitors the rotation direction of the rotating assembly 280 and changes the global display orientation based on the rotation direction. The global display orientation is the display orientation of the display 260, which can be either landscape or portrait.

[0222] For example, when display device 200's display 260 is in the state shown in FIG30 , a user is reading novel A on display device 200. At this point, the overall screen orientation of display device 200 is landscape. To facilitate reading novel A, the user adjusts display 260 to the state shown in FIG31 . At this point, the screen displayed on display 260 is shown in FIG31 , and the overall screen orientation of display device 200 is portrait.

[0223] It is understood that the display device 200 can automatically determine the horizontal or vertical state of the display 260 based on various parameters in the rotation component 280. For example, when the angle between the display 260 and the ground is 85°, the display 260 can be determined to be in the vertical state. This application does not impose any restrictions on this.

[0224] As can be seen from the above embodiment, when the rotation assembly 280 rotates, the display device 200 changes its global screen orientation. Similarly, when the global screen orientation changes, the applications displayed on the display device 200 also need to adjust their display orientation. To adapt the application's display window to the orientation of the display 260, when the application is launched, the application's display orientation must be set according to the global screen orientation of the display device 200 to match the screen content displayed on the display 260.

[0225] Therefore, in some embodiments, the display device 200 monitors the rotation direction of the rotation component 280 and changes the global screen orientation according to the rotation direction. The global screen orientation is the display screen orientation of the display 260. In response to the startup instruction for displaying the first application screen, the display device 200 reads the global screen orientation and controls the display 260 to display the user interface of the first application in full screen according to the global screen orientation. That is, the display device 200 can obtain the current display screen orientation of the display 260 through the rotation component 280 as the global screen orientation of the display device 200. When the user starts the first application, the display device 200 sets the display screen orientation of the first application according to the global screen orientation and displays the first application in full screen.

[0226] For example, let's take the first application as a fitness mirror application. When the display 260 of the display device 200 is in landscape orientation, the user controls the display device 200 via the remote control to launch the fitness mirror application. In response to the remote control key press, the display device 200 displays the screen shown in FIG32 on the display 260.

[0227] After the first application is launched, in order to display the applications on the same screen, a second application needs to be launched on the display device 200. Therefore, in some embodiments, the display device 200 also reads the screen size and global screen orientation of the second application in response to the launch instruction for displaying the second application screen. Based on the screen size and global screen orientation, a floating window is planned on the upper layer of the first application, and then the display 260 is controlled to display the user interface of the second application in the floating window.

[0228] For example, let's say the first application is a fitness mirror application, and the second application is a projection application. Display device 200 is in landscape mode and the fitness mirror application is activated. Display 260 displays the screen shown in FIG32 . The user activates the projection application on display device 200 via the remote control. Display device 200 then creates a floating window for the projection application based on the portrait orientation and the projection application's screen size, and displays the screen shown in FIG33 on display 260.

[0229] In order to facilitate obtaining the screen size of the second application, in some embodiments, the display device 200 obtains the media data of the second application and extracts the frame images in the media data. Then, the screen height and screen width are parsed from the frame images to obtain the screen size of the second application.

[0230] For example: Taking the second application as a screen mirroring application. The display device 200 can obtain the video stream sent by the screen mirroring application, extract a frame of the screen image from the video stream, and detect the width and height of the image to generate width data and height data. The display device 200 can then obtain the screen size of the screen mirroring application based on the width data and height data.

[0231] Moreover, since the media data in the second application may be of different size ratios, in order to set a floating window with a suitable size, in some embodiments, when the display device 200 plans a floating window on top of the first application according to the screen size and the global screen orientation, it also analyzes the screen width and screen height of the screen size, and calculates the screen ratio of the second application based on the screen width and screen height. If the screen width is greater than or equal to the screen height, the window size is set to the default width, and the window height is calculated according to the default width and the screen ratio; if the screen width is less than the screen height, the window size is set to the default height, and the window height is calculated according to the default width and the screen ratio. That is, when planning the floating window of the second application, the display device 200 can scale the user screen of the second application according to the screen ratio of the second application and a preset default value.

[0232] For example: Taking the second application as a screen mirroring application, obtaining the media data of the screen mirroring application, that is, the video stream of the screen mirroring application. Denote the width of the video stream as VideoWidth and the height of the video stream as VideoHeight. When the display device 200 plans a floating window on top of the first application, it obtains the width and height of the current parent layout, denotes the width of the parent layout as ViewWidth and the height of the parent layout as ViewHeight. And calculate the width and height of the floating window in the following way:

[0233] If VideoWidth≥VideoHeight, determine that the width of the parent layout is a fixed width, denoted as X, then the height of the parent layout is X*VideoHeight / VideoWidth, and the width and height of the floating window are the same as those of the parent layout; if VideoWidth<VideoHeight, determine that the height of the parent layout is a fixed height, denoted as Y, then the width of the parent layout is Y*VideoWidth / VideoHeight, and the width and height of the floating window are the same as those of the parent layout. The display device 200 can set the floating window according to the planned width and height dimensions to display the user screen of the screen mirroring application through the floating window.

[0234] Furthermore, since the second application is displayed on top of the first application via a floating window, and the floating display state is a special display state, when the display device 200 displays an application in a floating state, the display device 200 needs to wait until the underlying application has entered a state that supports floating and is ready before the display device 200 can display the application's user interface via the floating window.

[0235] Therefore, in some embodiments, when launching the second application, the display device 200 also obtains the preparation status of the first application. If the preparation status is ready, as shown in FIG33 , the display device 200 controls the display 260 to display the second application in a floating manner above the first application, so that the display device 200 can display the user interfaces of multiple applications on the same screen. If the preparation status is not ready, the display device 200 controls the display 260 to display the second application in full screen, so that the user can view the user interface of the second application on a large screen.

[0236] For example, the user interface of a first application is being displayed on display device 200. At this time, the user sends a startup instruction to display a second application screen to display device 200. Display device 200 detects the readiness status of the first application and detects that the first application is not yet ready. If the first application is not yet ready, it means that the current state does not support the simultaneous display of the user interfaces of the first and second applications on display device 200. Then, display device 200 controls display 260 to display the screen shown in FIG. 34 in full screen for the user to view.

[0237] Furthermore, in order to facilitate the display device 200 to obtain the status of the first application, in some embodiments, the display device 200 also receives a status message of the first application, and parses the identifier contained in the status message to determine the readiness status of the current application.

[0238] For example, the readiness status identifier of the first application includes IDLE - preparing, PREPARE - preparation completed, FOCUS - preparation completed, FINISH - interface exit, ERROR - exception. The display device 200 can parse the identifier contained in the status message and then judge the readiness status of the application. When the readiness status of the first application is PREPARE preparation completed or FOCUS preparation completed, it means that the first application is ready and can be suspended. At this time, the display device 200 can control the display 260 to display the second application in a floating manner above the first application.

[0239] In some embodiments, the display device 200 may also set a corresponding identifier to indicate the ready state of the first application according to the type of the first application. Since the background service application itself is configured with background operation and suspension functions, if the first application is a background service application, such as a screen projection application that relies on wireless network communication technology, the first application can be directly indicated as ready through the identifier. If the first application is not a background service application and requires a startup interface to start the application, it is necessary to control the suspension function switch and indicate that the first application is ready through the suspension function switch.

[0240] For example, background service applications can use the PREPARED state to represent the prepared state; non-background service applications can control their function switches through FOCUS, and use FOCUS to notify the current application that it can support floating interface display.

[0241] For applications that can only be launched from the startup interface, the display device 200 requires the user to perform corresponding operations on the startup interface before the second application is displayed in a floating manner. Therefore, in some embodiments, if the second application is not a background service application, the display device 200 controls the display 260 to display an operation prompt interface when the second application is launched. After the display 260 displays the operation prompt interface, the display device 200 also monitors the startup status of the second application. If the second application is launched, the display device 260 is controlled to close the operation prompt interface.

[0242] To facilitate users in viewing the operation prompt page, in some embodiments, when the display 260 is in a horizontal state, the display device 200 displays the operation prompt interface in a fixed-size layout at the display position on the right side of the display 260; when the display 260 is in a vertical state, the display device 200 displays the operation prompt interface in a fixed-size layout at the display position in the upper right corner of the display 260, so as to clearly present the operation prompt page on the display 260, making it convenient for users to operate according to the prompt information.

[0243] Furthermore, when the display device 200 displays the operation prompt interface, the user can also input an operation instruction to the display device 200 to change the corresponding operation prompt interface to select the startup mode of the application. That is, in some embodiments, the display device 200 receives an operation instruction input by the user to change the operation interface, and in response to the operation instruction, changes the corresponding operation prompt interface to switch the startup mode.

[0244] For example: Take the second application as a screen projection application, and the display 260 as a horizontal screen state. After the user starts the screen projection function in the display device 200, the display device 200 displays the operation prompt page of the screen projection connection as shown in Figure 35. After the user clicks the designated button on the remote control, the display device 200 responds to the user's operation and switches to the operation prompt page as shown in Figure 36 in the display 260. At this time, the user adjusts the display 260 to a vertical screen state through the rotating component 280, and the display device 200 displays the operation prompt page as shown in Figure 37. The user can perform corresponding operations on the screen projection device side synchronously according to the operation prompt page to establish a screen projection relationship with the display device 200. In addition, after the screen projection device establishes a screen projection connection relationship with the display device 200, the display device 200 closes the operation prompt page.

[0245] It can be understood that the first application and the second application in the embodiment of the present application are applications that establish a data intercommunication relationship with each other and both support floating display.

[0246] Therefore, in order to establish a data intercommunication relationship between the first application and the second application, in some embodiments, the display device 200 also traverses the applications in the display device 200, and filters out applications that support the floating function as the first application or the second application. Then, the client services in the first application and the second application are bound to establish a data intercommunication relationship between the first application and the second application. That is, as shown in Figure 38, the display device 200 traverses all applications that it has deployed, and filters out applications that support the floating function from all applications. After filtering out the applications that support the floating function, the client services between the applications are bound, so that the data intercommunication relationship between the first application and the second application can be established, so that the first application and the second application can communicate data.

[0247] For example: the first application is a background service application. After the second application is started, the display device 200 can traverse the current whole machine application through metadata information to determine whether there is an application that supports suspension. If there is an application that supports suspension, the agreed client services are bound in sequence to complete the data intercommunication between the applications that support suspension in the display device 200. The status information of the first application is received, and the preparation state of the first application is parsed as PREPARE. As shown in Figure 33, the display device 200 displays the user interface of the second application in a floating window on the upper layer of the first application.

[0248] As can be seen from the above embodiment, after the first application and the second application are launched, the display device 200 is in a state where the first application is displayed full screen and the second application is displayed in a floating window, as shown in Figure 33. At this time, when a rotation event of the rotating component 280 is detected, the display device 200 immediately changes the display direction of the first application and re-arranges the floating window of the second application.

[0249] S292: Generate layout information according to the converted display screen orientation and window size of the first application.

[0250] When re-arranging the floating window of the second application, display device 200 needs to obtain the screen orientation after the first application is converted to determine the display orientation of display 260 after rotation component 280 is rotated. In addition, since the floating window of the second application already exists on the current display interface of display 260, the layout information of the floating window needs to be re-determined based on the size of the currently displayed floating window.

[0251] S293: Planning a floating window on the upper layer of the first application according to the layout information and the screen size, and controlling the display to display the user interface of the second application through the floating window.

[0252] After determining the layout information of the floating window, the display device 200 calculates the size of the floating window based on the layout information and the screen size to plan the floating window. After planning the floating window, the user interface of the second application is displayed on top of the first application through the floating window, so that multiple user interfaces can be quickly and dynamically adjusted when the display device 200 is rotated.

[0253] In some embodiments, the layout information includes a preset display position, a default width, and a default height. The display device 200 can also obtain the display position according to the global screen orientation, and display the user interface of the second application on the display position through the floating window. For example, when the screen is in portrait orientation, the display position of the floating window is the right area; when the screen is in landscape orientation, the display position of the floating window is the upper right corner area, etc. The default height and default width are both fixed values ​​preset in the display device 200 to facilitate determining the size and display position of the floating window.

[0254] In some embodiments, when the display device 200 plans a floating window on top of the first application based on the layout information and the screen size, it also parses the window width and window height of the window size and calculates the screen ratio of the second application based on the screen size. If the window width is greater than or equal to the window height, the window height is extracted and the width and height of the floating window are calculated based on the window height and the screen ratio; if the window width is less than the window height, the window width is extracted and the width and height of the floating window are calculated based on the window width and the screen ratio.

[0255] For example: Obtain the media assets data of the second application, that is, the video stream in the second application. Denote the width of the video stream as VideoWidth and the height of the video stream as VideoHeight. When planning a floating window on the upper layer of the first application, obtain the width and height of the current parent layout. Denote the width of the parent layout as ViewWidth and the height of the parent layout as ViewHeight. When the display screen of the display 260 changes, calculate the width and height of the floating window in the following manner:

[0256] If ViewWidth ≥ ViewHeight and the video stream VideoWidth ≥ VideoHeight, determine that the width of the parent layout is a fixed width, denoted as X, and the height is X * VideoHeight / VideoWidth. The width and height of the floating window are the same as those of the parent layout. If ViewWidth ≥ ViewHeight and the video stream VideoWidth < VideoHeight, determine that the width of the parent layout is a fixed width, denoted as X, and the height is X * VideoWidth / VideoHeight. The width of the floating window is X * VideoWidth * VideoWidth / VideoHeight / VideoHeight, and the height of the floating window is X * VideoWidth / VideoHeight.

[0257] If ViewWidth < ViewHeight and the video stream VideoWidth ≥ VideoHeight, determine that the height of the parent layout is a fixed height, denoted as Y. Then the width of the parent layout is Y * VideoHeight / VideoWidth, and the width of the floating window is Y * VideoHeight / VideoWidth. The height of the floating window is Y * VideoHeight * VideoHeight / VideoWidth / VideoWidth. If ViewWidth < ViewHeight and the video stream VideoWidth < VideoHeight, determine that the height of the parent layout is a fixed height, denoted as Y. Then the height of the parent layout is Y * VideoWidth / VideoHeight, and the width and height of the floating window are the same as those of the parent layout.

[0258] The above calculation method is applicable to both the case where the display 260 changes from landscape to portrait and from portrait to landscape. After the display device 200 plans the floating window of the second application, set the floating window according to the planned size to display the user interface of the second application through the floating window.

[0259] For example, let's say the first application is a fitness mirror application, and the second application is a projection application. If the display 260 of the display device 200 is in landscape mode, the screen shown in FIG33 is displayed. At this point, if the user adjusts the display 260 to portrait mode, the display device 200 changes the orientation of the fitness mirror application's display screen and re-arranges the floating window of the projection application to display the projection application in the floating window. That is, after the display 260 is switched to portrait mode, the screen shown in FIG39 is displayed.

[0260] In addition, in order to ensure the user's viewing experience, some applications may only support one display screen orientation. For example, the screen ratio of the media data in the first application is horizontal. If it is played in vertical mode, the displayed screen will be too small or incomplete. Therefore, in some embodiments, the display device 200 also detects the configured screen orientation of the first application, and the configured screen orientation is the display screen orientation supported by the first application. If the global screen orientation is not within the range of the configured screen orientation, the floating window is traversed, and the target application is filtered according to the preset priority, and the target application is the second application that supports the global screen orientation. Switch the hierarchy of the first application and the target application to display the target application in full screen. Plan the floating window of the first application on the upper layer of the target application, and display the user interface of the first application through the floating window. Then, when the user rotates the direction of the display 260, the display device 200 can automatically display the first application and the second application according to the optimal display layout.

[0261] For example: Take the first application as a fitness mirror application and the second application as a projection application. The configuration screen orientation of the fitness mirror application is landscape, and the configuration screen orientation of the projection application is portrait. As shown in Figure 33, the display device 200 is playing the fitness mirror application and the projection application in landscape mode. At this time, the user rotates the display device 200 to portrait mode. Since the fitness mirror application does not support portrait display, as shown in Figure 40, the display device 200 switches the levels of the fitness mirror application and the projection application, displays the projection application in full screen, and displays the fitness mirror application in a suspended state.

[0262] In some embodiments, the preset priority rules can be set based on the screen ratio of the second application, with a higher weight being assigned to the second application that is closer to the screen ratio of the display 260, and a higher weight being assigned to the second application that is currently in portrait mode. The second application with the highest cumulative weight value is marked as the target application, so that a unique target application can be selected from the second applications corresponding to the floating window and switched with the display level of the first application.

[0263] It is understandable that if the level of the first application in the display device 200 is swapped, the display device 200 re-plans the floating window of the first application. The method of planning the floating window of the first application is the same as the method of planning the floating window of the second application, which will not be repeated here.

[0264] Furthermore, to facilitate user interaction with the display device 200, in some embodiments, the display device 200 can also receive various control commands input by the user and respond to the control commands by performing the corresponding operations. The switching command can be issued by a terminal device 300, such as a mobile phone, or by a remote control included with the display device 200, or by a touch-sensitive component external to the display device 200. In other words, the first application and the second application can respond to corresponding operations based on the control commands.

[0265] To facilitate the first and second applications in executing the control instructions, in some embodiments, the display device 200 further monitors focus events of the second application, encapsulates the focus events into control instructions corresponding to the events, and controls the second application to send the control instructions to the first application. The first application is then controlled to forward the control instructions to other applications, other than the first and second applications, in the display device, to notify the other applications to execute the control instructions.

[0266] That is, because the first application and the second application are in a data-interconnected relationship, the display device 200 can always capture focus events through the top-level application. After capturing the focus event of the screen switching position, the focus event is forwarded to the lower-level application. After receiving the focus event, the lower-level application can notify the upper-level application to perform the operation corresponding to the focus event.

[0267] Furthermore, in some embodiments, after detecting a focus event, the display device 200 further parses the focus event. Based on the parsing results, the focus event is packaged into a corresponding control instruction. For example, an event for rotating a horizontal or vertical screen image is packaged into a rotation instruction. The display device 200 can parse the application targeted by the control instruction, execute the control instruction through the first application, and / or control the first application to forward the control instruction to a second application, so that the application corresponding to the control instruction performs the corresponding operation.

[0268] For example, take the first application as the fitness mirror application and the second application as the screen mirroring application. First, the user starts the fitness mirror application on the display device 200. The fitness mirror application starts an Activity, captures the focus event, and binds to the client service for communication at the same time. Then, after the display device 200 receives the screen mirroring message, the screen mirroring application also starts an Activity and captures the focus event. At this time, the fitness mirror application no longer captures the focus event and changes to capturing the focus event through the screen mirroring application. After the screen mirroring application captures the focus event, it forwards or executes the relevant focus event through the Activity of the fitness mirror application and notifies other intermediate applications to perform UI switching.

[0269] In addition, in some embodiments, the user can send a switching instruction to the display device 200 through a control device such as a remote control to switch the display screen direction of the second application. The display device 200 then responds to the switching instruction and detects the window size of the second application. If the window width of the second application is greater than or equal to the window height, obtain the window height, and plan the display window according to the window height and the screen size; if the window width of the second application is less than the window height, obtain the window width, and plan the display window according to the window width and the screen size.

[0270] For example: still take the second application as the screen mirroring application. The display device 200 obtains the media data of the second application, that is, the video stream of the second application. Denote the width of the video stream as VideoWidth and the height of the video stream as VideoHeight. The placement state of the screen mirroring device has changed, so the displayed screen state has also changed. When it is detected that the screen state of the screen mirroring application has changed, obtain the width and height of the current parent layout. Denote the width of the parent layout as ViewWidth and the height of the parent layout as ViewHeight. Calculate the width and height of the window in the following way:

[0271] If ViewWidth≥ViewHeight and VideoWidth≥VideoHeight, determine that the height of the parent layout is a fixed height, denoted as Y. Then the width of the parent layout is Y*VideoHeight / VideoWidth, the width of the floating window is Y*VideoHeight / VideoWidth, and the height of the floating window is Y*VideoHeight*VideoHeight / VideoWidth / VideoWidth; if ViewWidth≥ViewHeight and VideoWidth<VideoHeight, determine that the height of the parent layout is a fixed height, denoted as Y. Then the height of the parent layout is Y*VideoWidth / VideoHeight, and the width and height of the window are the same as those of the parent layout.

[0272] If ViewWidth < ViewHeight and VideoWidth ≥ VideoHeight, determine that the width of the parent layout is a fixed width, denoted as X. Then the height of the parent layout is X * VideoHeight / VideoWidth, and the width and height of the floating window are the same as those of the parent layout. If ViewWidth < ViewHeight and VideoWidth < VideoHeight, determine that the width of the parent layout is a fixed width, denoted as X. Then the height of the parent layout is X * VideoWidth / VideoHeight, the width of the SurfaceView is X * VideoWidth * VideoWidth / VideoHeight / VideoHeight, and the height of the window is X * VideoWidth / VideoHeight. After re-planning the floating window of the second application, the display device 200 sets the floating window according to the planned size to display the user interface of the second application through the floating window.

[0273] In some embodiments, during the process of the applications being displayed on the same screen, the user can also close any one of the applications at any time to close its corresponding display window. That is, the display device 200 receives a close instruction and sends the close instruction to the first application. Then, the first application executes the close instruction, and / or controls the first application to forward the close instruction to the second application to control the second application to execute the close instruction.

[0274] Obviously, after the application is closed, the user interface of the application will no longer be displayed on the display 260. Then, if the user closes the first application, the display 260 will only display the user interface of the second application. Since the second application is in a floating display state, if only the first application is closed, the display 260 will still display the second application through the floating window, which will cause most of the display area on the display 260 to be wasted and reduce the user experience.

[0275] Therefore, in some embodiments, the display device 200 also detects the close states of the first application and the second application. If the first application is in a closed state and the second application is in an unclosed state, it controls the display 260 to display the user interface of the second application full screen.

[0276] For example: taking the first application as a fitness mirror application and the second application as a screen mirroring application. When the display device 200 displays the user interface as shown in FIG. 33, the user closes the fitness mirror application of the display device 200. The display device 200 immediately destroys the display window of the fitness mirror application and exits the display screen of the fitness mirror application. At the same time, as shown in FIG. 34, the display device 200 also displays the user interface of the screen mirroring application full screen.

[0277] It is understood that the close instruction is not limited to user input, and the application may be closed due to other reasons, such as network disconnection, conflict disconnection, etc. Therefore, in some embodiments, after the display device 200 detects an abnormal connection between the application and the display device 200, it automatically generates a close instruction to cause the display device 200 to close the application.

[0278] In the case of non-user-initiated shutdown, to facilitate the user's understanding of the reason for the application's shutdown, in some embodiments, the shutdown instruction also includes the reason for the application's shutdown. When encapsulating the shutdown instruction, the display device 200 also encapsulates the shutdown reason into the shutdown instruction. In this way, when executing the shutdown instruction, the display device 200 can also analyze the shutdown reason and control the display 260 to display a corresponding prompt message based on the shutdown reason.

[0279] For example, the first application is still a fitness mirror application, and the second application is a screen projection application. When the display device 200 displays the user interface shown in FIG33, it receives a shutdown instruction from the screen projection application due to network reasons. The display device 200 parses the shutdown reason from the shutdown instruction and notifies other applications of the display device 200 of the shutdown reason and shutdown instruction through the underlying screen projection application, so that the display device 200 can display a prompt message "Network connection abnormality, screen projection connection disconnected" on the display 260 according to the shutdown reason.

[0280] Based on the third processing method for a display device described above, a display device 200 in some embodiments of the present application, as shown in FIG41 , includes: a display 260, a rotation component 280, and at least one processor 250. The display 260 is configured to display the user interface of a first application, and to display the user interface of a second application via a floating window on top of the first application; the rotation component 280 is configured to rotate the display 260. As shown in FIG18 , the at least one processor 250 is configured to execute computer instructions stored in a memory of the display device so that the display device 200:

[0281] S291: In response to a rotation event of the rotating component, convert the display screen direction of the first application and read the window size and screen size of the second application; the first application and the second application are applications that establish a data intercommunication relationship with each other and both support floating display;

[0282] S292: Generate layout information according to the display screen orientation and window size after conversion of the first application;

[0283] S293: Planning a floating window on the upper layer of the first application according to the layout information and the screen size, and controlling the display to display the user interface of the second application through the floating window.

[0284] It can be seen from the above technical solutions that the display device according to some embodiments of the present application and the third processing method for the display device can respond to the rotation event of the rotating component, convert the display screen direction of the first application, and read the window size and screen size of the second application. Among them, the first application and the second application are applications that establish a data intercommunication relationship with each other and both support floating display. The method can generate the layout information of the floating window according to the display screen direction and window size after the conversion of the first application. Then, based on the layout information and the screen size of the second application, a floating window is planned on the upper layer of the first application. After planning the floating window, the display 260 is controlled to display the user interface of the second application through the floating window, so that when the display device 200 rotates, the user interface of the application can be quickly adjusted to improve the user experience.

[0285] It can be seen from the foregoing content that in some embodiments, for a display device 200 having the same or similar software and hardware configuration as the foregoing device, the terminal device 300 (screen projection sending end) can push media resources to the display device 200 (screen projection receiving end), and the media resources include but are not limited to video, image, audio, text and other types.

[0286] In some embodiments, the user can project the media resources of interest to the display device 200 for large-screen playback through certain applications with screen projection functions in the terminal device 300. Alternatively, the terminal device 300 and the display device 200 can establish a screen projection connection through certain protocols, such as the DLNA (Digital Living Network Alliance) protocol. Alternatively, the terminal device 300 can mirror the screen with the display device 200, for example, through wireless screen projection methods such as Miracast and Airplay, the current screen content of the terminal device 300 can be synchronized to the display device 200 to achieve screen sharing and multi-screen interaction.

[0287] In one implementation, before the terminal device 300 initiates a screen projection command to the display device 200, the terminal device 300 and the display device 200 can be connected to the same local area network (e.g., WiFi). Taking mirrored screen projection as an example, FIG42 shows a screen projection settings page 50 displayed by the terminal device 300 according to an embodiment of the present application. The screen projection settings page 50 includes a screen projection switch 51. As shown in FIG42, the user can set the on / off state of the screen projection switch 51 by touch operation according to usage requirements.

[0288] In response to the user turning on the screen projection switch 51, the terminal device 300 controls the display screen to switch the screen projection switch 51 in the screen projection setting page 50 to the on state, as shown in FIG43, and searches for a screen projection device that can establish a screen projection connection with the terminal device 300, and controls the display screen to display a screen projection device list 61, an initiate screen projection button 62, and a cancel button 63 in the screen projection setting page 50. The screen projection device list 61 includes N screen projection devices, which are display devices that are connected to the same local area network as the terminal device 300 and support the screen projection function, where N is greater than or equal to zero.

[0289] If N is equal to zero, that is, there is currently no screen projection device, the user can refresh the screen projection settings page 50 or the screen projection device list 61. If N is greater than zero, the user can select the target device from the screen projection device list and click the Start Screen Projection button 62.

[0290] In response to the user clicking the initiate screen projection button 62, the terminal device 300 obtains the device information of the target device and sends a screen projection request to the target device. The screen projection request may carry the device information of the terminal device 300, including but not limited to the device name or model, MAC address, etc.

[0291] In response to the user clicking the cancel button 63, the terminal device 300 may close the screen projection device list 61, or may switch the screen projection switch 51 to the off state. In this scenario, the terminal device 300 does not initiate a screen projection instruction to any display device.

[0292] Figure 44 shows a screen projection request page displayed by display device 200 according to an embodiment of the present application. After receiving the screen projection request, display device 200 (i.e., the target device) can control display 260 to display a screen projection request page 71', as shown in Figure 44. The screen projection request page 71' includes a prompt message 711', a reject button 712', and an accept button 713'.

[0293] The prompt information 711 ′ includes the device information of the terminal device 300 , which is used to prompt that the terminal device 300 has initiated a screen projection instruction to the local device and inquire whether the user accepts the screen projection instruction.

[0294] The display device 200 can receive user input operations through the control device 100 (e.g., a remote control). If the display device 200 receives a click operation on the reject button 712' from the user, it will not establish a screen projection connection with the terminal device 300 and may send a rejection screen projection message to the terminal device 300. Upon receiving the rejection screen projection message, the terminal device 300 may prompt the display device 200 that the screen projection request has been rejected.

[0295] If the display device 200 receives a click operation on the accept button 713' by the user, it establishes a screen projection connection with the terminal device 300. Figure 45 shows the display interface of the display device 200 and the terminal device 300 after the screen projection connection is successful according to an embodiment of the present application. As shown in Figure 45, after the screen projection connection is successful, the terminal device 300 displays the first screen projection page 81, which may include screen content 811 and a first end screen projection button 812.

[0296] After the screen projection connection is successful, the terminal device 300 can start the screen recording program to record the current screen content of the terminal device, thereby obtaining the projection data, and based on the projection connection, synchronize the projection data to the display device 200 in the form of video stream transmission. In this way, after the display device 200 receives the projection data, as shown in Figure 45, it controls the display 260 to display the second projection page 82, which may include the video content 821 corresponding to the projection data and the second end projection button 822.

[0297] In response to the user clicking the first end screen projection button 812, the terminal device 300 controls the display screen to exit the first screen projection page 81, stops executing the screen recording program (so that the video stream corresponding to the screen projection data is terminated), disconnects the screen projection connection with the display device 200, and sends a first end screen projection message to the display device 200. The display device 200 receives the first end screen projection message, controls the display 260 to exit the second screen projection page 82, and displays the page corresponding to the current signal source, thereby ending the current screen projection normally.

[0298] In some implementations, the user can also perform an end screen projection operation on the display device 200. In response to the user clicking the second end screen projection button 822, the display device 200 controls the display 260 to exit the second screen projection page 82, display the page corresponding to the signal source before the screen projection, disconnect the screen projection connection with the terminal device 300, and send a second end screen projection message to the terminal device 300. The terminal device 300 receives the second end screen projection message, controls the display screen to exit the first screen projection page 81, and stops executing the screen recording program, thereby ending the current screen projection normally.

[0299] In some implementations, when the display device 200 is displaying the second projection page 82, if the user switches the signal source, for example, the user switches the signal source to a signal channel such as ATV (Analog TV), DTV (Digital TV) or HDMI, the display device can control the display 260 to exit the second projection page 82, display the page corresponding to the switched target signal source, disconnect the projection connection with the terminal device 300, and send a third projection end information to the terminal device 300. The terminal device 300 receives the third projection end information, controls the display screen to exit the first projection page 81, and stops executing the screen recording program. In this implementation, if the user switches the signal source on the display device 200, the current projection can be forcibly ended.

[0300] In some implementations, when the display device 200 is displaying the second projection page 82, if it receives an operation from the user to switch the signal source, it may not disconnect the projection connection with the terminal device 300, continue to receive the projection data transmitted by the terminal device 300, run the current projection application in background mode, and control the display 260 to display the page corresponding to the switched target signal source in the foreground. In this way, when the display device 200 receives an operation from the user to switch back to the projection application, for example, the user clicks the return key on the control device 100, it exits the target signal source, switches the projection application from the background to the foreground, and causes the display 260 to refresh and display the second projection page 82. In this implementation, if the user switches the signal source on the display device 200 end, the projection connection can be maintained, and the projection application can be switched from the foreground to the background without forcibly ending the current projection.

[0301] After the display device 200 receives the projection data transmitted by the terminal device 300, it generally controls the display to display the projection data in the second projection page 82 using the system default and fixed picture quality parameters, where the picture quality parameters include but are not limited to: brightness, contrast, color, clarity, white balance, sharpness, etc.

[0302] During the research process, the applicant found that factors such as the resolution of the projection data and the processing capabilities of the display device itself will affect the image quality and clarity presented by the second projection page 82, where the resolution may include the width and height of the video frame, such as 1920*1080.

[0303] In some embodiments, the screen projection image quality adjustment logic executed by the display device 200 varies depending on the screen projection protocol, including but not limited to the following three situations:

[0304] The first scenario: For certain standard screen projection protocols, after the display device 200 accepts the projection request, the terminal device 300 can notify the display device 200 of the resolution used for the projection data. This resolution can be a fixed value. Even if the terminal device 300 detects a change in screen orientation during the projection process, such as from portrait to landscape, the resolution change is not notified to the display device 200. In this way, the display device 200 displays the projection data according to the preset fixed resolution and the corresponding fixed image quality parameters. In this scenario, the display device 200 does not dynamically adjust the projection image quality parameters.

[0305] The second scenario: For some standard screen projection protocols, the first resolution corresponding to the horizontal screen orientation and the second resolution corresponding to the vertical screen orientation can be preset. For this screen projection protocol and scenario, in the embodiment of the present application, after the display device 200 accepts the screen projection request, the terminal device 300 can detect the current screen orientation. For example, if the current screen orientation is horizontal, the display device 200 is notified that the resolution of the projection data is the first resolution, so that the display device can match the first image quality parameters according to the first resolution, and start broadcasting the projection data according to the first image quality parameters. When the terminal device 300 detects that the screen orientation is switched to vertical, it notifies the display device 200 that the resolution of the projection data has changed to the second resolution, so that the display device can match the second image quality parameters according to the second resolution, and display the projection data according to the second image quality parameters. In this scenario, the display device 200 can dynamically adjust the image quality parameters of the projection data synchronously when the screen orientation of the terminal device 300 changes.

[0306] The third scenario: This field can also formulate a screen projection protocol corresponding to the screen projection application based on the user's screen projection needs, for example: a screen projection protocol that supports the mirrored screen projection sender (i.e., the terminal device 300) to perform screen recording settings, etc. Figure 46 is a first screen projection page 81 displayed by another terminal device 300 according to an embodiment of the present application. As shown in Figure 46, the first screen projection page 81 can also include a screen recording setting button 813. In response to the user clicking the screen recording setting button 813, the terminal device controls the display screen to display the screen recording setting page.

[0307] FIG47 is a schematic diagram of the terminal device 300 displaying the screen recording setting page 83 according to an embodiment of the present application. As shown in FIG47 , the screen recording setting page 83 may include, but is not limited to: a resolution setting button 831 and a screen direction setting button 832 .

[0308] In response to the user clicking the resolution setting button 831, the terminal device 300 can display at least one resolution option, such as options including 1920*1080, 720*1280, etc., and can also include a custom option so that the user can customize the width and height of the video.

[0309] In response to the user clicking the screen orientation setting button 832, the terminal device 300 can display a screen orientation option to adjust the direction of the recorded video. The screen orientation options may include landscape, portrait, and automatic. Among them, setting the landscape mode will make the aspect ratio of the recorded video greater than or equal to 1, and setting the portrait mode will make the aspect ratio of the recorded video less than 1. The "auto" option can be configured to automatically match the screen orientation according to the resolution of the screen recording. For example, if the currently set resolution of the screen recording (video width * video height) is 608 * 1080, the screen orientation is automatically set to "portrait". Alternatively, the "auto" option can also be configured so that the screen orientation follows the terminal device 300. For example, if the screen orientation of the terminal device 300 is currently portrait, if it is detected that the terminal device 300 is flipped 90 degrees, the screen orientation can be automatically switched to landscape.

[0310] Based on the resolution setting button 831 and / or the screen direction setting button 832 in the screen recording setting page 83, the user can more flexibly and dynamically adjust the resolution of the projection data on the terminal device 300, that is, the resolution of the projection data can be dynamically changed. For the second and third situations mentioned above, if the display device 200 displays the projection data with the default fixed image quality parameters, the image quality parameters cannot be dynamically adjusted as the resolution of the projection data changes, resulting in the projection content presented by the display device 200 not being clear enough and the image quality effect being poor, affecting the user's projection viewing experience.

[0311] To solve the problems described above, an embodiment of the present application provides an implementation solution that can dynamically and adaptively adjust the image quality parameters used when displaying projection data: the display device 200 can preset and store the mapping relationship between the screen direction, preset resolution and image quality parameters based on the CPU chip configured locally.

[0312] In one implementation, assuming that the display device 200 is configured with a CPU 1, in landscape mode, the CPU 1 adjusts the image quality parameters based on the vertical resolution (i.e., video height) of the projection data. The preset mapping relationship is, for example:

[0313] If the video height is greater than or equal to 1080, the image quality parameters include but are not limited to: brightness A1, contrast B1, chroma C1, clarity D1, etc.;

[0314] If 1080> video height ≥ 720, the image quality parameters include but are not limited to: brightness A2, contrast B2, chroma C2, clarity D2, etc.;

[0315] If 720> video height ≥ 480, then the image quality parameters include but are not limited to: brightness A3, contrast B3, chroma C3, clarity D3, etc.;

[0316] If the video height is less than 480, the image quality parameters include but are not limited to: brightness A4, contrast B4, chroma C4, clarity D4, etc.

[0317] In one implementation, assuming that the display device 200 is configured with a CPU 1, in portrait mode, the CPU 1 adjusts the image quality parameters based on the horizontal resolution (i.e., video width) of the projection data. The preset mapping relationship is, for example:

[0318] If the video width is greater than or equal to 1920, the image quality parameters include but are not limited to: brightness A5, contrast B5, chroma C5, clarity D5, etc.;

[0319] If 1920> video width ≥ 1280, the image quality parameters include but are not limited to: brightness A6, contrast B6, chroma C6, clarity D6, etc.;

[0320] If 1280> video width ≥ 1080, then the image quality parameters include but are not limited to: brightness A7, contrast B7, chroma C7, clarity D7, etc.;

[0321] If 1080> video width ≥ 720, the image quality parameters include but are not limited to: brightness A8, contrast B8, chroma C8, clarity D8, etc.;

[0322] If 720> video width ≥ 480, then the image quality parameters include but are not limited to: brightness A9, contrast B9, chroma C9, clarity D9, etc.;

[0323] If the video width is less than 480, the image quality parameters include but are not limited to: brightness A10, contrast B10, chroma C10, clarity D10, etc.

[0324] For example, after the display device 200 receives the projection data, it queries that the resolution of the projection data is 608*1080, which is a vertical video stream, and the width 608 of the video stream is in the range of [480,720). Then, the display is controlled to display the projection content corresponding to the video stream according to the picture quality parameters such as brightness A9, contrast B9, chroma C9, and clarity D9.

[0325] For another example, the resolution of the projection data received by the display device 200 is 1280*960, which is a horizontal video stream, and the height 960 of the video stream is within the range of [720,1080). Then the display is controlled to display the projection content corresponding to the video stream according to the picture quality parameters such as brightness A2, contrast B2, chroma C2, and clarity D2.

[0326] It should be noted that the mapping relationship provided above in this application is only an example and does not represent or limit the configuration of the actual mapping relationship. This field can set the interval distribution of preset resolutions according to the application scenario and user requirements. The preset resolution can fully cover the possible value range of the video width / height of the projection data, and can also make the resolution interval more refined, thereby improving the precision of the image quality adjustment. In addition, it is also possible to expand more types of image quality parameters to improve the projection image quality of the display device 200.

[0327] In some implementations, the display device 200, serving as the screen projection receiving end, may be configured with one or more CPU chips. Different CPU chip models have different processing capabilities, resulting in different image quality effects presented by the display device 200 for the projection data. If the display device 200 has multiple CPU chips (i.e., a multi-core device), corresponding image quality parameter configuration information can be set for each CPU chip based on its processing capability. The image quality parameter configuration information may include a mapping relationship between the CPU chip model, screen orientation, preset resolution, and image quality parameters.

[0328] In some implementations, for example, a dual-core display device includes two CPU chips (CPU1 and CPU2), and corresponding image quality parameter configuration information can be set for CPU1 and CPU2 respectively. Assume that CPU1 corresponds to image quality parameter configuration information A and CPU2 corresponds to image quality parameter configuration information B. The embodiments of the present application do not limit the number and model of processors.

[0329] In some implementations, after the display device 200 receives a screen projection request from the terminal device 300, it receives information such as an SPS (Sequence Parameter Set) sent by the terminal device 300. The SPS is an information set that records parameters related to screen projection, and may include image format information, codec parameter information, temporal grading information, and other content. The image format information may include the resolution of the video stream.

[0330] Figure 48 is a schematic diagram of the principle of the display device 200 dynamically adjusting the image quality parameters according to an embodiment of the present application. As shown in Figure 48, after the display device 200 parses the SPS information sent by the terminal device 300, it obtains the resolution of the video stream and the corresponding screen direction, queries the target CPU currently processing the screen projection process, and obtains the target image quality parameter configuration information corresponding to the target CPU. The display device 200 further obtains the matching target image quality parameter 1 from the target image quality parameter configuration information based on the screen direction and resolution of the video stream, and receives the projection data sent by the terminal device 300 in the form of a video stream after the SPS, and controls the display to start broadcasting the projection data according to the target image quality parameter 1.

[0331] In some implementations, the terminal device 300 can send SPS information to the display device 200 in frames based on the recorded video stream, and carry the resolution of the current frame in the SPS information. In this way, the display device 200 can obtain the resolution of each frame and autonomously detect whether the resolution of the video stream changes, thereby dynamically adjusting the image quality parameters.

[0332] In some implementations, the terminal device 300 may also send SPS information to the display device 200 on a frame-by-frame basis. Referring to Figure 48, when the terminal device 300 detects that the resolution of the video stream has changed (for example, the user adjusts the resolution and / or screen direction of the screen recording), it may send SPS′ to the display device, and the SPS′ includes the updated resolution, so that after the display device receives the SPS′, it re-matches the target image quality parameter 2 from the target image quality parameter configuration information according to the updated resolution and / or screen direction, and displays the projection data received after the SPS′ according to the target image quality parameter 2. Before receiving the SPS′, the display device 200 assumes that the resolution of the projection data has not changed, so that the display device 200 will not dynamically adjust the image quality parameters. When the terminal device 300 detects that the resolution of the video stream has changed, it may notify the display device 200 of the updated resolution in the form of SPS′, or it may use other notification forms, such as resolution change indication information. The embodiment of the present application enables the display device to follow the changes in the resolution of the projection data and dynamically adjust the image quality parameters synchronously, so that the projection image quality is no longer limited to the system default image quality parameters, thereby improving the display effect of the projection content.

[0333] In some implementations, for other non-mirror screen projections, since the terminal device 300 does not need to record the screen, when the media resources being projected remain unchanged, the resolution of the projection data is generally unchanged. Taking application projection as an example, after the user starts the video application Q, the movie M is played. The current playback clarity of the movie M is high-definition (for example, the resolution is 480P, 720*480). The user can also click the projection button on the playback page of the movie M and choose to project the movie M to the display device 200 for playback. After the terminal device 300 establishes a projection connection with the display device 200, it can send the URL1 of the 480P source of the movie M to the display device 200. The display device 200 receives the URL1, obtains the resolution and screen orientation (landscape) corresponding to the URL1, and automatically matches the target image quality parameter 3 according to the aforementioned scheme. According to the target image quality parameter 3 and the URL1, it downloads and starts playing the source 1 of the movie M corresponding to the URL1. Since the resolution (480P) corresponding to URL1 is fixed, the display device 200 adjusts the image quality parameters only once when starting to play the video source 1, and does not dynamically adjust the image quality parameters during subsequent playback of the video source 1.

[0334] In some implementations, a user can switch the playback resolution of movie M on terminal device 300, for example, from HD to UHD (e.g., 720P resolution, 1280*720). Upon receiving the user's operation to switch the resolution, terminal device 300 can send the URL2 of the 720P source of movie M to display device 200. Display device 200 receives URL2, obtains the resolution and screen orientation corresponding to URL2, matches target image quality parameters 4, and adjusts the image quality of the movie M being played based on target image quality parameters 4.

[0335] In some implementations, after receiving URL2, the display device 200 can still maintain the playback process of the film source 1 corresponding to URL1, only match the target image quality parameter 4 based on URL2, and update the image quality parameter from target image quality parameter 3 to target image quality parameter 4.

[0336] In some implementations, after receiving URL2, display device 200 may also stop playing source 1 corresponding to URL1 and record the current playback progress of source 1. Based on target image quality parameters 4 and URL2, display device 200 downloads and starts playing source 2 of movie M corresponding to URL2, automatically adjusting source 2 to the playback progress of source 1 at the start of playback. Because the resolution (720P) corresponding to source 2 is fixed, display device 200 only adjusts the image quality parameters once when starting source 2 and does not dynamically adjust the image quality parameters during subsequent playback of source 2.

[0337] In some implementations, the display device 200 can save the image mode and image quality parameters before screen projection (hereinafter referred to as: first image mode and first image quality parameters) before starting the screen projection service. After controlling the display to close the second screen projection page 82, the display device 200 can control the display to display the page before screen projection according to the first image mode and the first image quality parameters. In this way, the embodiment of the present application realizes: during the screen projection process, the display device 200 dynamically and adaptively matches and adjusts the image quality parameters according to the local CPU configuration and the resolution of the projection data, and automatically restores to the image quality before projection (image quality restoration) when exiting the screen projection, thereby avoiding affecting the display effects of other source scenes.

[0338] In some implementations, the display device 200 can support multiple types of sources, including but not limited to ATV, DTV, HDMI, local pre-installed applications, and third-party applications. When connected to a physical signal source such as ATV, DTV, or HDMI, the image quality parameters used for video playback are generally preset by the source. While such a source is active, the image quality parameters cannot be changed at will. For example, the display device 200 is playing DTV according to the picture quality parameter X. At this time, it receives a mirror screen projection request initiated by the terminal device 300 and starts broadcasting the screen projection data with the picture quality parameter Y. During this period, it is detected that the resolution of the screen projection data has changed and is adjusted to the picture quality parameter Z. Before the normal end of this screen projection, the display device 200 suddenly receives a shutdown command or a restart command, and controls the display device 200 to shut down. After the display device 200 is restarted, since the picture quality restoration operation is not performed, the current effective picture quality parameter of the display device 200 is the picture quality parameter Z before the previous shutdown. In this way, if the display device 200 receives an operation to switch to DTV, when starting DTV, it needs to change from the picture quality parameter Z back to the DTV default picture quality parameter X. When presented on the display interface, the user will see the display suddenly flicker. The user does not know that it is caused by adjusting the picture quality, and may think that the display device 200 has a malfunction, affecting the user experience.

[0339] In view of this, the source can be classified in the embodiment of the present application, for example, divided into the first category source and the second category source. Among them, the first category source can support dynamic adjustment of picture quality parameters, and the first category source includes but is not limited to: preset sources such as preset applications and third-party applications in the system, customized sources, etc. The second category source is a signal source (non-preset source) whose picture quality parameters cannot be adjusted dynamically. This type of source often has picture quality parameters with default configuration. The second category source includes but is not limited to: physical signal sources such as DTV, ATV, and HDMI. In this way, after accepting the screen projection request, the display device can decide the control logic of the picture quality parameters based on the type of the current source.

[0340] In some implementations, the application layer of the display device 200 can set a first status bit, which can be used to store the first image quality parameters used by the source before the projection data starts. The first status bit, for example, records the first image quality parameters such as "brightness|contrast|chroma|clarity|..." in a String type; at the end of each projection, the display device 200 can control the display to restore to the image quality before projection according to the first image quality parameters. After the image quality restoration is completed, the first image quality parameters stored in the first status bit are cleared, and the first status bit is set to the first target value. The first target value is used to indicate that the image quality restoration has been completed after the projection is completed.

[0341] In some implementations, the application layer of the display device 200 may further set a second status bit, and the second status bit is used to store the first image mode used by the source before the projection data starts playing. The second status bit records the identifier of the first image mode, for example, in the form of an int type. At the end of each projection, the display device 200 may further control the display to restore to the image mode before projection according to the first image mode. After the image mode restoration is completed, the identifier of the first image mode stored in the second status bit is cleared, and the second status bit is set to the second target value. The second target value is used to indicate that the image mode restoration has been completed after the projection is completed. The categories of image modes include, but are not limited to, standard, vivid, natural, automatic, eye protection and other modes. The first target value and the second target value may be the same or different. For example, the first target value and the second target value are both set to -1. The embodiments of the present application do not limit the values ​​of the first target value and the second target value.

[0342] FIG49 is a flowchart of a fourth processing method for a display device according to an embodiment of the present application, which is executed by at least one processor 250 in the display device 200. As shown in FIG49 , after the display device 200 receives the screen projection request sent by the terminal device 300, the method includes:

[0343] S491: Determine whether the screen projection process is handled by the application layer.

[0344] This embodiment of the present application provides two screen projection processing methods: the first is to call a system preset interface to enable the underlying system to execute the screen projection process; the second is to have the application layer execute the screen projection process, which includes the processing logic for dynamically adaptively adjusting image quality parameters. The display device 200 can store platform capability information, which is used to indicate which screen projection processing logic the display device executes.

[0345] In the specific implementation, when executing step S491, the platform capability information is obtained; if the platform capability information records a first capability value, it is determined that the screen projection process is handled by the system bottom layer, and step S492 is executed; if the platform capability information records a second capability value, it is determined that the screen projection process is handled by the application layer, and step S493 is executed.

[0346] S492: Call the system preset interface. In step S492, the status value of the system preset interface is False, indicating that the screen projection service has not been started. After step S492, step S499 is executed.

[0347] S493: Register Source Listener, listen to the current source, and identify the category of the current source.

[0348] S494: Determine whether the current source is a first-category source. If the current source is a first-category source, execute step S495; if the current source is a second-category source, execute step S499.

[0349] S495, determining whether the first state bit stores the first target value.

[0350] If the first status bit does not store the first target value, it indicates that the image quality restoration was not completed at the end of the previous screen projection. At this time, the first status bit stores the image quality parameters recorded before the start of the previous screen projection, then execute step S496; if the first status bit stores the first target value, it indicates that the image quality restoration was completed at the end of the previous screen projection, then execute step S497.

[0351] S496: Setting the image mode to the first target mode, setting the first image quality parameter in the first target mode, and deleting the image quality parameter currently stored in the first status bit so that the first status bit stores the first target value. The first target mode is the image mode set when the screen projection is initialized. The first target mode can be a standard image mode. This allows the screen projection image quality parameters to be dynamically adjusted only in the standard image mode, and not in other image modes, thereby reducing the impact on the image display of the display device.

[0352] S497: Determine whether the second status bit stores the second target value.

[0353] If the second status bit does not store the second target value, indicating that the image mode restoration was not completed when the previous screen projection ended, then execute step S498; if the second status bit stores the second target value, indicating that the image mode restoration was completed when the previous screen projection ended, then execute step S499.

[0354] S498: The image mode is set to the second target mode currently indicated by the second status bit, and the identifier of the second target mode stored in the second status bit is deleted, so that the second status bit stores the second target value. Through the above process, the image mode and image quality parameters are initialized before starting the screen projection service. When the application layer detects that the first status bit stores the first target value and the second status bit stores the second target value, the screen projection service can be started.

[0355] S499: Start the screen projection service and obtain the resolution of the projection data based on the SPS information sent by the terminal device.

[0356] S4910: Determine whether the screen projection process is handled by the application layer.

[0357] In the specific implementation, when executing step S4910, the platform capability information is obtained; if the platform capability information records a first capability value, it is determined that the screen projection process is handled by the system bottom layer, and step S4911 is executed; if the platform capability information records a second capability value, it is determined that the screen projection process is handled by the application layer, and step S4912 is executed.

[0358] S4911: Call the system preset interface to start broadcasting the projection data according to the second image mode and the second image quality parameters.

[0359] Among them, the second image mode is the image mode used by default when the system bottom layer executes the screen projection process, and the second image quality parameter is the image quality parameter used by default when the system bottom layer executes the screen projection process. In step S4911, the status value of the system preset interface is True, indicating that the screen projection service has been started.

[0360] S4912: Determine whether the current source is a first-category source. If the current source is a second-category source, execute step S4913; if the current source is a first-category source, execute step S4914.

[0361] S4913: Start broadcasting the screen casting data according to the third image mode and third image quality parameters preset for the current source. If the current source is a second-category source, since the second-category source is a non-preset source that does not support dynamic adjustment of image quality parameters, but has a preset, default image mode (referred to as the third image mode), the third image mode includes preset image quality parameters (referred to as the third image quality parameters), and therefore, the screen casting can be started according to the third image mode and the third image quality parameters.

[0362] S4914: Enable the second status bit to store the identifier of the first image mode, and call the middleware capability interface to set the image mode to the first target mode.

[0363] Among them, the first image mode is the image mode set by the current source before the projection data is started. The first image mode may be: the first target mode, the second target mode, or any one of the third target modes to which the user switches before the projection data is started.

[0364] S4915: Acquire the first image quality parameter currently included in the first target mode, and enable the first state bit to store the first image quality parameter. At this time, the display device 200 displays the user interface in the first target mode and the first image quality parameter.

[0365] By executing steps S4914 and S4915, before starting screen projection, the first image mode and the first picture quality parameters used by the current first type of source are saved. In this way, when the display device 200 ends screen projection, the picture quality parameters can be restored to the first picture quality parameters before screen projection in the first target mode (for example, the standard image mode), and the image mode can be restored to the first image mode before screen projection, thereby completing the image mode restoration and image quality restoration.

[0366] S4916: Match the target image quality parameters according to the current CPU type, the resolution of the projection data, and the image quality parameter configuration information.

[0367] S4917: Start broadcasting the projection data according to the first target mode and target image quality parameters.

[0368] S4918: When it is detected that the resolution of the projection data has changed, dynamically adjust the target image quality parameters in the first target mode.

[0369] In the above method flow, the reason for repeatedly judging whether the current source is a first-class source is that: the premise for allowing dynamic adjustment of picture quality parameters during the screen projection process is that the current source is a first-class source (including designated sources such as screen projection applications). However, on the display device 200 side, the user can switch sources through the control device 100 or voice, for example, the user can click the DTV button on the remote control to quickly play digital TV programs. The second-class source does not support the application layer to execute the aforementioned picture quality adjustment logic when running. Therefore, before adjusting the picture quality parameters or performing picture quality restoration, the application layer needs to first judge the category of the current source based on the Source Listener to ensure the accuracy of the source playback control and the picture quality effect.

[0370] FIG50 is a second flowchart of a fourth processing method for a display device according to an embodiment of the present application, which is executed by at least one processor 250 in the display device 200. As shown in FIG50 , based on the method flow illustrated in FIG49 , when the display device 200 detects exiting screen projection, the method includes:

[0371] S501: Determine whether the screen projection process is handled by the application layer.

[0372] When any of the following conditions is met, it is detected that the screen projection needs to be exited: the display device 200 receives the first end screen projection information sent by the terminal device 300, or the display device 200 detects that the screen projection connection with the terminal device 300 is disconnected, or the display device 200 receives the user's input based on the second screen projection page 82, clicking the second end screen projection button 822, etc.

[0373] In the specific implementation, when executing step S501, the platform capability information is obtained; if the platform capability information records a first capability value, it is determined that the screen projection process is processed by the system bottom layer, and step S502 is executed; if the platform capability information records a second capability value, it is determined that the screen projection process is processed by the application layer, and step S503 is executed.

[0374] S502: Call the system preset interface, stop playing the screen projection data, and exit the screen projection service. In step S502, since the program to exit the screen projection is executed, the status value of the system preset interface is changed to False.

[0375] S503: Determine whether the current source is a first-category source. If the current source is a second-category source, execute step S504; if the current source is a first-category source, execute step S505.

[0376] S504: The application layer stops playing the screen projection data and exits the screen projection service.

[0377] S505: Determine whether the first state bit stores the first target value.

[0378] If the first status bit does not store the first target value, indicating that the image quality restoration has not been completed, then execute step S506; if the first status bit stores the first target value, indicating that the image quality restoration has been completed at the end of this screen projection, then execute step S507.

[0379] S506: After the image quality parameters are set to the first image quality parameters in the first target mode, the first image quality parameters currently stored in the first status bit are deleted, and the first status bit is set to store the first target value. During step S506, if the currently effective image mode is not the first target mode, the image mode can be set to the first target mode, and then the image quality parameters are restored in the first target mode.

[0380] S507: Determine whether the second state bit stores the second target value.

[0381] If the second status bit does not store the second target value, indicating that the image mode restoration is not completed, then execute step S508; if the second status bit stores the second target value, indicating that the image mode restoration is completed at the end of this screen projection, then execute step S509.

[0382] S508: After the image mode is set to the first image mode, the identifier of the first image mode currently stored in the second state bit is deleted, so that the second state bit stores the second target value.

[0383] In the first target mode, the picture quality parameters are restored to the first picture quality parameters used before the screen projection starts, and then the image mode is restored from the first target mode to the first image mode used before the screen projection starts. In this way, when the screen projection ends, the display presents the user interface according to the first image mode and the picture quality parameters contained in the first image mode.

[0384] S509: The application layer stops playing the screen projection data, exits the screen projection service, and cancels the registration of the Source Listener.

[0385] In some implementations, during the screen projection process, the Source Listener can listen to events where the source switches, such as when the current source changes to the target source. If the application layer determines that the target source is a second-category source, it will not restore to the first image quality parameters before the screen projection when switching the source, that is, it will not execute the image quality restoration program. It can start the target source according to the default third image mode and third image quality parameters preset by the target source, thereby avoiding abnormalities when the target source starts. The abnormality can be presented as, for example, a flickering of the target source screen or a sudden change in image quality. When returning to the screen projection interface from the target source, or restarting the screen projection, you can refer to steps S491 to S4918 to perform initialization settings before starting the screen projection, and then according to steps S499 to S4918, execute the start of the screen projection data and the dynamic adjustment of the image quality parameters during the screen projection.

[0386] If the application layer determines that the target source is a first-class source, for example, switching from a screen projection application to Launcher (desktop, or homepage), then when switching the source, it can refer to steps S505 to S509 to restore to the first image mode and first image quality parameters before projection, thereby restoring the image mode and image quality.

[0387] The embodiment of the present application can match the image quality parameters suitable for screen projection according to the CPU chip type, the resolution of the projection data and the preset image quality parameter configuration information in the display device, and can dynamically adjust the image quality parameters synchronously with the resolution change of the projection data to improve the image quality and clarity of the projection, thereby providing users with a better projection experience. The display device end can monitor the event of switching the source, and by identifying the platform processing capability (the execution subject is the application layer or the bottom layer of the system) and the category of the current source (first-class source or second-class source), it provides a corresponding projection process and image quality processing logic to ensure the accuracy of the source playback control and the image quality effect. When the display device switches the source or exits the projection, it can decide whether to execute the image mode and image quality restoration based on the category of the current source and the platform processing capability to avoid affecting the display effect of other source scenes. The embodiment of the present application improves the projection performance and picture display effect of the display device end.

[0388] In addition, the UI drawings shown in the embodiments of the present application are merely schematic. Based on the aforementioned processing logic for dynamically adjusting the projection image quality, this field can expand and adjust the projection process and projection image quality adjustment scheme according to different projection types, projection protocols, user operations, and source types to adapt to more possible scenario requirements.

[0389] In some implementations, the present application further provides a computer-readable non-volatile storage medium that may store a computer program. When the computer-readable non-volatile storage medium is configured in the display device 200, the computer program, when executed by at least one processor, implements the various processes of any of the methods in the above-described embodiments and can achieve the same technical effects. To avoid repetition, the details are not described here. The computer-readable non-volatile storage medium may be a magnetic disk, an optical disk, ROM, or RAM, etc.

Claims

1. A display device, comprising: a display configured to display a user interface and / or projection data; A communicator is configured to communicate with an external device and / or connect to a terminal device for screen projection according to a communication protocol; a memory configured to store computer instructions and / or data associated with the display device; At least one processor, connected to the display, the communicator, and the memory, is configured to execute the computer instructions to cause the display device to: In response to a screen casting instruction, obtaining a readiness status of a first application and launching a second application; wherein the first application is an application displayed in a current user interface, and the second application is a screen casting application; the readiness status is used to indicate whether the first application supports floating display; If the ready state is a state that supports floating, controlling the display to display the second application in a floating manner above the first application; If the ready state is a state that does not support suspension, control the display to display the second application in full screen.

2. The display device of claim 1 , wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: Traverse all applications in the display device; Filter out the second application that supports the floating function from all applications; Bind the first application and the client service in the second application to establish a data intercommunication relationship between the first application and the second application.

3. The display device of claim 1 , wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: Before floating the second application above the first application, controlling the display to display an operation prompt interface above the first application; After the second application is displayed in a floating manner above the first application, a floating receipt is generated, and the floating receipt is sent to the first application; Exit the operation prompt interface according to the floating receipt.

4. The display device according to claim 1 , wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: Controlling the display to display the first application in full screen; Setting the theme of the second application to transparent; The display is controlled to overlay and display the second application with a transparent theme on top of the first application.

5. The display device of claim 1 , wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: Controlling the second application to capture a focus instruction in the user interface; Sending the focus instruction to the first application; Parsing the focus instruction; The parsing result of the focus instruction is sent to other applications, so that the other applications perform operations according to the parsing result, and the other applications are applications other than the first application and the second application in the display device.

6. The display device of claim 1 , wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: In response to the screen projection disconnection event, generating a disconnection instruction; sending the disconnect instruction to the first application; The display is controlled to exit the second application according to the disconnect instruction.

7. The display device of claim 6, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: parsing a disconnection reason of the disconnection instruction from the disconnection instruction; generating a prompt message according to the disconnection instruction and the disconnection reason; When exiting the second application, the display is controlled to display the prompt message.

8. The display device of claim 1 , wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: detecting the running status of the first application; If the running state is abnormal, detecting the screen projection state of the second application; If the second application is in the screen projection state, exit the second application; If the second application is in a non-projection state, control the display to display the second application in full screen.

9. The display device of claim 1 , wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: In response to a screen switching instruction for a second application to play media data, obtaining a playback window size of the second application; Modify the playback window size according to the screen switching instruction; The media asset data in the second application is played according to the modified playback window size.

10. The display device according to claim 1, wherein the display is further configured to display the user interface of the first application, and display the user interface of the second application on an upper layer of the first application; The at least one processor is further configured to execute the computer instructions to cause the display device to: receiving a switching instruction for switching an application display position; In response to the switching instruction, switching the levels of the first application and the second application to control the display to display the user interface of the second application in full screen; The first application and the second application are applications that establish a data intercommunication relationship with each other and both support floating display; Reading the screen size of the first application, and planning a floating window on the upper layer of the second application according to the screen size; The display is controlled to display the user interface of the first application through the floating window.

11. The display device of claim 10, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: Receive a startup instruction for displaying an application screen; In response to the start instruction, obtaining a readiness state of the first application, and starting the second application; If the preparation state is a state of complete preparation, controlling the display to display the second application in a floating manner on top of the first application; If the preparation state is an unprepared state, the display is controlled to display the second application in full screen.

12. The display device of claim 10, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: monitoring a focus event of the first application; Encapsulating an event for switching screen positions as the switching instruction; The first application is controlled to execute the switching instruction, and the switching instruction is forwarded to the second application, so that the second application executes the switching instruction.

13. The display device of claim 12, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: After executing the switching between the levels of the first application and the second application, detecting the level position of the first application; Marking the window of the first application as a target window according to the hierarchical position; The focus event of the target window is monitored, and the focus event is encapsulated into a control instruction corresponding to the event.

14. The display device of claim 10, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: Acquire media asset data of a target application, where the target application is the first application or the second application; Parsing the screen size in the media resource data to obtain the screen state of the target application screen, wherein the screen state includes a horizontal screen state and a vertical screen state; If the screen state of the target application screen changes, a screen rotation instruction for the horizontal or vertical screen is generated, so that the target application executes the rotation instruction.

15. The display device of claim 14, wherein the at least one processor is configured to execute the computer instructions to cause the display device to: If the screen state of the target application screen changes, obtaining the window size of the target application; If the window width of the target application is greater than or equal to the window height, obtaining the window height, and planning a floating window on the upper layer of the second application according to the window height and the screen size; If the window width of the target application is smaller than the window height, the window width is acquired, and a floating window is planned on the upper layer of the second application according to the window width and the screen size.

16. The display device of claim 10, wherein the at least one processor is configured to execute the computer instructions to cause the display device to: Receive a close instruction; Sending the close instruction to the first application; The shutdown instruction is executed by the first application, and / or the first application is controlled to forward the shutdown instruction to the second application, so as to control the second application to execute the shutdown instruction.

17. The display device of claim 16, wherein the at least one processor is configured to execute the computer instructions to cause the display device to: detecting a closed state of the first application and the second application; If the second application is in a closed state and the first application is in an open state, the display is controlled to display the user interface of the first application in full screen.

18. The display device according to claim 1, further comprising: a rotating assembly configured to rotate the display; The display is further configured to display a user interface of a first application, and to display a user interface of a second application in a floating window on top of the first application; The at least one processor is further configured to execute the computer instructions to cause the display device to: In response to a rotation event of the rotating component, the display screen direction of the first application is changed, and the window size and screen size of the second application are read; the first application and the second application are applications that establish a data intercommunication relationship with each other and both support floating display; generating layout information according to the display screen orientation and the window size after conversion of the first application; A floating window is planned on the upper layer of the first application according to the layout information and the screen size, and the display is controlled to display the user interface of the second application through the floating window.

19. The display device of claim 18, wherein the at least one processor is configured to execute the computer instructions to cause the display device to: monitoring a rotation direction of the rotating component, and changing a global screen orientation according to the rotation direction, wherein the global screen orientation is a display screen orientation of the display; In response to a startup instruction for displaying a first application screen, reading the global screen orientation; The display is controlled to display the user interface of the first application in full screen according to the global screen.

20. The display device of claim 19, wherein the at least one processor is configured to execute the computer instructions to cause the display device to: In response to a start-up instruction for displaying a second application screen, reading a screen size of the second application and the global screen orientation; Planning a floating window on an upper layer of the first application according to the screen size and the global screen direction; The display is controlled to display the user interface of the second application through the floating window.

21. The display device according to claim 20, wherein the at least one processor is configured to plan a floating window on a layer above the first application according to the screen size and the global screen orientation, and is configured to execute the computer instructions to cause the display device to: parsing a screen width and a screen height of the screen size, and calculating an aspect ratio of the second application according to the screen width and the screen height; If the screen width is greater than or equal to the screen height, setting the window size to a default width, and calculating the window height according to a ratio of the default width to the screen height; If the screen width is smaller than the screen height, the window size is set to a default height, and the window height is calculated according to a ratio of the default width to the screen height.

22. The display device of claim 20, wherein the at least one processor is configured to execute the computer instructions to cause the display device to: Detecting a configured screen orientation of the first application, where the configured screen orientation is a display screen orientation supported by the first application; If the global screen orientation is not within the range of the configured screen orientation, traversing the floating windows and filtering a target application according to a preset priority, the target application being the second application that supports the global screen orientation; Switching the hierarchy of the first application and the target application to display the target application in full screen; A floating window of the first application is planned on an upper layer of the target application, and a user interface of the first application is displayed through the floating window.

23. The display device according to claim 18, wherein the at least one processor is configured to plan a floating window on a layer above the first application according to the layout information and the screen size, and is configured to execute the computer instructions to enable the display device to: parsing a window width and a window height of the window size, and calculating an aspect ratio of the second application according to the screen size; If the window width is greater than or equal to the window height, extracting the window height, and calculating the width and height of the floating window according to the window height and the screen ratio; If the window width is smaller than the window height, the window width is extracted, and the width and height of the floating window are calculated according to the window width and the screen ratio.

24. The display device of claim 10 or 18, wherein the at least one processor is configured to execute the computer instructions to cause the display device to: Traversing the applications in the display device; Filtering out applications that support the floating function to serve as the first application or the second application; Bind the first application and the client service in the second application to establish a data intercommunication relationship between the first application and the second application.

25. The display device of claim 18, wherein the at least one processor is configured to execute the computer instructions to cause the display device to: monitoring a focus event of the second application; Encapsulating the focus event into a control instruction corresponding to the event, and controlling the second application to send the control instruction to the first application; The first application is controlled to forward the control instruction to other applications to notify the other applications to execute the control instruction, where the other applications are applications other than the first application and the second application in the display device.

26. The display device of claim 18, wherein the at least one processor is configured to execute the computer instructions to cause the display device to: Acquire media resource data of the second application; Extracting a frame image from the media asset data; The screen height and screen width are parsed from the screen frame image to obtain the screen size of the second application.

27. The display device according to claim 1, wherein At least one processor is further configured to execute the computer instructions to cause the display device to: Upon receiving a screen projection request sent by a terminal device, obtaining image quality parameter configuration information, wherein the image quality parameter configuration information includes a mapping relationship between a preset resolution and image quality parameters; Receive the projection parameter information and projection data sent by the terminal device, the projection parameter information includes the projection data resolution; Determining target image quality parameters used during screen projection based on the screen projection parameter information and the image quality parameter configuration information; According to the target image quality parameters, the display is controlled to display the projection data.

28. The display device according to claim 27, wherein the at least one processor comprises at least one, and the image quality parameter configuration information comprises a mapping relationship between a processor model, a preset resolution, and an image quality parameter, and after obtaining the image quality parameter configuration information, the at least one processor is further configured to execute the computer instructions to cause the display device to: Get the model of the target processor currently executing the screen projection process; Obtain target image quality parameter configuration information corresponding to the target processor model.

29. The display device according to claim 27, wherein the at least one processor is configured to execute the computer instructions so as to cause the display device to determine the target image quality parameter in the following manner: Determining the screen direction corresponding to the screen projection data according to the screen projection parameter information; If the screen orientation is landscape, determining the target image quality parameter according to the video height of the projection data and the target image quality parameter configuration information; If the screen orientation is vertical, the target image quality parameter is determined according to the video width of the projection data and the target image quality parameter configuration information.

30. The display device according to claim 27, after controlling the display to display the projection data, the at least one processor is further configured to execute the computer instructions to enable the display device to: In response to ending the screen projection operation, disconnecting the screen projection connection with the terminal device and obtaining the current signal source; If the current signal source is a preset source, setting the image mode to the first target mode; wherein the preset source is a signal source type with adjustable image quality parameters; Read the first image quality parameter stored in the first status bit before starting to broadcast the screen projection data; Setting the image quality parameter to the first image quality parameter; Delete the first image quality parameter stored in the first status bit, and make the first status bit store a first target value; wherein, the first target value is used to indicate that the display has been restored to the image quality presented before the projection data started.

31. The display device of claim 30, wherein after the first status bit stores the first target value, the at least one processor is further configured to execute the computer instructions to cause the display device to: Read the identifier of the first image mode stored in the second state bit before starting to play the screen projection data; Setting the image mode to the first image mode; The identifier of the first image mode stored in the second state bit is deleted, and the second state bit is made to store a second target value; wherein, The second target value is used to indicate that the display has recovered to the image mode presented before the projection data was played; Control the display to stop displaying the projection data.

32. The display device according to claim 31, before receiving the projection parameter information and projection data sent by the terminal device, the at least one processor is further configured to execute the computer instructions to enable the display device to: Get the current signal source; If the current signal source is a preset source and the first state bit does not store the first target value, setting the image mode to the first target mode and setting the first image quality parameter included in the first target mode; Deleting the image quality parameter currently stored in the first status bit and used before the previous screen projection started, so that the first status bit stores the first target value, and determining whether the second status bit stores the second target value; If the second status bit does not store the second target value, setting the image mode to the second target mode currently indicated by the second status bit, and deleting the identifier of the second target mode stored in the second status bit, so that the second status bit stores the second target value; If the second status bit stores the second target value, the screen projection service is started.

33. The display device according to claim 31, before receiving the projection parameter information and projection data sent by the terminal device, the at least one processor is further configured to execute the computer instructions to enable the display device to: If the current signal source is a preset source and the first state bit stores the first target value, determining whether the second state bit stores the second target value; If the second status bit does not store the second target value, setting the image mode to the second target mode currently indicated by the second status bit, and deleting the identifier of the second target mode stored in the second status bit, so that the second status bit stores the second target value; If the second status bit stores the second target value, the screen projection service is started.

34. The display device according to claim 32 or 33, before determining the target image quality parameters used in screen projection, the at least one processor is further configured to execute the computer instructions to cause the display device to: If the current signal source is a preset source, the second state bit is made to store the identifier of the first image mode, and the image mode is set to the first target mode; wherein, The first image mode is one of the first target mode, the second target mode, and a third target mode switched by the user before the screen projection data starts playing; Acquire the first image quality parameter currently included in the first target mode, and enable the first status bit to store the first image quality parameter; Alternatively, if the current signal source is not a preset source, the projection data is broadcast according to the second image mode and the second image quality parameters preset by the current signal source.

35. The display device according to claim 31, after controlling the display to display the projection data, the at least one processor is further configured to execute the computer instructions to enable the display device to: When detecting that the resolution of the projection data has changed, updating the target image quality parameter, and displaying the projection data according to the updated target image quality parameter; In response to an operation of switching to a target source, if the target source is a preset source, setting the image mode to the first target mode and setting the image quality parameter to the first image quality parameter; deleting the first image quality parameter stored in the first status bit and causing the first status bit to store the first target value; Setting the image mode to the first image mode, deleting the identifier of the first image mode stored in the second state bit, and causing the second state bit to store the second target value; Control the display to stop displaying the projection data and display the target source.

36. A processing method for a display device, comprising: In response to a screen casting instruction, obtaining a readiness status of a first application and launching a second application; wherein the first application is an application displayed in a current user interface, and the second application is a screen casting application; the readiness status is used to indicate whether the first application supports floating display; If the ready state is a state that supports floating, controlling the display to display the second application in a floating manner above the first application; If the ready state is a state that does not support suspension, control the display to display the second application in full screen.

Images