Display device and screen projection display method for display device

Abstract

The application provides a display device and a screen projection display method for the display device. The display device can establish a communication connection with a terminal device and receive screen projection data sent by the terminal device. The screen projection data can include a picture frame used to form a screen projection picture. The picture frame can include a valid picture area and an invalid picture area. The display device can detect the invalid picture area of the picture frame, calculate target display parameters of the screen projection picture according to the invalid picture area, control the display to display the screen projection picture according to the target display parameters, and set the center point of the valid picture area in the picture frame on the center point of the user interface of the display device, and make the valid picture area of the picture frame have the maximum proportion in the user interface. In this way, the invalid picture area displayed by the display device can be removed to the maximum extent, the picture in the valid picture area can be displayed in the center, and the user's viewing experience is improved.

Description

Display device and screen projection display method for display device

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent applications filed on March 29, 2024, with application number 202410386294.X; filed on March 29, 2024, with application number 202410381310.6; filed on March 29, 2024, with application number 202410381264.X; and filed on March 29, 2024, with application number 202410386084.0, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the technical field of display devices, and in particular to a display device and a screen projection display method for the display device. Background Art

[0004] Display devices are intelligent devices that can present user interfaces and support user interaction. For example, smart TVs are based on Internet application technologies, feature open operating systems and chips, and boast an open application platform. They enable two-way human-computer interaction and integrate multiple functions, including audio, video, entertainment, and data, to meet diverse and personalized user needs. Display devices can establish communication connections with mobile phones and other terminal devices. Based on specific projection protocols, they can obtain projection data from terminal devices and display the projection image on the display device.

[0005] During the process of establishing screen projection between the display device and the terminal device, the terminal device can record the screen content displayed by itself to form projection data in the form of a video stream, and then send the projection data to the display device for display. However, in some scenarios, when the terminal device projects the screen to the display device, the screen displayed on the terminal device may have invalid screen areas, such as solid color areas such as black with no display content. For example, when the terminal device switches from landscape to portrait mode, black areas may appear at the top and bottom of the terminal device screen. Alternatively, the media played on the terminal device itself may also have invalid screen areas. In this way, when the display device displays the projection screen corresponding to the projection data, the invalid screen area included in the projection data will also be fully displayed on the display device, resulting in a waste of display space on the display device, affecting the user's viewing experience. Summary of the Invention

[0006] The present application provides a display device and a method for displaying a projected screen to solve the problem of low display space utilization when a terminal device projects a screen onto a display device.

[0007] According to some embodiments of the present application, a display device may include a display, a communication device, a memory and at least one processor. The display may be configured to display a user interface, and the user interface may include a projection screen of a terminal device; the communication device may be configured to establish a communication connection with the terminal device, and may receive projection data sent by the terminal device; the projection data may include a picture frame for forming the projection screen, and the picture frame may include a valid picture area and an invalid picture area; the memory may be configured to store a computer program; the at least one processor, connected to the display, the communication device and the memory, may be configured to execute the following computer program to enable the display device to execute: detecting the invalid picture area of ​​the picture frame; calculating the target display parameters of the projection screen based on the invalid picture area; the target display parameters may be used to set the center point of the valid picture area to the reference center point of the user interface, and to maximize the display proportion of the valid picture area in the user interface; and controlling the display to display the projection screen according to the target display parameters.

[0008] According to some embodiments of the present application, a screen projection display method for a display device is provided, and the display device may include a display, a communication device and at least one processor; wherein, the display may be configured to display a user interface, and the user interface may include a projection screen of the terminal device; the communication device may be configured to establish a communication connection with the terminal device, and receive projection data sent by the terminal device; the projection data may include a picture frame for forming the projection screen, and the picture frame may include a valid picture area and an invalid picture area; the method may include: detecting the invalid picture area of ​​the picture frame; calculating the target display parameters of the projection screen based on the invalid picture area; the target display parameters can be used to set the center point of the valid picture area to the reference center point of the user interface, and to maximize the display proportion of the valid picture area in the user interface; and controlling the display to display the projection screen according to the target display parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] FIG1 is a schematic diagram of a usage scenario of a display device according to some embodiments of the present application;

[0010] FIG2 is a hardware configuration block diagram of a control device according to some embodiments of the present application;

[0011] FIG3 is a hardware configuration block diagram of a display device according to some embodiments of the present application;

[0012] FIG4 is a schematic diagram of software configuration in a display device according to some embodiments of the present application;

[0013] FIG5 is a schematic diagram of mirror screen projection provided according to some embodiments of the present application;

[0014] FIG6 is a schematic diagram of push screen projection provided according to some embodiments of the present application;

[0015] FIG7 is a schematic diagram of functional modes of a display device according to some embodiments of the present application;

[0016] FIG8 is a schematic diagram of a screen projection data frame and a screen projection image during screen projection according to some embodiments of the present application;

[0017] FIG9a is a schematic diagram of a screen projection display effect provided according to some embodiments of the present application;

[0018] FIG9 b is a schematic diagram of another screen projection display effect provided according to some embodiments of the present application;

[0019] FIG9c is a schematic diagram of another screen projection display effect provided according to some embodiments of the present application;

[0020] FIG9 d is a schematic diagram of another screen projection display effect provided according to some embodiments of the present application;

[0021] FIG9e is a schematic diagram of another screen projection display effect provided according to some embodiments of the present application;

[0022] FIG10 is a schematic diagram showing an effect of optimizing a projection screen according to some embodiments of the present application;

[0023] FIG11 is a schematic diagram of a process for optimizing screen projection according to some embodiments of the present application;

[0024] FIG12 is a flowchart of a screen projection display method for a display device according to some embodiments of the present application;

[0025] FIG13 is a schematic diagram of a process for detecting valid image areas and invalid image areas according to some embodiments of the present application;

[0026] FIG14 is a schematic diagram of the margin of an invalid screen area according to some embodiments of the present application;

[0027] FIG15 is a second flow chart of a screen projection display method for a display device according to some embodiments of the present application;

[0028] FIG16 is a schematic diagram showing the display effect of an optimized projection screen according to some embodiments of the present application;

[0029] FIG17 is a third flow chart of a screen projection display method for a display device according to some embodiments of the present application;

[0030] FIG18 is a schematic diagram showing another effect of optimizing screen projection according to some embodiments of the present application;

[0031] FIG19 is a fourth flow chart of a screen projection display method for a display device according to some embodiments of the present application;

[0032] FIG20a is a schematic diagram showing another effect of optimizing screen projection according to some embodiments of the present application;

[0033] FIG20 b is a display principle diagram of the screen projection process shown in FIG20 a according to some embodiments of the present application;

[0034] FIG21a is a schematic diagram showing another effect of optimizing screen projection according to some embodiments of the present application;

[0035] FIG21 b is a display principle diagram of the screen projection process shown in FIG21 a according to some embodiments of the present application;

[0036] FIG22a is a schematic diagram showing another effect of optimizing screen projection provided according to some embodiments of the present application;

[0037] FIG22 b is a display principle diagram of the screen projection process shown in FIG22 a according to some embodiments of the present application;

[0038] FIG23a is a schematic diagram of an optimized screen projection display effect provided according to some embodiments of the present application;

[0039] FIG23 b is a schematic diagram of another optimized screen projection display effect provided according to some embodiments of the present application;

[0040] FIG23c is a schematic diagram of another optimized screen projection display effect provided according to some embodiments of the present application;

[0041] FIG23d is a schematic diagram of another optimized screen projection display effect provided according to some embodiments of the present application;

[0042] FIG23e is a schematic diagram of another optimized projection display effect provided according to some embodiments of the present application;

[0043] FIG24a is a schematic diagram of a picture frame in a subtitle scenario according to some embodiments of the present application;

[0044] FIG24 b is a schematic diagram of a picture frame in a bullet screen scenario according to some embodiments of the present application;

[0045] FIG25a is an example diagram of a screen projection effect corresponding to the picture frame shown in FIG24a according to some embodiments of the present application;

[0046] FIG25 b is an example diagram of a screen projection effect corresponding to the picture frame shown in FIG24 b according to some embodiments of the present application;

[0047] FIG26 is a schematic flow chart of another method for screen projection display on a display device according to some embodiments of the present application;

[0048] FIG27 is a schematic diagram showing another effect of optimizing screen projection according to some embodiments of the present application;

[0049] FIG28 is a schematic diagram showing another effect of optimizing screen projection according to some embodiments of the present application;

[0050] FIG29 is a schematic diagram showing another effect of optimizing screen projection according to some embodiments of the present application;

[0051] FIG30 a is a schematic diagram showing another effect of optimizing screen projection according to some embodiments of the present application;

[0052] FIG30 b is a schematic diagram illustrating the principle of calculating target display parameters during the screen projection process shown in FIG30 a according to some embodiments of the present application;

[0053] FIG31 is a schematic diagram showing another effect of optimizing screen projection according to some embodiments of the present application;

[0054] FIG32 is a flowchart diagram 1 of a third screen projection display method for a display device provided according to some embodiments of the present application;

[0055] FIG33 is an example diagram showing the effect of frequent switching of projection images according to some embodiments of the present application;

[0056] FIG34 is a schematic diagram of a process for calculating target display parameters according to some embodiments of the present application;

[0057] FIG35 is a schematic diagram showing another effect of optimizing screen projection according to some embodiments of the present application;

[0058] FIG36 is a schematic diagram illustrating a principle for calculating a center offset during the screen projection process shown in FIG35 according to some embodiments of the present application;

[0059] FIG37 is a schematic diagram showing another effect of optimizing screen projection according to some embodiments of the present application;

[0060] FIG38 is a second flow chart of a third method for screen projection display on a display device according to some embodiments of the present application;

[0061] FIG39 is a schematic diagram showing another effect of optimizing screen projection according to some embodiments of the present application;

[0062] FIG40 a is a diagram illustrating an effect of a screen mirroring process according to some embodiments of the present application;

[0063] FIG40 b is a diagram illustrating another effect of a mirror screen projection process according to some embodiments of the present application;

[0064] FIG41 is a schematic flow chart of a fourth screen projection display method for a display device according to some embodiments of the present application;

[0065] FIG42 a is a schematic diagram of a first picture frame set provided according to some embodiments of the present application;

[0066] FIG42 b is a schematic diagram of a second picture frame set provided according to some embodiments of the present application;

[0067] FIG43 is a schematic diagram of a third picture frame set provided according to some embodiments of the present application;

[0068] FIG44a is a schematic diagram showing the effect of screen projection in Example 1 provided according to some embodiments of the present application;

[0069] FIG44 b is a schematic diagram showing the effect of screen projection in Example 1 according to some embodiments of the present application;

[0070] FIG45 a is a schematic diagram showing the effect of screen projection in Example 2 provided according to some embodiments of the present application;

[0071] Figure 45b is a schematic diagram of the effect of the screen projection in Example 2 provided according to some embodiments of the present application. DETAILED DESCRIPTION

[0072] In order to make the purposes, schemes and advantages of the various embodiments shown in this application clearer, the schemes in some embodiments of this application will be clearly and completely described below in conjunction with the drawings in one or more embodiments of this application. Obviously, the one or more embodiments described are only part of the embodiments of this application, not all embodiments. Based on some embodiments shown in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. In addition, although the disclosure in this application is introduced according to one or several exemplary examples, it should be understood that each aspect of these disclosures can also constitute a complete solution separately.

[0073] It should be understood that the terms "first," "second," "third," etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, for example, they can be implemented in an order other than those given in the diagrams or descriptions of the embodiments of this application. In addition, the terms "including" and "having" and any variations thereof are intended to cover, but not exclude, inclusion. For example, a product or device comprising a series of components is not necessarily limited to those components explicitly listed, but may include other components that are not explicitly listed or are inherent to these products or devices.

[0074] The display device provided in the embodiments of the present application can have various implementation forms, for example, it can be a television, a laser projection device, a monitor, an electronic bulletin board, an electronic table, etc. Figures 1 and 3 are examples of a specific implementation of the display device of the present application.

[0075] Figure 1 is a schematic diagram of an operation scenario between a display device and a control device according to some embodiments of the present application. As shown in Figure 1, a user can operate a display device 200 through a smart device 300 or a control device 100.

[0076] In some embodiments, the control device 100 may be a remote control. Communication between the remote control and the display device may include infrared protocol communication, Bluetooth protocol communication, or other short-range communication methods, and the display device 200 may be controlled wirelessly or wired. The user may control the display device 200 by inputting user commands through buttons on the remote control, voice input, control panel input, etc.

[0077] In some embodiments, a smart device 300 (such as a mobile terminal, tablet computer, computer, laptop computer, etc.) can also be used to control the display device 200. For example, the display device 200 can be controlled using an application running on the smart device.

[0078] In some embodiments, the display device 200 may not use the aforementioned smart device or control device to receive instructions, but may receive user control through touch or gestures.

[0079] In some embodiments, the display device 200 can also be controlled in ways other than the control device 100 and the smart device 300. For example, the user's voice command control can be received through a module for obtaining voice commands configured inside the display device 200, or the user's voice command control can be received through a voice control device set outside the display device 200.

[0080] In some embodiments, the display device 200 also communicates data with the server 400. The display device 200 can be connected to a local area network (LAN), a wireless local area network (WLAN), or other networks. The server 400 can provide various content and interactions to the display device 200. The server 400 can be a single cluster or multiple clusters, and can include one or more types of servers.

[0081] Figure 2 is a block diagram of a control device according to some embodiments of the present application. As shown in Figure 2, the control device 100 may include at least one processor 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 may receive user input commands and convert them into commands that the display device 200 can recognize and respond to, acting as an intermediary for interaction between the user and the display device 200.

[0082] As shown in FIG3 , the display device 200 may include at least one of a tuner and demodulator 210 , a communication device 220 , a detector 230 , a device interface 240 , at least one processor 250 , a display 260 , an audio output interface 270 , a memory, a power supply, and a communication device interface 280 .

[0083] In some embodiments, the processor may include at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), a random access memory (RAM), a read-only memory (ROM), a first interface to an nth interface for input / output, a communication bus (Bus), etc.

[0084] In some embodiments, the display 260 may include a display screen component for presenting images, and a driving component for driving image display, a component for receiving image signals output from a controller, and a component for displaying video content, image content, and a menu control interface and a user control UI interface.

[0085] In some embodiments, the display 260 may be a liquid crystal display, an organic light-emitting diode (OLED) display, a projection display, or a projection device and a projection screen.

[0086] In some embodiments, the communication device 220 is a component for communicating with an external device or server 400 according to various communication protocol types. The display device 200 may be provided with multiple communication devices 220 depending on the supported communication methods. For example, if the display device 200 supports wireless network communication, the display device 200 may be provided with a communicator 220 including WiFi functionality. If the display device 200 supports Bluetooth connection communication, the display device 200 may be provided with a communicator 220 including Bluetooth functionality.

[0087] In some embodiments, the communication device 220 can establish a communication connection between the display device 200 and the external device or server 400 via a wireless or wired connection. A wired connection can connect the display device 200 to the external device via components such as a data cable or an interface. A wireless connection can connect the display device 200 to the external device via a wireless signal or a wireless network. The display device 200 can establish a connection with the external device directly or indirectly through a gateway, router, or connection device.

[0088] In some embodiments, the display device 200 can also support simultaneous communication connections with multiple external devices. Multiple external devices, such as terminal devices 500 such as mobile phones and tablet computers, can establish communication connections with the display device 200 by accessing the wireless local area network where the display device 200 resides. Multiple external devices can also connect to the display device 200 using different types of connection methods. For example, a first terminal device can connect to the display device 200 via a wireless local area network; a second terminal device can connect to the display device 200 via Bluetooth.

[0089] To achieve communication between the display device 200 and the terminal device 500, the display device 200 and the terminal device 500 are each equipped with a communication device that supports the same type of communication. For example, if the display device 200 is equipped with a communication device 220 including a WiFi module, the terminal device 500 should also be equipped with a communication device including a WiFi module. Furthermore, while the same type of communication device is provided, a specific communication transmission protocol is required for data transmission between the display device 200 and the terminal device 500. Examples include the WiFi protocol, Bluetooth connection protocol, ZigBee protocol, and NFC protocol.

[0090] In some embodiments, the communication device interface 280 may be used to receive a control signal from the control device 100 (eg, an infrared remote controller, etc.).

[0091] In some embodiments, detector 230 can be used to collect signals from the external environment or external interactions. For example, detector 230 can include a light receiver, a sensor for collecting ambient light intensity; or detector 230 can include an image collector, such as a camera, for collecting external environmental scenes, user attributes, or user interaction gestures; or detector 230 can include a sound collector, such as a microphone, for receiving external sounds.

[0092] In some embodiments, the device interface 240 may include, but is not limited to, any one or more of the following: a High Definition Multimedia Interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (Composite Video Broadcast Signal or Composite Video Blanking and Sync, CVBS), a Universal Serial Bus (USB) input interface (USB), an RGB port, or the like. Alternatively, the device interface 240 may be a composite input / output interface formed by multiple of the aforementioned interfaces.

[0093] In some embodiments, the tuner-demodulator 210 may receive broadcast television signals via wired or wireless reception, and demodulate audio and video signals, such as an Electronic Program Guide (EPG) data signal, from multiple wireless or wired broadcast television signals.

[0094] In some embodiments, at least one processor 250 and the tuner / demodulator 210 may be located in different separate devices, that is, the tuner / demodulator 210 may also be located in an external device of the main device where the at least one processor 250 is located, such as an external set-top box.

[0095] In some embodiments, at least one processor 250 can control the operation of the display device and respond to user operations through various software control programs (or computer programs) stored in the memory. The at least one processor 250 can control the overall operation of the display device 200. For example, in response to receiving a user command to select a UI object to be displayed on the display 260, the at least one processor 250 can perform operations related to the object selected by the user command.

[0096] A user may input a user command through a graphical user interface (GUI) displayed on display 260, and the user input interface receives the user input command through the GUI. Alternatively, a user may input a user command through a specific voice or gesture, and the user input interface receives the user input command by recognizing the voice or gesture through a sensor.

[0097] For ease of description, some embodiments of the present application are described using the display device 200 configured with the Android system as an example, but this is not intended to limit it. The display device 200 provided in the embodiments of the present application can also be configured with other systems.

[0098] As shown in Figure 4, in some embodiments, the system can be divided into four layers, from top to bottom, namely, the application layer (referred to as "application layer"), the application framework layer (referred to as "framework layer"), the Android runtime (Android runtime) and system library layer (referred to as "system runtime library layer"), and the kernel layer.

[0099] In some embodiments, the application layer may include at least one application. These applications may include a window program, a system settings program, a clock program, or other programs included with the operating system, or applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0100] In some embodiments, the framework layer can provide applications with an application programming interface (API) and programming framework. The application framework layer may include predefined functions. The application framework layer acts as a processing center, determining actions for applications in the application layer. Applications can access system resources and services during execution through the API.

[0101] As shown in Figure 4, in some embodiments of the present application, the application framework layer may also include managers, content providers, etc., where the manager may include at least one of the following modules: an activity manager (Activity Manager) can be used to interact with all activities running in the system; a location manager (Location Manager) can be used to provide system services or applications with access to system location services; a package manager (Package Manager) can be used to retrieve various information related to the application packages currently installed on the device; a notification manager (Notification Manager) can be used to control the display and clearing of notification messages; a window manager (Window Manager) can be used to manage icons, windows, toolbars, wallpapers and desktop widgets on the user interface.

[0102] In some embodiments, the activity manager can also be used to manage the lifecycle of each application and common navigation back functions, such as controlling the exit, opening, and backing of an application. The window manager can also be used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing screenshots, and controlling display window changes (for example, shrinking the display window, shaking the display, distorting the display, etc.).

[0103] In some embodiments, the system runtime layer can provide support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system can run the C / C++ library contained in the system runtime layer to implement the functions to be implemented by the framework layer.

[0104] In some embodiments, the kernel layer can be a layer between hardware and software. As shown in FIG4 , the kernel layer can include at least one of the following drivers: an audio driver, a display driver, a Bluetooth driver, a camera driver, a Wi-Fi driver, a USB driver, an HDMI driver, a sensor driver (such as a fingerprint sensor, a temperature sensor, a pressure sensor, etc.), and a power driver.

[0105] After establishing a communication connection with the terminal device 500, the display device 200 can establish a screen projection connection channel and transmit projection data based on the screen projection connection, so that the display device 200 can display the projection image of the terminal device 500. In some embodiments, to establish a screen projection connection relationship, the user can issue a screen projection connection request through the terminal device 500, and the screen projection connection request can be sent to the display device 200 via the WiFi network. The display device 200 can then complete the transmission protocol configuration based on the screen projection connection request, thereby establishing a transmission channel for screen projection data between the terminal device 500 and the terminal device 500.

[0106] The display device 200 can also achieve a screen projection connection with the terminal device 500 through different connection methods. For example, when the display device 200 and the terminal device 500 are connected to the same wireless local area network, a screen projection connection can be established based on the WiFi network. For another example, when both the display device 200 and the terminal device 500 are provided with a Near Field Communication (NFC) component, a screen projection connection relationship can be established through the NFC component. Obviously, other wired or wireless connection methods can also be used between the display device 200 and the terminal device 500 to establish a screen projection connection relationship, such as radio frequency (RF) connection, infrared connection, cellular network and other connection methods that can be associated with those skilled in the art based on the connection methods provided in the above embodiments.

[0107] After establishing the screen projection connection, the display device 200 can receive the screen projection related data from the terminal device 500 through the screen projection data channel, thereby playing the screen projection related content on the display device 200. Under different screen projection data transmission protocols, the screen projection data obtained by the display device 200 is in different forms. For example, the display device 200 and the terminal device 500 can establish a screen projection connection through the Digital Living Network Alliance (DLNA) protocol, Miracast protocol, AirPlay push protocol, WiDi screen projection, NFC mirror screen projection, LeTV screen projection, and other protocols.

[0108] Among them, the DLNA protocol is a protocol for sharing media resources between devices. This protocol can use network technology (wired or wireless) to interconnect various devices and exchange data through standard protocols, allowing media resources to be shared between devices. The DLNA protocol is based on the UPnP protocol, allowing devices to discover and control each other, achieving seamless transmission and sharing of media resources. The DLNA protocol supports a variety of devices, including TVs, speakers, mobile phones, computers, etc., allowing users to seamlessly switch and share media content between different devices.

[0109] The Miracast protocol is a wireless display standard developed by the Wi-Fi Alliance and based on Wi-Fi Direct. The Miracast screen projection protocol is compatible with Android devices, allowing Android devices to share video images through Miracast. Smart devices running Android 5.0 and higher have the Miracast protocol built-in, allowing direct wireless screen projection without installing any software, using the Wi-Fi P2P function between the projection device and the receiving device's Wi-Fi network card. Miracast also supports simultaneous audio and video transmission.

[0110] AirPlay is a wireless playback technology introduced by Apple, suitable for wireless screen projection on iOS and Mac systems. Apple devices have built-in AirPlay projection functionality. When used with a screen projector that supports the AirPlay server service, the display is clear and stable. The protocol also includes a built-in projection code function. However, the Apple device and screen projector must be on the same local area network (LAN segment) and cannot be used across different network segments or VLANs.

[0111] WiDi is an Intel-developed wireless streaming technology based on Wi-Fi Direct. Windows 8 / 10 / 11 laptops natively support WiDi, allowing you to cast your screen directly without installing any software. It also supports USB reverse control.

[0112] In some embodiments, after the display device 200 establishes a screen projection connection with the terminal device 500, the projection data can be transmitted through mirror mode and push mode. Among them, the mirror mode means that the display device 200 synchronously displays the screen content displayed by the terminal device 500. For example, as shown in Figure 5, the display device 200 displays the main interface of the terminal device 500, and when the user performs an interactive operation on the terminal device 500, causing the content of the main interface to change, the content displayed by the display device 200 also changes. To this end, the terminal device 500 can generate projection data in the form of a video stream based on the displayed content, and can send the projection data to the display device 200 for display.

[0113] Push mode means that the display device 200 accesses the network link pushed by the terminal device 500 and obtains the corresponding projection screen. For example, as shown in Figure 6, when the terminal device 500 is playing video media, a projection control is provided in the upper right corner of the playback interface. After the user clicks the projection control and selects the display device 200 as the device to display the projection screen, the terminal device 500 can send the Uniform Resource Locator (URL) address of the currently playing video media to the display device 200. After receiving the URL address, the display device 200 can access the URL address to obtain the video media data and play the video media.

[0114] Among the screen projection protocols supported by the display device 200 and the terminal device 500, some protocols are based on the mirror mode, and some protocols are based on the push mode. For example, under the DLNA protocol, the display device 200 can obtain the link address of a resource in the terminal device 500 through the screen projection data channel, and obtain the media content data by accessing the link address, thereby playing the media data. The display device 200 can also obtain control instructions sent by the terminal device 500 through the screen projection data channel when playing media data, such as fast forward, pause, stop screen projection, etc. For example, after establishing a connection through the communication protocol, the terminal device 500 such as a mobile phone and the display device 200 will negotiate the device functions and network conditions to select a suitable audio and video transmission format. After establishing the online protocol (Session) of the audio and video streaming, a series of real-time streaming protocol (RTSP) control commands can be used to control the playback and termination of the audio and video streaming.

[0115] Under other screen projection connection protocols such as Miracast, Airplay, NFC mirroring, and LeTV, the display device 200 can obtain a video stream, which serves as projection data. The video stream can be the video data corresponding to the final display screen of the display unit of the terminal device 500, that is, the video stream may include the media content screen, UI interface, and other video screen content that can be displayed on the display unit of the terminal device 500. The video stream can also be only the video data that can be obtained from the terminal device 500, such as the media video data being played. In this case, the terminal device 500 can serve as a signal source for the display device 200, and the display device 200 can continuously obtain video data through the projection data channel and display it in real time on the display 260, achieving a synchronized display effect with the terminal device 500. Alternatively, the video stream can also be video data obtained from the cloud, that is, the terminal device 500 can send the URL address of the video media to the display device 200. After receiving the URL address, the display device 200 accesses the URL address based on the projection protocol to obtain the projection data.

[0116] It should be noted that the above-mentioned screen projection protocol can not only obtain the video stream in the terminal device 500, but also obtain the audio data stream in the terminal device 500, so that the video data and audio data can be sent to the display device 200 synchronously, so that the display device 200 can play the video and audio synchronously.

[0117] The terminal device 500 can obtain different projection data according to different projection protocols. For example, for system applications or direct projection at the system level, the terminal device 500 can use Miracast projection, Airplay projection or WiDi projection protocol to establish a projection data channel. Since the above-mentioned Miracast projection protocol and Airplay projection protocol require system-level application permissions, the Miracast projection protocol and Airplay projection protocol are only applicable to system applications, or the display device 200 and the terminal device 500 of the same manufacturer directly establish a projection connection through the built-in rules of the operating system.

[0118] The display device 200 can establish a screen projection connection with the terminal device 500 based on any of the screen projection methods provided in the above embodiments. In some embodiments, the display device 200 can also set the layout of the screen projection window. To this end, as shown in Figure 7, the display device 200 can have multiple functional modules built into the same application, such as a screen projection application. The screen projection application may include: part or all of: a data management module, a message processing module, a service management module, an algorithm processing module, a View management module, an application interaction processing module, a location management module, and a screen projection management module.

[0119] In some embodiments, the data management module manages all message data involved in the current template and can synchronize data updates. The message processing module can transmit related messages between different modules, such as transmitting data changes to the View layer for refresh display, or synchronizing View changes to the data management module for broadcast control data change processing. The service management module is responsible for enabling and disabling services across the entire application, as well as some customized processing, such as launching specific services in specific scenarios.

[0120] In some embodiments, the algorithm processing module may be responsible for all algorithm processing of the entire application. The View management module may be responsible for all UI displays of the entire template application, and receives data to refresh the View display. The application interaction processing module may be responsible for processing interactions with external applications. For example, any entry to start the current template application needs to interact with this module, and different customized management can be performed based on the differences in the startup entries. The screen projection management module may be responsible for screen projection protocol interaction, client list management, and status information processing of window display content.

[0121] It should be noted that the screen projection application, as an application that can display multi-protocol multi-channel mirror projection and push content, is an upper-layer application within the display device 200. The framework layer of the display device 200 provides a Java interface for docking with hardware, etc. The multi-channel screen projection viewing application can achieve docking with lower layers and obtain information through the Java interface of the Framework layer. Other layers such as the National, Runtime, HAL, and Linux kernel are all standard layers of the Android architecture, and each layer interacts according to the business process.

[0122] In some embodiments, the display device 200 first starts the application through the service management module and performs some initialization preparations, such as starting the monitoring mirror or pushing the corresponding protocol service; the View management module initializes the View displayed by the application layer; the data management module creates some data for the data processing module; the algorithm processing module can perform algorithmic layout calculations on the projection content, etc.

[0123] In some embodiments, the service management module can start the screen projection application and register the Session during the process of establishing the screen projection connection; Usession Connection Listener, Usession Prepare Listener, USession Display Listener and Usession Event Listener. Among them, Session represents a conversation between the client and the server, which can save the information of the Client. Usession Connection Listener can be used to monitor the connection and disconnection events of the Session. Usession Prepare Listener interacts with the screen projection preparation of the Client. Usession Display Listener can monitor the changes in playback information and status and the status changes of the Client. Usession Event Listener processes information such as Client control.

[0124] After the View is initialized, the display device 200 can display a default boot page when entering the application. The boot page can display some device information, such as the device name of the current display device 200 and the corresponding network information. The boot page is then displayed by controlling the system's standard controls TextView and ImageView and their corresponding Layout.

[0125] To facilitate the management of View and Session, in some embodiments, a management object MultiCastPlayView can be set, where the management object MultiCastPlayView can be associated with information such as Session, SurfaceView / TextureView, and mViewPosition. For example, for terminal device A, the data corresponding to its Client can be represented as: Session:SessionClient, View:SurfaceView / TextureView.

[0126] When the display device 200 prepares the View as the server, it can notify the terminal device 500 as the mobile terminal to record or send the video stream (i.e., the projection data) through UsesionPrepareListener. When the terminal device 500 is ready to send the video information, the display device 200 will know that the projection is to be played through onFirstFrame() of UsesionDisplayListener. Therefore, it can obtain the current Client video width and height from the Session, and determine the direction of the current video stream through the width and height. Then, the algorithm can be used to calculate the current position and size to be displayed, i.e., the original position and size.

[0127] In some embodiments, the display device 200 may obtain a screen projection session (Session) sent by the terminal device 500. The screen projection session may include the resolution of the picture frame. The screen resolution of the display device may be detected, and the screen display area of ​​the projection screen may be calculated based on the resolution of the picture frame and the screen resolution. Then, initial display parameters are generated based on the screen display area, and the display 260 is controlled to display the projection screen according to the initial display parameters.

[0128] For example, when the resolution of the terminal device 500 is 1920×1080 and the screen resolution of the display device 200 is 1920×1080, the projection data that the terminal device 500 can send to the display device 200 is shown in the left part of Figure 8. The algorithm processing module obtains the screen resolution of the screen frame in the projection data from the projection session (Session) as 1920×1080. Then, based on the screen resolution of 1920×1080 and the screen resolution of 1920×1080, it is calculated that the screen display area of ​​the projection screen is the full screen area (i.e., the entire user interface), that is, the original position and size of the projection screen are: width 1920, height 1080, and distance from the upper left corner position (0, 0). The data management module of the display device 200 can sort and store the data obtained by the algorithm processing module, and then transmit the data, which may include initial display parameters: Session: SessionClient, View: SurfaceView / TextureView, Position: (0, 0), Width: 1920, Height: 1080.

[0129] It should be noted that the data associated with the management objects and the parameters of the algorithm processing provided in the above embodiment are only examples and are not limitations thereto. Other data objects may also be used in some embodiments of the present application.

[0130] After receiving the data information, the View management module of the display device 200 sets the display parameters through LayoutParams, which may include the position (i.e., coordinates) from the upper and left boundaries, as well as the corresponding display width and height, and then displays the position through the setLayoutParams method. That is:

[0131] FrameLayout.LayoutParams lp1=new FrameLayout.LayoutParams(width,height,0); lp1.width=1920; lp1.height=1080; lp1.leftMargin=0; lp1.topMargin=0; mPlayerView.setLayoutParams(lp1).

[0132] By setting the display position of the projection SurfaceView / TextureView, the display device 200 can display the projection screen as shown in the right part of Figure 8.

[0133] In some embodiments, the projection data sent by the terminal device 500 to the display device 200 may include a picture frame for forming a projection screen, and the picture frame may include a valid picture area and an invalid picture area. Among them, from a visual perspective, the valid picture area can be used to present picture content for users to watch. That is, the valid picture area can be an area in the picture frame with picture content for users to watch, and the invalid picture area can be an area in the picture frame other than the valid picture area. For example, the invalid picture area can be located on the periphery of the valid picture area and has no substantial effect. In other words, the invalid picture area can be an area in the picture frame that has no practical meaning and is an area that the user does not pay attention to. In some embodiments, the invalid picture area will present a single color effect, such as the invalid picture area can be a solid color area such as black or gray.

[0134] In some embodiments, the display device 200 can scan all pixels in the picture frame, which can include row-by-row scanning and column-by-column scanning, such as first scanning each pixel of the picture frame row by row and then scanning each pixel of the picture frame column by column, or first scanning each pixel of the picture frame column by column and then scanning each pixel of the picture frame row by row. Of course, the above scanning order is not unique. Then, the area where the color values ​​of the entire row and array pixels are all the target values ​​is marked as an invalid picture area. The color value can be a characteristic value that represents the color of the pixel, such as an RGB value.

[0135] For example, a pixel with an RGB value of (0, 0, 0) is a pixel with no display content. The area of ​​the image frame consisting of pixels with an RGB value of (0, 0, 0) is an invalid image area, such as area a in Figure 8. The area consisting of pixels with other color values ​​is a valid image area, such as area b in Figure 8. Since the invalid image area has no RGB values ​​set for the pixels, it may appear as a solid color such as black or gray.

[0136] In some embodiments, when the target value is the color value corresponding to black, the pixel points of the entire row of the picture frame whose color value is the target value can be marked as an invalid picture area, and the pixel points of the entire column of the target value can be marked as an invalid picture area. At this time, the display effect presented by the invalid picture area is black, and the invalid picture area can also be referred to as a black edge or a black edge area. Similarly, when the target value is the color value corresponding to gray, the display effect presented by the invalid picture area is gray, and the invalid picture area can also be referred to as a gray edge or a gray edge area. It is understandable that the principles for other colors are the same as above and will not be elaborated here.

[0137] For example, as shown in Figure 9a, when the terminal device 500 is in a vertical screen state, the terminal device 500 displays the terminal interface 910; at this time, the terminal device 500 starts recording and sends the projection data to the display device 200. The picture frame 920 in the projection data includes a valid picture area 921 and an invalid picture area 922. The display device 200 displays the projection screen 930 as shown in Figure 9a based on the projection data.

[0138] As shown in Figure 9b, when the terminal device 500 is in landscape mode, there is an invalid screen area in the media played by the terminal device 500, and the terminal device 500 displays the terminal interface 910; at this time, the terminal device 500 starts recording and sends projection data to the display device 200. The screen frame 920 in the projection data includes an invalid screen area 922 and a valid screen area 921. The display device 200 displays the projection screen 930 shown in Figure 9b based on the projection data.

[0139] As shown in Figure 9c, when the terminal device 500 switches from the horizontal screen state of Figure 9b to the vertical screen state, the media played by the terminal device 500 still has an invalid screen area 922, and the terminal device 500 displays the terminal interface 910; at this time, the terminal device 500 starts recording and sends the projection data to the display device 200. The picture frame 920 in the projection data includes an invalid screen area 922 and a valid screen area 921. The display device 200 displays the projection screen 930 shown in Figure 9c based on the projection data.

[0140] As shown in Figure 9d, when the terminal device 500 is in landscape mode, the user adjusts the playback window of the terminal device 500 to the size of area d, and the terminal device 500 displays the terminal interface 910; at this time, the terminal device 500 starts recording and sends projection data to the display device 200. The picture frame 920 in the projection data includes an invalid picture area 922 and a valid picture area 921. The display device 200 displays the projection screen 930 as shown in Figure 9d based on the projection data.

[0141] As shown in Figure 9e, when the terminal device 500 is in portrait mode, the user adjusts the playback window of the terminal device 500 to the size of area e, and the terminal device 500 displays the terminal interface 910; at this time, the terminal device 500 starts recording and sends projection data to the display device 200. The picture frame 920 in the projection data includes an invalid picture area 922 and a valid picture area 921. The display device 200 displays the projection screen 930 as shown in Figure 9e based on the projection data.

[0142] Obviously, from the screen projection process shown in Figures 9a-9e, it can be seen that when the picture frame of the screen projection data sent by the terminal device 500 includes an invalid picture area, the display device 200 will also display the invalid picture area completely in the user interface, and the invalid picture area will occupy a certain display space of the display device 200.

[0143] As shown in FIG11 , in some embodiments, the display device 200 may detect an invalid screen area of ​​a picture frame ( S1201 ) and, based on the width and height of the invalid screen area, calculate a pre-processed width of the picture frame in a first direction and a pre-processed height of the picture frame in a second direction ( S1101 ). The pre-processed width may be the width of the valid screen area 921 , and the pre-processed height may be the height of the valid screen area 922 . The first direction is perpendicular to the second direction, such as the x-direction and the y-direction. The pre-processed width may be the difference between the picture frame width and the width of the invalid screen area in the first direction, and the pre-processed height may be the difference between the picture frame height and the height of the invalid screen area in the second direction. A first ratio of the picture frame width to the pre-processed width and a second ratio of the picture frame height to the pre-processed height may be calculated ( S1102 ). The first ratio may be set as the scaling ratio of the projected image in the first direction, and the second ratio may be set as the scaling ratio of the projected image in the second direction ( S1103 ). Finally, the display device 200 may scale the width of the projected image according to the first ratio and the height of the projected image according to the second ratio ( S1104 ). That is, the scaling ratios of the first direction and the second direction can be calculated separately, and the projection image can be scaled according to the scaling ratios.

[0144] For example, the resolution of terminal device 500 is 1920×1080, and the screen resolution of display device 200 is 1920×1080. As shown in FIG10 , the first direction is the x-direction, and the second direction is the y-direction. Terminal device 500 records its own terminal interface based on the screen projection protocol, generates projection data, and transmits the projection data to display device 200. The projection data frame 920 may include an invalid screen area 922 and an invalid screen area 922 . As shown in FIG10 , the size of invalid screen area 922 may be (128, 72). Therefore, when calculating the preprocessing height, the algorithm processing module of display device 200 may subtract 72 pixels from each side in the y-direction, resulting in a preprocessing height of 1080-2*72=936. Similarly, when calculating the preprocessing width, the display device 200 may subtract 128 pixels from each side in the x-direction, resulting in a preprocessing width of 1920-2*128=1664.

[0145] After the display device 200 calculates the preprocessed width and preprocessed height, it can calculate the ratio of the width and height of the original size of the picture frame to the preprocessed height and preprocessed width. The first ratio is 1920 / 1664, and the second ratio is 1080 / 936. That is, scaleX = (float) 1920 / (float)(1920-2*128) = 1.15f, scaleY = (float) 1080 / (float)(1080-2*72) = 1.15f. Then, the display device 200 can set the position of the View through the View management module. ScaleX can be set to the scaling ratio of the projection screen (View: SurfaceView / TextureView) in the x direction, and scaleY can be set to the scaling ratio of the projection screen in the y direction. In this way, when the display device 200 displays the picture frame of the projection data, the projection screen 930 shown on the right side of Figure 10 is presented, which can reduce the invalid screen area in the user interface of the display device 200.

[0146] However, it can be seen from the screen projection process shown in Figures 9a-9e that the position and size of the invalid screen area 922 in the picture frame 920 are uncertain. When the invalid screen area 922 is unevenly distributed or the invalid screen area 922 is asymmetric in the x-direction and y-direction, the projection screen 930 is scaled by subtracting the invalid screen areas in the x-direction and y-direction through the algorithm processing module. The projection screen 930 may appear to be offset in the display device 200.

[0147] Based on the above application scenarios, in some embodiments, as shown in FIG12 , the display device 200 may include a display 260, a communication device 220, a memory, and at least one processor 250. The display 260 may be configured to display a user interface 261, which may include a projection screen of the terminal device 500; the communication device 200 may be configured to establish a communication connection with the terminal device 500, and receive projection data sent by the terminal device 500, wherein the projection data may include a picture frame 920 for forming a projection screen, and the picture frame 920 may include a valid picture area 921 and an invalid picture area 922; the memory may be configured to store a computer program, and the at least one processor 250 may be configured to execute the computer program to enable the display device to perform steps including but not limited to the following:

[0148] S1201: Detecting an invalid picture area of ​​the picture frame.

[0149] The display device 200 can establish a screen projection connection with the terminal device 500 via the communication device 220, receive the projection data sent by the terminal device 500, and control the display 260 to display the screen frames of the projection data based on the projection window displayed in full screen. During the projection process, the display device 200 can also detect each screen frame 920 of the projection data and detect the invalid screen area 922 of the screen frame 920 in real time.

[0150] In some embodiments, as shown in FIG13 , the display device 200 can determine the invalid screen area 922 in the screen frame 920 by detecting the color values ​​(such as RGB values) of each screen frame 920 of the projection data. That is, the display device 200 can read the screen frame of the projection data (S1301) and detect the color values ​​of the screen frame, that is, traverse the color values ​​of each pixel in the screen frame 920 (S1302). Determine whether the color value of each pixel is equal to the target value (S1303). The pixel whose color value is the target value can be marked as the pixel of the invalid screen area to obtain the invalid screen area of ​​the screen frame 920 (S1304). The target value represents the color value of the pixel without display content. For example, taking the color value as the RGB value as an example, the target value can be (0, 0, 0). In other embodiments, the pixel of other color values ​​can be marked as the pixel of the valid screen area 921 to obtain the valid screen area 921 of the screen frame 920 (S1305).

[0151] S1202: Calculate the target display parameters of the projection screen based on the invalid screen area; the target display parameters are used to set the center point of the valid screen area to the reference center point of the user interface, and to maximize the display proportion of the valid screen area in the user interface.

[0152] In some embodiments, after detecting the invalid screen area of ​​the picture frame 920, the display device 200 can calculate the target display parameters of the projection screen according to the size of the invalid screen area 922. Among them, the display parameters can be parameters for setting the projection screen, such as the layout set by the View management module; the target display parameters of some embodiments of the present application can be used to set the effective screen area 921 of the picture frame 920 to the reference center point of the user interface, and to maximize the display proportion of the effective screen area 921 in the user interface. That is, the projection screen displayed according to the target display parameters can minimize the invalid screen area 922 in the user interface, and make the center of the effective screen area 921 be located at the center position of the user interface (center point of the screen), so that the effective screen area 921 can be displayed in the center. When the display device 200 displays the projection screen, the display parameters of the projection screen can be calculated according to the above principles to adjust the display effect of the projection screen.

[0153] For example, when the invalid screen area 922 is a black-border area, the projection screen displayed by the display device 200 according to the target display parameters can minimize the black-border area in the user interface, so that the effective screen area 921 can be displayed in the user interface to the maximum extent, while keeping the effective screen area 921 centered in the user interface.

[0154] S1203: Control the display to display the projection image according to the target display parameters.

[0155] In some embodiments, after the display device 200 calculates the target display parameters according to the above principles, it can control the display 260 to display the projection screen according to the target display parameters, thereby minimizing the invalid screen area of ​​the user interface while allowing the effective screen area 921 to be displayed symmetrically based on the center position of the user interface. For example, after the display device 200 calculates the target display parameters through the algorithm processing module of the projection application, it can store the target display parameters through the data management module and transmit the target display parameters to the View management module. The View management module can then set the position of the projection screen based on the target display parameters, so that the display effect corresponding to the target display parameters is presented in the user interface of the display 260.

[0156] In some embodiments, when calculating the target display parameters, the display device 200 can use the reference center point as the center of symmetry to compare the invalid screen areas in the first direction and the second direction to determine whether the invalid screen area 922 is symmetrical. When determining whether the invalid screen area 922 is symmetrical, the display device 200 can use an image processing algorithm to determine whether the invalid screen area 922 is centrally symmetrical based on the reference center point; or, the display device 200 can also use a geometric symmetry detection algorithm to determine whether the invalid screen area 922 is centrally symmetrical based on the reference center point; or, the display device 200 can also obtain the margins of the invalid screen area 922 from the upper, lower, left, and right boundaries of the user interface (or projection window) to determine whether the invalid screen area 922 is centrally symmetrical based on the reference center point. For ease of description, in some embodiments of the present application, the margins of the invalid screen area 922 from the upper, lower, left, and right boundaries of the user interface are represented as top, bottom, left, and right, respectively.

[0157] It is understood that some embodiments of the present application list two methods for determining whether the invalid screen area is symmetrical, but this is not intended to be limiting. The method for determining whether the invalid screen area 922 is symmetrical can also be implemented based on other methods, such as using a third-party algorithm library or service framework.

[0158] In some embodiments, as shown in Figure 14, the display device 200 can detect the first invalid screen area 922a and the second invalid screen area 922b in the first direction, and detect the third invalid screen area 922c and the fourth invalid screen area 922d in the second direction. Then calculate the first margin (left) of the first invalid screen area 922a in the first direction, the second margin (right) of the second invalid screen area 922b in the first direction, the third margin (top) of the third invalid screen area 922c in the second direction, and the fourth margin (bottom) of the fourth invalid screen area in the second direction. That is, the margin (first margin) corresponding to the first invalid screen area 922a is recorded as left, the margin (second margin) corresponding to the second invalid screen area 922b is recorded as right, the margin (third margin) corresponding to the third invalid screen area 922c is recorded as top, and the margin (fourth margin) corresponding to the fourth invalid screen area 922d is recorded as bottom.

[0159] After calculating the margins of the invalid screen area 922 in each direction, if the first margin is equal to the second margin, and the third margin is equal to the fourth margin, the invalid screen area 922 can be marked as being centrally symmetrical about the reference center point; conversely, if the first margin is not equal to the second margin, and / or the third margin is not equal to the fourth margin, the invalid screen area 922 can be marked as not being centrally symmetrical about the reference center point. That is, if the two margins in the x-direction (left and right) are not equal; or the two margins in the y-direction (top and bottom) are not equal; or the two margins in the x-direction are not equal and the two margins in the y-direction are also not equal, the display device 200 can mark the invalid screen area 922 as not being centrally symmetrical about the reference center point.

[0160] It should be noted that, in some embodiments of the present application, the picture frame may not have one or more of the first invalid picture area, the second invalid picture area, the third invalid picture area and the fourth invalid picture area, that is, the value of any one of top, bottom, left and right can be 0.

[0161] In some embodiments, as shown in FIG15 , if the invalid picture area 922 is centrally symmetrical about the reference center point, the display device 200 may calculate a pre-processed width of the picture frame 920 in a first direction and a pre-processed height of the picture frame in a second direction ( S1401 ). A first ratio of the picture frame width to the pre-processed width and a second ratio of the picture frame height to the pre-processed height may then be calculated ( S1402 ). A determination is then made as to whether the first ratio and the second ratio are equal ( S1403 ).

[0162] In some embodiments, when the display device 200 calculates that the first ratio is equal to the second ratio, one of the ratios can be set as the scaling ratio of the projection screen in the first direction and the second direction. That is, when the first ratio is equal to the second ratio, the display device 200 can set the first ratio as the scaling ratio of the projection screen in the first direction and the second direction, or the second ratio can be set as the scaling ratio of the projection screen in the first direction and the second direction to generate the target display parameters (S1404). The display device 200 can display the projection screen according to the target display parameters (S1407). At this time, the projection display effect of the display device 200 is shown in Figure 10.

[0163] In other embodiments, when the first ratio and the second ratio are not equal, in order to ensure that the projected image can be displayed according to the original ratio of the effective image area 921 after scaling, the display device 200 can obtain the minimum value of the first ratio and the second ratio (S1405), and then set the minimum value as the scaling ratio of the projected image in the first direction, and set the minimum value as the scaling ratio of the projected image in the second direction to generate the target display parameters (S1406). The display device 200 can display the projected image according to the target display parameters (S1407). At this time, the projection display effect of the display device 200 is shown in Figure 18.

[0164] The following embodiments are directed to several scenarios of areas without content display (invalid screen areas), and are described by taking the first direction as the x direction and the second direction as the y direction as an example.

[0165] Scenario 1: The invalid image area is centrally symmetrical about the reference center point, and the first ratio is equal to the second ratio.

[0166] In some embodiments, if the invalid screen area 922 is centrally symmetrical about the reference center point, the display device 200 may calculate the pre-processed width of the screen frame 920 in the first direction and the pre-processed height of the screen frame 920 in the second direction. Then, the first ratio of the screen frame width to the pre-processed width, and the second ratio of the screen frame height to the pre-processed height may be calculated. Since the first ratio is equal to the second ratio at this time, the display device 200 may set one of the ratios as the scaling ratio of the projection screen. That is, when the first ratio is equal to the second ratio, the display device 200 may set the first ratio to the scaling ratio of the projection screen in the first direction to the second direction, or set the second ratio to the scaling ratio of the projection screen in the first direction to the second direction to generate the target display parameters. At this time, the display device 200 may display the projection screen 930 as shown in Figure 16.

[0167] In some embodiments, when the first ratio is set as the scaling ratio of the projected image 930 in the first direction to the second direction, the display device 200 may scale the width of the projected image 930 according to the first ratio, and scale the height of the projected image 930 according to the first ratio. Similarly, when the second ratio is set as the scaling ratio of the projected image 930 in the first direction to the second direction, the display device 200 may scale the width of the projected image 930 according to the second ratio, and scale the height of the projected image 930 according to the second ratio.

[0168] For example, the resolution of terminal device 500 is 1920×1080, and the screen resolution of display device 200 is 1920×1080. As shown in FIG10 , display device 200 calculates that the invalid screen area is centrosymmetrical about the reference center point, and the size of the invalid screen area is (128, 72). Display device 200 can then calculate the preprocessing height as 1080-2*72=936 and the preprocessing width as 1920-2*128=1664. Then, the first and second scales are calculated, namely, scaleX=(float)1920 / (float)(1920-2*128)=1.15f and scaleY=(float)1080 / (float)(1080-2*72)=1.15f. Since the first and second scales are equal, the View position can be set through the View management module, and scaleX or scaleY can be set to the scale ratio of the projection screen (View: SurfaceView / TextureView) in the x-direction and y-direction. In this way, when the display device 200 displays the screen frame of the projection data, it presents the projection screen 930 shown on the right side of Figure 10, which can reduce the invalid screen area in the user interface of the display device 200.

[0169] Scenario 2: The invalid image area is centrally symmetrical about the reference center point, and the first and second ratios are not equal.

[0170] As shown in FIG17 , in some embodiments, if the invalid screen area 922 is centrally symmetrical about the reference center point, the display device 200 may calculate a pre-processed width of the screen frame in a first direction and a pre-processed height of the screen frame in a second direction ( S1501 ) after detecting the invalid screen area of ​​the screen frame 920 ( S1201 ). Then, a first ratio of the screen frame width to the pre-processed width and a second ratio of the screen frame height to the pre-processed height may be calculated ( S1502 ).

[0171] At this time, due to the overall uneven distribution of the invalid screen area in the x-direction and the y-direction, the values ​​of the first ratio and the second ratio are not equal. In order to ensure that the projected screen can be displayed according to the original ratio of the valid screen area 921 after scaling, when the first ratio and the second ratio are not equal, the display device 200 can obtain the minimum value of the first ratio and the second ratio (S1503), set the minimum value as the scaling ratio of the projected screen in the first direction, and set the minimum value as the scaling ratio of the projected screen in the second direction to generate the target display parameters (S1504).

[0172] In some embodiments, if the invalid screen area 922 is centrally symmetrical about the reference center point, but the first ratio is not equal to the second ratio, when the display device 200 displays the projected screen according to the target display parameters, the width of the projected screen 930 can be scaled according to the above minimum value, and the height of the projected screen 930 can be scaled according to the above minimum value (S1505).

[0173] For example, the resolution of terminal device 500 is 1920×1080, and the screen resolution of display device 200 is 1920×1080. As shown in FIG18 , display device 200 calculates that the invalid screen area is centrosymmetrical about the reference center point, with the left and right sides of the invalid screen area being 100, and the top and bottom sides being 50. Its size can be recorded as (100, 50). Display device 200 can then calculate the preprocessing height as 1080-2*50=980, and the preprocessing width as 1920-2*100=1720. Then, the first scale in the x-direction and the second scale in the y-direction are calculated, namely, scaleX=(float)1920 / (float)(1920-2*100)=1.12f, and scaleY=(float)1080 / (float)(1080-2*50)=1.10f. Since the first ratio is not equal to the second ratio, the display device 200 can obtain the minimum value of the first ratio and the second ratio, that is, 1.10f. The position of the View can then be set according to the ratio of 1.10f through the View management module; that is, the projection screen can be scaled through View.animate().scaleX(scaleY).scaleY(scaleY). In this way, when the display device 200 displays the screen frame of the projection data, it presents the projection screen 930 shown on the right side of Figure 18, which can reduce the invalid screen area in the user interface of the display device 200.

[0174] Scenario 3: The invalid image area is not centrally symmetrical about the reference center point

[0175] As shown in FIG19 , in some embodiments, if the invalid screen area 922 is not centrally symmetrical about the reference center point, that is, the two invalid screen areas in the x direction are asymmetrical with each other, or the two invalid screen areas in the y direction are asymmetrical with each other, or the two invalid screen areas in the x direction are asymmetrical and the two invalid screen areas in the y direction are also asymmetrical, the display device 200 can calculate the pre-processed width of the screen frame 920 in the first direction and the pre-processed height in the second direction (S1701) after detecting the invalid screen area of ​​the screen frame 920 (S1201), and then calculate the target center point of the screen frame 920 under the pre-processed width and pre-processed height (S1702), that is, the center point of the valid screen area. Then, the center offset can be calculated based on the target center point and the reference center point (S1703), and the position parameters of the projection screen can be set based on the center offset (S1704). The first ratio of the screen frame width to the pre-processed width and the second ratio of the screen frame height to the pre-processed height can be calculated (S1705). In order to display the projected screen according to the original ratio of the effective screen area 921, the minimum value of the first ratio and the second ratio can be obtained (S1706), and the minimum value can be set as the scaling ratio of the projected screen in the first direction, and the minimum value can be set as the scaling ratio of the projected screen in the second direction to generate the target display parameters (S1707).

[0176] In some embodiments, if the invalid screen area 922 is not centrally symmetrical about the reference center point, the display device 200 can move the position of the projection screen in the user interface according to the position parameters (S1708), and then scale the width of the projection screen according to the minimum value, and scale the height of the projection screen according to the minimum value (S1709).

[0177] Since the invalid image area is not centrally symmetrical about the reference center point, there may be various situations. The following uses three examples to illustrate the three situations of scenario three.

[0178] Example 1: The two invalid screen areas in the x-direction are asymmetric, but the two invalid screen areas in the y-direction are symmetric

[0179] For example, assuming that the resolution of the terminal device 500 is 1920×1080 and the screen resolution of the display device 200 is 1920×1080, the reference center point of the user interface is (960, 540). As shown in Figure 20a, the display device 200 detects that the invalid screen area is not centrally symmetrical, and detects that the four margins of the invalid screen area are left=60, right=100, top=65, and bottom=65. The display device 200 can calculate the preprocessing height as 1080-65-65=950 through the algorithm processing module, and calculate the preprocessing width as 1920-100-60=1760. Then, the target center point can be obtained as (940, 540) through the preprocessing height and preprocessing width. As shown in Figure 20b, after the display device 200 subtracts each margin, the center point of the remaining screen frame area (valid screen area) is offset. In order to center the remaining frame content, the display device 200 needs to shift it 20 pixels to the right before scaling. Therefore, the display device 200 can calculate the offset of 960-940=20 based on the target center point (940, 540) and the reference center point (960, 540) through the algorithm processing module, i.e., a rightward shift of 20. Then, the display device 200 can calculate the first and second ratios based on the preprocessed height and preprocessed width, i.e., scaleX=(float)1920 / (float)(1920-100-60)=1.09f, and scaleY=(float)1080 / (float)(1080-2*65)=1.14f.

[0180] The display device 200 selects the smallest ratio between the first ratio and the second ratio, i.e. 1.09f, through the algorithm processing module, and transmits it to the View management module as the scaling ratio, and transmits the rightward shift of 20 as a position parameter to the View management module. Finally, the position of the projection screen is set through the View management module to shift the projection screen 20 pixels to the right, and scale the width and height of the picture frame according to the ratio of 1.09f. In this way, when the display device 200 displays the picture frame of the projection data, it presents the projection screen 930 shown on the right side of Figure 20a, which can reduce the invalid picture area in the user interface of the display device 200, and at the same time, the valid picture area 921 of the picture frame can be displayed in the center of the user interface.

[0181] In some embodiments, when setting the position parameters of the projection screen based on the center offset, the display device 200 may calculate a first difference and a second difference between the target center point and the reference center point. The first difference may be the difference between the target center point and the reference center point in the first direction, and the second difference may be the difference between the target center point and the reference center point in the second direction. The distance between the left edge of the projection screen and the projection window may be set based on the first difference, and the distance between the top edge of the projection screen and the projection window may be set based on the second difference.

[0182] For example, when the projection screen needs to be shifted 20 pixels to the right and scaled at a ratio of 1.09f, the target display parameters set by the View management module may include: FrameLayout.LayoutParams lp1 = new FrameLayout.LayoutParams(width, height, 0); lp1.width = 1920; lp1.height = 1080; lp1.leftMargin = 20; lp1.topMargin = 0; mPlayerView.setLayoutParams(lp1); mPlayerView.animate().scaleX(scaleX).scaleY(scaleX).

[0183] Example 2: The two invalid screen areas in the x-direction are symmetrical, but the two invalid screen areas in the y-direction are asymmetrical

[0184] For example, assuming that the resolution of the terminal device 500 is 1920×1080 and the screen resolution of the display device 200 is 1920×1080, the reference center point of the user interface is (960, 540). As shown in Figure 21a, the display device 200 detects that the invalid screen area is not centrally symmetrical, and detects that the four margins of the invalid screen area are left=100, right=100, top=0, and bottom=66. The display device 200 can calculate the preprocessing height as 1080-66-0=1014 and the preprocessing width as 1920-100-100=1720 through the algorithm processing module. Then, the target center point can be obtained as (960, 507) through the preprocessing height and preprocessing width. As shown in Figure 21b, after the display device 200 subtracts each margin, the center point of the remaining screen frame area (valid screen area) is offset. In order to center the remaining frame content, the display device 200 needs to move it downward by 33 pixels before scaling. Therefore, the display device 200 can calculate the offset of 540-507=33 based on the target center point (960, 507) and the reference center point (960, 540) through the algorithm processing module, that is, a downward translation of 33. Then, the display device 200 can calculate the first and second scales based on the preprocessed height and preprocessed width, that is, scaleX=(float)1920 / (float)(1920-100-100)=1.12f, scaleY=(float)1080 / (float)(1080-65)=1.07f.

[0185] The display device 200 selects the smallest ratio between the first ratio and the second ratio, i.e. 1.07f, through the algorithm processing module, and transmits it to the View management module as the scaling ratio, and transmits the downward shift of 33 as the position parameter to the View management module. Finally, the position of the projection screen is set through the View management module to shift the projection screen downward by 33 pixels and scale the width and height of the screen frame according to the ratio of 1.07f. Among them, the target display parameters set by the View management module may include: FrameLayout.LayoutParams lp1 = new FrameLayout.LayoutParams(width, height, 0); lp1.width = 1920; lp1.height = 1080; lp1.leftMargin = 20; lp1.topMargin = 0; mPlayerView.setLayoutParams(lp1); mPlayerView.animate().scaleX(scaleY).scaleY(scaleY). In this way, when the display device 200 displays the picture frame of the projection data, it presents the projection screen 930 shown on the right side of Figure 21a, which can reduce the invalid screen area in the user interface of the display device 200, and at the same time allow the valid screen area 921 of the picture frame to be displayed in the center of the user interface.

[0186] Example 3: The two invalid screen areas in the x-direction are asymmetric, and the two invalid screen areas in the y-direction are asymmetric

[0187] For example, assuming that the resolution of the terminal device 500 is 1920×1080 and the screen resolution of the display device 200 is 1920×1080, the reference center point of the user interface is (960, 540). As shown in Figure 22a, the display device 200 detects that the invalid screen area is not centrally symmetrical, and detects that the four margins of the invalid screen area are left=100, right=80, top=30, and bottom=100. The display device 200 can calculate the preprocessing height as 1080-30-100=950 and the preprocessing width as 1920-100-80=1740 through the algorithm processing module. Then, the target center point can be obtained as (970, 475) through the preprocessing height and preprocessing width. As shown in Figure 22b, after the display device 200 subtracts each margin, the center point of the remaining screen frame area (valid screen area) is offset. In order to display the remaining frame content in the center, the display device 200 needs to move it 10 pixels to the right and 65 pixels downward before it can be scaled. Therefore, the display device 200 can calculate the offsets including 10 and 65 based on the target center point (970, 475) and the reference center point (960, 540) through the algorithm processing module, that is, 10 to the right and 65 downward. Then, the display device 200 can calculate the first ratio and the second ratio based on the preprocessed height and the preprocessed width, that is, scaleX = (float) 1920 / (float) (1920-100-80) = 1.10f, scaleY = (float) 1080 / (float) (1080-30-100) = 1.14f

[0188] The display device 200 selects the smallest ratio between the first ratio and the second ratio, i.e. 1.10f, through the algorithm processing module, and transmits it to the View management module as the scaling ratio, and transmits the position parameters of 10 to the right and 65 to the bottom to the View management module. Finally, the position of the projection screen is set through the View management module to shift the projection screen 10 pixels to the right and 65 pixels down, and scale the width and height of the screen frame according to the ratio of 1.10f. Among them, the target display parameters set by the View management module may include: FrameLayout.LayoutParams lp1 = new FrameLayout.LayoutParams(width, height, 0); lp1.width = 1920; lp1.height = 1080; lp1.leftMargin = 10; lp1.topMargin = -65; mPlayerView.setLayoutParams(lp1); mPlayerView.animate().scaleX(scaleX).scaleY(scaleX). In this way, when the display device 200 displays the picture frame of the projection data, it presents the projection screen 930 shown on the right side of Figure 22a, which can reduce the invalid screen area in the user interface of the display device 200, and at the same time allow the valid screen area 921 of the picture frame to be displayed in the center of the user interface.

[0189] Based on the above-mentioned display device 200, some embodiments of the present application also provide a screen projection display method for a display device, and the method may include but is not limited to the above-mentioned steps S1201-S1203. Based on the above-mentioned embodiments, for the screen projection process shown in Figures 9a-9b, the display device 200 can scale and move the projection screen through the method provided in some embodiments of the present application to display the projection effect shown in Figures 23a-23e. That is, after the display device 200 executes steps S1201-S1203, the screen projection process shown in Figure 9a presents the projection effect shown in Figure 23a; the screen projection process shown in Figure 9b presents the projection effect shown in Figure 23b; the screen projection process shown in Figure 9c presents the projection effect shown in Figure 23c; the screen projection process shown in Figure 9d presents the projection effect shown in Figure 23d; the screen projection process shown in Figure 9e presents the projection effect shown in Figure 23e.

[0190] It can be seen from the above scheme that some embodiments of the present application provide a display device and a screen projection display method for a display device, and the display device can establish a communication connection with a terminal device and receive projection data sent by the terminal device. Among them, the projection data may include a picture frame for forming a projection screen, and the picture frame may include a valid picture area and an invalid picture area. The display device can detect the invalid picture area of ​​the picture frame, and calculate the target display parameters of the projection screen based on the invalid picture area, and then control the display to display the projection screen according to the target display parameters. Among them, the target display parameters can be used to set the center point of the valid picture area in the picture frame on the center point of the display device user interface, and to maximize the proportion of the valid picture area of ​​the picture frame in the user interface. The invalid picture area displayed by the display device can be removed to the greatest extent, while maintaining the picture centering display effect of the valid picture area, thereby improving the user's viewing experience.

[0191] Since the projection data is composed of multiple frames, the invalid screen areas of different frames may also be different. In some embodiments, in the projection data sent by the terminal device 500, the position and width and height of the invalid screen area 922 in the current frame and the previous frame may change. For example, as shown in Figure 24a, the media played by the terminal device 500 may include dynamically changing subtitle content. As the playback progresses, the subtitles will intermittently appear in the invalid screen area 922 at the bottom of the terminal interface 910. Correspondingly, in the projection data recorded by the terminal device 500, the position of the subtitles will be changed to the valid screen area 921, so the valid screen area 921 and the invalid screen area 922 shown in Figure 24a are constantly changing.

[0192] For another example, as shown in FIG24b, when the terminal device 500 plays the media resource shown in FIG24a, the barrage function is turned on. The barrage content will continuously scroll to the left in the terminal interface 910. Accordingly, as the barrage scrolls, the location of the barrage in the projection data recorded by the terminal device 500 will change to the effective screen area 921, so the effective screen area 921 and the invalid screen area 922 shown in FIG24b are also constantly changing.

[0193] Based on steps S1201-S1203 shown in Figure 12, in some embodiments, when the invalid screen area 922 in the screen frame changes, the display device 200 will still display the projection screen 930 according to the calculated target display parameters. If the invalid screen area 922 of each screen frame in the projection data changes frequently, the projection screen 930 displayed by the display device 200 will also change continuously, resulting in the effect of switching the projection screen back and forth. For example, in the projection process shown in Figures 24a-24b, when the display device 200 receives the projection data shown in Figure 24a, it displays the projection screen according to the target display parameters, and will display the display effect shown in Figure 25a; when the display device 200 receives the projection data shown in Figure 24b, it displays the projection screen according to the target display parameters, and will display the display effect shown in Figure 25b. It can be seen from the projection screens shown in Figures 25a-25b that since each screen frame in the projection data is constantly changing, the projection screen presented by the display device 200 will show the effect of frequent switching. It is understandable that the above-mentioned scenario of changing the invalid screen area 922 based on the dynamic content of the bullet screen and subtitles is only an example, and the embodiment provided in this application can be applied to the scenario of changing the invalid screen area 922 of any picture frame. In this regard, this application does not impose any limitation.

[0194] In some embodiments, as shown in FIG26 , the at least one processor 250 included in the display device 200 may be further configured to execute the aforementioned computer program to enable the display device to perform the following program steps:

[0195] S2601: Detecting an invalid picture area of ​​the picture frame.

[0196] The method for establishing a screen projection connection between the display device 200 and the terminal device 500 can be found in step S1201 and will not be described in detail here. In some embodiments, during the screen projection process, the display device 200 also detects the invalid screen area 922 of the screen frame 920 in real time.

[0197] In some embodiments, after the display device 200 establishes a screen projection connection with the terminal device 200 and displays the projection screen corresponding to the projection data according to the initial display parameters, a timing function can be started to determine whether the invalid screen area in the screen frame has changed within a target time period. For example, the target time period can be 500ms, and the screen projection application of the display device 200 can perform polling checks through Handler and Runnable, using the postDelayed method to execute a Runnable task every 500ms; or, the display device 200 registers a listener to detect changes in parameters related to the valid screen area 921, etc.

[0198] It should be noted that the target time period value can be an empirical value and can be freely set based on actual conditions. Alternatively, the target time period value set by the display device 200 may also be different for different projection protocols. This application does not impose any restrictions on this.

[0199] In some embodiments, the display device 200 can detect the effective screen area 921 of the current screen frame and obtain the first area information of the effective screen area 921 in the current screen frame. The first area information may include the width and height dimensions of the effective screen area and the position of the effective screen area in the screen frame, such as the coordinates of the upper left corner or the center point. If the first area information of the current screen frame is inconsistent with that of the previous screen frame, the invalid screen area 922 may be marked as changed; if the first area information of the current screen frame is consistent with that of the previous screen frame, the invalid screen area 922 may be marked as unchanged. In other words, the display device 200 can continuously receive screen frames of projection data and can detect in real time the relevant parameters of the effective screen area 921 in the current screen frame and the previous screen frame, which may include but are not limited to the width and height dimensions of the effective screen area 921 and the position of the effective screen area 921 in the screen frame 920. If any of the relevant parameters of the valid picture area 921 in the current picture frame is inconsistent with the relevant parameters of the valid picture area 921 in the previous frame, the invalid picture area 922 can be marked as changed; otherwise, the invalid picture area 922 can be marked as unchanged.

[0200] In other embodiments, the display device 200 may further obtain second region information of the invalid screen area 922 in the current frame. The second region information may include the width and height dimensions of the invalid screen area 922 and the position of the invalid screen area 922 in the frame 920. If the second region information of the current frame is inconsistent with that of the previous frame, the invalid screen area 922 may be marked as changed; if the second region information of the current frame is consistent with that of the previous frame, the invalid screen area 922 may be marked as unchanged. In other words, the display device 200 may continuously receive frames of projection data and detect, in real time, parameters related to the invalid screen area 922 in the current frame and the previous frame, including but not limited to the width and height dimensions of the invalid screen area 922 and the position of the invalid screen area 922 in the frame 920. If any of the parameters related to the invalid screen area 922 in the current frame is inconsistent with the parameters related to the invalid screen area 922 in the previous frame, the invalid screen area 922 may be marked as changed; otherwise, the invalid screen area 922 may be marked as unchanged.

[0201] It should be noted that, in some embodiments, the display device 200 may also synchronously obtain the first region information and the second region information, and mark whether the invalid screen region 922 has changed based on the first region information and the second region information. This application does not impose any restrictions on this.

[0202] In some embodiments, when detecting the invalid screen area 922 or the valid screen area 921, the display device 200 can still detect the color value (such as RGB value) of each screen frame 920 of the projection data to determine the invalid screen area 922 in the screen frame 920, which will not be repeated here.

[0203] S2602: Calculate the target display parameters of the projection screen based on the invalid screen area; the target display parameters are used to set the center point of the valid screen area to the reference center point of the user interface, and to maximize the display proportion of the valid screen area in the user interface.

[0204] After detecting the invalid image area of ​​the image frame 920, the display device 200 calculates the target display parameters of the projection image based on the size of the invalid image area 922. The calculation method of the target display parameters can be referred to Figures 14 to 19 and related content, which will not be repeated here.

[0205] S2603: When the invalid screen area changes within the target time period, detecting whether the invalid screen area is centrally symmetrical about the reference center point.

[0206] When the display device 200 calculates the target display parameters of the current picture frame, it can also detect whether the invalid picture area 922 of the current picture frame has changed within the target time period. When the invalid picture area 922 of each picture frame in the projection data changes, it means that the each picture frame of the projection data is constantly changing, that is, there is dynamic content in the picture frame of the projection data. In order to fully display the valid picture area 921 in the user interface and prevent the projection screen from frequently switching, the display device 200 can determine whether the invalid picture area 922 of the picture frame is centrally symmetrical with the reference center point. In some embodiments, when the display device 200 determines whether the invalid picture area 922 is centrally symmetrical with the reference center point, it can still be determined by the values ​​of top, bottom, left and right; or, the display device 200 can also be determined by any one of the methods provided in the above embodiments (such as image processing algorithm, symmetry detection algorithm, third-party library or framework, etc.). This application will not go into details here.

[0207] S2604: If the invalid screen area is centrally symmetrical about the reference center point, control the display to display the projection screen according to the target display parameters.

[0208] During the target time period, when the invalid screen area 922 of the picture frame changes, and the display device 200 detects that the invalid screen area 922 is symmetrical about the reference center point, it indicates that the changing area of ​​the dynamic content in the picture frame 920 is symmetrical. At this time, the display device 200 can display the projection screen according to the target display parameters. For example, there is a moving bullet comment in the picture frame 920, but the area where the bullet comment moves is symmetrical about the reference center point during the target time period; or there are subtitles in the picture frame 920, but the display area of ​​the subtitles is a dynamic content scene that is symmetrical about the reference center point during the target time period.

[0209] The following three examples illustrate the scenarios in which the display device 200 performs step S2604. In the examples, the resolution of the terminal device 500 is 1920×1080, the screen resolution of the display device 200 is 1920×1080, and the preset target time period is 500ms.

[0210] Example 1: As shown in Figure 27, the display device 200 detects through the algorithm processing module that the invalid image area 922 in the current image frame 920 has left and right values ​​of 0 and top and bottom values ​​of 50, and its size can be recorded as (0, 50). The display device 200 can then execute steps S1401-S1407. That is, the display device 200 calculates the pre-processed height as 1080-2*50=980 and the pre-processed width as 1920-0=1920. Then, the display device 200 calculates a first scale in the x-direction and a second scale in the y-direction: scaleX=(float)1920 / (float)(1920)=1f, and scaleY=(float)1080 / (float)(1080-2*50)=1.10f. The display device 200 obtains the minimum value, 1f, of the first and second scales as the x- and y-scale ratios of the projected image. As the bullet comment moves, the inactive area of ​​the frame changes continuously within 500ms. However, the area where the bullet comment moves within 500ms is centrally symmetrical about the reference center point. Therefore, the display device 200 can set the position of the projected image according to the target display parameter (1f). Since the minimum value of the first ratio and the second ratio is 1, as shown on the right side of Figure 27, the projected image 930 will maintain its original display effect after scaling.

[0211] Example 2: As shown in Figure 28, the display device 200 detects through the algorithm processing module that the invalid screen area 922 in the current screen frame 920 is 120 for the left and right, and 50 for the top and bottom. The display device 200 then executes steps S1401-S1407. That is, the display device 200 calculates the pre-processing height as 1080-2*50=980, and calculates the pre-processing width as 1920-2*120=1680. Then, the first ratio in the x-direction and the second ratio in the y-direction are calculated, that is, scaleX=(float)1920 / (float)(1920-2*120)=1.14f, scaleY=(float)1080 / (float)(1080-2*50)=1.10f. The display device 200 can obtain the minimum value 1.10f of the first ratio and the second ratio as the scaling ratio of the projection screen in the x-direction and the y-direction. As the bullet screen moves, the invalid screen area of ​​the picture frame is constantly changing within 500ms, but the moving area of ​​the bullet screen within 500ms is centrally symmetrical at the reference center point, so at this time the display device 200 can still set the position of the projection screen according to the target display parameter (1.10f). That is, the display device 200 scales the projection screen according to 1.10f through the View management module. In this way, when the display device 200 displays the picture frame of the projection data, it displays the projection screen 930 shown on the right side of Figure 28.

[0212] Example 3: As shown in Figure 29, the display device 200 detects through the algorithm processing module that the invalid screen area 922 in the current screen frame 920 is 405 for the left and right, and 656.5 for the top and bottom. The display device 200 then executes steps S1401-S1407. That is, the display device 200 calculates the pre-processing height as 1080-2*405=270, and calculates the pre-processing width as 1920-2*656.5=607. Then, the first ratio in the x-direction and the second ratio in the y-direction are calculated, that is, scaleX=(float)1080 / (float)(1080-2*405)=4f, scaleY=(float)1920 / (float)(1920-2*656.5)=3.2f. The display device 200 can obtain the minimum value of 3.2f between the first ratio and the second ratio as the scaling ratio of the projected screen in the x-direction and the y-direction. As the bullet screen moves, the invalid screen area of ​​the picture frame is constantly changing within 500ms, but the moving area of ​​the bullet screen within 500ms is centrally symmetrical at the reference center point, so at this time the display device 200 can set the position of the projection screen according to the target display parameter (3.2f). That is, the display device 200 scales the projection screen according to 3.2f through the View management module. In this way, when the display device 200 displays the picture frame of the projection data, it displays the projection screen 930 shown on the right side of Figure 29.

[0213] S2605: If the invalid image area is not centrally symmetrical about the reference center point, discard the target display parameters so that the display maintains the display parameters of the projected image.

[0214] In other embodiments, during the target time period, when the invalid screen area 922 of the picture frame changes, but the display device 200 detects that the invalid screen area 922 is not symmetrical with respect to the reference center point, it indicates that the changing area of ​​the dynamic content in the picture frame 920 is asymmetrical. At this time, if the display device 200 still displays the projection screen according to the target display parameters, the projection screen will present the frequent switching effect shown in Figures 25a-25b.

[0215] Therefore, in some embodiments of the present application, during the target time period, when the display device 200 detects that the invalid screen area 922 of the picture frame has changed, and the invalid screen area 922 is not symmetrical about the reference center point, the calculated target display parameters can be discarded. That is, the display device 200 can ignore the target display parameters currently calculated by the algorithm processing module based on the invalid screen area in the current picture frame, maintain the display parameters of the projection screen, and thus not scale or move the projection screen. For example, there is a changing barrage in the picture frame 920, but the area where the barrage moves is not symmetrical about the reference center point; or, the barrage is a dynamic content scene that is random in a flash-in and flash-out manner.

[0216] The following describes two examples of the display device 200 executing step S2604. In the examples, the resolution of the terminal device 500 is 1920×1080, the screen resolution of the display device 200 is 1920×1080, and the preset target time period is 500 ms.

[0217] Example 4: As shown in Figure 30a, the display device 200 detects through the algorithm processing module that the current invalid screen area is not centrally symmetrical, and the display device 200 executes steps S1701-S1709. Within 500ms, the barrage sometimes appears on the left side of the screen frame and sometimes on the right side of the screen frame, and the invalid screen area is constantly changing. When the barrage appears on the left side of the screen frame, left is 0, right is 120, top and bottom are 50, and the display device 200 can calculate the preprocessing height as 1080-2*50=980, and calculate the preprocessing width as 1920-120=1800. Then, the target center point can be obtained as (900,540) through the preprocessing height and the preprocessing width; as shown in Figure 30b, after the display device 200 subtracts the margin of the invalid screen area, the center point of the remaining screen frame area (valid screen area) is offset. In order to display the remaining content of the picture frame in the center, the display device 200 needs to move it 60 pixels to the right before it can be scaled. Therefore, the display device 200 can calculate the offset of 960-900=60=60 based on the target center point (900,540) and the reference center point (960,540) through the algorithm processing module, that is, a 60-degree shift to the right. Similarly, when the barrage appears on the right side of the picture frame, left is 120, right is 0, top and bottom are 50; the algorithm processing module will calculate the offset of 900-960=-60, that is, a 60-degree shift to the left. Then, the first ratio in the x direction and the second ratio in the y direction are calculated, that is, scaleX=(float)1920 / (float)(1920-120)=1.06f, scaleY=(float)1080 / (float)(1080-2*50)=1.10f. The display device 200 can obtain the minimum value 1.06f between the first ratio and the second ratio as the scaling ratio of the projection image in the x direction and the y direction.

[0218] As shown in the picture frame 920 in Figure 30a, as the barrage moves, the invalid picture area of ​​the picture frame is constantly changing, and the moving area of ​​the barrage within 500ms is not centrally symmetrical at the reference center point. Therefore, at this time, the display device 200 can discard the target display parameters to maintain the original display parameters of the picture frame. That is, the display device 200 can delete the target display parameters calculated this time through the algorithm processing module, and the View management module does not modify the display parameters of the projection screen. In this way, when the display device 200 displays the picture frame of the projection data, it can maintain the display effect of the picture frame corresponding to the initial display parameters, and display the projection screen 930 shown on the right side of Figure 30a.

[0219] Example 5: As shown in Figure 31, the calculation method of the target display parameters is the same as the principle of the above embodiment, which will not be repeated here. In the screen projection process shown in Figure 31, the number of barrages is small, and the effective screen area 921 used to display the media screen in the picture frame 920 is also constantly changing. Therefore, within 500ms, the invalid screen area 922 of each picture frame 920 in the screen projection data changes, and the invalid screen area 921 is not centrally symmetrical at the reference center point. The display device 200 can discard the target display parameters calculated by the algorithm processing module and use the original display parameters of the projection screen 930, thereby displaying the projection screen 930 shown on the right side of Figure 31.

[0220] For the case where the invalid screen area 922 in the picture frame has not changed, in some embodiments, within the target time period, when the invalid screen area 922 of each picture frame 920 in the projection data changes, the display device 200 can control the display 260 to display the projection screen according to the target display parameters. That is, if the invalid screen area 922 of the picture frame 920 has not changed, it means that the current projection data does not include dynamic content such as barrage or subtitles located in the invalid screen area. At this time, the display device 200 can display the projection screen according to the target display parameters calculated according to the above principles, thereby minimizing the invalid screen area 922 displayed by the user interface, increasing the effective screen area 921 displayed by the user interface, and at the same time allowing the effective screen area 921 to be displayed in the center of the user interface.

[0221] However, for media assets that already have an invalid screen area 922, the invalid screen area 922 will surround the boundary of the valid screen area 921. After the terminal device 500 enables the bullet chat function, the bullet chat may be displayed in the invalid screen area 921 at the boundary of the media asset. For ease of distinction, the following embodiments refer to the valid screen area 921 of the media itself as the media asset screen area, and the invalid screen area 922 of the media itself as the black area.

[0222] Based on this, in some embodiments, the display device 200 can also detect the target barrage area in the effective screen area 921. Among them, the target barrage area can be the display area of ​​the barrage located in the black area. Since the barrage is non-essential display content, the display device 200 can update the target barrage area to the invalid screen area 922 after detecting the target barrage area, so as to calculate the target display parameters according to the updated invalid screen area 922. In this way, when the display device 200 displays the projection screen 930, it can remove the barrage part displayed in the black area, thereby increasing the display proportion of the media asset screen area in the user interface.

[0223] It should be noted that when detecting the target bullet screen area, the display device 200 can perform motion analysis on the picture frames based on at least one of the attributes of the bullet screen, such as the dynamics, movement speed, and text features, detect the frame differences between each picture frame, and then determine the above-mentioned target bullet screen area based on the ratio of pixels with displayed content to pixels without displayed content. Alternatively, the display device 200 can also adopt other methods to detect the target bullet screen area. This application does not impose any restrictions on this.

[0224] Based on the above-mentioned display device 200, some embodiments of the present application also provide another screen projection display method for a display device, which includes but is not limited to the aforementioned steps S2601-S2605. As can be seen from the above embodiments, the method can detect the invalid screen area of ​​the picture frame in the projection data, and can calculate the target display parameters of the projection screen based on the invalid screen area. Among them, the target display parameters can be used to set the center point of the valid screen area in the picture frame to the reference center point of the user interface, while maximizing the display proportion of the valid screen area in the user interface. During the target time period, when the invalid screen area changes, it is detected whether the invalid screen area is centrally symmetrical with the reference center point. If the invalid screen area is centrally symmetrical with the reference center point, the projection screen is displayed according to the target display parameters; if the invalid screen area is not centrally symmetrical with the reference center point, the target display parameters are discarded to maintain the display parameters of the projection screen. The method reduces the invalid screen area displayed on the projection screen and keeps the valid screen area displayed in the center, while improving the problem of frequent switching of the projection screen, thereby improving the user's viewing experience.

[0225] In some embodiments, as shown in FIG32 , the at least one processor 250 included in the display device 200 may be further configured to execute the aforementioned computer program to enable the display device to perform the following program steps:

[0226] S3201: Acquire a first target area of ​​a current picture frame, where the first target area is a valid picture area of ​​the current picture frame.

[0227] In some embodiments, during the screen projection process, the display device 200 may further detect the valid screen area 921 and the invalid screen area 922 in the picture frame 920 in real time, and then obtain the valid screen area 921 of the current picture frame. For ease of distinction, in some embodiments of the present application, the valid screen area 921 in the current picture frame is referred to as the first target area, and the invalid screen area 922 in the current picture frame is referred to as the second target area.

[0228] In some embodiments, when the display device 200 detects the valid screen area 921 and the invalid screen area 922 of each picture frame 920, it can detect the valid screen area 921 of the picture frame 920, and then subtract the valid screen area 921 from the picture frame 920 to obtain the invalid screen area 922; or, the display device 200 can also detect the invalid screen area 922 of the picture frame 920, and then subtract the invalid screen area 922 from the picture frame 920 to obtain the valid screen area 921; or, the display device 200 can also synchronously detect the valid screen area 921 and the invalid screen area 922 in the picture frame 920.

[0229] Since the position and size of the invalid screen area 922 in the picture frame 920 are uncertain, and when the invalid screen area 922 is unevenly distributed or the invalid screen area 922 is overall asymmetric in the x-direction and y-direction, the projection screen 930 is scaled by subtracting the invalid screen areas in the x-direction and y-direction by the algorithm processing module. The projection screen 930 may show an offset effect as shown in Figure 33 in the display device 200. Therefore, as shown in Figure 34, in some embodiments, the display device 200 can detect the second target area of ​​the current picture frame (S3401). Among them, the second target area is the invalid screen area 922 of the picture frame. Then, according to the second target area, its corresponding target display parameters (S3402) can be calculated, that is, the aforementioned steps S1201-step 1202. That is, when the display device 200 displays the projection screen, it calculates the target display parameters of the projection screen according to the above principles. In this way, the projection screen 930 formed corresponding to the target display parameters can enable the display device 200 to minimize the invalid screen area 922 in the user interface, while allowing the center of the effective screen area 921 to be displayed at the center position of the user interface (center point of the screen), so that the effective screen area 921 can be displayed in the center.

[0230] In some embodiments, when calculating the target display parameters, the display device 200 can still use the reference center point as the center of symmetry and compare the second target area (invalid image area 922) of the current image frame in the first direction and the second direction to determine whether the second target area is symmetrical. The specific method used by the display device 200 to determine whether the second target area is symmetrical has been described above and will not be repeated here.

[0231] In some embodiments, the display device 200 may detect the second target area (invalid screen area 922) of the current screen frame, and then compare the second target area in the first direction and the second direction with the reference center point as the center of symmetry. Still as shown in Figure 15, if the second target area is centrally symmetrical with the reference center point, the first ratio of the screen frame width to the pre-processed width, and the second ratio of the screen frame height to the pre-processed height are calculated. When the display device 200 calculates that the first ratio is equal to the second ratio, one of the ratios is set to the scaling ratio of the projection screen in the first direction and the second direction to generate the target display parameters corresponding to the current screen frame. When the first ratio is not equal to the second ratio, in order to ensure that the projection screen can be displayed according to the original ratio of the valid screen area 921 after scaling, the display device 200 obtains the minimum value of the first ratio and the second ratio, and then sets the minimum value to the scaling ratio of the projection screen in the first direction, and sets the minimum value to the scaling ratio of the projection screen in the second direction to generate the target display parameters corresponding to the current screen frame.

[0232] Still as shown in Figure 19, if the second target area is not centrally symmetrical at the reference center point, that is, the two second target areas in the x direction are asymmetrical with each other, or the two second target areas in the y direction are asymmetrical with each other, or the two second target areas in the x direction are asymmetrical and the two second target areas in the y direction are also asymmetrical, the display device 200 calculates the preprocessing width and the preprocessing height, and then calculates the target center point of the picture frame 920 under the preprocessing width and the preprocessing height, that is, the center point of the effective picture area 921. Then, the center offset is calculated based on the target center point and the reference center point, and the position parameters of the projection screen are set based on the center offset. Then, the first ratio of the picture frame width to the preprocessing width is calculated, and the second ratio of the picture frame height to the preprocessing height is calculated. Similarly, in order to display the projection screen according to the original ratio of the first target area (effective picture area 921), the minimum value of the first ratio and the second ratio is obtained, and the minimum value is set as the scaling ratio of the projection screen in the first direction, and the minimum value is set as the scaling ratio of the projection screen in the second direction to generate the target display parameters corresponding to the current picture frame.

[0233] S3202: Calculate the area difference between the first target area and the reference area; the reference area is generated based on the valid picture area of ​​the historical picture frame, and the historical picture frame is a picture frame in the projection data that is earlier than the current picture frame.

[0234] After the display device 200 obtains the first target area of ​​the current frame, it can also calculate the area difference between the first target area and the reference area based on the first target area. The reference area can be a recorded area in the display device 200 and can be generated based on the valid image area 921 of the historical image frame in the projection data; the historical image frame can be an image frame in the projection data that is earlier than the current image frame.

[0235] In some embodiments, after the display device 200 establishes a communication connection with the terminal device 500 and receives the projection data sent by the terminal device 500, the invalid screen area 922 and the valid screen area 921 of the initial screen frame in the projection data can be detected. Among them, the initial screen frame can be the earliest screen frame in the projection data, that is, the first frame in the projection data. The valid screen area 921 of the initial screen frame can then be recorded as a reference area, which can be used as the initial value of the reference area. Then, the target display parameters of the initial screen frame are calculated based on the invalid screen area 922 of the initial screen frame, and the display 260 is controlled to display the projection screen according to the target display parameters of the initial screen frame.

[0236] That is, when the display device 200 receives the first initial screen frame in the projection data, it can record the valid screen area 921 of the initial screen frame as the reference area, which serves as the initial value of the reference area during the current projection process. The target display parameters corresponding to the initial screen frame are then calculated using the invalid screen area 922 of the initial screen frame, and the projection image 930 is displayed according to the target display parameters corresponding to the initial screen frame, so that the display device 200 presents the initial projection image 930 of the current projection.

[0237] For example, taking the display device 200 in the vertical screen state as an example, the resolution of the terminal device 500 is 1920×1080, and the screen resolution of the display device 200 is 1080×1920. As shown in Figure 35, the display device 200 can capture the first frame of the screen projection data, that is, the initial screen frame (the mth frame). Then, the algorithm processing module can be used to obtain the left and right of the invalid screen area 922 in the initial screen frame as 360, and the top and bottom as 900. Correspondingly, the width of the valid screen area 921 in the initial screen frame is 360 and the height is 120, then the valid screen area of ​​the initial screen frame can be recorded as (360,120) as the initial value of the reference area.

[0238] In some embodiments, the display device 200 may clear the value of the reference area when monitoring a screen projection disconnection event between the terminal device 500 and the display device 200. For example, in the screen projection application of the display device 200, a specific flag may be used to record the setting of the reference area; when the display device 200 is disconnected from the terminal device 500, the screen projection application may set the value of the reference area flag to zero or a corresponding empty state.

[0239] In some embodiments, the area difference between the first target area and the reference area may include a height difference and a width difference. When calculating the area difference, the display device 200 may obtain a first width and height dimension and a second width and height dimension. The first width and height dimension may include the first height and first width of the first target area, and the second width and height dimension may include the second height and second width of the reference area. Then, the height difference between the first height and the second height is calculated, as well as the width difference between the first width and the second width is calculated. For example, the reference area recorded by the display device 200 is (360, 120), the second width of the reference area is 360, and the second height is 120. As shown in Figure 35, when the current frame obtained by the display device 200 is the nth frame, the first width of the first target area (effective screen area) is 600 and the first height is 360. Then, the height difference calculated by the algorithm processing module is 240 and the width difference is 240.

[0240] In other embodiments, the area difference between the first target area and the reference area may include an area difference. When calculating the area difference, the display device 200 may calculate a first area of ​​the first height and the first width, and a second area of ​​the second height and the second width. The area difference between the first area and the second area is then calculated as the area difference between the first target area and the reference area.

[0241] S3203: If the area difference is greater than a preset difference threshold, control the display to display the projection screen according to the target display parameters of the current screen frame, and update the reference area through the first target area; the target display parameters are used to set the center point of the effective screen area to the reference center point of the user interface, and to maximize the display proportion of the effective screen area in the user interface.

[0242] After calculating the area difference between the first target area and the reference area, the display device 200 can compare the area difference with the value of the preset difference threshold. Among them, the value of the difference threshold can be customized. In the projection process shown in Figure 32, in order to reduce unnecessary scaling processes, when the first target area of ​​the picture frame 920 is reduced, the display device 200 may no longer update the scaling ratio of the projection screen; or, when the first target area of ​​the picture frame 920 is slightly increased, the display device 200 may also not update the scaling ratio of the projection screen. Therefore, in some embodiments, the value of the difference threshold may be greater than or equal to 0, and the value of the difference threshold may be a small positive value. For example, the value range of the difference threshold may be [0,25].

[0243] When the area difference is greater than the difference threshold, it means that the first target area of ​​the current picture frame has increased in range compared to the reference area, and the range of change is greater than the difference threshold. That is, the effective picture area 921 of the current picture frame 920 has increased in range compared to the effective picture area 921 recorded last time, and the difference between the two ranges is large. At this time, the display device 200 can display the projection screen 930 corresponding to the current picture frame according to the target display parameters calculated based on the second target area (steps S3401-S3402).

[0244] In order to reduce the amount of calculation of the display device 200, in some embodiments, after the display device 200 calculates the area difference between the first target area and the reference area, if the area difference is greater than the difference threshold, the display device 200 may execute step S3401; otherwise, no processing is performed.

[0245] In some embodiments, when the regional difference includes a height difference and a width difference, if the height difference is greater than the difference threshold, and / or the width difference is greater than the difference threshold, the display device 200 can control the display 260 to display the projection screen according to the target display parameters of the current picture frame. If the height difference is less than or equal to the difference threshold, and the width difference is less than or equal to the difference threshold, the target display parameters of the current picture frame can be discarded to maintain the display parameters of the projection screen. That is, when the regional difference includes a height difference and a width difference, if one of the height difference and the height difference is greater than the preset difference threshold, the display device 200 can determine that the regional difference is greater than the difference threshold, and display the projection screen through the target display parameters corresponding to the current picture frame; otherwise, if it is determined that the regional difference is less than or equal to the difference threshold, the calculated target display parameters can be discarded, thereby maintaining the original display effect of the projection screen.

[0246] In other embodiments, when the region difference includes an area difference, if the area difference is greater than a difference threshold, the display device 200 may control the display 260 to display the projected image according to the target display parameters of the current frame. If the area difference is less than or equal to the difference threshold, the target display parameters of the current frame may be discarded to maintain the display parameters of the projected image. That is, when the region difference includes an area difference, if the area difference is greater than a preset difference threshold, the display device 200 may display the projected image using the target display parameters corresponding to the current frame; otherwise, the calculated target display parameters may be discarded to maintain the original display effect of the projected image.

[0247] It should be noted that the regional difference value can include more or fewer differences, such as only height differences or only width differences; the difference threshold value can also be one or more different values, such as a height difference threshold value and a width difference threshold value. The height difference threshold value can be used to compare height differences, and the width difference threshold value can be used to compare width differences. The comparison method for the regional difference value and the difference threshold value can be a combination of one or more of the above, or other numerical comparison methods can be used. This application does not impose any restrictions on this.

[0248] When the area difference is greater than the difference threshold, the display device 200 may display the projected image according to the target display parameters corresponding to the current frame, i.e., the target display parameters calculated in steps S3401-S3402. In some embodiments, the display device 200 may also update the first target area in the current frame as the reference area to facilitate comparison of the effective image area 921 in subsequent frames with the reference area.

[0249] For example, the resolution of the terminal device 500 is 1920×1080, the screen resolution of the display device 200 is 1920×1080, and the difference threshold is 20. As shown in Figure 35, the effective screen area 921 of the first frame (the mth frame) in the projection data is obtained with a width of 360 and a height of 120, so the initial value of the reference area is (360, 120). As shown in Figure 35, when the current frame obtained by the display device 200 is the nth frame, the first width of the first target area (effective screen area) is 600 and the first height is 360. The height difference calculated by the display device 200 through the algorithm processing module is 240 and the width difference is 240. Obviously, the height difference and the width difference are both greater than 20, and the display device 200 can display the projection screen according to the target display parameters corresponding to the nth frame, and the first width and the first height can be updated to the reference area. That is, the value of the reference area is updated to (600, 360).

[0250] S3204: If the area difference is less than or equal to the difference threshold, maintain the display parameters of the projection image.

[0251] In some embodiments, when the area difference is less than or equal to the difference threshold, it means that the range of change of the first target area of ​​the current picture frame compared to the reference area is less than or equal to the difference threshold. That is, the effective picture area 921 of the current picture frame 920 has not changed compared to the effective picture area 921 recorded last time, or the effective picture area 921 may be increased but the increased area is small. At this time, the display device 200 may no longer update the display parameters of the projected screen, maintain the display parameters of the projected screen, and not process it. In this way, when the effective picture area 921 changes less in each frame or the effective picture area 921 decreases, the display device 200 may not re-scale the projected screen, thereby reducing the frequency of changes in the projected screen and improving the overall display effect of the projected screen.

[0252] In some embodiments, the display device 200 may further set an acquisition cycle upon receiving the initial frame of the projection data. The step of acquiring the first target area of ​​the current frame (S3201) may then be performed according to the acquisition cycle to reduce the computational complexity of the projection application in the display device 200. For example, the acquisition cycle may be 500ms, and after receiving the first frame of the projection data, the display device 200 executes steps S3201-S3204 every 500ms.

[0253] In some embodiments, when setting the acquisition cycle, the display device 200 can perform polling checks using Handler and Runnable, using the postDelayed method to execute a Runnable task every 500ms. Alternatively, the display device 200 can use the Timer and TimerTask classes to create a Timer object and use the schedule method of the Timer object to set the execution time period.

[0254] The following three examples illustrate the scenarios in which the display device 200 performs steps S3201-S3205. In these examples, the resolution of the terminal device 500 is 1920×1080, the screen resolution of the display device 200 in landscape mode is 1920×1080, and the resolution in portrait mode is 1080×1920. The preset difference threshold is 20.

[0255] Example a: The media played by the terminal device 500 changes as shown in Figure 35. The display device 200 is in portrait mode, and the acquisition period set by the display device 200 is 500ms. The display device 200 can obtain the first frame of the screen (frame m) in the projection data. The algorithm processing module detects that the left and right of the invalid screen area 922 of the frame m are 360, the top and bottom are 900, and the valid screen area 921 of the frame m is 360 wide and 120 high. The reference area can be recorded as (360, 120). Then, the first ratio in the x direction and the second ratio in the y direction are calculated, that is, scaleX = (float) 1080 / (float) (1080-2*360) = 3f, scaleY = (float) 1920 / (float) (1920-2*900) = 16f. The display device 200 can obtain the minimum value 3f between the first ratio and the second ratio as the scaling ratio of the projection screen in the x direction and the y direction, so that the View management module can scale the projection screen according to 3f and display the display effect of the projection screen m as shown in the right part of Figure 35.

[0256] After 500ms, it is assumed that the display device 200 obtains the width of the valid picture area 921 in the picture frame n as 600 and the height as 360. The width difference is calculated to be 240 and the height difference is 240 through the algorithm processing module. Since 240>20, the display device 200 can execute step S3203, update the value of the reference area to (600,360), and calculate the target display parameters corresponding to the picture frame n through the algorithm processing module. That is, the invalid picture area 922 is not symmetrical at the reference center point, and the first ratio and the second ratio are calculated: scaleX=(float)1080 / (float)(1080-600)=1.8f, scaleY=(float)1920 / (float)(1920-2*780)=5.3f. The display device 200 can obtain the minimum value 1.8f as the scaling ratio of the projection picture. As shown in Figure 36, the center point of the effective screen area 921 is offset. The center offset can be calculated as 180 by the algorithm processing module, and the center offset can be transmitted to the View management module as a position parameter, so that the View management module can move the projection screen 180 to the right and scale it according to 1.8f to display the display effect of the projection screen n as shown on the right side of Figure 35.

[0257] Then, after another 500ms, assume that the display device 200 obtains that the width of the valid image area 921 in the image frame o is 860 and the height is 360. At this time, the recorded reference area is (600, 360), and the algorithm processing module calculates that the width difference is 260 and the height difference is 0. Since 260>20, the display device 200 can execute step S3203, update the value of the reference area to (860, 360), and calculate the target display parameters corresponding to the image frame o through the algorithm processing module. That is, the invalid image area 922 is centrally symmetrical about the reference center point, and the first and second ratios are calculated as scaleX=(float)1080 / (float)(1080-2*110)=1.26f, and scaleY=(float)1920 / (float)(1920-2*780)=5.3f. The display device 200 can obtain the minimum value 1.26f as the scaling ratio of the projection screen, so that the View management module can scale the projection screen according to 1.26f, thereby displaying the display effect of the projection screen o as shown on the right side of Figure 35 in area e.

[0258] Similarly, after another 500ms, the display device 200 obtains the width of the effective picture area 921 in the picture frame p as 360 and the height as 240. At this time, the recorded reference area is (860, 360), and the width difference is calculated by the algorithm processing module as -500 and the height difference is -120. Since -500<20 and -120<20, the display device 200 can execute step S3204, and does not update the value of the reference area through the effective picture area 921 of the picture frame p, and does not process the projection picture. That is, the value of the reference area remains (860, 360), and the projection picture still maintains the target display parameters calculated according to the picture frame o, and displays the display effect described in the projection picture p as shown on the right side of Figure 35.

[0259] Example b: As shown in Figure 37, the display device 200 is in landscape mode, and the user switches between different applications on the terminal device 500. The display device 200 detects the screen frame of the projection data in real time. The effective screen area 921 of the first frame is 640 in width and 1080 in height, so the reference area can be recorded as (640, 1080). The minimum value of the first ratio and the second ratio is 1f, and the display device 200 maintains the original ratio of the projection screen, and the effective screen area 921 is displayed in area f. Then, the display device 200 obtains that the effective screen area 921 of the second frame is 300 in width and 460 in height, with a width difference of -340 and a height difference of -620. Since the width difference and the height difference are both less than 20, no processing is performed. Then, the display device 200 obtains that the effective screen area 921 of the third frame is restored to a width of 640 and a height of 1080, and the height difference and the width difference are both 0, then the original display parameters are maintained and no processing is performed. In this way, as shown in the screen projection process in Figure 37, when the user switches different applications on the terminal device 500, the display device 200 can ensure the display effect of the projected screen while reducing its device performance consumption, thereby improving the user's viewing experience.

[0260] In order to be applicable to complex change scenarios, in some embodiments, as shown in Figure 38, if the regional difference is less than or equal to the difference threshold (S3801), the display device 200 can also detect the picture change status of the effective picture area 921 in the picture frame within a preset time period (S3802). Determine whether the picture change status is changed (S3803). If the picture change status of the effective picture area 921 in the picture frame is unchanged, it means that the picture frame of the projection data is stable within the range of the current effective picture area 921, then the display 260 can be controlled to display the projection picture according to the target display parameters of the current picture frame (S3804), and update the reference area through the first target area (S3805); otherwise, no processing is performed. Among them, when the display device 200 is set with an acquisition cycle, the preset time period may be greater than the acquisition cycle. For example, the acquisition cycle is 500ms, and the preset time period may be 1s.

[0261] Example c: As shown in Figure 39, the display device 200 is in landscape mode. When the user plays media on the terminal device 500, the control center view is called up, and the preset time period is 1s. The display device 200 detects the screen frame of the projection data in real time. The effective screen area 921 of the first frame is 1780 in width and 864 in height. The initial value of the reference area can be recorded as (1780, 864). The minimum value of the first ratio and the second ratio is 1.07f, and the display device 200 can scale the projection screen according to 1.07f. Then, the display device 200 obtains the effective screen area 921 of the second frame as 1920 in width and 1080 in height, with a width difference of 840 and a height difference of 216. Since both the width difference and the height difference are greater than 20, step S3203 can be executed, and the reference area can be updated to (1920, 1080). Then, the scaling ratio of the projection screen can be set to 1f, so that the projection screen is displayed according to the original ratio of the screen frame. Then, the display device 200 obtains that the effective picture area 921 of the third frame is restored to a width of 1780 and a height of 864, and the height difference and the width difference are both less than 20. The display device 200 can then execute step S3203 without performing any processing. However, as the media resources in the terminal device 500 are played, the display device 200 detects that within 1 second, the effective picture area 921 of the picture frame always remains at a width of 1780 and a height of 864, indicating that the effective picture area 921 has been stable at a width of 1780 and a height of 864 for a long time. The display device 200 can then execute steps S3804-S3805, update the value of the reference area to (1780, 864), and restore the zoom ratio to 1.07, so that the View management module can scale the projection image according to 1.07f, displaying the display effect shown on the right side of Figure 39. In this way, as shown in the screen projection process in Figure 39, when the user plays media on the terminal device 500 and temporarily switches to other different applications or views, the display device 200 can ensure the display effect of the projected screen while reducing its device performance consumption, thereby improving the user's viewing experience.

[0262] It can be understood that the display device 200 in the above example three does not set an acquisition cycle. When an acquisition cycle is set, the display device acquires picture frames according to the time interval of the acquisition cycle, and the time of the acquisition cycle is less than the preset time period. The rest of the processing logic is the same as that in example three and will not be repeated here.

[0263] Based on the above-described display device 200, some embodiments of the present application further provide a third screen projection display method for a display device, the method including but not limited to the aforementioned steps S3201-S1304. As can be seen from the above scheme, the method can obtain a first target area of ​​the current frame, where the first target area is the effective screen area of ​​the current frame. The area difference between the first target area and the reference area is then calculated. If the area difference is greater than a preset difference threshold, the display is controlled to display the projected image according to the target display parameters of the current frame and update the reference area using the first target area. Otherwise, the display parameters of the projected image are maintained and no processing is performed. The reference area is generated based on the effective screen area of ​​a historical frame, where the historical frame is an image frame in the projection data that is earlier than the current frame. The target display parameters are used to set the center point of the effective screen area to the reference center point of the user interface and maximize the display proportion of the effective screen area in the user interface. While maximally centering the effective screen area in the frame, the method can also dynamically adjust the display parameters of the projected image based on changes in the effective screen area, thereby improving the display effect of the projected image and enhancing the user's viewing experience.

[0264] In some embodiments, the display device 200 obtains a screen projection session (Session) sent by the terminal device 500. The screen projection session may include the resolution of the picture frame. The screen resolution of the display device can then be detected, and the initial display parameters of the projection screen can be calculated based on the resolution of the picture frame and the screen resolution. The initial display parameters can be used to set the center point of the picture frame to the center point of the user interface, to make the aspect ratio of the projection screen equal to the aspect ratio of the picture frame, and to maximize the proportion of the picture frame in the user interface. Then, the display 260 is controlled to display the projection screen according to the initial display parameters.

[0265] In some embodiments, during the screen projection process between the terminal device 500 and the display device 200, if the user manipulates the terminal device 500, the data information of each frame 920 in the projection data may also change, and the projection screen 930 displayed on the display device 200 may also change accordingly. For example, in the mirror mode projection process shown in FIG40a, the user opens the gallery of the terminal device 500 and quickly switches between different photos, and the invalid screen area displayed in the terminal interface is constantly changing; accordingly, in the process of the user switching photos, the projection data recorded by the terminal device 500, the invalid screen area 922 of the frame 930 is constantly changing; the display device 200 scales the projection screen 930 according to the invalid screen area 922, and the projection screen 930 presents the display effect shown on the right side of FIG40a. For another example, in the mirror mode projection process shown in FIG40b, the user switches the terminal device 500 from the portrait mode to the landscape mode, and the invalid screen area 922 in the terminal interface of the terminal device 500 is constantly changing; accordingly, in the projection data recorded by the terminal device 500, the invalid screen area 922 of each screen frame 930 will also change; the display device 200 scales the projection screen 930 according to the invalid screen area 922, and the display effect shown on the right side of FIG40b will be displayed.

[0266] As shown in the process of Figures 40a and 40b, when the invalid screen area 922 of the screen frame in the projection data changes frequently, the display device 200 needs to continuously calculate the scaling ratio based on the changed invalid screen area 922 and scale the projection screen according to the calculated scaling ratio. This not only increases the resource consumption of the display device 200, but also causes the effective screen area of ​​the projection screen 930 to appear to be larger and smaller in a short period of time, affecting the user's viewing experience.

[0267] Based on the above application scenarios, in some embodiments, as shown in FIG41 , the at least one processor 250 included in the display device 200 may be further configured to execute the aforementioned computer program to enable the display device to perform the following program steps:

[0268] S4101: Detect characteristic information of the projection data within a preset time period.

[0269] The display device 200 establishes a communication connection with the terminal device 500 through the communicator 220, and receives the projection data generated by the terminal device 500. Then, within the preset time period, the characteristic information in the projection data is detected. Among them, the characteristic information may include various data parameters of the picture frame 930, which is used to determine whether the projection data is stable within the preset time period. For example, the preset time period may be 500ms, and the display device 200 can detect the characteristic information of the projection data within 500ms. Within the preset time period, when the characteristic information changes, it can be determined that the state of the projection data in the preset time period is unstable; when the characteristic information does not change, it can be determined that the state of the projection data in the preset time period is stable.

[0270] When detecting feature information, the display device 200 can detect it according to one or more of the following principles: 1. Whether the valid picture area or invalid picture area in two or more consecutive picture frames 930 in the projection data has changed; 2. Whether the resolution of the projection data has changed; 3. Whether the size of the projection data has changed; 4. Whether the key frame of the projection data has changed; 5. Whether the projection data has changed, etc.

[0271] In some embodiments, the feature information may include at least one of the following: active screen area 921, inactive screen area 922, resolution, timestamp, keyframe, frame size, and video identifier. That is, the feature information may include one or more combinations of the above, all of which may affect the display quality of the projection screen 930.

[0272] The following describes the above-mentioned data information respectively.

[0273] For the valid screen area 921 and the invalid screen area 922: Since the valid screen area 921 and the invalid screen area 922 are two relative data parameters, when one of them changes, the other must also change. Therefore, the characteristic information may include only the valid screen area 921, may include only the invalid screen area 922, or may include both the valid screen area 921 and the invalid screen area. For example, during the screen projection process in mirror mode, when the user switches the screen displayed in the terminal device 500 or changes the placement status of the terminal device 500, the invalid screen area and the valid screen area in the terminal interface 910 change, and the invalid screen area 922 and the valid screen area 921 of each screen frame 930 in the projection data also change. When the valid screen area 921 or the invalid screen area 922 changes, it means that the user may be operating, and the display device 200 can determine that the status of the current projection data is unstable.

[0274] In some embodiments, the display device 200 can detect the valid screen area 921 of the picture frame and obtain the first area information of the valid screen area 921 in the picture frame. If the first area information of the picture frame is inconsistent, the valid screen area 921 is marked as changed; if the first area information of the picture frame is consistent, the valid screen area 921 is marked as unchanged. In some embodiments, the display device 200 can also detect the invalid screen area 922 of the picture frame and obtain the second area information of the invalid screen area 922 in the picture frame. If the second area information of the picture frame is inconsistent, the invalid screen area 922 is marked as changed; if the second area information of the picture frame is consistent, the invalid screen area 922 is marked as unchanged. It should be noted that, in some embodiments, the display device 200 can also synchronously obtain the above-mentioned first area information and second area information. This application is not limited to this.

[0275] Regarding resolution and screen frame size. For example, during the screen projection process, the user can modify the resolution of the projection data in the projection settings or projection application of the terminal device 500; and some projection protocols also allow users to customize the resolution and size of the projection screen. When the resolution and screen frame size change, it means that the user may be operating, and the display device 200 can determine that the state of the projection data is unstable.

[0276] Regarding the timestamp, key frame, and video identifier (a unique identifier of the projection data, which may be different for different devices, applications, and projection protocols), for example, during the projection process, if the timestamp, key frame, and video identifier of the projection data change, it indicates that the projection data sent by the terminal device 500 has changed, and the display device 200 may also determine that the state of the projection data is unstable.

[0277] It should be noted that the feature information is described above through several data information, but it is not limited thereto. The feature information provided in the embodiment of the present application can be other data information of the projection data. In some embodiments, the display device 200 can also set the feature information according to the projection screen protocol established with the terminal device 500.

[0278] In some embodiments, as shown in FIG42a , when the time between the current frame 1502 and the initial frame is less than or equal to a preset time period, the display device 200 uses the time of the initial frame 1401 as the starting point and obtains the first frame set of the preset time period. The initial frame is the earliest frame in the projection data (the first frame). Feature information of the frames in the first frame set is then parsed.

[0279] If the feature information of any two frames in the first frame set is inconsistent, the display device 200 may determine that the feature information has changed within a preset time period; otherwise, the display device 200 may determine that the feature information has not changed within the preset time period. That is, if the feature information is inconsistent, the display device 200 may mark that the feature information of the frames in the first frame set has changed within the preset time period; if the feature information is consistent, the display device 200 may mark that the feature information of the frames in the first frame set has not changed within the preset time period.

[0280] For example, the preset time period is 500ms, and the characteristic information may include the invalid screen area 922 and the screen frame size. After receiving the first frame of the projection data, the display device 200 can turn on the timing function. Taking the time of the first frame as the starting point, each frame of the projection data within 500ms can be obtained to generate a first frame set, and the characteristic information of each frame obtained can be stored. Then compare the characteristic information of each frame in the first frame set. If the invalid screen area of ​​any two frames in the first frame set, or the frame size of any two frames are inconsistent, the marked characteristic information changes within 500ms; otherwise, the marked characteristic information does not change within 500ms.

[0281] In some embodiments, when the duration between the current frame and the initial frame is greater than a preset time period, as shown in FIG42b , the display device 200 uses the time of the current frame 1402 as the timing endpoint and obtains a second set of frames within the preset time period. Feature information of the frames in the second set of frames can then be parsed. If the feature information of the frames in the second set of frames is inconsistent, the feature information is marked as having changed within the preset time period. If the feature information of the frames in the second set of frames is consistent, the feature information is marked as having not changed within the preset time period.

[0282] For example, the preset time period is 500ms, and the characteristic information may include the invalid screen area 922 and the screen frame size. After 500ms of the display device 200 receiving the initial screen frame time, the display device 200 can obtain each screen frame of the projection data within 500ms with the time of the current screen frame as the timing end point to generate a second screen frame set, and can store the characteristic information of each screen frame obtained. Then compare the characteristic information of each screen frame in the second screen frame set. If the invalid screen area of ​​any two screen frames in the second screen frame, or the screen frame size of any two screen frames are inconsistent, the characteristic information can be marked as changed within 500ms; otherwise, the characteristic information is marked as not changed within 500ms.

[0283] As shown in Figure 43, in other embodiments, the display device 200 may set an acquisition cycle when receiving the initial picture frame 1401 of the projection data. The initial picture frame may be the earliest picture frame in the projection data, and the acquisition cycle may be equal to the preset time period. Then, the picture frames within the preset time period may be acquired according to the acquisition cycle to generate a third picture frame set. The feature information of the picture frames in the third picture frame set may then be parsed. If the feature information of the picture frames in the third picture frame set is inconsistent, the feature information may be marked as having changed within the preset time period; if the feature information of the picture frames in the third picture frame set is consistent, the feature information may be marked as not having changed within the preset time period.

[0284] In some embodiments, when setting the acquisition cycle, the display device 200 can perform polling checks using Handler and Runnable, using the postDelayed method to execute a Runnable task every 500ms. Alternatively, the display device 200 can use the Timer and TimerTask classes to create a Timer object and use the schedule method of the Timer object to set the execution time period.

[0285] In some embodiments, the preset time period can be set based on the projection data sent by the terminal device 500. For example, the display device 200 can set the preset time period based on the frame rate of the projection data; the display device 200 can preset the number of image frames to be detected, and then set the value of the preset time period based on the preset number of image frames and the frame rate of the projection data.

[0286] S4102: If the characteristic information changes, maintain the display parameters of the projection image.

[0287] The display device 200 detects the characteristic information of the projection data. When it is detected that the characteristic information has changed within a preset time period, it indicates that the state of the projection data is unstable. The display device 200 can maintain the current display parameters of the projection screen and does not perform scaling or other processing on it, so that the projection screen maintains the original display effect.

[0288] For example, the resolution of the terminal device 500 is 1920×1080, the screen resolution of the display device 200 is 1920×1080, the characteristic information is the invalid screen area, and the preset time period is 500ms. As shown in Figure 44a, the terminal device 500 and the display device 200 establish a projection connection based on the mirror mode, and the user switches pictures in the gallery in the terminal device 500. As shown in the picture frame a-picture frame c in Figure 44a, the display device 200 detects that the invalid screen area of ​​the projection data has changed within 500ms, and the display parameters of the projection screen can be maintained without processing. In this way, the consumption of system resources of the display device 200 can be reduced, and the problem of the effective screen area 921 fluctuating in size in the user interface can be improved.

[0289] S4103: If the characteristic information has not changed, the target display parameters of the projected screen are calculated based on the invalid screen area of ​​the current screen frame, and the display is controlled to display the projected screen according to the target display parameters; the target display parameters are used to set the center point of the valid screen area in the current screen frame to the reference center point of the user interface, and to maximize the display proportion of the valid screen area in the user interface.

[0290] In other embodiments, the display device 200 detects characteristic information of the projection data within a preset time period. When it is detected that the characteristic information has not changed, it indicates that the state of the projection data has stabilized. The display device 200 can then calculate target display parameters for the projection image 930 based on the invalid image area 922 in the current image frame. The target display parameters are used to set the center point of the effective image area 921 to the reference center point of the user interface, while reducing the invalid image area 922 displayed in the user interface and maximizing the display area of ​​the effective image area 921 in the user interface.

[0291] That is, when the state of the projection data is stable, the display device 200 calculates the target display parameters of the projection screen according to the above principles. The projection screen 930 formed corresponding to the target display parameters can enable the display device 200 to minimize the invalid screen area 922 in the user interface, while allowing the center of the effective screen area 921 to be displayed at the center position of the user interface (the center point of the screen), so that the effective screen area 921 can be displayed in the center.

[0292] In some embodiments, when calculating the target display parameters, the display device 200 may still use the reference center point as the center of symmetry and compare the invalid screen area 922 in the first direction and the second direction in the current screen frame to determine whether the invalid screen area 922 is symmetrical. The specific method used by the display device 200 to determine whether the invalid screen area 922 is symmetrical has been described above and will not be repeated here.

[0293] In some embodiments, the display device 200 detects the invalid image region 922 of the current image frame, and then compares the invalid image region 922 in the first direction and the second direction with the reference center point as the center of symmetry. Whether the invalid image region 922 is centrally symmetric with the reference center point can be processed by referring to the processing flow shown in Figures 15 and 19, and will not be repeated here.

[0294] The following describes two examples of the display device 200 executing steps S4101-S4105. In the examples, the resolution of the terminal device 500 is 1920×1080, the screen resolution of the display device 200 is 1920×1080, and the preset time period is 500ms.

[0295] Example a': The characteristic information is the effective picture area 921 and the resolution. The user operates on the terminal device 500 to switch the picture content displayed in the terminal interface, and the terminal interface of the terminal device 500 presents the change process as shown in Figure 44a. The display device 200 detects the characteristic information of the projection data within 500ms, and detects that the resolution of the projection data remains unchanged at 1920×1080 within 500ms, but the effective picture area 921 from picture frame a to picture frame b is increasing, and the effective picture area 921 from picture frame b to picture frame c is decreasing. At this time, the display device 200 can maintain the display parameters of the projection screen 930 without processing.

[0296] When the user stops operating the terminal device 500, as shown in Figure 44b, the display device 200 detects the characteristic information of the projection data within 500ms, and detects that the resolution of the projection data remains unchanged at 1920×1080 within 500ms, and the effective picture area 921 from picture frame d to picture frame f has not changed. At this time, the display device 200 can calculate the target display parameters of picture frame f, and can display the projection picture 930 according to the calculated target display parameters. That is, the algorithm processing module detects that left and right are 656.5, and top and bottom are 154. The preprocessing height is 1080-2*154=772, and the preprocessing width is calculated to be 1080-2*656.5=607. Then, the first scale in the x-direction and the second scale in the y-direction are calculated: scaleX = (float)1920 / (float)(1920 - 2 * 656.5) = 3.16f, scaleY = (float)1080 / (float)(1080 - 2 * 154) = 1.40f. The minimum of the first and second scales, 1.40f, is then taken as the scaling factor for the projection image. This allows the View management module to scale the projection image 930 by 1.40f, resulting in the projection image f shown in the right portion of FIG. 44b.

[0297] Example b': The characteristic information is the effective screen area 921, resolution and video identification. The user switches the terminal device 500 from the portrait mode to the landscape mode, and the terminal interface of the terminal device 500 presents the change process shown in Figure 45a. The display device 200 detects the characteristic information of the projection data within 500ms, and detects that the resolution of the projection data remains unchanged at 1920×1080 within 500ms, and the video identification has not changed, but the effective screen area 921 from the picture frame g to the picture frame h is reduced, and the effective screen area from the picture frame h to the picture frame i is increasing. At this time, the display device 200 can maintain the display parameters of the projection screen without processing.

[0298] After the terminal device 500 is stabilized in the horizontal screen state, as shown in Figure 45b, the display device 200 detects the characteristic information of the projection data within 500ms, and detects that the resolution of the projection data remains unchanged at 1920×1080 within 500ms, and the effective screen area 921 from picture frame j to picture frame l has not changed. At this time, the display device 200 can calculate the target display parameters of picture frame l, and can display the projection screen 930 according to the target display parameters calculated. That is, the algorithm processing module detects that the left and right are 90, and the top and bottom are 50. The preprocessing height is 1080-2*50=980, and the preprocessing width is calculated to be 1920-2*90=1740. Next, the first scale in the x-direction and the second scale in the y-direction are calculated: scaleX = (float)1920 / (float)(1920 - 2 * 90) = 1.10f, and scaleY = (float)1080 / (float)(1080 - 2 * 50) = 1.10f. 1.10f is then used as the scale factor for the projection image, causing the View management module to scale the projection image according to 1.1f, resulting in the projection image 1 shown on the right side of Figure 45b.

[0299] Based on the above-mentioned display device 200, according to some embodiments of the present application, a fourth screen projection display method for a display device is also provided, and the method includes but is not limited to the aforementioned steps S1301-S1303. As can be seen from the above scheme, the method can receive the projection data sent by the terminal device and detect the characteristic information of the projection data in a preset time period. If the characteristic information changes within the preset time period, the display parameters of the projection screen are maintained; otherwise, the target display parameters of the projection screen are calculated based on the invalid screen area of ​​the current screen frame, and the display is controlled to display the projection screen according to the target display parameters. Among them, the target display parameters are used to set the center point of the effective screen area in the current screen frame to the reference center point of the user interface, and to maximize the display proportion of the effective screen area in the user interface. The method can remove the invalid screen area displayed by the display device to the greatest extent when the projection data is in a stable state, and maintain the center display effect of the effective screen area, thereby improving the display effect of the projection screen while reducing system resource consumption.

[0300] The same and similar parts between the various embodiments in this application can be referenced to each other and will not be described again here.

[0301] It should be noted that the above examples are only used to illustrate a few scenarios, but any screen projection display method for a display device provided in this application is not limited to the above scenarios, and the screen projection display methods for other scenarios also need to follow the above principles, which will not be elaborated here.

[0302] Those skilled in the art will clearly understand that the technology in some embodiments of the present application can be implemented by means of software plus a necessary general-purpose hardware platform. Based on this understanding, the solutions in some embodiments of the present application, or the parts that contribute to the relevant technology, can be embodied in the form of a software product. The computer software product can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present application or certain parts of the embodiments.

[0303] Finally, it should be noted that the above embodiments are only used to illustrate the solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding solutions from the scope of the embodiments of the present application.

Claims

1. A display device, comprising: A display configured to display a user interface, wherein the user interface includes a projection screen of the terminal device; A communication device configured to establish a communication connection with a terminal device and receive projection data sent by the terminal device; the projection data includes a picture frame for forming the projection picture, and the picture frame includes a valid picture area and an invalid picture area; a memory configured to store a computer program; At least one processor is connected to the display, the communication device, and the memory, and is configured to execute the computer program to cause the display device to perform: Detecting an invalid picture area of ​​the picture frame; Calculating target display parameters of the projection screen based on the invalid screen area; the target display parameters are used to set the center point of the valid screen area to the reference center point of the user interface, and to maximize the display proportion of the valid screen area in the user interface; Control the display to display the projection screen according to the target display parameters.

2. The display device according to claim 1, wherein the at least one processor is configured to calculate target display parameters of the projection image, and is further configured to execute the computer program to cause the display device to perform: Taking the reference center point as the center of symmetry, comparing the invalid screen areas in a first direction and a second direction, where the first direction is perpendicular to the second direction; If the invalid picture area is centrally symmetrical about the reference center point, calculating a pre-processed width of the picture frame in the first direction and a pre-processed height of the picture frame in the second direction; The pre-processing width is the difference between the picture frame width and the invalid picture area width in the first direction, and the pre-processing height is the difference between the picture frame height and the invalid picture area height in the second direction; Calculating a first ratio of the picture frame width to the pre-processing width, and calculating a second ratio of the picture frame height to the pre-processing height; When the first ratio is equal to the second ratio, the first ratio is set as the scaling ratio of the projection screen in the first direction and the second direction, or the second ratio is set as the scaling ratio of the projection screen in the first direction and the second direction to generate the target display parameters.

3. The display device according to claim 2, wherein the at least one processor is configured to control the display to display the projection image according to the target display parameters, and is further configured to execute the computer program to cause the display device to perform: When the first ratio is set as the scaling ratio of the projection screen in the first direction to the second direction, the width of the projection screen is scaled according to the first ratio, and the height of the projection screen is scaled according to the first ratio; When the second ratio is set as the scaling ratio of the projection screen in the first direction and the second direction, the width of the projection screen is scaled according to the second ratio, and the height of the projection screen is scaled according to the second ratio.

4. The display device according to claim 2, wherein the at least one processor is configured to calculate the target display parameters of the projection image, and is further configured to execute the computer program to cause the display device to perform: When the first ratio is not equal to the second ratio, obtaining a minimum value of the first ratio and the second ratio; The minimum value is set as the scaling ratio of the projection screen in the first direction, and the minimum value is set as the scaling ratio of the projection screen in the second direction to generate the target display parameters.

5. The display device according to claim 4, wherein the at least one processor controls the display to display the projection image according to the target display parameters, and is further configured to execute the computer program to cause the display device to perform: Scale the width of the projection screen according to the minimum value, and scale the height of the projection screen according to the minimum value.

6. The display device according to claim 2, wherein the at least one processor is further configured to execute the computer program to cause the display device to: If the invalid picture area is not centrosymmetric about the reference center point, calculating a preprocessing width of the picture frame in the first direction and a preprocessing height of the picture frame in the second direction; The pre-processing width is the difference between the picture frame width and the invalid picture area width in the first direction, and the pre-processing height is the difference between the picture frame height and the invalid picture area height in the second direction; Calculating a target center point of the picture frame under the preprocessing width and the preprocessing height; Calculating a center offset according to the target center point and the reference center point; Setting the position parameters of the projection image based on the center offset; Calculating a first ratio of the picture frame width to the pre-processing width, and calculating a second ratio of the picture frame height to the pre-processing height; Obtaining a minimum value of the first ratio and the second ratio; The minimum value is set as the scaling ratio of the projection screen in the first direction, and the minimum value is set as the scaling ratio of the projection screen in the second direction to generate the target display parameters.

7. The display device according to claim 6, wherein the at least one processor controls the display to display the projection image according to the target display parameters, and is further configured to execute the computer program to cause the display device to perform: Moving the projection image to a position on the user interface according to the position parameter; Scale the width of the projection screen according to the minimum value, and scale the height of the projection screen according to the minimum value.

8. The display device according to claim 6, wherein the projection screen is displayed in the user interface via a projection window; the at least one processor is further configured to execute the computer program to cause the display device to: Calculating a first difference and a second difference between the target center point and the reference center point, the first difference being the difference between the target center point and the reference center point in the first direction, and the second difference being the difference between the target center point and the reference center point in the second direction; The distance between the left edge of the projection screen and the projection window is set according to the first difference, and the distance between the upper edge of the projection screen and the projection window is set according to the second difference.

9. The display device according to claim 2, wherein the at least one processor is configured to compare the invalid screen areas in the first direction with the invalid screen areas in the second direction, and is further configured to execute the computer program to cause the display device to execute: detecting a first invalid screen area and a second invalid screen area in the first direction, and detecting a third invalid screen area and a fourth invalid screen area in the second direction; Calculating a first margin of the first invalid screen area in the first direction, a second margin of the second invalid screen area in the first direction, a third margin of the third invalid screen area in the second direction, and a fourth margin of the fourth invalid screen area in the second direction; If the first margin is equal to the second margin and the third margin is equal to the fourth margin, marking the invalid screen area as being centrally symmetrical about the reference center point; If the first margin is not equal to the second margin, and / or the third margin is not equal to the fourth margin, it is marked that the invalid image area is not centrally symmetrical about the reference center point.

10. The display device according to claim 1, wherein the at least one processor is further configured to execute the computer program to cause the display device to: Detecting an invalid picture area of ​​the picture frame; Calculating target display parameters of the projection screen based on the invalid screen area; the target display parameters are used to set the center point of the valid screen area to the reference center point of the user interface, and to maximize the display proportion of the valid screen area in the user interface; within a target time period, when the invalid screen area changes, detecting whether the invalid screen area is centrally symmetrical about the reference center point; If the invalid screen area is centrally symmetrical about the reference center point, controlling the display to display the projection screen according to the target display parameters; If the invalid screen area is not centrally symmetrical about the reference center point, the target display parameters are discarded so that the display maintains the display parameters of the projection screen.

11. The display device according to claim 10, wherein after receiving the projection data of the terminal device through the communication device, the at least one processor is further configured to execute the computer program to cause the display device to perform: Obtaining a screen projection session sent by the terminal device, where the screen projection session includes a resolution of the picture frame; Detect the screen resolution of the display device; Calculating a screen display area of ​​the projection screen according to the resolution of the screen frame and the screen resolution; Initial display parameters are generated based on the screen display area, and the display is controlled to display the projection screen according to the initial display parameters.

12. The display device according to claim 10, wherein after calculating the target display parameter, the at least one processor is further configured to execute the computer program to cause the display device to: When the invalid screen area has not changed, the display is controlled to display the projection screen according to the target display parameters.

13. The display device according to claim 10, wherein the at least one processor is further configured to execute the computer program to cause the display device to: Detect the valid picture area of ​​the current picture frame; Acquire first region information of the effective picture area in the current picture frame, where the first region information includes width and height of the effective picture area and a position of the effective picture area in the picture frame; If the first area information of the current picture frame is inconsistent with that of the previous picture frame, marking that the invalid picture area is changed; If the first region information of the current picture frame is consistent with the first region information of the previous picture frame, the invalid picture region is marked as unchanged.

14. The display device according to claim 10, wherein the at least one processor is further configured to execute the computer program to cause the display device to: Acquire second region information of the invalid picture area in the current picture frame, where the second region information includes width and height of the invalid picture area and a position of the invalid picture area in the picture frame; If the second area information of the current picture frame is inconsistent with that of the previous picture frame, marking that the invalid picture area is changed; If the second region information of the current picture frame is consistent with the second region information of the previous picture frame, the invalid picture region is marked as unchanged.

15. The display device according to claim 10, wherein the at least one processor is configured to detect whether the invalid screen area is centrally symmetric about the reference center point, and is further configured to execute the computer program to cause the display device to execute: Detecting a first invalid picture area and a second invalid picture area of ​​the picture frame in a first direction, and detecting a third invalid picture area and a fourth invalid picture area of ​​the picture frame in a second direction, wherein the first direction is perpendicular to the second direction; Calculating a first margin of the first invalid screen area in the first direction, a second margin of the second invalid screen area in the first direction, a third margin of the third invalid screen area in the second direction, and a fourth margin of the fourth invalid screen area in the second direction; If the first margin is equal to the second margin and the third margin is equal to the fourth margin, marking the invalid screen area to be centrally symmetrical with respect to the reference center point; If the first margin is not equal to the second margin, and / or the third margin is not equal to the fourth margin, the invalid screen area is marked as not being centrally symmetrical about the reference center point.

16. The display device according to claim 10, wherein the at least one processor is configured to calculate the target display parameters of the projection image: Taking the reference center point as the center of symmetry, comparing the invalid screen areas in a first direction and a second direction, where the first direction is perpendicular to the second direction; If the invalid picture area is centrally symmetric with respect to the reference center point, calculating a pre-processed width of the picture frame in the first direction and a pre-processed height of the picture frame in the second direction; The pre-processing width is the difference between the picture frame width and the invalid picture area width in the first direction, and the pre-processing height is the difference between the picture frame height and the invalid picture area height in the second direction; Calculating a first ratio of the picture frame width to the pre-processing width, and calculating a second ratio of the picture frame height to the pre-processing height; When the first ratio is equal to the second ratio, the first ratio is set to the scaling ratio of the projection screen in the first direction and the second direction, or the second ratio is set to the scaling ratio of the projection screen in the first direction and the second direction to generate the target display parameters.

17. The display device according to claim 1, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: Acquire a first target area of ​​a current picture frame, where the first target area is a valid picture area of ​​the current picture frame; Calculating an area difference between the first target area and the reference area; The reference area is generated based on a valid picture area of ​​a historical picture frame, where the historical picture frame is a picture frame in the projection data that is earlier than the current picture frame; If the area difference is greater than a preset difference threshold, controlling the display to display the projection image according to the target display parameters of the current image frame, and updating the reference area through the first target area; The target display parameter is used to set the center point of the effective screen area to the reference center point of the user interface, and to maximize the display area of ​​the effective screen area in the user interface; If the area difference is less than or equal to the difference threshold, the display parameters of the projection image are maintained.

18. The display device according to claim 17, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: Detecting an invalid picture area and a valid picture area of ​​an initial picture frame in the projection data, wherein the initial picture frame is the earliest picture frame in the projection data; Recording the effective picture area of ​​the initial picture frame as the reference area; Calculating target display parameters of the initial picture frame according to the invalid picture area of ​​the initial picture frame; Control the display to display the projection screen according to the target display parameters of the initial screen frame.

19. The display device according to claim 17, wherein the area difference comprises a height difference and a width difference, and the at least one processor is further configured to execute the computer instructions to cause the display device to: Obtaining a first width and height dimension and a second width and height dimension, wherein the first width and height dimension includes a first height and a first width of the first target area, and the second width and height dimension includes a second height and a second width of the reference area; A height difference between the first height and the second height is calculated, and a width difference between the first width and the second width is calculated.

20. The display device according to claim 19, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: If the height difference is greater than the difference threshold, and / or the width difference is greater than the difference threshold, controlling the display to display the projection image according to the target display parameters of the current image frame; If the height difference is less than or equal to the difference threshold, and the width difference is less than or equal to the difference threshold, the target display parameters of the current picture frame are discarded to maintain the display parameters of the projection picture.

21. The display device according to claim 17, wherein the at least one processor is configured to execute the step of acquiring the first target area of ​​the current picture frame, and further configured to execute the computer instructions to cause the display device to execute: When receiving the initial picture frame of the projection data, setting an acquisition cycle, the initial picture frame is the earliest picture frame in the projection data; The step of acquiring the first target area of ​​the current picture frame is performed according to the acquisition cycle.

22. The display device according to claim 21, wherein if the regional difference value is less than or equal to the difference threshold, the at least one processor is further configured to execute the computer instructions to cause the display device to: Detecting a picture change state of a valid picture area in the picture frame within a preset time period; If the picture change status of the valid picture area in the picture frame is unchanged, controlling the display to display the projection picture according to the target display parameters of the current picture frame, and updating the reference area through the first target area; The preset time period is greater than the acquisition period.

23. The display device according to claim 17, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: detecting a second target area of ​​the current picture frame, where the second target area is an invalid picture area of ​​the picture frame; Taking the reference center point as the center of symmetry, comparing the second target area in the first direction and the second direction, the first direction being perpendicular to the second direction; If the second target area is centrally symmetric with respect to the reference center point, calculating a pre-processed width of the picture frame in the first direction and a pre-processed height of the picture frame in the second direction; The pre-processing width is the difference between the picture frame width in the first direction and the width of the second target area, and the pre-processing height is the difference between the picture frame height in the second direction and the height of the second target area; Calculating a first ratio of the picture frame width to the pre-processing width, and calculating a second ratio of the picture frame height to the pre-processing height; When the first ratio is equal to the second ratio, the first ratio is set to the scaling ratio of the projection screen in the first direction and the second direction, or the second ratio is set to the scaling ratio of the projection screen in the first direction and the second direction to generate the target display parameters of the current screen frame.

24. The display device according to claim 23, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: When the first ratio is not equal to the second ratio, obtaining a minimum value of the first ratio and the second ratio; The minimum value is set as the scaling ratio of the projection screen in the first direction, and the minimum value is set as the scaling ratio of the projection screen in the second direction to generate the target display parameters of the current screen frame.

25. The display device according to claim 23, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to: If the second target area is not centrally symmetrical about the reference center point, calculating a pre-processed width of the picture frame in the first direction and a pre-processed height of the picture frame in the second direction; The pre-processing width is the difference between the picture frame width and the second target area width in the first direction, and the pre-processing height is the difference between the picture frame height and the second target area height in the second direction; Calculating a target center point of the picture frame under the preprocessing width and the preprocessing height; Calculating a center offset according to the target center point and the reference center point; Setting a position parameter of the projection screen based on the center offset, wherein the position parameter is used to set a display position of the projection screen; Calculating a first ratio of the picture frame width to the pre-processing width, and calculating a second ratio of the picture frame height to the pre-processing height; Obtaining a minimum value of the first ratio and the second ratio; The minimum value is set as the scaling ratio of the projection screen in the first direction, and the minimum value is set as the scaling ratio of the projection screen in the second direction to generate the target display parameters of the current screen frame.

26. The display device according to claim 1, wherein the at least one processor is further configured to execute the computer program to cause the display device to: Detecting characteristic information of the projection data within a preset time period; If the characteristic information changes, maintaining the display parameters of the projection screen; If the characteristic information has not changed, calculating the target display parameters of the projection image according to the invalid image area of ​​the current image frame, and controlling the display to display the projection image according to the target display parameters; The target display parameter is used to set the center point of the effective picture area in the current picture frame to the reference center point of the user interface, and to maximize the display proportion of the effective picture area in the user interface. 27 . The display device according to claim 26 , wherein the feature information comprises at least one of the valid picture area, the invalid picture area, resolution, timestamp, key frame, picture frame size, and video identification.

28. The display device according to claim 26, wherein when the duration between the current image frame and the initial image frame is less than or equal to the preset time period, the at least one processor detects characteristic information of the projection data in the preset time period, and is further configured to execute the computer program to cause the display device to: Taking the time of the initial picture frame as the timing starting point, obtaining the first picture frame set of the preset time period, wherein the initial picture frame is the earliest picture frame in the projection data; parsing feature information of the picture frames in the first picture frame set; If the feature information of the picture frames in the first picture frame set is inconsistent, marking that the feature information has changed within the preset time period; If the feature information of the frames in the first frame set is consistent, it is marked that the feature information has not changed within the preset time period.

29. The display device according to claim 28, wherein when the time difference between the current image frame and the initial image frame is greater than the preset time period, the at least one processor detects characteristic information of the projection data in the preset time period, and is further configured to execute the computer program to cause the display device to perform: Taking the time of the current picture frame as the timing end point, obtaining a second picture frame set of the preset time period; parsing feature information of the picture frames in the second picture frame set; If the feature information of the picture frames in the second picture frame set is inconsistent, marking that the feature information has changed within the preset time period; If the feature information of the frames in the second frame set is consistent, it is marked that the feature information has not changed within the preset time period.

30. The display device according to claim 26, wherein the at least one processor is further configured to execute the computer program to cause the display device to: When receiving the initial picture frame of the projection data, setting an acquisition period; the initial picture frame is the earliest picture frame in the projection data, and the acquisition period is equal to the preset time period; Acquire picture frames within the preset time period according to the acquisition cycle to generate a third picture frame set; parsing feature information of the picture frames in the third picture frame set; If the feature information of the picture frames in the third picture frame set is inconsistent, marking that the feature information has changed within the preset time period; If the feature information of the frames in the third frame set is consistent, it is marked that the feature information has not changed within the preset time period.

31. The display device according to claim 26, wherein the at least one processor is further configured to execute the computer program to cause the display device to: Obtaining a screen projection session sent by the terminal device, where the screen projection session includes a resolution of the picture frame; Detect the screen resolution of the display device; Calculating initial display parameters of the projection image according to the resolution of the image frame and the screen resolution; The initial display parameters are used to set the center point of the picture frame at the center point of the user interface, make the aspect ratio of the projected picture equal to the aspect ratio of the picture frame, and maximize the proportion of the picture frame in the user interface; Control the display to display the projection screen according to the initial display parameters.

32. The display device according to claim 26, wherein the at least one processor is configured to calculate target display parameters of the projection image, and is further configured to execute the computer program to cause the display device to: Taking the reference center point as the center of symmetry, comparing the invalid image areas of the current image frame in a first direction and a second direction, where the first direction is perpendicular to the second direction; If the invalid picture area is centrally symmetric with respect to the reference center point, calculating a pre-processed width of the picture frame in the first direction and a pre-processed height of the picture frame in the second direction; The pre-processing width is the difference between the picture frame width and the invalid picture area width in the first direction, and the pre-processing height is the difference between the picture frame height and the invalid picture area height in the second direction; Calculating a first ratio of the picture frame width to the pre-processing width, and calculating a second ratio of the picture frame height to the pre-processing height; When the first ratio is equal to the second ratio, the first ratio is set as the scaling ratio of the projection screen in the first direction and the second direction, or the second ratio is set as the scaling ratio of the projection screen in the first direction and the second direction to generate the target display parameters.

33. The display device according to claim 16 or 32, wherein the at least one processor is further configured to execute the computer program to cause the display device to: When the first ratio is not equal to the second ratio, obtaining a minimum value of the first ratio and the second ratio; The minimum value is set as the scaling ratio of the projection screen in the first direction, and the minimum value is set as the scaling ratio of the projection screen in the second direction to generate the target display parameters.

34. The display device according to claim 16 or 32, wherein the at least one processor is further configured to execute the computer program to cause the display device to: If the invalid picture area is not centrally symmetrical about the reference center point, calculating a pre-processed width of the picture frame in the first direction and a pre-processed height of the picture frame in the second direction; The pre-processing width is the difference between the picture frame width and the invalid picture area width in the first direction, and the pre-processing height is the difference between the picture frame height and the invalid picture area height in the second direction; Calculating a target center point of the picture frame under the preprocessing width and the preprocessing height; Calculating a center offset according to the target center point and the reference center point; Setting a position parameter of the projection screen based on the center offset, wherein the position parameter is used to set a display position of the projection screen; Calculating a first ratio of the picture frame width to the pre-processing width, and calculating a second ratio of the picture frame height to the pre-processing height; Obtaining a minimum value of the first ratio and the second ratio; The minimum value is set as the scaling ratio of the projection screen in the first direction, and the minimum value is set as the scaling ratio of the projection screen in the second direction to generate the target display parameters.

35. A screen projection display method for a display device, the display device comprising a display, a communication device, and at least one processor, the display being configured to display a user interface, the user interface comprising a projection screen of a terminal device; the communication device being configured to establish a communication connection with the terminal device and receive projection data sent by the terminal device, the projection data comprising a frame for forming the projection screen, the frame comprising a valid frame area and an invalid frame area; the method comprising: Detecting an invalid picture area of ​​the picture frame; Calculating target display parameters of the projection screen according to the invalid screen area; The target display parameter is used to set the center point of the effective screen area to the reference center point of the user interface, and to maximize the display area of ​​the effective screen area in the user interface; Control the display to display the projection screen according to the target display parameters.

Images