Display equipment, terminal equipment and equipment projection screen anti-control method

Abstract

The embodiment of the invention provides a display device, a terminal device and a device projection screen anti-control method, and the method comprises the steps: obtaining a rotation state and a projection screen pixel value of a display through responding to a control instruction input by a user, dividing a picture display region according to a preset projection screen size based on the rotation state, and analyzing a click coordinate of the control instruction in the picture display area. And calculating a normalized coordinate according to the screen projection pixel value, a preset screen projection size and the click coordinate, generating an anti-control instruction according to the normalized coordinate, sending the anti-control instruction to the terminal device, and controlling the display to display a screen projection picture after the terminal device responds to the anti-control instruction. The click coordinate is converted into the normalized coordinate according to the pixel value and the screen projection size during screen projection of the display device, the synchronization of the click positions of the display device and the terminal device is improved, the anti-control instruction is generated according to the normalized coordinate, and the accuracy of the anti-control process is improved.

Description

Technical Field

[0001] The present application relates to the field of device control technology, and in particular to a display device, a terminal device, and a device reverse control method. Background Art

[0002] Wireless display standard protocols (for example, Miracast protocol) can connect two devices and display the screen of one device on the screen of the other device. For example, a user can use the wireless display standard protocol to project the user interface on the terminal device to the screen of the display device, where the terminal device is the screen projection initiator and the display device is the screen projection receiver. Projecting the screen of the terminal device to a display device with a larger screen through the wireless display standard protocol can improve the user's viewing experience.

[0003] The wireless display standard protocol includes a User Input Back Channel (UIBC) feature, which allows the display device to send back-control commands to the terminal device to control the terminal device's interaction. However, after the back-control commands are sent to the terminal device, the accuracy of the back-control function may be reduced due to the difference in display mode between the terminal device and the display device. Summary of the Invention

[0004] The present application provides a display device, a terminal device, and a device screen projection reverse control method to solve the problem of poor reverse control accuracy.

[0005] In a first aspect, some embodiments of the present application provide a display device, including a display, a rotating assembly, and a controller, wherein the display is configured to display a projection image sent by a terminal device, the rotating assembly is connected to the display and configured to adjust the rotation state of the display, and the controller is configured to:

[0006] In response to a control instruction input by a user, obtaining a rotation state of the display and a projection pixel value, wherein the projection pixel value is a video stream pixel value of the projection screen;

[0007] Based on the rotation state, the screen display area is divided according to the preset projection screen size, and the screen display area is the area where the current projection screen is displayed;

[0008] Analyzing the click coordinates of the control instruction within the screen display area;

[0009] Calculating normalized coordinates according to the projection screen pixel value, the preset projection screen size, and the click coordinates, and generating a reverse control instruction according to the normalized coordinates;

[0010] The reverse control instruction is sent to the terminal device, and the display is controlled to display the projection screen after the terminal device responds to the reverse control instruction.

[0011] In a second aspect, some embodiments of the present application further provide a terminal device, comprising a display module, a communication module, and a processor, wherein the display module is configured to display a terminal screen, the communication module is configured to establish a communication connection with the display device, and the processor is configured to:

[0012] In response to a screen projection instruction input by a user, sending a projection image to the display device, so that the display device displays the projection image in a screen display area divided according to a preset projection size based on the rotation state of the display;

[0013] Obtaining a counter-control instruction fed back by the display device, and parsing the normalized coordinates in the counter-control instruction, the normalized coordinates being calculated based on the click coordinates and a preset projection screen size and projection screen pixel value, the click coordinates being parsed by the display device in response to a control instruction input by the user, and the projection screen pixel value being a pixel value of a video stream of the projection screen;

[0014] A projection screen is generated based on the terminal screen and the normalized coordinates, so that the display device displays the projection screen in response to the reverse control instruction.

[0015] In a third aspect, some embodiments of the present application provide a device screen projection reverse control method, which is applied to the display device described in the first aspect, wherein the display device includes a display, a rotating component, and a controller, wherein the display is configured to display a projection image sent by a terminal device, and the rotating component is connected to the display and is configured to adjust the rotation state of the display; the method includes:

[0016] In response to a control instruction input by a user, obtaining a rotation state of the display and a projection pixel value, wherein the projection pixel value is a video stream pixel value of the projection screen;

[0017] Based on the rotation state, the screen display area is divided according to the preset projection screen size, and the screen display area is the area where the current projection screen is displayed;

[0018] Analyzing the click coordinates of the control instruction within the screen display area;

[0019] Calculating normalized coordinates according to the projection screen pixel value, the preset projection screen size, and the click coordinates, and generating a reverse control instruction according to the normalized coordinates;

[0020] The reverse control instruction is sent to the terminal device, and the display is controlled to display the projection screen after the terminal device responds to the reverse control instruction.

[0021] In a fourth aspect, some embodiments of the present application provide a device screen projection reverse control method, which is applied to the terminal device described in the second aspect, wherein the terminal device includes a display module, a communication module, and a processor, wherein the display module is configured to display the terminal screen, and the communication module is configured to establish a communication connection with the display device, and the method includes:

[0022] In response to a control instruction input by a user, obtaining a rotation state of the display and a projection pixel value, wherein the projection pixel value is a video stream pixel value of the projection screen;

[0023] Based on the rotation state, the screen display area is divided according to the preset projection screen size, and the screen display area is the area where the current projection screen is displayed;

[0024] Analyzing the click coordinates of the control instruction within the screen display area;

[0025] Calculating normalized coordinates according to the projection screen pixel value, the preset projection screen size, and the click coordinates, and generating a reverse control instruction according to the normalized coordinates;

[0026] The reverse control instruction is sent to the terminal device, and the display is controlled to display the projection screen after the terminal device responds to the reverse control instruction.

[0027] It can be seen from the above technical solutions that the embodiments of the present application provide a display device, a terminal device, and a device screen projection reverse control method, which obtains the rotation state of the display and the projection pixel value by responding to the control command input by the user, divides the screen display area according to the preset projection size based on the rotation state, and parses the click coordinates of the control command in the screen display area. Normalized coordinates are calculated based on the projection pixel value, the preset projection size, and the click coordinates, and a reverse control command is generated based on the normalized coordinates. The reverse control command is sent to the terminal device, and the display is controlled to display the projection screen after the terminal device responds to the reverse control command. The present application converts the click coordinates into normalized coordinates based on the pixel value and projection size of the display device when projecting the screen, improves the synchronization of the click position of the display device and the terminal device, and generates a reverse control command based on the normalized coordinates to improve the accuracy of the reverse control process. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solution of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0029] Figure 1 This is a usage scenario of the display device in the embodiment of the present application;

[0030] Figure 2 This is a hardware configuration block diagram of the control device in the embodiment of the present application;

[0031] Figure 3 This is a hardware configuration diagram of the display device in the embodiment of the present application;

[0032] Figure 4 A software configuration diagram of the display device in the embodiment of the present application;

[0033] Figure 5 A schematic diagram of mirror screen projection provided in some embodiments of the present application;

[0034] Figure 6 Schematic diagram of push screen projection provided in some embodiments of this application;

[0035] Figure 7 This is a schematic structural diagram of a display device with a rotating component in an embodiment of the present application;

[0036] Figure 8 This is a schematic diagram of the structure of the rotating assembly in the embodiment of the present application;

[0037] Figure 9 This is a schematic diagram of the process of the display device performing the reverse control function in an embodiment of the present application;

[0038] Figure 10 This is a schematic diagram of dividing the screen display area according to the projection screen size in an embodiment of the present application;

[0039] Figure 11 Schematic diagram of the screen display area of ​​the display in the vertical state in an embodiment of the present application;

[0040] Figure 12 This is a schematic diagram of detecting multiple click coordinates in an embodiment of the present application;

[0041] Figure 13 This is a flowchart of calculating normalized coordinates based on a user's sliding action in an embodiment of the present application;

[0042] Figure 14 This is a screen projection display diagram of the horizontal screen split in the embodiment of the present application;

[0043] Figure 15 This is a screen projection display diagram of the split screen in vertical mode in an embodiment of the present application;

[0044] Figure 16 This is a distribution display diagram of non-content screens in an embodiment of the present application;

[0045] Figure 17 This is a screen display diagram of an enlarged content screen in an embodiment of the present application. DETAILED DESCRIPTION

[0046] The following embodiments are described in detail, with examples illustrated in the accompanying drawings. When the following description refers to the drawings, identical numbers in different figures represent identical or similar elements unless otherwise indicated. The embodiments described in the following embodiments are not intended to represent all possible implementations consistent with the present application. They are merely examples of systems and methods consistent with certain aspects of the present application, as detailed in the claims.

[0047] It should be noted that the brief descriptions of terms in this application are only for the purpose of facilitating the understanding of the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their ordinary and usual meanings.

[0048] In the specification and claims of this application and the accompanying drawings, the terms "first," "second," "third," etc. are used to distinguish similar or similar objects or entities, and are not necessarily intended to limit a particular order or sequence, unless otherwise noted. It should be understood that the terms used in this manner are interchangeable under appropriate circumstances.

[0049] The terms "comprise," "include," and "have," and any variations thereof, are intended to cover but not exclude inclusion; for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0050] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functionality associated with that element.

[0051] Figure 1 This is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG, a user can operate the display device 200 through touch operation, the mobile terminal 300 and the control device 100. The control device 100 may be a remote controller, a stylus pen, or the like.

[0052] In some embodiments, the mobile terminal 300 can install software applications on the display device 200 and connect to the display device 200 through a network communication protocol to achieve one-to-one control operations and data communication. It is also possible to transmit audio and video content displayed on the mobile terminal 300 to the display device 200 for synchronous display.

[0053] like Figure 1 As shown in FIG, the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0054] In addition to providing broadcast reception television functions, the display device 200 may also provide intelligent network television functions with computer support functions, including but not limited to network television, smart TV, Internet Protocol television (IPTV), etc.

[0055] Figure 2 Some embodiments of this application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0056] In some embodiments, the display device 200 includes at least one of a tuner 210 , a communicator 220 , a detector 230 , a device interface 240 , a controller 250 , a display 260 , an audio output interface 270 , a memory, a power supply, and a user interface.

[0057] In some embodiments, detector 230 is used to collect signals from the external environment or external interactions. For example, detector 230 may include a light receiver, such as a sensor for collecting ambient light intensity; or an image collector, such as a camera, for collecting external environmental scenes, user attributes, or user interaction gestures; or a sound collector, such as a microphone, for receiving external sounds.

[0058] In some embodiments, the display 260 includes a display screen component for presenting images, and a driving component for driving image display, a component for receiving image signals output from a controller, and a component for displaying video content, image content, and a menu control interface, as well as a user control UI interface, etc.

[0059] In some embodiments, the communicator 220 is a component for communicating with an external device or server 400 according to various communication protocol types. The display device 200 may be provided with multiple communicators 220 depending on the supported communication methods. For example, if the display device 200 supports wireless network communication, the display device 200 may be provided with a communicator 220 including WiFi functionality. If the display device 200 supports Bluetooth connection communication, the display device 200 may also be provided with a communicator 220 including Bluetooth functionality.

[0060] The communicator 220 can establish a communication connection between the display device 200 and an external device or server 400 via a wireless or wired connection. A wired connection can connect the display device 200 to an external device via a data cable, an interface, or other components. A wireless connection can connect the display device 200 to an external device via a wireless signal or wireless network. The display device 200 can establish a connection with an external device directly or indirectly through a gateway, router, or connection device.

[0061] In some embodiments, the display device 200 may also support establishing communication connections with multiple external devices at the same time. Multiple external devices may be connected through the same. For example, the external device is a terminal device 500 such as a mobile phone or a tablet computer. The first terminal device 510 and the second terminal device 520 may establish a communication connection with the display device 200 by accessing the wireless local area network where the display device 200 is located. Multiple external devices may also be connected to the display device 200 through different types of connection methods. For example, the first terminal device 510 is connected to the display device 200 via a wireless local area network; the second terminal device 520 is connected to the display device 200 via Bluetooth.

[0062] To achieve communication between the display device 200 and the terminal device 500, the display device 200 and the terminal device 500 are each equipped with a communicator supporting the same type of communication. For example, if the display device 200 is equipped with a communicator 220 including a WiFi module, the terminal device 500 should also be equipped with a communicator including a WiFi module. Furthermore, while the same type of communicator is provided, a specific communication transmission protocol is required for data transmission between the display device 200 and the terminal device 500. Examples include WiFi, Bluetooth, ZigBee, and NFC protocols.

[0063] In some embodiments, controller 250 includes a processor, a video processor, an audio processor, a graphics processor, RAM, ROM, and first to nth interfaces for input / output. Controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. Controller 250 controls the overall operation of display device 200.

[0064] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0065] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0066] In some embodiments, the user interface 280 is an interface that can be used to receive control input.

[0067] Figure 3 Some embodiments of this application provide Figure 1 The hardware configuration diagram of the control device in the figure. Figure 3As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0068] The control device 100 is configured to control the display device 200 , and can receive user input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200 .

[0069] In some embodiments, the control device 100 may be a smart device. For example, the control device 100 may be installed with various applications for controlling the display device 200 according to user needs.

[0070] In some embodiments, as Figure 1 As shown, the mobile terminal 300 or other intelligent electronic devices can play a similar function as the control device 100 after installing the application for controlling the display device 200 .

[0071] The controller 110 includes a processor 112, RAM 113, ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between internal components and external and internal data processing functions.

[0072] Under the control of the controller 110, the communication interface 130 communicates control signals and data signals with the display device 200. The communication interface 130 may include at least one of a WiFi chip 131, a Bluetooth module 132, an NFC module 133, or other near field communication modules.

[0073] The user input / output interface 140 includes at least one of a microphone 141 , a touch panel 142 , a sensor 143 , a button 144 and other input interfaces.

[0074] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with the communication interface 130, such as a WiFi, Bluetooth, or NFC module, to encode user input commands via the WiFi protocol, Bluetooth protocol, or NFC protocol and transmit them to the display device 200.

[0075] The memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0076] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0077] like Figure 4 In some embodiments, the system is divided into four layers, from top to bottom: the application layer (referred to as the "application layer"), the application framework layer (referred to as the "framework layer"), the Android runtime and system library layer (referred to as the "system runtime library layer"), and the kernel layer.

[0078] like Figure 4 As shown, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: an activity manager for interacting with all activities running in the system; a location manager for providing system services or applications with access to the system location service; a package manager for retrieving various information related to the application packages currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0079] In some embodiments, the activity manager is used to manage the lifecycle of each application and common navigation back functions, such as controlling application exit, opening, and back. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, taking screenshots, and controlling changes in display windows, such as shrinking, shaking, or distorting the display window.

[0080] In some embodiments, the system runtime layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system will run the C / C++ library contained in the system runtime layer to implement the functions to be implemented by the framework layer.

[0081] In some embodiments, the kernel layer is a layer between hardware and software. Figure 4As shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0082] After establishing a communication connection with the terminal device 500, the display device 200 can establish a screen projection connection channel and transmit projection data based on the projection connection so that the display device 200 can display the projection image of the terminal device 500. In some embodiments, to establish a screen projection connection relationship, the user can issue a screen projection connection request through the terminal device 500 and send the screen projection connection request to the display device 200 via the WiFi network. The display device 200 then completes the transmission protocol configuration based on the screen projection connection request, thereby establishing a transmission channel for projection data between the display device 200 and the terminal device 500.

[0083] The display device 200 can also realize the screen projection connection with the terminal device 500 through different connection methods. For example, when the display device 200 and the terminal device 500 are connected to the same wireless local area network, a screen projection connection can be established based on the WiFi network. For another example, when both the display device 200 and the terminal device 500 are provided with a near field communication (NFC) component, a screen projection connection relationship can be established through the NFC component. Obviously, other wired or wireless connection methods can also be used between the display device 200 and the terminal device 500 to establish a screen projection connection relationship, such as RF radio frequency connection, infrared connection, cellular network and other connection methods that those skilled in the art can associate with the connection methods provided in the above embodiments.

[0084] After establishing the screen projection connection, the display device 200 can receive the screen projection related data from the terminal device 500 through the screen projection data channel, thereby playing the screen projection related content on the display device 200. Under different screen projection data transmission protocols, the screen projection data obtained by the display device 200 is in different forms. For example, the display device 200 and the terminal device 500 can establish a screen projection connection through the Digital Living Network Alliance (DLNA) protocol, Miracast protocol, AirPlay push protocol, WiDi screen projection, NFC mirror screen projection, LeTV screen projection and other protocols.

[0085] The Miracast protocol is a wireless display standard developed by the Wi-Fi Alliance and based on Wi-Fi Direct. The Miracast protocol is compatible with Android devices, allowing Android devices to share video images through Miracast. Smart devices running Android 5.0 and higher have the Miracast protocol built-in, allowing for direct wireless screen projection without installing any software, using the Wi-Fi P2P function between the projecting device and the receiving device's Wi-Fi network card. Miracast also supports simultaneous audio and video transmission.

[0086] In some embodiments, after the display device 200 establishes a screen projection connection with the terminal device 500, the projection data can be transmitted through the mirror mode and the push mode. Among them, the mirror mode means that the display device 200 synchronously displays the screen content displayed by the terminal device 500. For example, Figure 5 As shown, the display device 200 displays the main interface of the terminal device 500, and when the user performs an interactive operation on the terminal device 500, resulting in a change in the content of the main interface, the content displayed by the display device 200 also changes. To this end, the terminal device 500 can generate a video-based projection data stream based on the displayed content and send the projection data stream to the display device 200 for display.

[0087] Push mode means that the display device 200 accesses the network link pushed by the terminal device 500 and obtains the corresponding projection screen. Figure 6 As shown, when the terminal device 500 plays video media, a screen projection control is provided in the upper right corner of the playback interface. After the user clicks the screen projection control and selects the display device 200 as the device for displaying the screen projection, the terminal device 500 can send the URL address of the currently playing video media to the display device 200. After receiving the URL address, the display device 200 accesses the URL address to obtain the video media data and plays the video media.

[0088] Under the Miracast screen projection connection protocol, the display device 200 can directly obtain the video data stream from the terminal device 500. The video data stream can be the video data stream corresponding to the final display screen of the display unit of the terminal device 500, that is, the video data stream includes the media content screen, UI interface and other video screen content that can be displayed in the display unit of the terminal device 500. The video data stream can also be only the video data that can be obtained in the terminal device 500, such as the media video data being played. At this time, the terminal device 500 can serve as a signal source for the display device 200, and the display device 200 can continuously obtain video data through the screen projection data channel, and display it in real time on the display 260, so as to achieve the effect of synchronous display with the terminal device 500.

[0089] It should be noted that the above-mentioned screen projection protocol can not only obtain the video data stream in the terminal device 500, but also obtain the audio data stream in the terminal device 500, so that the video data and audio data can be sent to the display device 200 synchronously, so that the display device 200 can play the video and audio synchronously.

[0090] The terminal device 500 can obtain different projection data according to different projection protocols. For example, for system applications or direct projection at the system level, the terminal device 500 can use the Miracast projection protocol to establish a projection data channel. Since the Miracast projection protocol requires system-level application permissions, the Miracast projection protocol is only applicable to system applications, or the display device 200 and the terminal device 500 of the same manufacturer directly establish a projection connection through the built-in rules of the operating system.

[0091] The Miracast screen projection protocol includes a reverse control function. The user can send a control instruction to the display device 200. The display device 200 can generate a reverse control instruction based on the control instruction and send the reverse control instruction to the terminal device 500 to control the terminal device to perform the interaction.

[0092] In some embodiments, the display device 200 includes a display 260 for displaying the projection screen sent by the terminal device 500. The display 260 can be rotated to display the projection screen in a horizontal or vertical position. To this end, the display device 200 also has a built-in or external rotation component 290, which is used to adjust the rotation state of the display 260, for example, Figure 7 As shown, the rotating assembly 290 can drive the display 260 to rotate to adjust the tilt angle of the display 260. The tilt angle adjustment helps to enable the display device 200 to present a display image at different angles, for example, to enable the display 260 to display in a horizontal or vertical position.

[0093] It should be noted that the landscape or portrait display in this embodiment does not refer to the landscape or portrait display state of the display 260, but rather to the screen placement of the display 260 after rotation by the rotation assembly 290. For example, when the display 260 is rotated 90° clockwise or counterclockwise around the screen, the display 260 rotates from the default landscape orientation to the portrait orientation, and the corresponding display image is adaptively scaled or filled according to the rotation angle.

[0094] In order to realize the rotation of the display 260, as shown in FIG. Figure 8As shown, in some exemplary embodiments, the rotating assembly 290 may include a bracket 291 or a steering assembly 292. The display 260 may be fixed to the bracket 291, and the steering assembly may include a power assembly, such as a drive motor, a transmission component, etc., to provide rotational power to the steering assembly 292. The transmission component may be a drive shaft, a transmission belt, etc., which is used to transmit the power generated by the drive motor to the entire steering assembly 292. The bracket 291 is fixed to the steering assembly 292 and drives the display 260 to rotate along the corresponding direction according to the direction of the drive motor. In order to achieve steering control of the display 260, the drive motor may be a servo motor, a stepper motor, etc. that supports steering control. For example, when the drive motor rotates forward, the steering assembly 292 drives the display 260 to rotate clockwise, and when the drive motor rotates backward, the steering assembly 292 drives the display 260 to rotate counterclockwise.

[0095] During the rotation of the display 260, the gravity generated by the display 260 itself will affect the accuracy of the tilt adjustment of the display 260. For this reason, the drive motor also supports a self-locking function, that is, the rotation shaft angle can be locked after the rotation is completed to maintain the display 260 at the current rotation angle, reduce the influence of the gravity of the display 260, and thus improve the accuracy of the tilt adjustment.

[0096] In some embodiments, the user can Figure 7 The manual rotation method shown adjusts the rotation state of the display 260. However, this adjustment method requires the user to be relatively close to the display device 200, resulting in poor portability of user adjustment. Therefore, when the user is not close enough to the display device 200 to manually adjust the rotation state of the display 260, the user can also send a control instruction for adjusting the rotation state of the display 260 to the controller 250 of the display device 200 via the control device 100.

[0097] The steering assembly 292 can also establish a communication connection with the controller 250. Specifically, the controller 250 can drive the steering assembly 292 to start, pause, stop, and reverse according to control commands sent by the user. For example, if the user wishes to change the tilt angle of the display 260 by rotating it clockwise, the user can generate a control command through the control device 100 or through interactive actions, and send it to the controller 250. After receiving the control command, the controller 250 can control the steering assembly 292 to rotate forward according to the control command, thereby causing the display 260 to rotate clockwise to adjust the tilt angle.

[0098] When the display 260 rotates to a display mode different from that of the terminal device 500, the reverse control instruction generated by the control instruction will reduce the accuracy of the reverse control function due to the difference in display mode between the terminal device 500 and the display device, causing the terminal device 500 to perform incorrect interaction.

[0099] To this end, in some embodiments, the controller 250 is configured to execute a device screen projection reverse control method to solve the problem of low accuracy of the reverse control function caused by the different display modes of the terminal device 500 and the display device 200. Figure 9 This is a flow chart of a device screen projection reverse control method provided in an embodiment of the present application. Figure 9 , the method comprising:

[0100] S100: In response to a control instruction input by a user, obtaining a rotation state of the display and a projection pixel value.

[0101] In this embodiment, the display device 200 is a screen projection receiving end, which is used to receive the projection screen sent by the terminal device 500 that has established a screen projection connection with the display device 200. The projection screen may include the application interface displayed by the terminal device 500, the media playback screen, etc. The user can use the control device 100 to send control instructions to the display device 200 based on the projection screen. For example, if an application icon A is displayed in the projection screen, when the user wants to run the application, he can use the control device 100 to select the application icon A in the display device 200 and click the "OK" button to generate a control instruction in response to the application icon A.

[0102] In some embodiments, the user can also manually click on the area of ​​the application icon A through the touch function to generate a control instruction. To implement the touch function, the display 260 can also be provided with a touch component, which is used to detect the touch action input by the user through the display 260, such as a click action, a pinch-to-zoom action, a slide action, etc., and generate a corresponding control instruction based on the touch action. For example, when a click action is detected, a control instruction is generated based on the user's click position.

[0103] The controller 250 can obtain the rotation status of the display 260 in response to the control instruction. In addition to horizontal screen display and vertical screen display, the user can also customize the rotation angle of the display 260 according to different viewing needs. With 0° as the initial placement angle of the display 260, the display 260 can be rotated 90° clockwise, 90° counterclockwise, or 180° counterclockwise, etc.

[0104] The controller 250 can also obtain the projection pixel value of the projection screen sent by the terminal device 500 while obtaining the rotation status of the display 260. The projection pixel value is the video stream pixel value of the projection screen. The video stream is the video stream formed when the terminal device 500 sends the terminal screen to the display device 200. For example, if the terminal device 500 is a smart phone, the smart phone can be placed in landscape or portrait mode. However, the video stream of the smart phone is always a horizontal video stream. When the user uses the terminal device 500 in portrait mode, the terminal screen will display part of the video stream screen. This part of the video stream screen is a video stream screen with display content. Since the video stream is a horizontal video stream, the remaining video stream screen is located outside the terminal screen, and the terminal device 500 does not display the remaining video stream screen.

[0105] The projection pixel value includes the height pixel value and the width pixel value. The width pixel value and the height pixel value are related to the clarity of the video. For example, when the clarity of the projection screen is standard definition, the projection pixel value is 1280×720, that is, the height pixel value is 720px and the width pixel value is 1280px; when the clarity of the projection screen is ultra-high definition, the projection pixel value is 1920×1080, that is, the height pixel value is 1080px and the width pixel value is 1920px.

[0106] S200: Based on the rotation state, divide the screen display area according to a preset projection size.

[0107] The screen display area is the area where the current projection screen is displayed. The display device 200 can divide the screen display area according to the preset projection screen size and display the projection screen within the screen display area. This embodiment uses the display 260 in the landscape mode and the portrait mode as examples. When the display 260 is in the landscape mode, the controller 250 can divide the screen display area according to the screen size of the display 260, that is, the display 260 is displayed in full screen in the landscape mode.

[0108] When the preset projection screen size is the screen size of the display 260, since the screen sizes of the display device 200 and the terminal device 500 are different, the display device 200 needs to enlarge the projection screen uploaded by the terminal device 500 before displaying it on the display 260. To this end, the controller 250 can obtain the screen pixel value of the display 260 and calculate the projection screen size based on the ratio of the screen pixel value and the projection screen pixel value. The screen pixel value is the number of pixels in the horizontal and vertical directions of the display 260 screen. For example, when the screen pixel value is 1366×768, the screen pixel value has 1366 pixels in the horizontal direction and 768 pixels in the vertical direction.

[0109] It should be noted that when the controller 250 performs subsequent size ratio calculations, pixel values ​​must be used as a unified unit for calculation. Therefore, in this embodiment, the projection size, screen size and other dimensional data are all in pixel values, rather than the actual measured size of the projection screen on the display 260 and the actual screen size of the display 260, so as to reduce calculation errors caused by inconsistent units.

[0110] like Figure 10 As shown, the projection screen size is the display size of the terminal screen in the display 260. The projection screen size includes the projection screen height and the projection screen width. The controller 250 can calculate the ratio of the width pixel value and the height pixel value respectively. For example, the projection screen height is calculated according to the height pixel value and the screen height value, and the projection screen width is calculated according to the width pixel value and the screen width value. The zoom ratio of the video stream is obtained according to the projection screen height and the projection screen width to obtain the projection screen size. The controller 250 can divide the screen display area in the display 260 based on the projection screen size. In order to display the projection screen in the center, the controller 250 can detect the center point coordinates of the display 260, and use the center point coordinates as the center of the screen display area. The screen display area is divided according to the projection screen size and the projection screen is displayed within the screen display area.

[0111] Figure 11 This is a schematic diagram of the projection screen in the vertical screen state of the display in the embodiment of this application. Figure 11 When the display 260 is in portrait mode, the screen pixel value of the display 260 will change according to the portrait mode. For example, when the display 260 is in landscape mode, the height pixel value of the screen pixel value is 768px, and the width pixel value of the screen pixel value is 1366px. When the display 260 is rotated to portrait mode, the height pixel value of the screen pixel value is 1366px, and the width pixel value of the screen pixel value is 768px. To this end, the controller 250 can recalculate the projection screen size according to the rotation state of the display 260 after rotation, and re-divide the image display area according to the calculated projection screen size to reduce the response deviation caused by the rotation of the display 260 screen and improve the accuracy of the control instruction response.

[0112] S300: Analyze the click coordinates of the control instruction within the screen display area.

[0113] The controller 250 can establish a coordinate system within the display 260 to generate click coordinates based on the response position of the control command. The coordinate system can be established based on any reference point within the display 260 as the coordinate origin, for example, the coordinate system can be established with the upper left corner, lower right corner, or center point of the display 260 screen as the coordinate origin.

[0114] After establishing the coordinate system, the controller 250 can parse the click coordinates of the control instruction in the screen display area. In some embodiments, the controller 250 can also determine the click coordinates based on the option icon in the projection screen. For example, when the controller 250 detects that the location where the control instruction is clicked is within the response range of the application icon A, the controller 250 can obtain the coordinate range of the application icon A in the coordinate system based on the coordinate system of the display 260, and use the coordinates of the center point of the coordinate range as the click coordinates.

[0115] S400: Calculating normalized coordinates according to the projection pixel value, the preset projection size, and the click coordinates, and generating a reverse control instruction according to the normalized coordinates.

[0116] The controller 250 can calculate a normalized ratio based on the projection pixel value and the preset projection screen size, and calculate normalized coordinates based on the click coordinates and the normalized ratio. The normalized coordinates are the coordinates obtained by normalizing the click coordinates according to the Miracast protocol and can be used to generate reverse control instructions. Before performing the normalization process, the controller 250 can detect the support status of the terminal device 500 for the Miracast protocol through the projection screen to ensure that both the terminal device 500 and the display device 200 support the Miracast protocol.

[0117] Normalized coordinates include a normalized horizontal coordinate and a normalized vertical coordinate. The preset projection screen size includes a projection screen height and a projection screen width. The controller 250 can calculate a normalized height value based on the height pixel value and the projection screen height according to the analyzed projection screen pixel value, to obtain a normalized ratio in the height direction of the display 260. The controller 250 calculates a normalized width value based on the width pixel value and the projection screen width to obtain a normalized ratio in the width direction of the display 260. The controller 250 calculates the normalized coordinates based on the click coordinates and the calculated normalized ratio.

[0118] For example, when the click coordinates are (300, 600), the click horizontal coordinate is 300, and the click vertical coordinate is 600. The horizontal coordinate corresponds to the normalized ratio in the width direction, such as 1 / 2, and the vertical coordinate corresponds to the normalized ratio in the height direction, such as 1 / 3. The normalized horizontal coordinate can be calculated to be 300×1 / 2=150, and the normalized vertical coordinate is 600×1 / 3=200. Then the normalized coordinates are (150, 200). When the coordinate value is negative, it means that the click coordinate is located in the quadrant of the coordinate system. When calculating the normalized coordinates, only the numerical value of the coordinate value is calculated, and the positive and negative values ​​of the coordinate value are retained.

[0119] In some embodiments, as Figure 12As shown, the user can generate control instructions by triggering multiple click positions at the same time, for example, pinch zoom, or trigger additional game effects by combining multiple virtual keys when running a game program. At this time, the controller 250 can detect multiple click coordinates on the display 260 at the same time, such as Figure 11 Click coordinates 1 and 2 are shown. To synchronize the functional effects of the conversion control instructions, the controller 250 can obtain multiple click coordinates respectively and convert these click coordinates into normalized coordinates simultaneously according to the normalization ratio, thereby improving the normalization efficiency of the click coordinates.

[0120] When the user clicks on multiple locations of the projected screen in sequence, the controller 250 can monitor the click events of the display 260 in the projected screen according to the control instructions, sort the click events according to the event generation time, and perform normalization calculations in sequence based on the click coordinates of the sorted click events, so that the normalized coordinates of the click coordinates are calculated in sequence according to the time series, thereby improving the normalization accuracy of the click coordinates.

[0121] like Figure 13 As shown, when the user zooms in and out using two fingers, the user can slide on the projection screen with their fingers. For example, when performing a zoom-in operation, two fingers are slid relatively outward on the projection screen, and when performing a zoom-out operation, two fingers are slid relatively inward on the projection screen. During the sliding process, the click coordinates generate a sliding displacement due to the sliding, and the sliding displacement is used to represent the zoom ratio of the zoom operation. In the process of calculating the normalized coordinates based on the click coordinates, in order to synchronously convert the functional effect of the sliding, the controller 250 can detect the sliding action received by the display 260, and parse the starting point and end point of the sliding action. The controller 250 can obtain the coordinate values ​​of the starting point and the end point respectively, and calculate the normalized starting point coordinates and the normalized end point coordinates based on the coordinate values.

[0122] The controller 250 can also obtain the trajectory path of the sliding action. When the sliding action is a straight line or a broken line, the controller 250 can obtain the normalized starting point coordinates and the normalized ending point coordinates, and then connect the corresponding normalized coordinate points in sequence according to the sliding order of the trajectory path. Since both the starting point and the ending point perform normalization calculations, the trajectory path connecting the normalized coordinate points has the same normalized ratio as the normalized trajectory path. When the sliding action is a regular curve, the controller 250 can use edge detection algorithms, mathematical models, etc. to detect the collection points of the trajectory path, such as arc vertices, circles, etc. When the sliding action is an irregular curve, the controller 250 can obtain the coordinate points that need to be normalized by detecting tangent points and other methods, thereby performing normalization calculations.

[0123] In some embodiments, upon detecting that the display 260 has rotated, the controller 250 may obtain rotation information of the rotation component 290, including the rotation direction and rotation angle. The controller 250 may synchronously correct the coordinate system information of the display 260 based on the rotation information. For example, when the steering angle is 90°, the controller 250 may convert the horizontal coordinate of the coordinate system before rotation into the vertical coordinate after conversion, and convert the vertical coordinate of the coordinate system before rotation into the horizontal coordinate after conversion. The controller 250 may also perform normalization calculations on the subsequently detected click coordinates using the rotated coordinate system to improve the accuracy of the normalized coordinate calculations when the display 260 is in different rotation states.

[0124] The controller 250 can reversely control the terminal device 500 through the display device 200 according to the reverse control function. Since the display device 200 is in communication with the terminal device 500, after obtaining the normalized coordinates, the controller 250 can obtain the coordinate conversion rules of the terminal device 500. The coordinate conversion rules are used to convert the normalized coordinates to the screen coordinate points on the terminal device 500, thereby generating a reverse control instruction based on the screen coordinate points to improve the accuracy of the reverse control instruction response on the terminal device 500.

[0125] S500: Sending the reverse control instruction to the terminal device, and controlling the display to display the projection screen after the terminal device responds to the reverse control instruction.

[0126] After generating the counter-control instruction, the controller 250 can send the counter-control instruction to the terminal device 500 and wait for the terminal device 500 to respond. After receiving the counter-control instruction, the terminal device 500 can initiate a response at the location specified by the counter-control instruction on the terminal screen, for example, moving interface icons, running applications, pausing and playing media screens, etc. After the terminal device 500 responds to the counter-control instruction, the terminal screen also changes accordingly. In order to improve the screen projection synchronization between the terminal device 500 and the display device 200, the terminal device 500 can send the terminal screen after responding to the counter-control instruction as a projection screen to the display device 200. After the display device 200 receives the projection screen sent by the terminal device 500 after the terminal device 500 responds to the counter-control instruction, the controller 250 controls the display 260 to display the projection screen after responding to the counter-control instruction in the screen display area.

[0127] In some embodiments, the controller 250 can also fill the projection screen in the display 260 according to the rotation state of the display 260 to achieve the effect of displaying the entire video stream screen in full screen. At this time, the projection size is the screen size of the display. The controller 250 can respond to the control instruction to obtain the rotation state of the display 260. When the rotation state is the horizontal screen state, the screen width of the display 260 is greater than the screen height. In order to keep the video stream screen intact, the controller 250 uses the screen height as a reference for scaling the projection screen. That is, when the controller 250 performs scaling on the projection screen, when the projection screen height reaches the screen height of the display 260, the display 260 is controlled to display the current projection screen.

[0128] When the rotation state is the portrait state, the screen width of the display 260 is greater than the screen height. At this time, the controller 250 uses the screen width as a reference for scaling the projection screen. That is, in the process of the controller 250 scaling the projection screen, when the projection width of the projection screen reaches the screen width of the display 260, the display 260 is controlled to display the current projection screen, so as to realize the full-screen display of the complete video stream screen in the horizontal or vertical state.

[0129] like Figure 14 As shown, the display device 200 may also have a split-screen function. The controller 250 may divide the display 260 into multiple display areas through the split-screen function, and display different screens in different display areas. For example, when the display 260 is in a horizontal state, the display 260 may be divided into two display areas of the same size, and a first application screen may be displayed in the first display area, and a second application screen different from the first application screen may be displayed in the second display area, so that the user can view at least two application screens on the same display 260.

[0130] When the controller 250 displays the projected image in the split-screen state, the size of the projected image will change according to the distribution of the split-screen windows. Therefore, in some embodiments, the controller 250 can detect the split-screen windows in the currently displayed image. If there are split-screen windows in the currently displayed image, the controller 250 can determine the target split-screen window as the split-screen window that displays the projected image based on the display area ratio of the target split-screen window to the full-screen window.

[0131] The display area ratio of the split-screen windows can be determined based on the number of split-screen windows. When there are two split-screen windows, for example, two split-screen windows with the same display area are distributed left and right on display 260 in landscape mode. In this case, the target split-screen window is one of the two split-screen windows, and the display area ratio of the target split-screen window to the full-screen window is 2 / 1. In landscape mode, the projection height of the target split-screen window is the same as the screen height, and the projection width of the target split-screen window is 1 / 2 of the screen width, thereby obtaining the window size of the split-screen window.

[0132] The controller 250 can calculate the window pixel value of the target split-screen window based on the window size and the screen pixel value. For example, if the screen pixel value is 1366×768, the screen width pixel value is 1366px and the screen height pixel value is 768px. Since the projection height of the target split-screen window is the same as the screen height, the controller 250 only needs to calculate the width pixel value of the target split-screen window, that is, the product of the screen width pixel value and the display area ratio 1366px×1 / 2=683px. Then the window length pixel value is 768px, the window width pixel value is 683px, and the window pixel value is 683×768, that is, there are 683 pixel values ​​in the horizontal direction and 768 pixel values ​​in the vertical direction in the split-screen window. The controller 250 calculates the projection screen size based on the ratio of the window pixel value and the projection screen pixel value, and controls the display 260 to display the projection screen in the target split-screen window based on the projection screen size.

[0133] In some embodiments, the controller 250 can set the split screen layout rule according to the rotation state of the display 260, for example, Figure 14 As shown, in the horizontal screen state, the left and right split screen rules are set, and when the split screen instruction is received, the controller 250 displays the split screen windows in the display 260 in a left-right distribution manner. Figure 15 As shown, in the portrait mode, the top-bottom split-screen rule is set, and the controller 250 displays the split-screen windows in a top-bottom distribution on the display 260. The controller 250 can set multiple split-screen windows according to the user's split-screen requirements, and evenly adjust the display areas of the split-screen windows according to the split-screen layout rule. For example, when four split-screen windows are detected, the split-screen windows are set to be displayed with 1 / 4 of the display area of ​​the full-screen window, and all the split-screen windows are arranged in an array.

[0134] It should be noted that the split-screen window can include a variety of arrangements, such as array arrangement or straight line arrangement, and the controller 250 can calculate the projection size of the split-screen window according to different arrangements. The user can also manually adjust the projection size of one or more split-screen windows, for example, enlarge one of the two split-screen windows and reduce the other split-screen window. Each split-screen window can display a projection screen uploaded by a terminal device 500. The split-screen window can also include a floating window, which can be displayed in a floating manner in the full-screen display, wherein the projection screen can be selected to be projected to the floating window or the full-screen projection display according to the user's selection instruction.

[0135] In some embodiments, as Figure 16As shown, the projection screen includes the content screen and the non-content screen, wherein the content screen is the screen in the video stream that displays the projection content, and the non-content screen is the blank screen other than the content screen in the video stream. The terminal device 500 only displays the content screen in the terminal screen and does not display the non-content screen. When the controller 250 receives the projection screen sent by the terminal device 500, it controls the display 260 to simultaneously display the content screen and the non-content screen in the projection screen.

[0136] After the controller 250 receives the projected screen, it can control the display 260 to display the projected screen with different display effects, such as filling effect, stretching effect, adaptation effect, tiling effect, etc., wherein, when the pixel value of the projected screen is smaller than the screen pixel value, the filling effect will enlarge the projected screen to fill the entire screen display area. The stretching effect can change the width or height of the projected screen according to the screen pixel value, so that the aspect ratio of the projected screen is the same as the aspect ratio of the screen pixel value, thereby improving the display quality of the projected screen. The adaptation effect will display the maximum size of the projected screen in the screen display area while the aspect ratio of the projected screen remains unchanged. When the projection pixel value of the projected screen is smaller than the screen pixel value, the tiling effect will repeatedly arrange the projected screen and display it in the screen display area.

[0137] When displaying the projected screen using different display effects, the content screen may deviate from the center area of ​​the display 260, causing the non-content screen to occupy too much display area. To this end, the controller 250 can obtain the coordinates of the center point of the display 260 based on the screen coordinate system of the display 260, and align the center point of the content screen with the coordinates of the center point of the display 260 so that the content screen is displayed in the center area of ​​the display 260. The controller 250 can also perform non-screen filling on the non-content screen, such as filling with a solid background, a black border area, or a shadow area.

[0138] Since the video stream is displayed horizontally, when the display 260 displays the projection screen, the non-content screen will occupy most of the display area. Figure 17As shown, the controller 250 can obtain the zoom ratio input by the user, and scale the projection screen according to the zoom ratio based on the center point coordinates, and adjust the preset projection screen size according to the scaled projection screen, so as to increase the proportion of the content screen displayed by the display 260. In order to fully display the content screen, the controller 250 can set a maximum zoom ratio. The controller 250 can limit the user's excessive zooming of the projection screen through the maximum zoom ratio to avoid the user inputting a zoom ratio that is too large or too small, resulting in part of the projection screen being located outside the screen display area, resulting in the user being unable to view the complete projection screen. For example, when the maximum zoom ratio is 5:1, when the user inputs a 50-fold zoom ratio value, the controller 250 can perform a zoom operation on the projection screen at the maximum zoom ratio, so that the content screen is located within the screen display area, thereby improving screen utilization.

[0139] In some embodiments, in order to fully display the content screen at the maximum ratio on the display 260, the controller 250 can also obtain the screen size of the content screen and calculate the fill scaling ratio based on the screen size and the screen pixel value. The fill scaling ratio is the scaling ratio for controlling the display 260 to fill the displayed content screen, and controls the display 260 to display the scaled content screen according to the fill scaling ratio.

[0140] The fill scaling ratio can be adjusted according to different rotation states. For example, when the display 260 is in a landscape state, the fill scaling ratio is calculated using the height pixel value of the screen pixel value. When the display 260 is in a portrait state, the fill scaling ratio is calculated using the width pixel value of the screen pixel value to ensure that the display 260 can fully display the content screen at the maximum ratio in different rotation states.

[0141] Some embodiments of the present application further provide a terminal device 500, which includes a display module, a communication module, and a processor. The display module may be a display screen, such as an LED display screen, a liquid crystal display screen, etc., for displaying a terminal screen. The terminal screen may include application icons, desktop components, user interfaces, etc. The communication module is configured to establish a communication connection with the communicator 220 of the display device 200. The processor is configured to:

[0142] In response to the projection instruction input by the user, the projection screen is sent to the display device 200, so that the display device 200 displays the projection screen in the screen display area divided according to the preset projection size based on the rotation state of the display 260. When the display device 200 displays the projection screen, the controller 250 of the display device 200 can also monitor the control instructions sent by the user to the display device 200, and parse the click coordinates of the control instructions, calculate the normalized coordinates according to the projection pixel value, the preset projection size and the click coordinates, and generate a reverse control instruction based on the normalized coordinates and send it to the terminal device 500. Among them, the normalized coordinates are calculated based on the click coordinates and the preset projection size and projection pixel value, and the click coordinates are parsed by the display device in response to the control instruction input by the user, and the projection pixel value is the video stream pixel value of the projection screen.

[0143] After the terminal device 500 receives the reverse control instruction, the processor will parse the normalized coordinates in the reverse control instruction, and generate a projection screen based on the terminal screen and the normalized coordinates, so that the display device displays the projection screen after responding to the reverse control instruction.

[0144] Some embodiments of the present application also provide a device screen projection reverse control method, the method comprising:

[0145] S100: In response to a control instruction input by a user, obtaining a rotation state of the display and a projection pixel value.

[0146] The projection pixel value is the pixel value of the video stream of the projection screen.

[0147] S200: Based on the rotation state, divide the screen display area according to a preset projection size.

[0148] The screen display area is the area where the current projection screen is displayed.

[0149] S300: Analyze the click coordinates of the control instruction within the screen display area.

[0150] S400: Calculating normalized coordinates according to the projection pixel value, the preset projection size, and the click coordinates, and generating a reverse control instruction according to the normalized coordinates.

[0151] S500: Sending the reverse control instruction to the terminal device, and controlling the display to display the projection screen after the terminal device responds to the reverse control instruction.

[0152] It can be seen from the above technical solutions that the embodiments of the present application provide a display device, a terminal device, and a device screen projection reverse control method, which obtains the rotation state of the display and the projection pixel value by responding to the control command input by the user, divides the screen display area according to the preset projection size based on the rotation state, and parses the click coordinates of the control command in the screen display area. Normalized coordinates are calculated based on the projection pixel value, the preset projection size, and the click coordinates, and a reverse control command is generated based on the normalized coordinates. The reverse control command is sent to the terminal device, and the display is controlled to display the projection screen after the terminal device responds to the reverse control command. The present application converts the click coordinates into normalized coordinates through the pixel value and projection size of the display device when projecting the screen, thereby improving the synchronization of the click position of the display device and the terminal device, and generates a reverse control command based on the normalized coordinates to improve the accuracy of the reverse control process.

[0153] Similar parts between the embodiments provided in this application can be referenced to each other. The specific implementation methods provided above are only a few examples under the overall concept of this application and do not constitute a limitation on the scope of protection of this application. For those skilled in the art, any other implementation methods expanded based on the scheme of this application without expending creative work shall fall within the scope of protection of this application.

Claims

1. A display device, characterized in that: include: A display configured to display the projection image sent by the terminal device; a rotating assembly connected to the display and configured to adjust a rotation state of the display; The controller is configured as: In response to a control instruction input by a user, obtaining a rotation state of the display and a projection pixel value, wherein the projection pixel value is a video stream pixel value of the projection screen; Based on the rotation state, the screen display area is divided according to the preset projection screen size, and the screen display area is the area where the current projection screen is displayed; Analyzing the click coordinates of the control instruction within the screen display area; Calculating normalized coordinates according to the projection screen pixel value, the preset projection screen size, and the click coordinates, and generating a reverse control instruction according to the normalized coordinates; The reverse control instruction is sent to the terminal device, and the display is controlled to display the projection screen after the terminal device responds to the reverse control instruction.

2. The display device according to claim 1, wherein The normalized coordinates include a normalized horizontal coordinate and a normalized vertical coordinate, the preset projection screen size includes a projection screen height and a projection screen width, and the click coordinates include a click horizontal coordinate and a click vertical coordinate; The controller calculates normalized coordinates according to the projection pixel value, the preset projection size, and the click coordinates, and is configured to: Analyze the height pixel value and width pixel value of the projection pixel value; Calculating a normalized height value according to the height pixel value and the projection screen height, and calculating a normalized width value according to the width pixel value and the projection screen width; The product of the normalized height value and the click vertical coordinate is calculated to generate a normalized vertical coordinate, and the product of the normalized width value and the click horizontal coordinate is calculated to generate a normalized horizontal coordinate.

3. The display device according to claim 2, wherein The controller is further configured to: If the preset projection screen size is the screen size of the display, obtaining the screen pixel value of the display; Calculate the projection screen size according to the ratio of the screen pixel value to the projection screen pixel value; Based on the projection screen size, control the display to display the projection screen.

4. The display device according to claim 3, wherein The rotation state includes a horizontal screen state or a vertical screen state, the projection screen size includes a projection screen height and a projection screen width, and the controller controls the display to display the projection screen, and is configured to: Obtaining the rotation state of the display; When the rotation state is the horizontal screen state, controlling the display to fill the projection screen according to the projection screen height; When the rotation state is a portrait state, the display is controlled to fill the projection screen according to the projection width.

5. The display device according to claim 3, wherein The controller calculates the projection screen size according to the ratio of the screen pixel value to the projection screen pixel value, and is configured to: Detect the split-screen window in the current display; If the split-screen window exists in the current display screen, the display area ratio of the target split-screen window to the full-screen window is calculated, and the target split-screen window is the split-screen window that displays the projection screen; Calculating a window pixel value of the target split-screen window according to the display area ratio and the screen pixel value; The projection screen size is calculated based on the ratio of the window pixel value to the projection screen pixel value.

6. The display device according to claim 1, wherein The projection screen includes a content screen and a non-content screen, wherein the content screen is a screen in the video stream that displays the projection content, and the non-content screen is a blank screen other than the content screen in the video stream; the controller controls the display to display the projection screen, and is configured to: Acquiring center point coordinates based on a screen coordinate system of the display, where the screen coordinate system is a coordinate system established according to a preset origin; The content frame is displayed according to the center point coordinates, and non-frame filling is performed on the non-content frame.

7. The display device according to claim 6, wherein: The controller is further configured to: Get the zoom ratio entered by the user; Based on the center point coordinates, scaling the projection image according to the scaling ratio; Adjust the preset projection screen size according to the scaled projection screen.

8. The display device according to claim 6, wherein: The controller is further configured to: Get the screen size of the content screen; Calculating a fill scaling ratio according to the screen size and the screen pixel value, the fill scaling ratio being a scaling ratio at which the display fills the content screen; The display is controlled to display the zoomed content image according to the fill zoom ratio.

9. A terminal device, characterized in that: include: A display module configured to display a terminal screen; a communication module configured to establish a communication connection with a display device; The processor is configured to: In response to a screen projection instruction input by a user, sending a projection image to the display device, so that the display device displays the projection image in a screen display area divided according to a preset projection size based on the rotation state of the display; Obtaining a counter-control instruction fed back by the display device, and parsing the normalized coordinates in the counter-control instruction, the normalized coordinates being calculated based on the click coordinates and a preset projection screen size and projection screen pixel value, the click coordinates being parsed by the display device in response to a control instruction input by the user, and the projection screen pixel value being a pixel value of a video stream of the projection screen; A projection screen is generated based on the terminal screen and the normalized coordinates, so that the display device displays the projection screen in response to the reverse control instruction.

10. A device screen projection reverse control method, characterized in that: The display device according to any one of claims 1 to 8 comprises a display, a rotating component, and a controller, wherein the display is configured to display a projection image sent by a terminal device, and the rotating component is connected to the display and configured to adjust the rotation state of the display; the method comprising: In response to a control instruction input by a user, obtaining a rotation state of the display and a projection pixel value, wherein the projection pixel value is a video stream pixel value of the projection screen; Based on the rotation state, the screen display area is divided according to the preset projection screen size, and the screen display area is the area where the current projection screen is displayed; Analyzing the click coordinates of the control instruction within the screen display area; Calculating normalized coordinates according to the projection screen pixel value, the preset projection screen size, and the click coordinates, and generating a reverse control instruction according to the normalized coordinates; The reverse control instruction is sent to the terminal device, and the display is controlled to display the projection screen after the terminal device responds to the reverse control instruction.

Images