Display device and wired same-screen anti-control method

Abstract

The invention provides a display device and a wired same-screen anti-control method, and the method can store an initial display area of a media asset picture on a display when the display presents the media asset picture sent by a terminal device through a device interface. The method comprises the following steps: in response to a touch event input by a user, generating touch data of the touch event, then obtaining an initial display area, and detecting a current display area of a media asset picture; and comparing the current display area with the initial display area, and if the current display area is the same as the initial display area, sending the touch data to the terminal equipment. If the current display area is different from the initial display area, coordinate mapping is executed on the touch data according to the initial display area, and the touch data obtained after mapping is sent to the terminal equipment. According to the method, the touch data can be remapped when the change of the display area of the media asset picture is detected, so that the problem that the touch position is changed due to the conversion between the horizontal screen and the vertical screen of the display equipment during wired same-screen reverse control is solved.

Description

Technical Field

[0001] The present application relates to the technical field of display devices, and in particular to a display device and a wired same-screen reverse control method. Background Art

[0002] Display devices refer to terminal devices that can output specific display images, which can be terminal devices such as smart TVs, communication terminals, smart advertising screens, projectors, etc. Taking smart TVs as an example, smart TVs are based on Internet application technology, have open operating systems and chips, have open application platforms, can realize two-way human-computer interaction functions, and are TV products that integrate multiple functions such as audio and video, entertainment, and data to meet the diverse and personalized needs of users.

[0003] The display device also supports multi-screen interaction. The terminal device can project media images to the display device through multi-screen applications (such as Miracast, Aireplay, etc.) and control the display device. The display device (which supports touch) can also reversely control the terminal device through the multi-screen application. For display devices that do not have pre-installed multi-screen applications, the terminal device can connect to the display device through the device interface (such as the Type-C interface), and then project media images to the display device through the device interface.

[0004] However, when the display device controls the terminal device in reverse based on the device interface connection, after the display device rotates and / or scales the media screen, the original data of the touch screen in the display device does not change, but because the media screen is rotated and / or scaled, the actual touch position of the media screen displayed by the display device changes. And for the terminal device, the rotation and / or scaling of the media screen is unknown. At this time, if the user inputs a touch command based on the media screen displayed on the display device, the touch data captured by the display device will have an error with the touch data actually input by the user, so that the terminal device performs the touch data related operations different from the touch operations the user wants to perform. Summary of the invention

[0005] The present application provides a display device and a wired same-screen reverse control method to solve the problem of touch position change caused by horizontal and vertical screen conversion of the display device during wired same-screen reverse control.

[0006] In a first aspect, some embodiments of the present application provide a display device, including a display, a device interface, and a controller, wherein the device interface is connected to a terminal device, and the device interface is configured to transmit a media screen and touch data; the display is configured to display the media screen sent by the terminal device through the device interface, the display is connected to a touch interaction module, and the touch interaction module is used to receive a touch event input by a user, and the touch point coordinates of the touch interaction module correspond to the pixel point coordinates of the display. The controller is configured to execute the following program steps:

[0007] When the display presents the media resource picture sent by the terminal device through the device interface, storing the initial display area of ​​the media resource picture on the display;

[0008] In response to a touch event input by a user, generating touch data of the touch event;

[0009] Acquire the initial display area, and detect the current display area of ​​the media resource screen when the touch event is input;

[0010] If the current display area is the same as the initial display area, sending the touch data to the terminal device;

[0011] If the current display area is different from the initial display area, coordinate mapping is performed on the touch data according to the initial display area, and the mapped touch data is sent to the terminal device.

[0012] In a second aspect, some embodiments of the present application further provide a wired same-screen reverse control method, which is applied to the above-mentioned display device, including:

[0013] When the display presents the media resource picture sent by the terminal device through the device interface, storing the initial display area of ​​the media resource picture on the display;

[0014] In response to a touch event input by a user, generating touch data of the touch event;

[0015] Acquire the initial display area, and detect the current display area of ​​the media resource screen when the touch event is input;

[0016] If the current display area is the same as the initial display area, sending the touch data to the terminal device;

[0017] If the current display area is different from the initial display area, coordinate mapping is performed on the touch data according to the initial display area, and the mapped touch data is sent to the terminal device.

[0018] It can be seen from the above technical solutions that the present application provides a display device and a wired same-screen reverse control method, which can store the initial display area of ​​the media screen on the display when the display presents the media screen sent by the terminal device through the device interface. In response to the touch event input by the user, the touch data of the touch event is generated, and then the initial display area is obtained, and the current display area of ​​the media screen is detected. The current display area is compared with the initial display area. If the current display area is the same as the initial display area, the touch data is sent to the terminal device. If the current display area is different from the initial display area, coordinate mapping is performed on the touch data according to the initial display area, and the touch data obtained after mapping is sent to the terminal device. The method can remap the touch data when a change in the display area of ​​the media screen is detected, so as to solve the problem of touch position change caused by the horizontal and vertical screen conversion of the display device during wired same-screen reverse control. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 A schematic diagram of a usage scenario of a display device provided in some embodiments of the present application;

[0021] Figure 2 A hardware configuration block diagram of a display device provided in some embodiments of the present application;

[0022] Figure 3 A schematic diagram of the connection relationship of external terminal devices provided in some embodiments of the present application;

[0023] Figure 4 A hardware configuration block diagram of a control device provided in some embodiments of the present application;

[0024] Figure 5 A schematic diagram of software configuration in a display device provided in some embodiments of the present application;

[0025] Figure 6 A structural diagram of a dual-screen display system provided for some embodiments of the present application;

[0026] Figure 7 A diagram showing the relationship between touch data transmission in a dual-screen display system provided in some embodiments of the present application;

[0027] Figure 8 A schematic diagram of synchronous display based on a wired manner provided by some embodiments of the present application;

[0028] Fig. 9 A schematic diagram of the structure of a reverse control terminal device provided in some embodiments of the present application;

[0029] Fig.10 A schematic diagram of a horizontal and vertical screen display of a terminal device and a display device provided in some embodiments of the present application;

[0030] Fig.11 Another schematic diagram of horizontal and vertical screen display of a terminal device and a display device provided in some embodiments of the present application;

[0031] Fig.12 A schematic diagram of a display provided in some embodiments of the present application being converted from a horizontal posture to a vertical posture;

[0032] Fig.13 A schematic diagram of rotating and enlarging a media screen based on a display usage posture provided in some embodiments of the present application;

[0033] Fig.14 A schematic diagram of touch data remapping when converting a horizontal screen display to a vertical screen display provided in some embodiments of the present application;

[0034] Fig.15 A schematic flow chart of a wired same-screen reverse control method provided in some embodiments of the present application;

[0035] Fig.16 A schematic diagram of touch data remapping when a vertical screen display is converted to a horizontal screen display provided in some embodiments of the present application;

[0036] Fig.17 A flowchart of the judgment of the reverse control provided in some embodiments of the present application;

[0037] Fig.18 A schematic diagram of a touch data forwarding process provided in some embodiments of the present application;

[0038] Fig.19 A schematic diagram of the touch data processing flow provided in some embodiments of the present application. DETAILED DESCRIPTION

[0039] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0040] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0041] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0042] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0043] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0044] Figure 1 This is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG. 1 , a user can operate the display device 200 through the mobile terminal 300 and the control device 100 .

[0045] In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

[0046] like Figure 1 As also shown in FIG. 4 , the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0047] In addition to providing the broadcast receiving television function, the display device 200 may also provide an intelligent network television function with computer support functions, including but not limited to network television, smart television, Internet Protocol television (IPTV), etc.

[0048] Figure 2 Some embodiments of the present application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0049] In some embodiments, the display device 200 includes at least one of a tuner 210 , a communicator 220 , a detector 230 , a device interface 240 , a controller 250 , a display 260 , an audio output interface 270 , a memory, a power supply, and a communication device interface 280 .

[0050] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., for receiving external sounds.

[0051] In some embodiments, the display 260 includes a display screen component for presenting images, and a driving component for driving image display, which is used to receive image signals output from a controller, display video content, image content, and a menu control interface component and a user control UI interface, etc.

[0052] The display 260 may be a liquid crystal display, an OLED display, or a projection display, and may also be a projection device and a projection screen.

[0053] In some embodiments, the display device 200 may also support touch interaction, that is, a touch interaction module may be provided on the display device 200, and the touch interaction module may be a built-in module or an external module. For example, in order to implement touch interaction operations, a touch panel (TP) may be provided on the display 260 of the display device 200, and the touch panel may receive touch signals input by fingers, contacts, styluses, etc., and execute touch interaction responses according to a pre-set interaction strategy. Of course, the display device 200 may also be a touch-integrated screen.

[0054] In some embodiments, the display device 200 is further provided with a built-in or external rotating assembly, and the rotating assembly is used to rotate the display 260. For example, the rotating assembly can drive the display 260 to rotate around the screen to adjust the inclination angle of the display 260.

[0055] In some embodiments, the communicator 220 is a component for communicating with an external device or a server 400 according to various communication protocol types.

[0056] In some embodiments, the device interface 240 may include, but is not limited to, any one or more of the following interfaces: a high-definition multimedia interface (HDMI), an analog or digital high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. It may also be a composite input / output interface formed by the above multiple interfaces.

[0057] For example, in order to enrich the display types of the display device 200, the HDMI interface can connect devices with video and audio input / output functions, and implement the corresponding audio and video output / input functions through the HDMI driver in the hardware layer. The USB interface can connect devices with signal and data transmission functions and interactive functions, and access signals, transmit data, and perform specific interactive actions based on the USB driver in the hardware layer.

[0058] In some embodiments, the display device 200 can support multiple interface accesses and perform corresponding functions according to the specific device type accessed. For example, the display device 200 can be connected to another display or terminal device 500 via an HDMI interface to form display functions such as dual screen, split screen, screen transmission, conference, whiteboard, NFC, etc. The display device 200 can also be connected to the terminal device 500 via a USB interface, and activate the Open Pluggable Specification (OPS) channel to connect the terminal device 500. When the terminal device 500 is connected through the OPS channel, since the display device 200 has an independent operating system (system A), and the terminal device 500 can also have an independent operating system (system B), a dual system mode can be formed.

[0059] In some embodiments, the display device 200 and / or the connected terminal device 500 may both support touch interaction. When the display device 200 and the connected terminal device 500 both support touch interaction operations, a data transmission and conversion module may be provided in the display device 200 to transmit and convert touch data generated during the touch interaction process, so as to achieve channel reverse control, that is, to control the display device 200 through the touch-interactive terminal device 500. For example, Figure 3 As shown, a switch module can be set in the touch interaction system, and the TP module with touch function can be connected to the main control system (system A) of the display device 200 through the USB interface, and at the same time, connected to the terminal device 500 (system B), the external host, and the type-c device. In this way, the OPS channel formed by the HDMI interface can transmit display content data, and the touch data can be transmitted through the conversion module and the USB interface.

[0060] In some embodiments, the controller 250 includes a processor, a video processor, an audio processor, a graphics processor, a RAM, a ROM, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0061] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0062] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0063] In some embodiments, the communication device interface 280 is an interface that can be used to receive control input.

[0064] Figure 4 Some embodiments of the present application provide Figure 1 The hardware configuration diagram of the control device in the figure. Figure 4 As shown, the control device 100 includes a controller 110 , a communication interface 130 , a user input / output interface, a memory 190 , and a power supply 180 .

[0065] The control device 100 is configured to control the display device 200 , and can receive user input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200 .

[0066] In some embodiments, the control device 100 may be a smart device, for example, the control device 100 may be installed with various applications for controlling the display device 200 according to user needs.

[0067] In some embodiments, Figure 1 As shown, the mobile terminal 300 or other intelligent electronic devices can play a similar function as the control device 100 after installing the application for controlling the display device 200 .

[0068] The controller 110 includes a processor 112, a RAM 113, a ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between the internal components and the external and internal data processing functions.

[0069] The communication interface 130 implements communication of control signals and data signals with the display device 200 under the control of the controller 110. The communication interface 130 may include at least one of other near field communication modules such as a WiFi chip 131, a Bluetooth module 132, and an NFC module 133.

[0070] The user input / output interface 140 , wherein the input interface includes at least one of other input interfaces such as a microphone 141 , a touch panel 142 , a sensor 143 , and a button 144 .

[0071] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with a communication interface 130, such as a WiFi, Bluetooth, NFC or other module, and can encode the user input command through the WiFi protocol, Bluetooth protocol, or NFC protocol and send it to the display device 200.

[0072] The memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0073] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0074] Figure 5 Some embodiments of the present application provide Figure 1 Schematic diagram of software configuration in the display device. In some embodiments, the system of the display device 200 can be divided into three layers, namely, application layer, middleware layer and hardware layer from top to bottom.

[0075] The application layer mainly includes commonly used applications on TV and application frameworks. Common applications are mainly applications developed based on browsers, such as HTML5 APPs and native APPs.

[0076] The Application Framework is a complete program model that has all the basic functions required by standard application software, such as file access, data exchange, etc., as well as the user interfaces of these functions (toolbars, status bars, menus, dialog boxes).

[0077] Native apps can support online or offline, message push or local resource access.

[0078] The middleware layer includes various TV protocols, multimedia protocols, system components and other middleware. The middleware can use the basic services (functions) provided by the system software to connect various parts of the application system or different applications on the network, and can achieve the purpose of resource sharing and function sharing.

[0079] The hardware layer mainly includes the hardware abstraction layer (HAL) interface, hardware and drivers. The HAL interface is a unified interface for all TV chips to connect, and the specific logic is implemented by each chip. Drivers mainly include: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver.

[0080] In some embodiments, the display device 200 can connect an external display via HDMI output to form a dual-screen system with the display 260 itself to achieve a dual-screen display mode. Figure 6 As shown, the dual-screen system may include an image generation (ImageProducer) module, an image rendering (surface flinger) module, a synthesis (Composer) module, a video process (VideoProcessing Unit hardware, VPU hardware) module, a main screen (panel screen) and a secondary screen (HDMI screen).

[0081] During the display process, the media player (media player), camera peripheral (camera per view) and graphics interface (openGL ES) of the image generation module can generate image content and pass the image content to the image rendering module. The image rendering module includes a primary display unit (primary display) and an extended display unit (external display). The primary display unit is used to control the display content of the main screen, that is, the main display unit renders the image content through the user interaction layer (UI layer) or the video layer (video layer) according to the image content. In order to obtain a better display effect, the image content of the UI layer can be rendered by the GPU component, so as to form a picture frame for display in the synthesis module. Among them, the synthesis module can form the picture content by software or hardware. For example, the synthesis module can be a hardware synthesis (Hardware Composer, HWC) module.

[0082] In the synthesis module, the picture frames can be stored through the frame buffer and the video buffer, and the picture content can be formed in the corresponding layers through the video process module. That is, the image frames generated by the video playback process form the picture content through the video plane, and the image frames of other image pictures (such as touch tracks, graphics, etc.) form the picture content through the primary OSD plane. Finally, the display device 200 forms the final picture through the blending unit of the video process module and displays it on the main screen.

[0083] The extended display unit is used to control the display content of the secondary screen, that is, after the image generation module passes the image content to the image rendering module, the extended display unit also renders the image content through the UI layer or the video layer. According to the display process of the main screen, the image frame is formed by the image rendering module, the synthesis module and the video process module respectively, and displayed on the secondary screen.

[0084] In some embodiments, in order to obtain a better picture display effect, an additional extended display layer (external_OSD plane) can be set in the video process module of the display device 200 on the basis of the main OSD layer and the video layer. The extended display layer can be configured with different layer characteristics according to specific display effect requirements. For example, the bitmap refresh mode of the extended display layer can be modified so that the extended display layer is refreshed in real time according to the region, thereby improving the display speed of the graphics. At this time, the extended display layer can be used as an acceleration layer in scenes with high dynamic display effects.

[0085] Taking the display device 200 running a whiteboard drawing application as an example, in order to improve the hand tracking of the line drawing process, after detecting the user's touch operation, the touch track can be drawn in real time on the extended display layer (acceleration layer). The display level of the acceleration layer can be higher than other layers, and when the user completes the touch input, the line drawn by the acceleration layer is updated to other layers, thereby reducing input delay and improving the real-time display effect of the whiteboard application drawing process.

[0086] In some embodiments, the extended display layer can also work together with other graphics to improve display quality. For example, the acceleration layer can respond to the touch events input by the user at the same time as the main OSD layer, and draw graphics separately. However, since the acceleration layer is displayed above the main OSD layer, the graphics drawn by the acceleration layer can block the graphics drawn by the OSD layer to maintain real-time response speed. After the user inputs the touch interaction action, the content displayed by the acceleration layer can be released directly, and the OSD layer directly forms the drawn graphics, so there is no need for the acceleration layer to refresh the drawn graphics to the OSD layer, reducing the graphics mixing time. It not only improves the real-time display effect of the graphics drawing process, but also improves the final presentation quality of the graphics drawing.

[0087] Based on the above dual-screen display system, the user can control the display device 200 to execute different display strategies as needed. For example, the main screen is connected to the secondary screen through HDMIOUT to realize dual-screen display, and the secondary screen integrates a touch component to realize touch-control of the secondary screen content. In addition, the main screen and the secondary screen are also connected via USB, and the USB connection channel can be used to transmit touch data, that is, the secondary screen touch is connected to the USB of the main screen. After the connection is completed, the display device 200 can control the display of the main screen and the secondary screen. Among them, the main screen and the secondary screen can support the same display, that is, the display 260 (main screen) of the display device 200 and the external display display the same content. The main screen and the secondary screen can also support different displays, that is, the main screen and the secondary screen display different contents. Different display is also called extended display, and the main scenarios include: the physical signal source screen is displayed on the secondary screen; the player screen is displayed on the secondary screen; the wireless screen transmission screen displays two screen transmission screens; the conference application displays the shared content on the secondary screen, etc.

[0088] When the secondary screen connected to the display device 200 also supports touch interaction operations, the user can also reversely control the display device 200 or the external terminal device 500 by inputting touch operations on the secondary screen. That is, in some embodiments, after the user inputs a touch interaction action on the secondary screen, the secondary screen will pass the touch data in the touch event to the display device 200 through the USB channel. When the display device 200 is also connected to the external terminal device 500, the touch data can be forwarded to the USB channel connected to the terminal device 500 through the conversion module built into the display device 200, so that the terminal device 500 responds to the touch event input by the user on the secondary screen.

[0089] For example, Figure 7As shown, the user can connect a display with a touch function to the HDMI1 interface of the display device 200 based on system A as the secondary screen of the display device 200. At the same time, the OPS computer based on system B is connected to the HDMI2 interface of the display device 200 as the external terminal device 500 of the display device 200. In addition, the display device 200 is also connected to the secondary screen device through the USB1 interface and the OPS computer through the USB2 interface to form an interactive system that can support the reverse control function. The display 260 of the display device 200 serves as the main screen. During the display process, the main screen and the secondary screen can respectively display the screen content output by the OPS computer in the same display or different display modes.

[0090] When the user inputs a touch operation on the main screen, the display device 200 can transmit the touch data generated by the touch operation to the OPS computer through the USB2 interface channel, and the OPS computer responds to the click touch data. When the user touches and clicks the interactive control on the secondary screen, the generated click touch data can be transmitted to the display device 200 through the USB1 interface channel, and then transmitted to the USB2 interface channel through the conversion module (switch) inside the display device 200, and then transmitted to the OPS computer, and the OPS computer responds to the click touch data.

[0091] It should be noted that the touch reverse control function can generate different interaction logics according to different application scenarios. That is, the reverse control can include the following scenarios:

[0092] There is only a main screen and a secondary screen, that is, the display device 200 is connected to an external display to access the secondary screen, and the display device 200's own display 260 is used as the main screen. In this scenario, the display device 200 needs to distinguish between the touch of the whole device and the external USB touch, configure the touch of the whole device as internal, and configure the external USB touch as external to achieve dual-screen touch control, and the application dynamically adjusts the display according to whether the extended mode is turned on.

[0093] OPS device, main screen and sub-screen, that is, the display device 200 is connected to an external monitor and an OPS device at the same time, the external monitor is used as the sub-screen, the display 260 built into the display device 200 is used as the main screen, and the OPS device is used as the signal source. In this scenario, it is necessary to distinguish the display screen of the OPS signal. When the OPS signal is displayed on the sub-screen, the USB needs to be switched to the OPS channel to realize the reverse control function of the sub-screen to the OPS channel, and the main screen's external touch function needs to be turned off. When the OPS signal is displayed on the main screen, the main screen touch needs to be turned on and switched to the OPS device. Because the USB channel has been switched to the OPS device, the touch of the sub-screen will also be recognized and responded to by the OPS.

[0094] For this scenario, the touch component needs to support the function of turning off the current channel output through an external channel. When switching from USB to OPS channel, it is necessary to determine whether USB touch is connected. If USB touch is connected, the output needs to be turned off first, and then switched to OPS channel. The hardware design of the above touch reverse control system can control all USB3.0 interfaces through the same switch.

[0095] The following describes the display process based on the above-mentioned dual-screen system through specific examples. It should be understood that the following display process is only an example and does not limit the specific display method of the dual-screen system. Other display methods and interface contents that can be associated with by technical personnel in this field based on the following examples all belong to the protection scope of this application.

[0096] In some embodiments, the display device 200 also supports multi-screen interaction, and the terminal device 500 can project the media screen to the display device 200 through a multi-screen application (such as Miracast, Aireplay, etc.), so as to achieve the effect of synchronous display of the terminal device 500 and the display device 200. The user can control the display device 200 through the terminal device 500. The terminal device 500 can also be reversely controlled through the display device 200 (which itself supports touch control).

[0097] For a display device 200 without pre-installed multi-screen application, such as Figure 8 As shown, the terminal device 500 can be connected to the display device 200 through the device interface 240, and then the media data is placed on the display device 200 through the device interface 240. For example, the display device 200 is connected to the terminal device 500 through the Type-C interface, and the display device 200 can continuously obtain the media data through the interface channel of the Type-C interface, and display it in real time on the display 260, so as to achieve the effect of synchronous display with the terminal device 500.

[0098] When the user inputs a touch command based on the terminal device 500, the terminal device 500 responds to the touch command, performs a corresponding touch operation, changes the displayed media screen, and synchronizes it to the display device 200 in real time through the interface channel of the Type-C interface, thereby realizing the control of the display device 200 by the terminal device 500.

[0099] Based on the above-mentioned wired connection method, in order to realize the reverse control of the display device 200 over the terminal device 500, a human interface device (Hid) can be virtualized in the display device 200, so that the display device 200 can be recognized by the terminal device 500 inserted into the device interface 240 as an input device that complies with the Hid standard, thereby realizing the communication and control between the terminal device 500 and the display device 200. At the same time, a communication core service (HSPCore) and a touch data forwarding service (TpService) are also set to synchronize the touch data to the terminal device 500, thereby realizing the reverse control of the terminal device 500.

[0100] like Fig. 9 As shown, the display 260 of the display device 200 may be provided with a touch screen as a touch display, connected to the HidDriver driver via the Inter-Integrated Circuit (I2C), and the terminal device 500 is connected to the HidDriver driver via the Type-C interface, wherein HidDriver refers to a driver that provides a communication interface and interactive functions for the Hid device. The HidDriver driver can communicate with the input interface of the display device 200, the touch data forwarding service (TpSer vice) and the terminal device 500, the touch data forwarding service is used to interact with the application and forward touch data, and the communication core service is used to transfer parameters with the application (APP) of the application layer of the display device 200.

[0101] Thus, the user inputs a touch command to the display device 200, and the display device 200 can capture the touch data of the touch command, including but not limited to the touch position, touch type, touch pressure, touch time, touch speed, touch track, etc. The touch data is sent to the terminal device 500 through the interface channel of the Type-C interface, so that the terminal device 500 performs the operation corresponding to the touch data, changes the displayed media screen, and synchronizes the display to the display device 200 in real time through the interface channel of the Type-C interface, so as to realize the reverse control of the terminal device 500 by the display device 200.

[0102] In some embodiments, Fig.10 , 11 As shown, the display device 200 and the terminal device 500 connected via the device interface 240 can both support horizontal and vertical screen display. The display device 200 is connected to the terminal device 500 via the Type-C interface, and the terminal device 500 can display media data in horizontal or vertical screen. When the media data is projected onto the display device 200, the display device 200 can also display in horizontal or vertical screen according to the usage posture of the display 260.

[0103] Among them, the usage posture of the display 260 can be controlled according to the rotating component of the display device 200. The user can control the rotation of the rotating component through a specific interaction strategy, so that the rotating component drives the display 260 to rotate around the screen, thereby adjusting the usage posture of the display 260.

[0104] In some embodiments, in order to improve the display effect, the display device 200 can also adjust the media picture according to the use posture of the display 260. That is, after the display device 200 controls the display 260 to display the media picture delivered by the terminal device 500, if the user controls the rotation component to rotate through a specific interaction strategy, a rotation event can be generated in response to the rotation event. The display device 200 can detect the rotation direction and rotation angle of the rotation event, and rotate the media picture by the rotation angle in the opposite direction of the rotation direction so that the media picture is displayed forward. And the media picture is scaled according to the display size of the display 260 so that the media picture is presented on the display 260 with the maximum display ratio.

[0105] For example, Fig.12 As shown in FIG. 1 , when the display device 200 receives the media screen delivered by the terminal device 500, the display 260 is in a horizontal position and can display the media screen in a horizontal screen. At this time, the user controls the display 260 to rotate 90 degrees clockwise through the rotation component, so that the display 260 is converted from a horizontal position to a vertical position. Since the display 260 does not rotate after rotating 90 degrees clockwise, the displayed media screen appears as follows: Fig.12 The tilt effect shown.

[0106] In order to display the media picture in the forward direction, the display device 200 can rotate the media picture 90 degrees counterclockwise to realize vertical screen display. At the same time, the media picture can be enlarged according to the display size of the display 260, so that the displayed media picture appears as follows: Fig.13 The full screen display effect is shown.

[0107] During the above adjustment process, the original data of the touch screen on the display 260 of the display device 200 does not change, but because the media screen is rotated and enlarged, the actual touch position of the media screen displayed by the display 260 changes. And for the terminal device 500, the rotation and enlargement of the media screen is unknown. At this time, if the user inputs a touch command based on the media screen displayed on the display device 200, the touch data captured by the display device 200 will have an error with the touch data that the user actually wants to input.

[0108] For example, Fig.14 As shown, Fig.12 , 13 The original picture of the Chinese media resource has not been rotated or enlarged ( Fig.14 The image on the right in the middle) and the processed image after rotation and enlargement ( Fig.14 The user inputs a click instruction for area A in the media screen through the touch interaction module of the display device 200. The touch position of area A in the original screen is (x1, y2, z3), and the touch position in the processed screen after the enlargement is (x4, y5, z6), that is, the actual touch position changes after the media screen is rotated and enlarged.

[0109] If the user inputs a click instruction for area A based on the original screen presented when the display 260 is not rotated, the display device 200 can capture the touch data with the touch position (x1, y2, z3) and send it to the terminal device 500, so that the terminal device 500 performs a click operation on area A. If the user inputs a click instruction for area A based on the processed screen presented after the display 260 is rotated, the display device 200 can capture the touch data with the touch position (x4, y5, z6) and send it to the terminal device 500. Since the media asset screen is adaptively adjusted based on the display device 200, it is unknown to the terminal device 500. Therefore, after receiving the touch data sent by the display device 200, the terminal device 500 will perform a click operation on the area (x4, y5, z6). Fig.14 Area B) performs a click operation. Thus, the operation related to the touch data performed by the terminal device 500 is different from the touch operation that the user wants to perform.

[0110] In order to solve the problem that the touch position of the media screen changes due to the horizontal and vertical screen conversion of the display device 200, a wired same-screen reverse control method is provided in some embodiments of the present application, which is applied to the display device 200. Among them, the display device 200 that can apply the wired same-screen reverse control method includes: a display 260, a device interface 240 and a controller 250. Among them, the device interface 240 is connected to the terminal device 500. The device interface 240 is configured to transmit media screens and touch data. The display 260 is configured to display the media screen sent by the terminal device 500 through the device interface 240. The display 260 is connected to a touch interaction module, and the touch interaction module is used to receive touch events input by the user. The touch point coordinates of the touch interaction module correspond to the pixel point coordinates of the display 260. The controller 250 is configured to execute the above-mentioned wired same-screen reverse control method.

[0111] like Fig.15 As shown, Fig.15A flowchart of a wired same-screen reverse control method provided in an embodiment of the present application. After the display device 200 synchronously displays the media data of the terminal device 500 through the interface channel of the device interface 240. The user can input a touch event through the touch interaction module of the display device 200, and the display device 200 can generate touch data of the touch event and send it to the terminal device 500 through the interface channel of the device interface 240, thereby realizing reverse control of the terminal device 500.

[0112] In order to solve the problem of touch position change caused by media screen change. After the display device 200 obtains the media screen sent by the terminal device 500 through the device interface 240, it controls the display 260 to display the media screen, and records the display area when the media screen is displayed as the initial display area, and stores the initial display area. When the user inputs a touch event through the touch interaction module, the display device 200 responds to the touch event, generates touch data of the touch event, and detects the current display area of ​​the media screen. Then the initial display area is obtained, and the current display area and the initial display area are compared. If the current display area is the same as the initial display area, that is, the media screen has not changed after the terminal device 500 is placed on the display 260 for display, the touch data is sent to the terminal device 500.

[0113] If the current display area is different from the initial display area, that is, the media image is rotated and / or scaled after being displayed on the display 260 by the terminal device 500, coordinate mapping is performed on the touch data according to the initial display area, and the mapped touch data is sent to the terminal device 500.

[0114] For example, Fig.14 Take the display effect shown as an example. The user controls the display 260 to rotate 90 degrees clockwise through a specific interaction strategy. The display 260 is converted from a horizontal posture to a vertical posture. The display device 200 rotates the media screen counterclockwise by 90 degrees, and enlarges the media screen according to the display size of the display 260 to display the media screen in full screen. At this time, the display device 200 will detect the current display area of ​​the media screen, which is different from the initial display area. When the user inputs a click instruction for area A in the media screen. The display device 200 obtains the touch position in the touch data as (x4, y5, z6), performs coordinate mapping on the touch data according to the initial display area, and obtains the touch position in the mapped touch data as (x1, y2, z3). The mapped touch data is sent to the terminal device 500, so that the terminal device 500 performs the same touch operation related to the touch data as the user wants to perform.

[0115] Since only the touch position of the media image changes after the media image is rotated and / or scaled, the display device 200 can remap the touch data by performing coordinate mapping on the touch data according to the initial display area and calculating its position in the initial display area.

[0116] like Fig.14 As shown above Fig.12 , 13 The case where the display 260 shown is converted from a horizontal screen posture to a vertical screen posture is taken as an example. A touch box may be provided on the display 260 of the display device 200 to implement touch interactive operations. Taking the upper left corner of the display 260 screen as the starting point, xP is the x coordinate of the pixel point of the display 260 screen, xP1 is the left edge coordinate of the media screen, xP2 is the right edge coordinate of the media screen, and xPmax is the pixel point width of the display 260 screen. yP is the y coordinate of the pixel point of the display 260 screen, and yPmax is the pixel point height of the display 260 screen. xT is the x coordinate of the touch box, xTmax is the width value of the touch box, yT is the y coordinate of the touch box, and yTmax is the height value of the touch box.

[0117] The pixel point position corresponding to the touch position can be calculated based on the coordinates of the touch frame and the height and width of the pixel points of the display 260:

[0118] xP=(xT / xTmax)*xPmax;

[0119] yP=(yT / yTmax)*yPmax;

[0120] Since the display device 200 adjusts the media screen according to the usage posture of the display 260, it is based on the display size of the display 260 and is proportionally enlarged. The touch data obtained by the display device 200 is original data, so the display device 200 only needs to calculate the position and proportion of the touchable area of ​​the touch frame in the initial display area. That is, the display device 200 can calculate the position relationship and proportion relationship of the touchable area of ​​the touch frame in the initial display area, and perform coordinate mapping on the touch data according to the position relationship and proportion relationship.

[0121] Among them, the positional relationship and the proportional relationship can be calculated based on the vertex coordinates of the touchable area and the vertex coordinates of the initial display area. That is, the display device 200 obtains the vertex coordinates of the touchable area (the vertex coordinates of the touch frame) and the vertex coordinates of the initial display area (the pixel coordinates of the vertices of the media screen display area). According to the rotation direction of the display 260, the touchable area is rotated by the rotation angle. Then, the conversion relationship between the vertex coordinates of the touchable area after rotation and the vertex coordinates of the initial display area is calculated, so as to obtain the positional relationship and proportional relationship of the touchable area in the initial display area.

[0122] like Fig.14 As shown, when the display 260 is not rotated, the vertex coordinates of the touchable area are (0, 0), (xTmax, 0), (0, yTmax) and (xTmax, yTmax) respectively. After the display 260 is rotated 90 degrees clockwise, the vertex coordinates of the touchable area of ​​the display 260 are (0, yTmax), (0, 0), (xTmax, yTmax) and xTmax, 0) respectively.

[0123] In order to calculate the positional relationship and proportional relationship of the touchable area of ​​the touch frame in the initial display area, the display device 200 may rotate the vertex coordinates of the touchable area by 90 degrees clockwise based on the following formula:

[0124] xT=yTmax-yT;

[0125] yT=xT;

[0126] The vertex coordinates after clockwise rotation are (0, 0), (xTmax, 0), (0, yTmax) and (xTmax, yTmax). After the rotation, the positional relationship and proportional relationship of the touchable area of ​​the touch frame in the initial display area can be calculated according to the vertex coordinates of the touchable area and the vertex coordinates of the initial display area, and coordinate mapping is performed on the touch data according to the positional relationship and proportional relationship to obtain remapped touch data.

[0127] The display device 200 may calculate the remapped touch data based on the following formula:

[0128] xTdest=(yTmax-yT)*(xP2-xP1) / xPmax+(xTmax*xP1 / xPmax);

[0129] yTdest = xT;

[0130] Among them, xT, yT are the x coordinate and y coordinate of the touch frame acquired by the display device 200, and xTdest, yTdest are the x coordinate and y coordinate of the touch frame after remapping.

[0131] Similarly, based on the above method, the conversion formula of touch data when the display 260 rotates at different angles can be obtained:

[0132] The conversion formula for rotating the display 260 90 degrees clockwise (corresponding to rotating the terminal device 500 90 degrees counterclockwise) is:

[0133] xTdest=(yTmax-yT)*(xP2-xP1) / xPmax+(xTmax*xP1 / xPmax);

[0134] yTdest = xT;

[0135] The conversion formula for rotating the display 260 clockwise by 180 degrees (corresponding to rotating the terminal device 500 counterclockwise by 180 degrees) is:

[0136] xTdest = xT;

[0137] yTdest = yTmax - yT;

[0138] The conversion formula for display 260 to rotate 270 degrees clockwise (corresponding to terminal device 500 to rotate 270 degrees counterclockwise) is:

[0139] xTdest=yT*(xP2-xP1) / xPmax+(xTmax*xP1 / xPmax);

[0140] yTdest = xTmax - xT;

[0141] Similarly, when the display 260 is converted from a vertical screen display to a full screen display of media data, the display device 200 receives the media screen delivered by the terminal device 500 and controls the display 260 to display the media screen in a vertical screen. When the user controls the display 260 to rotate 90 degrees clockwise through the rotation component, the display 260 is converted from a vertical posture to a horizontal posture. In order to display the media screen in the forward direction, the display device 200 rotates the media screen counterclockwise by 90 degrees and enlarges the media screen according to the display size of the display 260 to achieve full screen display.

[0142] like Fig.16 As shown, the original image of the media asset image is not rotated or enlarged ( Fig.16 The image on the right in the middle) and the processed image after rotation and enlargement ( Fig.16 (left side of the figure), refer to the above Fig.14 The display device 200 can obtain the conversion formula of the touch data when the display 260 rotates at different angles through the related conversion steps:

[0143] The conversion formula for rotating the display 260 90 degrees clockwise (corresponding to rotating the terminal device 500 90 degrees counterclockwise) is:

[0144] xTdest = yTmax - yT;

[0145] yTdest=(xT-(yP1 / yPmax)*yTmax)*(yPmax / (yP2-yP1));

[0146] The conversion formula for rotating the display 260 clockwise by 180 degrees (corresponding to rotating the terminal device 500 counterclockwise by 180 degrees) is:

[0147] xTdest = xT;

[0148] yTdest = yTmax - yT;

[0149] The conversion formula for display 260 to rotate 270 degrees clockwise (corresponding to terminal device 500 to rotate 270 degrees counterclockwise) is:

[0150] xTdest = yT;

[0151] yTdest=((xTmax-xT)-(yP1 / yPmax)*yTmax)*(yPmax / (yP2-yP1)).

[0152] In some embodiments, in order to reduce the conflict between the touch operation of the display device 200 itself and the touch operation of the reverse control terminal device 500, the touch data forwarding service (TpService) of the display device 200 can be controlled to be turned on and off to control whether the display device 200 reversely controls the terminal device 500.

[0153] It can be understood that since the touch data forwarding service (TpService) of the display device 200 is used for interaction with applications and touch data forwarding, when the touch data forwarding service is turned on, it can interact with the application layer of the display device 200 and send touch data to the terminal device 500. When the touch data forwarding service is turned off, it does not interact with the application layer of the display device 200 and does not send touch data to the terminal device 500. That is, only when the touch data forwarding service is turned on, can the terminal device 500 be reversely controlled.

[0154] Among them, the on and off states of the touch data forwarding service can be manually controlled by the user, that is, the state setting control of the touch data forwarding service can be encapsulated in the user interface of the display device 200, so that the user can manually set the state of the touch data forwarding service. It can also be that the display device 200 sets it by itself based on the current display content, that is, when the signal source of the display device 200 switches to the interface channel of the device interface 240 and there is a signal in the interface channel, the touch data forwarding service is set to the on state. When the signal source of the display device 200 switches out of the interface channel of the device interface 240, or the display device 200 starts other interfaces, the touch data forwarding service is set to the off state.

[0155] Among them, the interface channel to which the signal source of the display device 200 is currently switched can be represented based on the channel identification. When the display device 200 supports multiple interface accesses, the channel identification value can be set for it according to the type of the interface. Therefore, when the terminal device 500 is connected to the display device 200 through the device interface 240, the display device 200 can respond to the plug-in signal generated when the terminal device 500 is connected to the device interface 240, and set the channel identification to the channel identification value representing the device interface 240, thereby switching the signal source of the display device 200 to the interface channel of the device interface 240. For example, the display device 200 includes a Type-C interface and a USB interface. When the channel identification of the display device 200 is a channel identification value representing the Type-C interface, it indicates that the signal source of the display device 200 is switched to the Type-C interface.

[0156] Whether there is a signal on the interface channel of the device interface 240 can also be indicated based on the status identifier. After the signal source of the display device 200 is switched to the interface channel of the device interface 240, the display device 200 can access the data signal sent by the terminal device 500 through the device interface 240 through the hardware layer according to the interface protocol and data transmission requirements of the device interface 240. When the data signal is received, the status identifier is set to the first status identifier value, indicating that there is a signal on the interface channel of the device interface 240.

[0157] Therefore, the display device 200 can read the channel identifier and the state identifier, and if the channel identifier is the channel identifier value of the display device interface 240 and the state identifier is the first state identifier value, the reverse control identifier of the touch data forwarding service is set to the first identifier value. If the channel identifier is not the channel identifier value of the display device interface 240, the reverse control identifier of the touch data forwarding service is set to the second identifier value.

[0158] The on and off states of the touch data forwarding service can be represented based on the reverse control identifier. That is, when the signal source of the display device 200 is switched to the interface channel of the device interface 240 and there is a signal in the interface channel, the reverse control identifier of the touch data forwarding service is set to the first identifier value, indicating that the service state of the touch data forwarding service is on. When the signal source of the display device 200 switches out of the interface channel of the device interface 240, or the display device 200 starts other interfaces, the reverse control identifier of the touch data forwarding service is set to the second identifier value, indicating that the service state of the touch data forwarding service is off. Among them, other interface channels are interface channels other than the device interface 240, and other interfaces are window interfaces other than the media resource screen.

[0159] For example, setting the reverse control flag to a number "1" indicates that the touch data forwarding service is in an on state, and setting the reverse control flag to a number "0" indicates that the touch data forwarding service is in a off state.

[0160] Therefore, if Fig.17 The figure shows a reverse control judgment flow chart provided in the embodiment of the application. After the display device 200 obtains the touch data of the touch event, it can read the reverse control identifier of the touch data forwarding service. If the reverse control identifier is the first identifier value, it means that the service state of the touch data forwarding service is on and the terminal device 500 needs to be reverse controlled, and the subsequent steps of detecting the current display area of ​​the media asset screen and sending the touch data to the terminal device 500 are performed. If the reverse control identifier is the second identifier value, it means that the service state of the touch data forwarding service is off and the terminal device 500 does not need to be reverse controlled, and the subsequent steps are not performed.

[0161] In some embodiments, when the touch data forwarding service is turned on, when the user controls the terminal device 500 through the display device 200, there may be a problem of interruption of touch data sent to the terminal device 500. For example, when the user controls the terminal device 500 through the display device 200, an application interface of an application may be started in the display device 200. At this time, the display device 200 detects the start of the application interface and sets the touch data forwarding service to the off state, which will cause the touch data sent to the terminal device 500 to be interrupted.

[0162] In order to ensure that the touch data can be sent normally, a waiting-to-close state can be added to the touch data forwarding service. The waiting-to-close state is invisible to the user and is a state set when the touch data is sent to the terminal device 500.

[0163] When the touch data forwarding service of the display device 200 is in the on state, the user inputs a touch event through the touch interaction module of the display device 200. In response to the touch event, the display device 200 can generate touch data of the touch event and send the touch data or the remapped touch data to the terminal device 500. At the same time, the display device 200 can detect the interface state of the current user interface. If a window interface is started, the touch data forwarding service is switched from the on state to the to-be-closed state.

[0164] The pending shutdown state may also be based on the reverse control flag. For example, the reverse control flag is set to the number "2", indicating that the touch data forwarding service is in the pending shutdown state.

[0165] Therefore, in the process of executing touch data forwarding, the display device 200 can read the reverse control identifier of the touch data forwarding service. If the reverse control identifier is the third identifier value, it means that the service status of the touch data forwarding is in the to-be-closed state, indicating that the reverse control needs to be ended. Then, the end identifier is marked for the touch data to be sent, and the touch data marked with the end identifier is sent to the terminal device 500. At the same time, the touch data forwarding service is switched from the to-be-closed state to the closed state, so that the touch data can be sent normally.

[0166] It is understandable that marking the end mark for the touch data changes the touch event into an end event, similar to the up event in the touch event detected by the touch interaction module, to notify the terminal device 500 that the touch event is over.

[0167] like Fig.18 As shown, it is a schematic diagram of the process of touch data forwarding provided in an embodiment of the present application. The display device 200 can detect whether the service status of the touch data forwarding service is in the closed state to determine whether the terminal device 500 needs to be reversely controlled. When the terminal device 500 needs to be reversely controlled, the touch data forwarding service of the display device 200 reads the touch data transmitted from the hardware layer in a blocking mode every 10ms. And detect the current display area of ​​the media asset screen, and compare the current display area with the initial display area to determine whether the touch data needs to be remapped. At the same time, it can be detected whether the service status of the touch data forwarding service is in the to-be-closed state to determine whether to end the reverse control.

[0168] If the reverse control does not need to be terminated, the touch data is forwarded normally to the terminal device 500. If the reverse control needs to be terminated, an end flag is marked for the touch data to be sent to change the touch event to an end event, and the data is forwarded to the terminal device 500. At the same time, the reverse control flag is set, and the touch data forwarding service is switched from the pending shutdown state to the shutdown state, so that the touch event ends normally.

[0169] In some embodiments, in order to reduce invalid touch data forwarding. After acquiring touch data or remapped touch data, the display device 200 can calculate the touch area of ​​the touch data. If the touch area is located in the non-controllable area, the touch data is invalidated and not sent to the terminal device 500. If the touch area is located in the controllable area, the subsequent step of sending the touch data to the terminal device 500 is performed. The controllable area is the display area of ​​the media asset screen, and the non-controllable area is the display area other than the controllable area. For example, Figure 8 Taking the display interface shown as an example, the controllable area is the display area of ​​the media resource screen, and the non-controllable area is the shadow area on both sides of the media resource screen.

[0170] like Fig.19As shown, it is a flow chart of touch data processing provided in an embodiment of the present application. When a user inputs a touch event through the touch interaction module of the display device 200, the touch interaction module will receive the touch data generated by the touch event, and the touch data will be sequentially transmitted upward by the hardware layer of the display device 200 in the form of a data packet (frame data). After reading one frame of touch data, the display device 200 detects whether the service status of the touch data forwarding service is in a closed state. If it is in a closed state, it sleeps for 10ms to continue to read the next frame of touch data, and continues to detect whether it is in a closed state. If it is not in a closed state, it detects whether the current display area of ​​the media asset screen is the same as the initial display area to determine whether the media asset screen delivered by the terminal device 500 has been rotated and / or scaled. If it has been rotated and / or scaled, the touch data is remapped.

[0171] Then the touch area of ​​the touch data can be calculated to determine whether the touch area is a non-controllable area. If it is a non-controllable area, the touch data is marked with an end flag, the touch event is changed to an end event, the closed state is set, and the touch data is forwarded to the terminal device 500. If it is not a non-controllable area, it is detected whether the service state of the touch data forwarding service is a pending shutdown state. If it is a pending shutdown state, the touch data is marked with an end flag, the touch event is changed to an end event, the closed state is set, and the touch data is forwarded to the terminal device 500, so that the touch data sent to the terminal device 500 ends normally. If it is not a pending shutdown state, the touch data is forwarded to the terminal device 500, and the next touch data reading operation is performed after sleeping for 10ms.

[0172] Based on the above-mentioned wired same-screen reverse control method, a display device 200 is also provided in some embodiments of the present application, including: a display 260, a device interface 240 and a controller 250. Among them, the device interface 240 is connected to the terminal device 500. The device interface 240 is configured to transmit media images and touch data. The display 260 is configured to display the media images sent by the terminal device 500 through the device interface 240. The display 260 is connected to a touch interaction module, and the touch interaction module is used to receive touch events input by the user. The touch point coordinates of the touch interaction module correspond to the pixel point coordinates of the display 260. The controller 250 is configured to execute the following program steps:

[0173] When the display 260 presents the media resource picture sent by the terminal device 500 through the device interface 240 , the initial display area of ​​the media resource picture on the display 260 is stored.

[0174] In response to a touch event input by a user, touch data of the touch event is generated.

[0175] Get the initial display area and detect the current display area of ​​the media asset screen when a touch event is input.

[0176] If the current display area is the same as the initial display area, the touch data is sent to the terminal device 500 .

[0177] If the current display area is different from the initial display area, coordinate mapping is performed on the touch data according to the initial display area, and the mapped touch data is sent to the terminal device 500 .

[0178] It can be seen from the above technical solutions that the display device and wired same-screen reverse control method provided in the above embodiments can store the initial display area of ​​the media picture on the display when the display presents the media picture sent by the terminal device through the device interface. In response to the touch event input by the user, the touch data of the touch event is generated, and then the initial display area is obtained, and the current display area of ​​the media picture is detected. The current display area is compared with the initial display area. If the current display area is the same as the initial display area, the touch data is sent to the terminal device. If the current display area is different from the initial display area, coordinate mapping is performed on the touch data according to the initial display area, and the touch data obtained after mapping is sent to the terminal device. The method can remap the touch data when a change in the display area of ​​the media picture is detected, so as to solve the problem of touch position change caused by the horizontal and vertical screen conversion of the display device during wired same-screen reverse control.

[0179] Those skilled in the art can clearly understand that the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform. Based on this understanding, the technical solution in the embodiments of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various embodiments of the present invention or certain parts of the embodiments.

[0180] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0181] For the convenience of explanation, the above description has been made in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that: include: A device interface, the device interface is connected to the terminal device, and the device interface is configured to transmit media images and touch data; A display, configured to display the media resource screen sent by the terminal device through the device interface, the display is connected to a touch interaction module, the touch interaction module is used to receive touch events input by a user, and the touch point coordinates of the touch interaction module correspond to the pixel point coordinates of the display; The controller is configured as: When the display presents the media resource picture sent by the terminal device through the device interface, storing the initial display area of ​​the media resource picture on the display; In response to a touch event input by a user, generating touch data of the touch event; Acquire the initial display area, and detect the current display area of ​​the media resource screen when the touch event is input; If the current display area is the same as the initial display area, sending the touch data to the terminal device; If the current display area is different from the initial display area, coordinate mapping is performed on the touch data according to the initial display area, and the mapped touch data is sent to the terminal device.

2. The display device according to claim 1, characterized in that The controller is also configured to: Reading a reverse control flag of the touch data forwarding service, the reverse control flag comprising a first flag value and a second flag value, the first flag value indicating that the service state of the touch data forwarding service is in an on state, and the second flag value indicating that the service state of the touch data forwarding service is in a off state; If the reverse control identifier is the first identifier value, executing the step of sending the touch control data to the terminal device; If the reverse control identifier is the second identifier value, the step of sending the touch data to the terminal device is not performed.

3. The display device according to claim 1, characterized in that The controller is also configured to: Reading a channel identifier and a state identifier, wherein the channel identifier indicates an interface channel to which a signal source of the display device is switched; If the channel identifier is a channel identifier value representing the device interface and the state identifier is a first state identifier value, setting the reverse control identifier of the touch data forwarding service to the first identifier value, the first state identifier value indicating that a signal exists in the interface channel of the device interface; If the channel identifier is not a channel identifier value representing the device interface, the reverse control identifier of the touch data forwarding service is set to a second identifier value.

4. The display device according to claim 2, characterized in that The controller is also configured to: When the service state of the touch data forwarding service is in the on state, in response to the start instruction of the window interface, the reverse control flag of the touch data forwarding service is set to a third flag value, and the third flag value indicates that the service state of the touch data forwarding service is in the to-be-closed state.

5. The display device according to claim 2, characterized in that The controller executes sending the touch data to the terminal device, and is further configured to: Read the reverse control identifier of the touch data forwarding service; If the reverse control flag is the third flag value, it is the touch data marking end flag; The touch data marked with an end mark is sent to the terminal device.

6. The display device according to claim 1, characterized in that The display device further includes a rotating assembly configured to rotate the display; The controller is also configured to: In response to a rotation event generated by the rotation of the rotating component, detecting a rotation direction and a rotation angle of the rotation event; Rotating the media picture by the rotation angle in the opposite direction of the rotation direction, and scaling the media picture according to the display size of the display; The display is controlled to display the rotated and scaled media asset image.

7. The display device according to claim 6, characterized in that The controller performs coordinate mapping on the touch data according to the initial display area, and is further configured to: Calculating the positional relationship and proportional relationship of the touchable area in the initial display area; Coordinate mapping is performed on the touch data according to the position relationship and the proportional relationship.

8. The display device according to claim 7, characterized in that The controller calculates the positional relationship and proportional relationship of the touchable area in the initial display area, and is further configured to: Acquire vertex coordinates of the touchable area and vertex coordinates of the initial display area; According to the rotation direction, rotating the touchable area by the rotation angle; The conversion relationship between the vertex coordinates of the rotated touchable area and the vertex coordinates of the initial display area is calculated to obtain the position relationship and the proportional relationship.

9. The display device according to claim 1, characterized in that: The controller is also configured to: Calculating a touch area of ​​the touch data; If the touch area is located in a controllable area, a step of sending the touch data to the terminal device is performed, and the controllable area is a display area of ​​the media resource screen; If the touch area is located in a non-controllable area, the step of sending the touch data to the terminal device is not performed, and the non-controllable area is a display area excluding the controllable area.

10. A wired same-screen reverse control method, characterized in that: Applied to the display device according to any one of claims 1 to 9, the method comprising: When the display presents the media resource picture sent by the terminal device through the device interface, storing the initial display area of ​​the media resource picture on the display; In response to a touch event input by a user, generating touch data of the touch event; Acquire the initial display area, and detect the current display area of ​​the media resource screen when the touch event is input; If the current display area is the same as the initial display area, sending the touch data to the terminal device; If the current display area is different from the initial display area, coordinate mapping is performed on the touch data according to the initial display area, and the mapped touch data is sent to the terminal device.

Images