Display device and display device control method

Abstract

The invention relates to display equipment and a display equipment control method, and relates to the technical field of display equipment. The display device includes a display configured to display a user interface; the at least one controller is connected with the display and is configured to respond to a control message sent by the remote controller and analyze the control message; when it is analyzed that the control message is generated based on key operation on a trigger area in the remote controller when the remote controller is in a touch mode, focus view detection is carried out on a first interface currently displayed by the display; simulating a key event corresponding to the control message based on a key value corresponding to the trigger area under the condition that the focus view does not exist in the first interface; and executing the key event based on the first interface. By adopting the technical scheme, the compatibility of key event interaction logic in the touch mode of the remote controller can be realized, the smooth interaction between the remote controller and the display equipment is ensured, and the user experience and the application compatibility of the display equipment are improved.

Description

Technical Field

[0001] This application relates to the field of display device technology, and in particular to a display device and a display device control method. Background Technology

[0002] With the widespread adoption of smart TVs, set-top boxes, and other multimedia terminal devices, remote controls, as a primary human-computer interaction tool, are constantly evolving in their input methods. Currently, in addition to traditional button remotes, more and more devices support various types of remotes, including pointing remotes. However, display devices handle button events differently for different types of remotes. Specifically, when a pointing remote enters pointing mode, pressing a specific button (such as the OK button) no longer generates traditional button events on the display device. This prevents some interaction logic that relies on specific button events (such as showing / hiding views when the OK button is pressed) from being implemented between the pointing remote and the display device, impacting user experience and application compatibility. Summary of the Invention

[0003] This application provides a display device and a display device control method, which can achieve compatibility with button event interaction logic in remote control touch mode, ensure smooth interaction between remote control and display device, and improve user experience and application compatibility of display device.

[0004] In a first aspect, some embodiments provide a display device, including:

[0005] The display is configured to show a user interface; wherein the user interface has a cursor, the position of which is determined by the position of the remote control in three-dimensional space.

[0006] At least one controller, connected to a display, is configured to: parse a control message sent by a remote control; if the parsed control message was generated by a button operation on a trigger area of ​​the remote control while the remote control is in touch mode, perform focus view detection on a first interface currently displayed on the display; if no focus view is detected on the first interface, simulate a button event corresponding to the control message based on the button value corresponding to the trigger area; wherein the button value is the code value corresponding to a button operation on the trigger area while the remote control is in button mode; and execute the button event based on the first interface.

[0007] In the above technical solution, the display device includes a display for displaying a user interface, which has a cursor, and the position of the cursor on the user interface is determined by the position pointed to by the remote control in three-dimensional space. Then, when the display is currently displaying a first interface, at least one controller included in the display device can respond to a control message sent by the remote control, parse the control message, and if it is determined that the control message was generated by a button operation on a trigger area of ​​the remote control while the remote control is in touch mode, perform focus view detection on the first interface. If it is detected that the first interface does not have a focus view, it simulates the button event corresponding to the control message based on the button value corresponding to the trigger area (i.e., the code value corresponding to the button operation on the trigger area when the remote control is in button mode), and executes the button event based on the first interface. In this way, when the remote control is in touch mode and the display device is controlled by button operations on the trigger area of ​​the remote control, the display device can simulate the button events corresponding to the received control messages even when there is no focus view in the currently displayed interface. By executing the button events, it responds to the remote control control. Thus, in remote control touch mode, the interaction logic between the remote control and the display device based on the button operations on the trigger area of ​​the remote control is realized. This ensures compatibility of the button event interaction logic in remote control touch mode, guarantees smooth interaction between the remote control and the display device, and improves user experience and application compatibility of the display device.

[0008] Secondly, some embodiments provide a display device control method, applied to a display device, including:

[0009] In response to a control message sent by the remote control, the control message is parsed. The display device's screen is used to display the user interface, which has a cursor whose position is determined by the remote control's position in three-dimensional space. If the parsed control message was generated by a button operation on a trigger area of ​​the remote control while the remote control is in touch mode, a focus view detection is performed on the first interface currently displayed on the screen. If no focus view is detected in the first interface, a button event corresponding to the control message is simulated based on the button value corresponding to the trigger area. The button value is the code value corresponding to a button operation on the trigger area while the remote control is in button mode. The button event is then executed based on the first interface.

[0010] Thirdly, some embodiments also provide a display device control apparatus, applied to a display device, comprising:

[0011] The message parsing module is used to parse control messages sent by the remote control. The display device's screen displays the user interface, which has a cursor whose position is determined by the remote control's position in three-dimensional space. The view detection module performs focus view detection on the currently displayed first interface when the control message is parsed and the remote control is in touch mode, based on button operations on a trigger area of ​​the remote control. The event simulation module simulates the button event corresponding to the control message based on the button value corresponding to the trigger area when no focus view is detected on the first interface. The button value is the code value corresponding to the button operation on the trigger area when the remote control is in button mode. The event execution module executes the button event based on the first interface.

[0012] Fourthly, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the steps of the methods provided in some embodiments of the second aspect.

[0013] Fifthly, a computer program product is provided, comprising: a computer program that, when executed by a processor, implements the steps of the methods provided in some embodiments of the second aspect. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 A flowchart illustrating a defect image generation method provided in some embodiments of this application;

[0016] Figure 2 This is a schematic diagram of the hardware configuration of a display device provided in some embodiments of this application;

[0017] Figure 3 This is a schematic diagram of the hardware configuration of the control device provided in some embodiments of this application;

[0018] Figure 4 This is a schematic diagram of the software configuration of a display device provided in some embodiments of this application;

[0019] Figure 5A A schematic diagram illustrating the position mapping between a remote controller and a display device provided for some embodiments of this application;

[0020] Figure 5B A schematic diagram of a click provided for some embodiments of this application;

[0021] Figure 5C A schematic diagram of sliding provided for some embodiments of this application;

[0022] Figure 5D A schematic diagram illustrating continuous upward sliding provided in some embodiments of this application;

[0023] Figure 6 The diagram illustrates the execution of key events provided in some embodiments of this application.

[0024] Figure 7A A schematic flowchart illustrating a display device control method provided in some embodiments of this application;

[0025] Figure 7B This is a schematic diagram illustrating the execution of operations corresponding to control messages provided in some embodiments of this application;

[0026] Figure 7C This application provides schematic diagrams illustrating the interaction between a remote control and a display device in some embodiments.

[0027] Figure 8 A flowchart illustrating the execution of key events provided in some embodiments of this application;

[0028] Figure 9 A flowchart illustrating the execution of key events provided for other embodiments of this application;

[0029] Figure 10 A flowchart illustrating the detection of the focus view provided in some embodiments of this application;

[0030] Figure 11 A flowchart illustrating simulated key events provided in some embodiments of this application;

[0031] Figure 12 A flowchart illustrating the execution of key events provided in some embodiments of this application;

[0032] Figure 13 A flowchart illustrating the detection of a focus view provided for other embodiments of this application;

[0033] Figure 14 This is a flowchart illustrating a display device control method provided in other embodiments of this application. Detailed Implementation

[0034] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application as detailed in the claims.

[0035] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0036] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate. The terms "comprising" and "having," and any variations thereof, are intended to be comprehensive but not exclusive; for example, a product or device comprising a series of components is not necessarily limited to all components explicitly listed, but may include other components not explicitly listed or inherent to such product or device. The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code capable of performing the functions associated with that element.

[0037] In this embodiment, a remote control device refers to a device that controls a display device remotely. A display device generally refers to a device with screen display and data processing capabilities. For example, display devices include, but are not limited to, smart TVs, mobile terminals, computers, monitors, advertising screens, wearable devices, virtual reality devices, and augmented reality devices. The remote control device acts as the control device for the display device, controlling the display device through its movement.

[0038] Figure 1 This is a schematic diagram illustrating an operational scenario between a display device and a control device provided in some embodiments of this application. For example... Figure 1As shown, a user can operate the display device 200 via touch operation, a mobile terminal 300, and a control device 100. For example, the control device 100 can be a remote control, a stylus, a gamepad, etc. The mobile terminal 300 can serve as a control device for human-computer interaction between the user and the display device 200. The mobile terminal 300 can also serve as a communication device for establishing a communication connection with the display device 200 and exchanging data. In some embodiments, the mobile terminal 300 can have software applications installed on it and connect to the display device 200 via network communication protocols to achieve one-to-one control operation and data communication. Audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve synchronous display. Figure 1 The diagram also shows that the display device 200 communicates with the server 400 via various communication methods. The display device 200 can communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks. The display device 200 can provide broadcast television reception functionality and can also be additionally equipped with intelligent network television functionality that provides computer support, including but not limited to, internet television, smart television, and Internet Protocol television (IPTV).

[0039] Figure 2 Provided for some embodiments of this application Figure 1The diagram shows the hardware configuration of the display device 200. In some embodiments, the display device 200 may include at least one of a tuner / demodulator 210, a communication device 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface. In some embodiments, the detector 230 is used to collect signals from the external environment or to interact with the external environment. For example, the detector 230 includes a light receiver, a sensor for collecting ambient light intensity; or, the detector 230 includes an image acquisition device, such as a camera, which can be used to collect external environmental scenes, user attributes, or user interaction gestures; or, the detector 230 includes a sound acquisition device, such as a microphone, for receiving external sounds. In some embodiments, the display 260 includes display function components for presenting images and driving components for driving image display. The display 260 is used to receive and display image signals output from the controller 250. For example, the display 260 can be used to display video content, image content, menu control interface components, and user control UI interfaces, etc. In some embodiments, the communication device 220 is a component used to communicate with external devices or the server 400 according to various communication protocol types. The display device 200 may have multiple communication devices 220 depending on the supported communication methods. For example, when the display device 200 supports wireless network communication, it may have a communication device 220 with WiFi functionality. When the display device 200 supports Bluetooth connection communication, it needs to have a communication device 220 with Bluetooth functionality. In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and provide input / output interfaces from a first interface to an nth interface. The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in its memory. The controller 250 controls the overall operation of the display device 200. In some embodiments, the controller 250 and the tuner / demodulator 210 may be located in different separate devices; that is, the tuner / demodulator 210 may also be located in an external device of the main device containing the controller 250, such as an external set-top box. In some embodiments, a user can input user commands through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input commands through the graphical user interface (GUI). In some embodiments, the audio output device 270 can be a built-in speaker of the display device 200 or an external audio output device connected to the display device 200.In some embodiments, the display device 200 may also be provided with an external audio output terminal for connecting an external audio output device to the display device 200, thereby outputting sound from the display device 200. In some embodiments, the user input interface 280 may be used to receive instructions input by a user.

[0040] Figure 3 Provided for some embodiments of this application Figure 1 Hardware configuration block diagram of the central control device. (Example) Figure 3 As shown, the control device 100 may include: a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply. The control device 100 is configured to control the display device 200 and to receive user input operation commands, converting these commands into commands that the display device 200 can recognize and respond to, thus acting as an intermediary for interaction between the user and the display device 200. Under the control of the controller 110, the communication interface 130 enables communication of control signals and data signals with the display device 200. The communication interface 130 may include at least one of a WiFi chip 131, a Bluetooth module 132, an NFC (Near Field Communication) module 133, or other near-field communication modules. The user input / output interface 140 includes at least one of a microphone 141, a touch area 142, a sensor 143, a button 144, or other input interfaces. In some embodiments, to perform user interaction, the display device 200 may run an operating system. The operating system is a computer program used to manage and control the hardware and software resources of the display device 200. The operating system can (control the display device) provide a user interface, allowing users to interact with the display device 200 and supporting the running of various applications. It should be noted that the operating system can be a native operating system based on a specific operating platform, a deeply customized third-party operating system based on a specific operating platform, or an independent operating system specifically developed for the display device.

[0041] An operating system can be divided into different modules or levels based on the functions it implements, for example... Figure 4As shown, in some embodiments, the system can be divided into four layers, from top to bottom: the Applications layer (hereinafter referred to as the "Application Layer"), the Application Framework layer (hereinafter referred to as the "Framework Layer"), the System Library layer, and the Kernel layer. In some embodiments, the Application Layer provides services and interfaces for applications, enabling the display device 200 to run applications and interact with users based on the applications. At least one application can run in the Application Layer. These applications can be Windows programs, system settings programs, or clock programs that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the application packages in the Application Layer are not limited to the examples above. The Framework Layer provides Application Programming Interfaces (APIs) and programming frameworks for applications. The Application Framework Layer includes some predefined functions. The Application Framework Layer acts as a processing center, which determines the actions of applications in the Application Layer. Applications can access system resources and obtain system services during execution through the API interface. Figure 4As shown, the application framework layer in this embodiment includes a view system, managers, and content providers. The view system designs and implements the application's interface and interactions, and includes lists, grids, text boxes, buttons, etc. The managers include at least one of the following modules: an activity manager for interacting with all running activities in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to application packages currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface. In some embodiments, the activity manager manages the lifecycle of each application and common navigation and back functions, such as controlling application exit, opening, and back actions. The window manager manages all window programs, such as obtaining the screen size, determining the presence of a status bar, locking the screen, capturing the screen, and controlling changes to the displayed windows, such as shrinking, dithering, or distorting the display. In some embodiments, the system runtime library layer can provide support for the framework layer. When the framework layer is used, the operating system runs the instruction libraries contained in the system runtime library layer, such as C / C++ instruction libraries, to implement the functions required by the framework layer. In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. For example, ... Figure 4 As shown, the kernel layer can be configured with hardware drivers. The kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB (Universal Serial Bus) driver, HDMI (High Definition Multimedia Interface) driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver. It should be noted that the above example is merely a simple division of operating system functions and does not limit the specific operating system form of the display device 200 in this embodiment. Depending on the functions of the display device, the type of operating system, and other factors, the number and specific types of layers included in the operating system can take other forms.

[0042] In some embodiments, a display device is provided, comprising: a display configured to display a user interface, the user interface including at least one control; the user interface further having a cursor, the position of which is determined by the position of a remote control pointing in three-dimensional space. Optionally, a user can control the display device to start via voice, a remote control, or the power switch of a television device. After the display device is started, the user interface corresponding to the homepage is displayed on the display. The user can switch to other user interfaces via voice or a remote control, such as the user interface corresponding to a TV series tab, the user interface corresponding to a movie tab, or the user interface corresponding to a page provided by an application, etc. This application embodiment does not limit this. Optionally, among all the user interfaces displayed on the display, some user interfaces include controls, while others may not. The solution provided in this application embodiment is for user interfaces that include controls. The number of controls on a user interface that includes controls may be one or more, and this application embodiment does not limit this. Optionally, the remote control used in conjunction with the display device can be a remote control with pointing function, hereinafter referred to as a pointing remote control. When the pointing function is enabled, the user interface displayed on the display also has a cursor, the position of which is determined by the position of the pointing remote control pointing in three-dimensional space. In this embodiment, the position of the remote controller in three-dimensional space is also referred to as the position data of the remote controller in three-dimensional space. The specific implementation process of determining the position of the cursor on the user interface based on the position data of the remote controller is described below.

[0043] The position data of the pointing remote control in three-dimensional space can include position coordinates and pointing direction. The position coordinates are the physical position of the pointing remote control relative to the display device (e.g., horizontal distance, vertical height, etc.). The pointing direction is the pointing angle of the pointing remote control (e.g., horizontal and vertical angles relative to the display device). The pointing remote control has a built-in gyroscope to detect the rotation angle of the remote control and an accelerometer to detect the acceleration of the remote control's movement. Through these sensors, the pointing remote control can determine its own position data in three-dimensional space in real time. In some embodiments, after acquiring its own position data in three-dimensional space, the pointing remote control can send this position data to the display device. The display device can perform screen coordinate transformation based on this position data, thereby converting the position data of the pointing remote control in three-dimensional space to the screen coordinate system. The obtained screen coordinates can be used as the cursor position on the user interface, and the cursor can be drawn at this position to display the cursor. In some embodiments, the remote controller includes a coordinate transformation module. After acquiring its own position data in three-dimensional space, the remote controller can input this position data into the coordinate transformation module. The coordinate transformation module can perform screen coordinate transformation based on this position data, thereby converting the position data into the screen coordinate system to obtain screen coordinates. These screen coordinates can be used as the cursor position on the user interface, and the cursor position on the user interface is sent to the display device. After receiving the cursor position on the user interface, the display device draws the cursor at that position, thereby displaying the cursor. For example, if the cursor position on the user interface is coordinates (a, b), the display device draws the cursor at coordinates (a, b) on the user interface, thereby displaying the cursor.

[0044] For example, see Figure 5A As shown, when the position data of the remote control in 3D space is A, the cursor position on the user interface is at coordinates (a1, b1); when the position data is B, the cursor position on the user interface is at coordinates (a2, b2). When the position data is C, the screen coordinates calculated in the above way exceed the screen range, and the display device no longer draws the cursor; therefore, the cursor disappears. See also... Figure 5B As shown, the remote control supports tapping. When the cursor hovers over a control, the user can press the touch area (OK button) on the remote control. The display device then responds to this operation by executing the corresponding logic. These controls can be cards, buttons, input boxes, radio buttons, switches, dialog boxes, multi-select buttons, sliders, playback progress bars, scroll bars, drop-down menus, labels, lists, etc. This application embodiment does not limit the specific controls. See also Figure 5CAs shown, the remote control supports swiping. When a cursor is displayed on the user interface, the user can swipe up, down, left, or right on the touch area of ​​the remote control, and the display device will respond to these swipes. See also Figure 5D As shown, the remote control supports continuous swiping. When the cursor is displayed on the user interface, the user can continuously swipe up, down, left, or right on the touch area of ​​the remote control. The display device will respond to these continuous swipes and display a damped swipe effect on the user interface.

[0045] With the widespread adoption of smart TVs, set-top boxes, and other multimedia terminal devices, remote controls, as a primary human-computer interaction tool, are constantly evolving in their input methods. Currently, in addition to traditional button remotes, more and more devices support various types of remotes, including pointing remotes. However, display devices handle button events differently for different types of remotes. Specifically, when a pointing remote enters pointing mode, pressing a specific button (such as the OK button) no longer generates traditional button events on the display device. This prevents some interaction logic that relies on specific button events (such as showing / hiding views when the OK button is pressed) from being implemented between the pointing remote and the display device, impacting user experience and application compatibility.

[0046] In traditional technology, when using a remote control to control a display device, the remote control sends control messages to the display device in response to button operations. Current remote controls have two modes: button mode and touch mode (also known as pointing mode), which can be switched using a mode switch on the remote control.

[0047] In the remote control's button mode, the control signals include a keycode and an action. The keycode is the code value of the button being operated on; the action indicates the button's state. For example, an action of ACTION_DOWN (downward) means the button is pressed, and an action of ACTION_UP (upward) means the button is released after being pressed. The display device then parses the control signals to obtain the keycode and action, encapsulates them into an event (KeyEvent), and executes the KeyEvent to control the display device via the remote control.

[0048] Correspondingly, when the remote control is in touch mode, the display screen includes a cursor, and the cursor moves with the remote control to quickly reach the desired location. Furthermore, the OK button area on the remote control is converted into a touch area, allowing the user to perform actions such as... Figures 5C-5D The sliding motion shown can also be performed by pressing with a second pressure greater than the first pressure to perform a touch click operation on the touch area. In this case, the control signal generated by the remote control in response to the above button operation includes Source Type and button Action, but no longer includes the keycode of the OK button. Here, Source Type indicates the source of the control signal, such as a remote control, touch screen, mouse, etc. in touch mode.

[0049] For example, such as Figure 6 As shown, when the remote control is in button mode and the display device shows interface 601 generated based on DTV (Digital Television) signals, there are no view components in interface 601. The user can press the OK button on the remote control to render a menu including signal source 1, signal source 2, and signal source 3 on the upper layer of interface 601, allowing switching between the displayed signal sources. However, when the remote control is in touch mode and the display device also shows interface 601, pressing the OK button causes the display device to parse the control signal to obtain the Source Type and button Action. Since the button value is not parsed, the display device does not encapsulate the button event, and therefore does not respond to the control signal. In touch mode, the interaction logic relying on the button event corresponding to the OK button operation cannot be implemented, and the user cannot control the display device by pressing the OK button. Consequently, no menu is displayed on the screen.

[0050] Based on this, in some embodiments, a display device control method is provided. This display device control method can be implemented by the cooperation of a display device and a remote controller, etc.

[0051] In one exemplary embodiment, the display device control method is applied to a controller in a display device as an example.

[0052] The display device includes a monitor and at least one controller. The communication connection between the monitor and the controller can be a wired or wireless communication connection. The monitor is configured to display a user interface with a cursor, the position of which is determined by the position of a remote control in three-dimensional space. When a user operates the display device using a remote control (such as a pointing remote or air mouse remote), the cursor can be moved on the display device screen by moving the remote control. The user can move the cursor to any position on the display device screen, and the cursor's position on the screen is determined by the position of the remote control in three-dimensional space. The cursor's position on the display device screen can be understood as its position within the user interface displayed on the display device. Optionally, the user interface can be an interface generated based on audio and video data sent by an external device communicating with the display device, such as an HDMI device, DTV device, or CEC (Consumer Electronics Control) device; alternatively, the user interface can be an interface pre-stored in the display device.

[0053] like Figure 7A As shown, a display device control method applied to a controller in a display device may include the following steps:

[0054] S701 responds to control messages sent by the remote control and parses the control messages.

[0055] Depending on the communication method between the remote control and the display device, the remote control can send control messages adapted to the communication connection method to the display device in various forms. For example, it can send Bluetooth signals (such as BLE (Bluetooth Low Energy) data packets), infrared codes, or radio frequency signals as control messages. Optionally, when the remote control and the display device are connected via Bluetooth, the remote control sends control messages to the display device in the form of Bluetooth signals; or, when the remote control and the display device are connected via infrared, the remote control sends control messages to the display device in the form of infrared codes.

[0056] When the display device's screen is currently displaying a first interface, upon receiving a control message from the remote control, the display device's controller (hereinafter referred to as the controller) first parses the control message to obtain its content. Specifically, when the remote control is in button mode, the control message includes the button value and button action. Correspondingly, when the remote control is in touch mode, the control message includes at least the source type and button action.

[0057] Optionally, when the remote control is in touch mode, the control message may also include at least one of the following: timestamp, remote control identifier, identifier of the display screen associated with the remote control (if the display device has multiple displays), pointer information (coordinates and / or pressure) of button operation.

[0058] Optionally, the Source Type is represented by a bitmask. Optionally, in the case that the same control message includes multiple input sources, the Source Type can be the result of a bitwise OR operation of the masks of the multiple sources.

[0059] The first interface includes a cursor, the position of which is determined by the position pointed to by the remote control in three-dimensional space, and the cursor moves as the remote control moves. Optionally, the first interface is generated based on audio and video data sent by an external device communicating with the display device, or the first interface is a pre-stored interface in the display device.

[0060] In one optional embodiment, the display device further includes an external device interface configured to connect to an external device to receive audio and video data sent by the external device. In this case, S701 may include parsing a control message sent by a remote control when the display is controlling the display to show the first interface based on the audio and video data. The display device can connect to the external device via the external device interface to receive audio and video data sent by the external device, thereby controlling the display to show the first interface based on the aforementioned audio and video data.

[0061] For example, if a display device is connected to a set-top box via an HDMI interface, in a live streaming scenario, the server corresponding to the live stream source sends the live audio and video data to the set-top box. The set-top box then sends this live audio and video data to the display device via the HDMI interface, allowing the display device to control the monitor to display the live stream image as the first interface. Alternatively, if the display device is connected to a CEC device via an HDMI interface, the CEC device sends its own operating interface's audio and video data to the display device via the HDMI interface, allowing the display device to control the monitor to display the CEC device's operating interface as the first interface. Furthermore, during the process of the monitor displaying the first interface based on the audio and video data, the controller can parse control messages received from the remote control.

[0062] For example, a display device connects to a digital television signal source device via a DTV interface. The display device obtains the program list by receiving Program Specific Information (PSI) and Service Information (SI) from the digital television signal. This information is periodically sent to the display device by the digital television signal source device in a specific table format during the transport stream. This process includes the following stages: Initial search: After entering the DTV service or application, the display device first searches for the main frequency, obtains the Program Association Table (PAT), and then finds the Program Map Table (PMT) for all programs based on the PAT; Program information parsing: Obtains the audio / video PID (Packet Identifier) ​​of the program from the PMT table, obtains the program name from the Service Description Table (SDT), and obtains the program schedule from the Event Information Table (EIT); Electronic Program Guide (EPG) generation: The SI information is organized in an orderly manner to generate a program bulletin-like format, which is then displayed on the display device in real time according to the user's specific actions.

[0063] S702, when it is determined that the control message was generated by the remote control being in touch mode and based on the button operation of the trigger area on the remote control, performs focus view detection on the first interface currently displayed on the display.

[0064] As mentioned earlier, when the remote control is in touch mode, specific button areas (such as the OK button) on the remote control are converted into touch areas. Therefore, whether the user applies a first-pressure touch or a second-pressure button operation on the touch area, the display device interprets it as a touch click operation on that touch area. The control message generated by the remote control in response to this button operation includes the Source Type and the button Action, but not the button value. Therefore, when the controller parses the received control message and obtains the Source Type and button Action, but lacks the button value, the controller can determine that the control message was generated based on a button operation on a trigger area of ​​the remote control while the remote control is in touch mode. This trigger area is the button area of ​​a specific button on the remote control, which is converted into a touch area when the remote control is in touch mode. Thus, a user's button operation on a button in the trigger area is equivalent to a touch click operation on that trigger area.

[0065] Therefore, after parsing the control message, since no key value was parsed, the controller cannot encapsulate the key event to implement the interaction logic corresponding to the key operation. To achieve this interaction logic, the controller first performs focus view detection on the currently displayed first interface. The focus view is the currently interactive view. Optionally, the focus view can be the view component on which the focus is hovering in the first interface, or a view component whose focus attribute indicates it is in focus.

[0066] Optionally, if the control message is parsed as being generated by a button operation on the trigger area of ​​the remote control while the remote control is in touch mode, and the button Action indicates that the action state of the button operation on the touch area is pressed and released (such as ACTION_UP), then focus view detection is performed on the first interface currently displayed on the screen.

[0067] Optionally, the controller can traverse each view component of the first interface one by one to perform focus view detection on each view component based on the visibility and focus attribute of each view component, so as to determine the view component whose visibility is visible and whose focus attribute is in focus as the focus view; or, the controller can perform image detection on the first interface to determine the view component in the first interface where the cursor is located as the focus view.

[0068] S703, when it is detected that there is no focus view in the first interface, simulates the key event corresponding to the control message based on the key value corresponding to the trigger area.

[0069] Among them, the button value is the code value corresponding to the button operation on the trigger area when the remote control is in button mode.

[0070] If no focused view is detected in the first interface, in order to control the display device, the controller can determine that the remote control is in button mode. Based on the code value corresponding to the button operation on the trigger area, it takes the key value corresponding to the trigger area as the key value. Then, based on the key value of the trigger area and the key action in the control message, it encapsulates the key event to obtain a Key Event. Here, since the key value of the trigger area is not parsed from the control message, but is encapsulated based on the key value of the trigger area stored in button mode and the key action in the control message, it is equivalent to a simulated key event obtained by encapsulating the key event based on the key value and key action in the control message sent by the remote control when the remote control is in button mode. The controller can then execute the key event based on the first interface to complete the interaction logic of the key event corresponding to the button operation on the trigger area of ​​the remote control, and realize the control of the display device. For example, the second interface corresponding to the original key event is presented in the first interface.

[0071] Optionally, the button value corresponding to the trigger area can be the code value of the button located in the trigger area on the remote control. For example, when the remote control is in touch mode and the OK button area on the remote control is converted into a touch area, the button value corresponding to the trigger area is the code value of the OK button.

[0072] Optionally, the display device may pre-store the key values ​​corresponding to the aforementioned trigger areas. Thus, when it is detected that there is no focus view in the first interface, the key value can be directly read to generate the key event corresponding to the aforementioned control message.

[0073] In one optional embodiment, the display device control method further includes, when a focus view is detected in the first interface, executing the operation corresponding to the control message based on the focus view. In this embodiment, as mentioned above, the focus view is the currently interactive view. Therefore, when a focus view is detected in the first interface, the controller can execute the operation corresponding to the control message based on the focus view without simulating the button event corresponding to the control message. The focus view can directly trigger events such as OnClick events corresponding to the touch area. The controller can execute the operation corresponding to the control message in various ways based on different focus views. No specific limitation is made in this regard; all operations that the display device can implement are within the scope of this embodiment. Optionally, when a focus view is detected in the first interface, the controller encapsulates the event according to the Source Type and button Action in the control information to obtain a touch event, and then executes the touch event based on the focus view to realize the operation corresponding to the control message.

[0074] Optionally, the controller renders and displays the interface corresponding to the focused view on top of the first interface currently displayed on the monitor, such as... Figure 7B As shown, when the "Frequently Used Functions" view is present in the first interface, the controller executes the operation corresponding to the control message, controlling the display to show the interface including the sub-menus "Sound Mode" and "Screen Brightness Mode" of "Frequently Used Functions". Alternatively, the controller executes the operation corresponding to the control message based on the display function corresponding to the focused view. For example, if the focused view is the pause / pause button, the controller executes the operation corresponding to the control message, controlling the display to stop playing the video stream of the first interface, or controlling the display to start playing the video stream of the first interface, etc.

[0075] S704 executes button events based on the first interface.

[0076] After simulating the button events corresponding to the control messages based on the button values ​​corresponding to the trigger area, the controller can execute the button events based on the first interface to complete the interaction logic of the button events corresponding to the button operations on the trigger area of ​​the remote control, thereby realizing the control of the display device. The controller can execute button events in various ways. For example, rendering other interfaces on top of the first interface, stopping or starting playback of the video stream containing the first interface, adjusting the playback volume, displaying a progress bar, etc., without specific limitations. For example, such as... Figure 6 As shown, menu 602 is rendered on top of the displayed interface 601 (first interface).

[0077] For example, such as Figure 7C As shown, in the WMS (Window Manager Service) layer of the display device, many Windows are managed. Each interface of an interactive application on the display device is an Activity, and each Activity holds a Window. Each interactive application can contain multiple Activities. For example, clicking to open an interactive application will first open a login screen, which is a LoginActivity. After successful login verification, the main screen will appear, which is the HomeActivity. When an Activity is created, a root view is created simultaneously. The root view is responsible for managing other views displayed on the interface.

[0078] Taking the OK button area as an example, when the user presses the OK button, the remote control can determine the corresponding data packet header source marker according to its own mode, and encapsulate it into Bluetooth data packets of different modes, which are then sent to the display device as remote control events (i.e., control messages). The remote control event is captured by the device node corresponding to the remote control in the Linux kernel of the display device and uploaded to the InputReader in the InputManagerService of the display device. After receiving the remote control event, the InputReader parses the event to generate a key event or a touch event based on the data packet header source marker and message content (keycode and button action; or, source type and button action), and sends the generated event to the InputDispatcher. The InputDispatcher uploads the received key event or motion event to the foreground window (Window1), which is the Window held by the first interface currently displayed on the display device's monitor. This foreground window is visible and interactive to the user. The foreground window (Window) sends a Key Event or Motion Event to the root view (DecorView) of the first interface (Activity1, the Activity that holds the foreground window) through ViewRootImpl (View Root Implementation, which can be understood as an event forwarder). After receiving the Key Event or Motion Event, DecorView iterates through all Views to find the View that has focus and can handle the event.If a View with focus and the ability to handle events is found, the Key Event or Motion Event is directly sent to the View for processing. After receiving the Key Event, if the View determines that it is a key event corresponding to the OK button, it will trigger the View's OnClick callback event by default (i.e., the key operation corresponding to the OK button is pre-mapped to the View's OnClick event). Therefore, only the interaction logic for the View (such as pausing playback) needs to be added to the OnClick callback event. This added interaction logic is the interaction logic for the key event corresponding to the OK button's key operation. Then, the View will trigger the OnClick click event (similar to a mouse click) to implement the above interaction logic. If the View determines that it is a touch event corresponding to the OK button, the View will also trigger the View's OnClick callback event. Through the same logic as the key event corresponding to the OK button, the OnClick click event will be triggered (similar to a mouse click) to implement the above interaction logic. In this case, for the case where a View exists, there is no need to perform additional event callback adaptation for the touch event corresponding to the OK button. If no focused View (focused view) is found, and the key event is determined to be an OK button press, the OnKeyUp callback event of the View's Activity is invoked. Within this OnKeyUp callback, the Activity determines that the key operation corresponds to the OK button press and executes the pre-defined interaction logic for that OK button press, such as displaying a channel list in a live streaming scenario. If the touch event is determined to be an OK button press, a touch event callback is invoked. If the touch event callback determines that the key operation corresponds to the OK button press, the OK button press event is simulated, and the logic of invoking the OnKeyUp callback event of the View's Activity is returned to execute the pre-defined interaction logic for that OK button press, such as displaying a channel list in a live streaming scenario.

[0079] In this embodiment, the display device includes a display for displaying a user interface, which has a cursor whose position on the user interface is determined by the position pointed to by the remote controller in three-dimensional space. Furthermore, when the display is currently displaying a first interface, at least one controller included in the display device can respond to a control message sent by the remote controller, parse the control message, and if it is determined that the control message was generated by a button operation on a trigger area of ​​the remote controller while the remote controller is in touch mode, perform focus view detection on the first interface. If it is detected that the first interface does not have a focus view, it simulates the button event corresponding to the control message based on the button value corresponding to the trigger area (i.e., the code value corresponding to the button operation on the trigger area when the remote controller is in button mode), and executes the button event based on the first interface. In this way, when the remote control is in touch mode and the display device is controlled by button operations on the trigger area of ​​the remote control, the display device can simulate the button events corresponding to the received control messages even when there is no focus view in the currently displayed interface. By executing the button events, it responds to the remote control control. Thus, in remote control touch mode, the interaction logic between the remote control and the display device based on the button operations on the trigger area of ​​the remote control is realized. This ensures compatibility of the button event interaction logic in remote control touch mode, guarantees smooth interaction between the remote control and the display device, and improves user experience and application compatibility of the display device.

[0080] Based on the above embodiments, in an exemplary embodiment, the execution of the key event in S704 is further refined. Optionally, as... Figure 8 As shown, the following steps may be included:

[0081] S801, query the second interface corresponding to the pre-stored first interface under the key event.

[0082] When the monitor displays different interfaces, the new interface displayed can differ depending on the button operations in the trigger area of ​​the remote control. Therefore, for different interfaces, the corresponding new interface can be pre-set under the control of button operations in the trigger area of ​​the remote control, and the correspondence between each interface and the new interface can be stored. For example, when the monitor displays an interface generated based on a DTV signal, the DTV signal sub-menu can be displayed under the control of button operations in the trigger area of ​​the remote control; or, for example, when the monitor displays the video frames of the currently playing video stream in full screen, the playback pause operation and / or the playback progress of the video stream can be executed under the control of button operations in the trigger area of ​​the remote control.

[0083] Based on this, after simulating the above-mentioned button events, the system first queries the pre-stored second interface corresponding to the first interface currently displayed on the display under the button event, that is, queries the pre-stored second interface displayed under the control of the button operation of the trigger area in the remote control.

[0084] S802 renders and displays the second interface on top of the first interface.

[0085] Once the second interface is found, the controller can render and display the second interface on top of the first interface currently displayed on the monitor.

[0086] In this embodiment, when the remote control is in touch mode, by pre-storing the second interface corresponding to the first interface under the button event, the interaction logic between the remote control and the display device regarding screen switching based on the button operation of the trigger area in the remote control is realized. This achieves compatibility of the button event interaction logic in the remote control touch mode, ensures smooth interaction between the remote control and the display device, and improves the user experience and application compatibility of the display device.

[0087] Based on the above embodiments, in an exemplary embodiment, the execution of the key event in S704 is further refined. Optionally, as... Figure 9 As shown, the following steps may be included:

[0088] S901, based on the communication protocol between the display device and external devices, converts key events into control commands that conform to the communication protocol and sends the control commands to the external devices.

[0089] As mentioned earlier, in some cases, the display device controls the monitor to display the first interface based on audio and video data sent by an external device. Therefore, after simulating the aforementioned button events, the display device can execute the button events to obtain the corresponding interface from the external device for display, thereby realizing the interactive logic between the remote control and the display device regarding screen switching based on button operations in the trigger area of ​​the remote control. Based on this, the controller can convert the button events into control commands conforming to the communication protocol between the display device and the external device, and send these control commands to the external device through the external device interface. For example, if the display device communicates with a CEC device via an HDMI interface, the communication protocol between the display device and the CEC device is the CEC protocol, and the controller can convert the button events into control commands conforming to the CEC protocol. As another example, if the display device communicates with a set-top box via an HDMI interface, and the communication protocol between the display device and the CEC device is the TMDS (Transition Minimized Differential Signaling) protocol, the controller can convert the button events into control commands conforming to the TMDS protocol.

[0090] S902, a third interface is rendered and displayed on top of the first interface in response to control commands sent by the external device.

[0091] After receiving the control command, the external device determines the third interface in response to the control command and sends the third interface to the display device through the external device interface. Thus, after receiving the third interface, the controller can render and display the third interface on top of the first interface.

[0092] In this embodiment, when the remote control is in touch mode, the interaction logic between the remote control and the display device based on the communication protocol is realized through information interaction between the display device and the external device. This logic enables the switching of screens based on the button operations of the trigger area on the remote control, thereby achieving compatibility of the button event interaction logic in the touch mode of the remote control, ensuring smooth interaction between the remote control and the display device, and improving the user experience and application compatibility of the display device.

[0093] Based on the above embodiments, in an exemplary embodiment, the detection of the focus view in S702 is further refined. Optionally, if the controller runs the current application and system modules, then as follows... Figure 10 As shown, the following steps may be included:

[0094] S1001 generates touch events corresponding to control messages through system modules and sends touch events to the current interactive application.

[0095] A system module is a functional module that can encapsulate events and communicate with the current interactive application. It can be an independent module in the controller or integrated with other functional modules in the controller.

[0096] The current interactive application is the application to which the first interface currently displayed on the screen belongs. For example, when playing a video stream, the video frames currently displayed on the screen are the first interface currently displayed, and the video player used to play the video stream is the current interactive application.

[0097] When the control message is parsed to be generated by a button operation on the trigger area of ​​the remote control while the remote control is in touch mode, the system module in the controller encapsulates the event according to the Source Type and button Action in the control information to obtain the touch event corresponding to the control message, and sends the touch event to the current interactive application.

[0098] Optionally, each interface of the current interactive application has an event listener registered to listen for touch events generated by the system module when the interface is displayed on the screen.

[0099] S1002, the touch event is parsed through the current interactive application, and if the specified information is parsed, the focus view detection is performed on the first interface currently displayed on the display.

[0100] The specified information is used to characterize button operations on the trigger area when the remote control is in touch mode.

[0101] Upon receiving the aforementioned touch event, the current interactive application can parse it to determine its source. As mentioned earlier, the touch event is generated by encapsulating the Source Type and button Action from the control information. Therefore, the touch event may include specified information corresponding to the Source Type, representing button operations on the trigger area when the remote control is in touch mode. Optionally, the touch event may include Source Type and button Action, or it may include specified parameters. When the value of the specified parameter is a specified value (such as 1), the specified information is determined. For example, the touch event carries a data structure including a button Action field and a Source Type field. By parsing the touch event, the field values ​​of the button Action field and the Source Type field can be obtained. The field value of the Source Type field is the specified information. Thus, the field value of the Source Type field represents the focus view detection of the first interface currently displayed on the screen when the remote control is in touch mode and a button operation on the trigger area is performed. Since the key value cannot be obtained after parsing the touch event, the current interactive application cannot execute the touch event. Therefore, the current interactive application can perform focus view detection on the first interface currently displayed on the screen. Optionally, the current interactive application can traverse each view component of the first interface one by one, and perform focus view detection on each view component according to the visibility and focus attribute of each view component, so as to determine the view component that is visible and focused as the focus view.

[0102] Optionally, the current interactive application starts from the root view component of the first interface currently displayed on the screen and traverses each view component of the first interface; when traversing each view component, focus view detection is performed on the view component and its subview components until all view components have been traversed or a focus view has been detected.

[0103] In this embodiment, when the control message generated by the remote control is generated by the button operation on the trigger area of ​​the remote control while the remote control is in touch mode, the focus view detection of the first interface is realized through the interaction between the system module running in the controller and the current interactive application based on touch events. This enables the current interactive application to which the first interface belongs to to participate in the response to the control message, thereby improving the response speed and the specificity of the response.

[0104] Based on the above embodiments, in an exemplary embodiment, the simulation of the key event in S703 is further refined. Optionally, if the controller is running the current application and system modules, then as follows... Figure 11 As shown, the following steps may be included:

[0105] S1101, if the current interactive application detects that there is no focus view in the first interface, query the button value corresponding to the trigger area through the current interactive application.

[0106] If the current interactive application detects that there is no focus view in the first interface, in order to control the display device, the current interactive application queries the code value corresponding to the key operation of the trigger area when the remote control is in key mode, and uses it as the key value corresponding to the trigger area.

[0107] Optionally, the button value corresponding to the aforementioned trigger area can be the code value of a button located within the trigger area on a remote control. Optionally, the storage area allocated to the current interactive application in the display device can pre-store the button value corresponding to the aforementioned trigger area. Thus, when the current interactive application detects that there is no focused view in the first interface, it directly reads the button value from its own storage area and sends the button value to the system module to notify the system module to generate a button event.

[0108] Optionally, the current interactive application sends an event generation notification carrying the key value corresponding to the trigger area to the system module; or, the current interactive application sends the key value to the system module.

[0109] S1102 simulates the key events corresponding to control messages based on key values ​​through the system module.

[0110] After receiving the key value corresponding to the trigger area sent by the current interactive application, the system module encapsulates the event based on the key value and the key action in the control message to obtain the Key Event, so as to simulate the key event corresponding to the control message.

[0111] In this embodiment, through the interaction between the current interactive application and the system module based on the key value corresponding to the trigger area, when there is no focus view in the currently displayed interface, the key event corresponding to the received control message is simulated. By executing the key event in the remote control touch mode, the interaction logic between the remote control and the display device based on the key operation of the trigger area in the remote control is realized. This achieves compatibility of the key event interaction logic in the remote control touch mode, ensures smooth interaction between the remote control and the display device, and improves the user experience and application compatibility of the display device.

[0112] Based on the above embodiments, in an exemplary embodiment, the execution of the key event in S704 is further refined. Optionally, if the controller is running the current application and system modules, then as follows... Figure 12 As shown, the following steps may be included:

[0113] S1201 sends key events to the current interactive application through the system module.

[0114] After the system module simulates the key event corresponding to the above control message based on the key value corresponding to the trigger area sent by the current interactive application, the system module can send the key event to the current interactive application.

[0115] S1202, execute button events based on the first interface of the current interactive application.

[0116] Upon receiving the aforementioned button event, the current interactive application can execute the button event based on the first interface to complete the interaction logic corresponding to the button operation on the trigger area of ​​the remote control, thereby controlling the display device. The current interactive application can execute the button event in various ways. For example, it can render other interfaces on top of the first interface, stop playing the video stream containing the first interface, start playing the video stream containing the first interface, adjust the playback volume, display a progress bar, etc. No specific limitations are imposed on these methods. Optionally, the current interactive application can query the pre-stored second interface corresponding to the first interface under the button event and render and display the second interface on top of the first interface.

[0117] In this embodiment, through the interaction between the current interactive application and the system module based on key events, the current interactive application executes the key event based on the first interface currently displayed on the display. Even when there is no focus view in the currently displayed interface, by executing the key event in the remote control touch mode, the interaction logic between the remote control and the display device is realized based on the key operation of the trigger area in the remote control. This achieves compatibility of key event interaction logic in the remote control touch mode, ensures smooth interaction between the remote control and the display device, and improves user experience and application compatibility of the display device.

[0118] Based on the above embodiments, in an exemplary embodiment, the detection of the focus view in S702 is further refined. Optionally, as follows: Figure 13 As shown, the following steps may be included:

[0119] S1301, starting from the root view component of the first interface currently displayed on the monitor, traverse each view component of the first interface.

[0120] Typically, the first interface currently displayed by the display device includes at least one view component, and this at least one view component is set in at least one view component layer. In the case of multiple view components and multiple view component layers, each view component layer includes at least one view component, and the multiple view component layers have a hierarchical nesting relationship. Therefore, the controller can first determine the root view component of the first interface, and then, starting from the root view component, sequentially traverse each view component of the first interface. Optionally, starting from the root view component, traverse each view component of the first interface layer by layer.

[0121] S1302, when traversing each view component, perform focus view detection on the view component and its subview components until all view components have been traversed or a focus view has been detected.

[0122] When traversing each view component, you can first perform focus view detection on the view component and its child view components, that is, detect the focus view in the view component and its child view components.

[0123] Optionally, when traversing each view component, the visibility of that view component is checked, i.e., whether the view component is in a displayed state. Specifically, if the visibility of a view component indicates that it is invisible, the view components are traversed again; correspondingly, if the visibility of a view component indicates that it is displayed, the focus attribute of that view component is further checked, i.e., whether the view component is in focus. If the focus attribute of a view component indicates that it is in focus, then that view component is the focus view; correspondingly, if the focus attribute of a view component indicates that it is not in focus, then that view component is not the focus view. Therefore, the focus attributes of each subview component of the view component are further checked. If the focus attribute of any subview component indicates that it is in focus, then that subview component is the focus view; if the focus attribute of all subview components indicates that they are not in focus, the next view component is traversed. The process is repeated until all view components have been traversed or a focus view has been detected. If all view components have been traversed but no focus view has been detected, it is determined that no focus view is detected in the first interface currently displayed on the monitor.

[0124] In this embodiment, by traversing each view component of the first interface and focusing on the view component, the omission of the focus view in the first interface due to detection omission can be avoided, thereby improving the detection accuracy of the focus view detection of the first interface currently displayed on the display, and ensuring the accuracy of the subsequent response to the above control information.

[0125] Based on the above embodiments, in an exemplary embodiment, the implementation of the above display device control method is described through the interaction between the remote controller, external device, and display device. Figure 14 As shown, the following steps may be included:

[0126] S1401, the remote control sends a control message to the display device.

[0127] S1402, an external device sends audio and video data to a display device.

[0128] S1403, the display device controls the display to show the first interface based on audio and video data, and parses the control messages sent by the remote control in response to the control messages sent by the remote control.

[0129] S1404, when the display device parses the control message and finds that the remote control is in touch mode, based on the button operation of the trigger area on the remote control, the system module generates the touch event corresponding to the control message and sends the touch event to the current interactive application.

[0130] S1405, The display device parses the touch event through the current interactive application.

[0131] S1406, when the display device resolves the specified information, it starts from the root view component of the first interface currently displayed on the display through the current interactive application, traverses each view component of the first interface, and performs focus view detection on the view component and its subview components when traversing each view component, until all view components have been traversed or a focus view has been detected.

[0132] S1407, when the display device detects that there is no focus view in the first interface through the current interactive application, it queries the key value corresponding to the trigger area through the current interactive application, and simulates the key event corresponding to the control message through the system module based on the key value.

[0133] S1408, the display device sends a key event to the current interactive application through the system module, and the current interactive application executes the key event based on the first interface.

[0134] The specific implementation methods of S1401-S1408 are the same as those in the above embodiments, and will not be repeated here.

[0135] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0136] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0137] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0138] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0139] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this application. The above embodiments only illustrate several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of this application. It should be noted that for those skilled in the art, several modifications and improvements can be made without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A display device, characterized in that, include: A display is configured to display a user interface; wherein the user interface has a cursor whose position on the user interface is determined by the position pointed to by a remote control in three-dimensional space; At least one controller, connected to the display, is configured to: In response to a control message sent by the remote controller, the control message is parsed; When the control message is parsed as being generated by a button operation on a trigger area of ​​the remote control while the remote control is in touch mode, focus view detection is performed on the first interface currently displayed on the display. If no focus view is detected in the first interface, the key event corresponding to the control message is simulated based on the key value corresponding to the trigger area; wherein, the key value is the code value corresponding to the key operation on the trigger area when the remote control is in key mode; Based on the first interface, the button event is executed.

2. The display device according to claim 1, characterized in that, The display device further includes: An external device interface is configured to connect to an external device to receive audio and video data sent by the external device; The controller executes a control message in response to the remote control, and when parsing the control message, it is configured as follows: When controlling the display to show the first interface based on the audio and video data, the control message is parsed in response to the control message sent by the remote control.

3. The display device according to claim 2, characterized in that, When the controller executes the key event based on the first interface, it is configured as follows: Query the pre-stored second interface corresponding to the first interface under the key event, and render and display the second interface on top of the first interface; or, Based on the communication protocol between the display device and the external device, the key event is converted into a control command that conforms to the communication protocol, the control command is sent to the external device, and a third interface in response to the control command is rendered and displayed on the upper layer of the first interface.

4. The display device according to any one of claims 1-3, characterized in that, The controller runs the current interactive application and system modules. When the controller performs focus view detection on the first interface currently displayed on the display, it is configured as follows: The system module generates a touch event corresponding to the control message and sends the touch event to the current interactive application. The touch event is parsed by the current interactive application, and when the specified information is parsed, the focus view of the first interface currently displayed on the display is detected; wherein, the specified information is used to characterize the button operation on the trigger area when the remote control is in touch mode.

5. The display device according to claim 4, characterized in that, When the controller executes a key event corresponding to the control message based on the key value corresponding to the trigger area, it is configured as follows: If the current interactive application detects that there is no focus view in the first interface, the current interactive application queries the button value corresponding to the trigger area. The system module simulates the key events corresponding to the control message based on the key value.

6. The display device according to claim 4, characterized in that, When the controller executes the key event based on the first interface, it is configured as follows: The system module sends the key event to the current interactive application. The button event is executed by the current interactive application based on the first interface.

7. The display device according to any one of claims 1-3, characterized in that, When the controller performs focus view detection on the first interface currently displayed on the display, it is configured to: Starting from the root view component of the first interface currently displayed on the display, traverse each view component of the first interface; When each view component is traversed, focus view detection is performed on the view component and its subview components until all view components have been traversed or a focus view has been detected.

8. The display device according to any one of claims 1-3, characterized in that, The controller is also configured to: If a focus view is detected in the first interface, the operation corresponding to the control message is executed based on the focus view.

9. A method for controlling a display device, characterized in that, Applied to a display device, the method includes: In response to a control message sent by a remote controller, the control message is parsed; wherein, the display device's screen is used to display a user interface, the user interface having a cursor, the position of the cursor on the user interface being determined by the position pointed to by the remote controller in three-dimensional space; When the control message is parsed as being generated by a button operation on a trigger area of ​​the remote control while the remote control is in touch mode, focus view detection is performed on the first interface currently displayed on the display. If no focus view is detected in the first interface, the key event corresponding to the control message is simulated based on the key value corresponding to the trigger area; wherein, the key value is the code value corresponding to the key operation on the trigger area when the remote control is in key mode; Based on the first interface, the button event is executed.

10. The method according to claim 9, characterized in that, The step of responding to a control message sent by the remote controller and parsing the control message includes: When the control display shows the first interface of audio and video data sent by an external device, the control message is parsed in response to the control message sent by the remote control.

Images