Display device and cursor-based interaction method

Abstract

The invention relates to a display device and a cursor-based interaction method. The display apparatus includes: a display configured to display an image and / or a user interface; the controller is configured to obtain a cursor coordinate of the remote controller and a screen range of the display; under the condition that the coordinate component exceeding the screen range exists in the cursor coordinate, modifying the coordinate component exceeding the screen range into the screen range, and displaying the cursor of the remote controller according to the modified cursor coordinate; displaying a pop-up prompt used for representing that the cursor coordinates exceed the screen range before modification, and displaying a visual effect prompt used for representing that the cursor coordinates exceed the screen range before modification at the screen edge corresponding to the cursor coordinates before modification. Due to the fact that when the cursor exceeds the screen range, the cursor can still be maintained to be displayed in the screen range, the cursor is still visible, and a user can know the position of the cursor in real time through the pop-up window and the visual effect on the edge of the screen.

Description

Technical Field

[0001] The present application relates to the field of computer application technology, and in particular to a display device and a cursor-based interaction method. Background Art

[0002] When a pointing remote control is currently in use, the cursor of the pointing remote control may go beyond the screen range. On this basis, the user may not realize that the previous operation has moved the cursor beyond the screen range, and the cursor position may be lost, causing inconvenience in operation. Summary of the Invention

[0003] Based on this, it is necessary to provide a display device and a cursor-based interaction method that provide users with convenient operation to address the above technical problems.

[0004] In a first aspect, the present application provides a display device, comprising:

[0005] A display configured to display a user interface; the user interface having a cursor, wherein a position of the cursor on the user interface is determined by a position pointed to by the remote control in three-dimensional space;

[0006] The controller is configured as:

[0007] Get the cursor coordinates of the remote control and the screen range of the display;

[0008] If a coordinate component of the cursor coordinates exceeds the screen range, the coordinate component exceeding the screen range is modified to be within the screen range, and the cursor of the remote control is displayed according to the modified cursor coordinates;

[0009] A pop-up prompt is displayed to indicate that the cursor coordinates before the modification exceed the screen range, and a visual effect prompt is displayed at the screen edge corresponding to the cursor coordinates before the modification to indicate that the cursor coordinates before the modification exceed the screen range.

[0010] The above technical solution has the following advantages or beneficial effects: since the cursor coordinates can be automatically modified to within the screen range when the cursor exceeds the screen range, even if the user moves the cursor of the remote control beyond the screen range, the cursor can still be displayed within the screen range and remain visible, so that the user can understand the cursor position in real time, which provides convenience for user operation. On this basis, although the cursor coordinates are modified to within the screen range, the user may not realize that the previous operation has moved the cursor outside the screen range, so that the visual feedback to the user can be enhanced through pop-ups and visual effects at the edge of the screen, making the user more intuitive about the interaction status between the remote control and the display device. Overall, the user's operating experience on the display device can be improved. In addition, the above functions can all be implemented through software without the need for additional hardware support, thereby reducing costs.

[0011] In one embodiment, the area beyond the screen range is divided into an upper left corner area, a left edge area, a lower left corner area, a lower edge area, a lower right corner area, a right edge area, an upper right corner area, and an upper edge area; the controller modifies the coordinate components beyond the screen range to within the screen range, and is configured to:

[0012] When the cursor coordinates are in the upper left corner area, the horizontal coordinate component of the cursor coordinates is modified to the horizontal coordinate component of the upper left vertex, and the vertical coordinate component of the cursor coordinates is modified to the vertical coordinate component of the upper left vertex;

[0013] When the cursor coordinates are in the left edge area, the abscissa component of the cursor coordinates is modified to the abscissa component of the point on the left edge;

[0014] When the cursor coordinates are in the lower left corner area, the horizontal coordinate component of the cursor coordinates is modified to the horizontal coordinate component of the lower left vertex, and the vertical coordinate component of the cursor coordinates is modified to the vertical coordinate component of the lower left vertex;

[0015] When the cursor coordinates are in the lower edge area, the ordinate component of the cursor coordinates is modified to the ordinate component of the point on the lower edge;

[0016] When the cursor coordinates are in the lower right corner area, the abscissa component of the cursor coordinates is modified to the abscissa component of the lower right corner vertex, and the ordinate component of the cursor coordinates is modified to the ordinate component of the lower right corner vertex;

[0017] When the cursor coordinates are in the right edge area, the horizontal coordinate component of the cursor coordinates is modified to the vertical coordinate component of the point on the right edge;

[0018] When the cursor coordinates are in the upper right corner area, the abscissa component of the cursor coordinates is modified to the abscissa component of the upper right corner vertex, and the ordinate component of the cursor coordinates is modified to the ordinate component of the upper right corner vertex;

[0019] When the cursor coordinates are in the upper edge area, the ordinate component of the cursor coordinates is modified to the ordinate component of the point on the upper edge.

[0020] The above technical solution has the following advantages or beneficial effects: since the area beyond the screen range can be divided into multiple areas, the cursor coordinates are automatically modified to within the screen range according to the area in which the cursor coordinates fall. This allows the cursor to be maintained within the screen range even if the user moves the cursor of the remote control beyond the screen range, and the cursor remains visible, thereby enabling the user to understand the cursor position in real time, providing convenience for user operation.

[0021] In one embodiment, the controller is further configured to:

[0022] When it is determined according to the cursor coordinates of the remote controller that the cursor is hovering over the control, description information for describing the control in the properties of the control is displayed through a floating window.

[0023] The above technical solution has the following advantages or beneficial effects: when the cursor hovers over a control, description information for describing the control can be displayed according to the control properties, thereby providing a prompt for key content of the control where the cursor is hovering, thereby enhancing the intuitiveness of the interaction.

[0024] In one embodiment, the controller is further configured to:

[0025] When it is determined based on the cursor coordinates of the remote controller that the cursor is hovering over a control, a corresponding matching cursor shape rule is determined based on the associated information of the control;

[0026] Modifies the cursor shape when the cursor hovers over a control, according to the cursor shape rules.

[0027] The above technical solution has the following advantages or beneficial effects: since the cursor shape can be changed according to the associated information of the control when the cursor hovers over the control, the user is prompted by the cursor shape, while the visual feedback to the user is enhanced, and the fun and recognition are increased.

[0028] In one embodiment, the controller is further configured to:

[0029] When it is detected that the cursor of the remote control has been staying for a predetermined time, a visual effect prompt indicating that the cursor will be hidden is displayed;

[0030] When the cursor is hidden, display a visual effect prompt indicating that the cursor is hidden;

[0031] When movement of the cursor of the remote controller is detected, the cursor is restored to display.

[0032] The above technical solution has the following advantages or beneficial effects: When the cursor remains stationary for a long time, the cursor can be gradually hidden and a visual effect can be used to prompt the user, thereby reducing distractions when the user is viewing the display device and improving the viewing experience. In addition, when the cursor on the remote control is needed, the hidden cursor can be restored by moving it, thereby simultaneously meeting the user's control needs.

[0033] In one embodiment, the visual effect prompt used to indicate that the cursor is about to be hidden is a countdown number displayed on the cursor; the visual effect prompt used to indicate that the cursor has been hidden is a pop-up prompt.

[0034] The above technical solution has the following advantages or beneficial effects: Because the visual effect prompt used to indicate that the cursor is about to be hidden is displayed on the cursor, it is more closely related to the hidden object and is more directly and effectively perceived by the user. The pop-up prompt can clearly inform the user that the cursor has been hidden, allowing the user to effectively understand the cursor status.

[0035] In one embodiment, the controller performs a process of detecting movement of a cursor of a remote controller and is configured to:

[0036] When the changes in the coordinate components of the detected cursor coordinates all exceed the preset changes, it is determined that the cursor movement is detected; when the changes in the coordinate components of the detected cursor coordinates do not exceed the preset changes, it is determined that the cursor movement is not detected.

[0037] The above technical solution has the following advantages or beneficial effects: since it is possible to determine whether the cursor is triggered by mistake when the cursor movement is detected after the cursor is hidden, it is possible to reduce the situation where the cursor is restored due to false triggering and reduce the interference when the user watches the display device.

[0038] In one embodiment, the controller is further configured to:

[0039] In response to a function setting operation on the remote controller, setting at least one of an on / off state or a prompt style of a prompt function to update a local configuration file;

[0040] Synchronize local profiles to the cloud so that profiles can be synchronized when logging in on other devices.

[0041] Since the above functions can be set to meet customized needs, the display is more flexible.

[0042] In a second aspect, the present application further provides a cursor-based interaction method, which is applied to the display device provided by various possible implementations of the first aspect, and the method includes:

[0043] Get the cursor coordinates of the remote control and the screen range of the display;

[0044] In the case where a coordinate component of the cursor coordinates exceeds the screen range, the coordinate component exceeding the screen range is modified, and the cursor of the remote controller is displayed according to the modified cursor coordinates;

[0045] According to the cursor coordinates before the modification, a pop-up prompt is displayed to indicate that the cursor coordinates before the modification exceed the screen range, and a visual effect prompt is displayed at the screen boundary corresponding to the cursor coordinates before the modification to indicate that the cursor coordinates before the modification exceed the screen range.

[0046] The above technical solution has the following advantages or beneficial effects: since the cursor coordinates can be automatically modified to within the screen range when the cursor exceeds the screen range, even if the user moves the cursor of the remote control beyond the screen range, the cursor can still be displayed within the screen range and remain visible, so that the user can understand the cursor position in real time, which provides convenience for user operation. On this basis, although the cursor coordinates are modified to within the screen range, the user may not realize that the previous operation has moved the cursor outside the screen range, so that the visual feedback to the user can be enhanced through pop-ups and visual effects at the edge of the screen, making the user more intuitive about the interaction status between the remote control and the display device. Overall, the user's operating experience on the display device can be improved. In addition, the above functions can all be implemented through software without the need for additional hardware support, thereby reducing costs.

[0047] In one embodiment, the area beyond the screen range is divided into an upper left corner area, a left edge area, a lower left corner area, a lower edge area, a lower right corner area, a right edge area, an upper right corner area, and an upper edge area; and modifying the coordinate components beyond the screen range to within the screen range includes:

[0048] When the cursor coordinates are in the upper left corner area, the horizontal coordinate component of the cursor coordinates is modified to the horizontal coordinate component of the upper left vertex, and the vertical coordinate component of the cursor coordinates is modified to the vertical coordinate component of the upper left vertex;

[0049] When the cursor coordinates are in the left edge area, the abscissa component of the cursor coordinates is modified to the abscissa component of the point on the left edge;

[0050] When the cursor coordinates are in the lower left corner area, the horizontal coordinate component of the cursor coordinates is modified to the horizontal coordinate component of the lower left vertex, and the vertical coordinate component of the cursor coordinates is modified to the vertical coordinate component of the lower left vertex;

[0051] When the cursor coordinates are in the lower edge area, the ordinate component of the cursor coordinates is modified to the ordinate component of the point on the lower edge;

[0052] When the cursor coordinates are in the lower right corner area, the abscissa component of the cursor coordinates is modified to the abscissa component of the lower right corner vertex, and the ordinate component of the cursor coordinates is modified to the ordinate component of the lower right corner vertex;

[0053] When the cursor coordinates are in the right edge area, the horizontal coordinate component of the cursor coordinates is modified to the vertical coordinate component of the point on the right edge;

[0054] When the cursor coordinates are in the upper right corner area, the abscissa component of the cursor coordinates is modified to the abscissa component of the upper right corner vertex, and the ordinate component of the cursor coordinates is modified to the ordinate component of the upper right corner vertex;

[0055] When the cursor coordinates are in the upper edge area, the ordinate component of the cursor coordinates is modified to the ordinate component of the point on the upper edge.

[0056] The above technical solution has the following advantages or beneficial effects: since the area beyond the screen range can be divided into multiple areas, the cursor coordinates are automatically modified to within the screen range according to the area in which the cursor coordinates fall. This allows the cursor to be maintained within the screen range even if the user moves the cursor of the remote control beyond the screen range, and the cursor remains visible, thereby enabling the user to understand the cursor position in real time, providing convenience for user operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0058] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

[0059] Figure 2 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0060] Figure 3A schematic diagram of the hardware configuration of a control device provided in some embodiments of the present application;

[0061] Figure 4 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0062] Figure 5 A schematic diagram of dividing the area beyond the screen range provided in some embodiments of the present application;

[0063] Figure 6 1 is a flow chart of a cursor-based interaction method according to an embodiment;

[0064] Figure 7 A schematic diagram of interaction for establishing a connection between a pointing remote control and a display device in one embodiment;

[0065] Figure 8 This is a schematic diagram of an interaction for sending data to a display device in one embodiment;

[0066] Figure 9 An interactive diagram of modifying the cursor coordinates of a remote controller in one embodiment;

[0067] Figure 10 An interactive diagram showing control description information in one embodiment;

[0068] Figure 11 An interactive schematic diagram of modifying a cursor shape in one embodiment;

[0069] Figure 12 A schematic diagram of interaction of hiding and restoring a cursor in one embodiment;

[0070] Figure 13 A schematic diagram of a cursor hiding countdown in one embodiment;

[0071] Figure 14 A schematic diagram of a prompt showing that the cursor is hidden in one embodiment;

[0072] Figure 15 FIG. 4 is a structural block diagram of a cursor-based interaction device in one embodiment. DETAILED DESCRIPTION

[0073] The following embodiments are described in detail, with examples illustrated in the accompanying drawings. When the following description refers to the drawings, identical numbers in different figures represent identical or similar elements unless otherwise indicated. The embodiments described in the following embodiments are not intended to represent all possible implementations consistent with the present application. They are merely examples of systems and methods consistent with certain aspects of the present application, as detailed in the claims.

[0074] It should be noted that the brief descriptions of terms in this application are only for the purpose of facilitating the understanding of the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their ordinary and usual meanings.

[0075] In the specification and claims of this application and the accompanying drawings, the terms "first," "second," "third," etc. are used to distinguish similar or similar objects or entities, and are not necessarily intended to limit a particular order or sequence, unless otherwise noted. It should be understood that the terms used in this manner are interchangeable under appropriate circumstances.

[0076] The terms "comprise," "include," and "have," and any variations thereof, are intended to cover but not exclude inclusion; for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0077] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functionality associated with that element.

[0078] In the embodiments of the present application, the display device 200 generally refers to a device capable of displaying images and processing data. For example, the display device 200 includes but is not limited to a smart TV, a mobile terminal, a computer, a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc.

[0079] Figure 1 This is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG, a user can operate the display device 200 through touch operation, the mobile terminal 300 and the control device 100. For example, the control device 100 can be a remote controller, a stylus pen, a handle, etc.

[0080] The mobile terminal 300 can function as a control device for performing human-computer interaction between a user and the display device 200. The mobile terminal 300 can also function as a communication device for establishing a communication connection with the display device 200 and exchanging data. In some embodiments, the mobile terminal 300 can install software applications with the display device 200, enabling connection and communication via a network communication protocol, enabling one-to-one control operations and data communication. Audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 for synchronized display.

[0081] like Figure 1As shown in FIG, the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0082] The display device 200 may provide a broadcast receiving television function, and may also additionally provide an intelligent network television function with a computer support function, including but not limited to network television, smart TV, Internet Protocol television (IPTV), etc.

[0083] Figure 2 Some embodiments of this application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0084] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0085] In some embodiments, detector 230 is used to collect signals from the external environment or external interactions. For example, detector 230 may include a light receiver, such as a sensor for collecting ambient light intensity; or an image collector, such as a camera, for collecting external environmental scenes, user attributes, or user interaction gestures; or a sound collector, such as a microphone, for receiving external sounds.

[0086] In some embodiments, the display 260 includes a display component for presenting images and a driver component for driving image display. The display 260 is configured to receive image signals output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, menu control interface components, and user control UI interfaces.

[0087] In some embodiments, the communication device 220 is a component used to communicate with an external device or server 400 according to various communication protocol types. The display device 200 can be provided with multiple communication devices 220 depending on the supported communication methods. For example, if the display device 200 supports wireless network communication, the display device 200 can be provided with a communication device 220 including WiFi functionality. If the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including Bluetooth functionality.

[0088] The communication device 220 can establish a communication connection between the display device 200 and an external device or server 400 via a wireless or wired connection. A wired connection can connect the display device 200 to an external device via a data cable, an interface, or other components. A wireless connection can connect the display device 200 to an external device via a wireless signal or wireless network. The display device 200 can establish a connection with an external device directly or indirectly through a gateway, router, or connection device.

[0089] In some embodiments, the controller 250 may include at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processor, and a power processor, and first to nth interfaces for input / output. The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in a memory. The controller 250 controls the overall operation of the display device 200.

[0090] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0091] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0092] In some embodiments, the audio output device 270 may be a local speaker of the display device 200, or an external audio output device connected to the display device 200. For the external audio output device connected to the display device 200, the display device 200 may further be provided with an external audio output terminal, through which the audio output device may be connected to the display device 200 to output the sound of the display device 200.

[0093] In some embodiments, the user input interface 280 may be configured to receive instructions from a user.

[0094] Figure 3 Some embodiments of this application provide Figure 1 The hardware configuration diagram of the control device in the figure. Figure 3 As shown, the control device 100 may include: a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0095] The control device 100 is configured to control the display device 200 , and can receive user input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200 .

[0096] In some embodiments, the control device 100 may be a smart device. For example, the control device 100 may be installed with various applications for controlling the display device 200 according to user needs.

[0097] In some embodiments, as Figure 1 As shown, the mobile terminal 300 or other intelligent electronic devices can play a similar function as the control device 100 after installing the application for controlling the display device 200 .

[0098] The controller 110 includes a processor 112, RAM 113, ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between internal components and external and internal data processing functions.

[0099] Under the control of the controller 110, the communication interface 130 communicates control signals and data signals with the display device 200. The communication interface 130 may include at least one of a WiFi chip 131, a Bluetooth module 132, an NFC module 133, or other near field communication modules.

[0100] The user input / output interface 140 includes at least one of a microphone 141 , a touch panel 142 , a sensor 143 , a button 144 and other input interfaces.

[0101] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with the communication interface 130, such as a WiFi, Bluetooth, or NFC module, to encode user input commands via the WiFi protocol, Bluetooth protocol, or NFC protocol and transmit them to the display device 200.

[0102] The memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0103] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0104] To facilitate user interaction, in some embodiments, the display device 200 may run an operating system. An operating system is a computer program used to manage and control the hardware and software resources of the display device 200. The operating system may provide a user interface (control the display device), allow the user to interact with the display device 200, and support the running of various application programs.

[0105] It should be noted that the operating system can be a native operating system based on a specific operating platform, a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for display devices.

[0106] The operating system can be divided into different modules or layers according to the functions implemented, e.g. Figure 4 As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (referred to as "application layer"), the application framework layer (referred to as "framework layer"), the system library layer and the kernel layer.

[0107] In some embodiments, the application layer provides services and interfaces for applications, enabling the display device 200 to run applications and interact with the user based on the applications. The application layer can host at least one application, which can include built-in window programs, system settings programs, clock programs, and the like, or applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the examples above.

[0108] The framework layer provides applications with an application programming interface (API) and programming framework. The application framework layer includes predefined functions. The application framework layer acts as a processing center, determining the actions taken by applications in the application layer. Through the API, applications can access system resources and services during execution.

[0109] like Figure 4As shown, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: an activity manager for interacting with all activities running in the system; a location manager for providing system services or applications with access to the system location service; a package manager for retrieving various information related to the application packages currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0110] In some embodiments, the activity manager is used to manage the lifecycle of each application and common navigation back functions, such as controlling application exit, opening, and back. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, taking screenshots, and controlling changes in display windows, such as shrinking, shaking, or distorting the display window.

[0111] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction library contained in the system runtime layer, such as the C / C++ instruction library, to implement the functions to be implemented by the framework layer.

[0112] In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. Figure 4 As shown, the kernel layer can be configured with hardware drivers, and the drivers included in the kernel layer can be at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0113] It should be noted that the above example is only a simple division of the operating system functions and does not constitute a limitation on the specific operating system form of the display device 200 in the embodiment of the present application. Depending on factors such as the function of the display device and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.

[0114] In some embodiments, the control device 100 may be a pointing remote control, and the display device 200 may be running an Android operating system. In Android, the OnHoverListener interface can be used to monitor hover events of user interface controls. This interface is typically used to detect when a pointer device (such as a mouse, stylus, or pointing remote control) is hovering over a control. Using the OnHoverListener interface, it is possible to monitor hover events of a pointing remote control cursor over a user interface control.

[0115] Among them, the OnHoverListener interface can obtain:

[0116] (1) Basic information of the control: including the control's ID, type, size, and position.

[0117] (2) Control status: Status information of the current control, such as whether it is selected, whether it is available, etc.

[0118] (3) Attribute information: custom attribute information, such as the operational attributes of the control (happy, sad, recommended, etc.). These attributes are usually stored through the control's tag or custom attribute field.

[0119] The implementation principle of the OnHoverListener interface includes:

[0120] (1) Event monitoring: When the cursor pointing to the remote control hovers over the control, the system triggers a hover event and calls the onHover method of OnHoverListener.

[0121] (2) Event processing: In the onHover method, the basic information and status of the control are obtained through the event object.

[0122] (3) Attribute extraction: Use the getTag method of the control or other custom methods to extract the operational attributes of the control.

[0123] (4) Feedback mechanism: trigger corresponding visual feedback (such as color change, animation effect) based on the acquired attribute information.

[0124] In some embodiments, the display device 200 and the control device 100 can bidirectionally transmit data via the Bluetooth protocol. Bluetooth is a short-range wireless communication technology that supports bidirectional data transmission between devices. It is widely used to connect various electronic devices, such as mobile phones, headphones, remote controls, and smart TVs.

[0125] The implementation process of bidirectional data transmission between the display device 200 and the control device 100 is as follows:

[0126] (1) Pairing and connection: First, the control device 100 and the display device 200 need to be paired via Bluetooth to establish a secure connection channel. This usually involves steps such as device discovery, pairing code verification, and connection establishment.

[0127] (2) Data channel establishment: Once the connection is established, a data channel can be established between devices through the L2CAP (logical link control and adaptation protocol) layer in the Bluetooth protocol stack for the transmission of data packets.

[0128] (3) Sending data: When the user selects a control on the display device 200, the TV sends the control's attribute data packet to the control device 100 via Bluetooth. The data packet usually contains the control's ID, attribute type, and status information.

[0129] (4) Receiving data: After receiving the data packet, the control device 100 parses the attribute information therein and triggers corresponding tactile feedback (such as vibration, buzzing, etc.) according to a preset feedback mechanism.

[0130] (5) Feedback confirmation: To ensure the reliability of data transmission, the Bluetooth protocol supports an ACK (acknowledgement) mechanism. The receiving device sends a confirmation signal back to the sending device after successfully receiving the data packet.

[0131] In addition, during the bidirectional data transmission process, optimization can also be performed:

[0132] (1) Data compression: To improve transmission efficiency, data packets can be compressed to reduce the amount of data transmitted.

[0133] (2) Error detection and correction: The Bluetooth protocol has built-in error detection and correction mechanisms to ensure the accuracy and integrity of data transmission.

[0134] (3) Low power mode: BLE (Bluetooth Low Energy) mode can reduce the energy consumption of the device and extend the battery life without affecting the transmission performance.

[0135] Through the above mechanism, the Bluetooth protocol realizes efficient two-way data transmission between the remote control and the TV, providing users with a rich interactive experience.

[0136] In some embodiments, the principle of cursor detection for pointing remote controller may be as follows:

[0137] 1. Sensor data collection

[0138] Pointing remote controls usually integrate multiple types of sensors:

[0139] (1) IMU (Inertial Measurement Unit):

[0140] It consists of a three-axis accelerometer and a gyroscope (some also include a magnetometer), and calculates the device's attitude angle (pitch angle, yaw angle) by measuring angular velocity (gyroscope) and linear acceleration (accelerometer).

[0141] Example data fusion: The gyroscope updates the attitude at high frequency, the accelerometer provides gravity direction calibration, and the Kalman filter or complementary filter is used to eliminate gyroscope drift.

[0142] (2) Optical positioning module:

[0143] The pointing remote control may have a built-in camera or infrared sensor that captures reference markers (such as infrared LEDs) on the edge of the TV screen and calculates the pointing position based on triangulation or image recognition.

[0144] 2. Attitude calculation and coordinate conversion

[0145] (1) Spatial posture modeling:

[0146] Treat the pointing remote control as a ray source in three-dimensional space and establish a local coordinate system centered on the device. Use quaternions or Euler angles to describe the device's pointing direction (ray vector).

[0147] (2) Screen coordinate system mapping:

[0148] Define the screen of the display device 200 as a two-dimensional plane coordinate system (usually with the upper left corner as the origin), and project the device pointing ray onto the screen plane. The field of view (FOV) calibration needs to be considered, as follows:

[0149] x_screen=(arctan(θ_x)) / α_x×W+C_x

[0150] y_screen=(arctan(θ_y)) / α_y×H+C_y

[0151] Where θ is the device deflection angle, α is the preset field of view angle parameter, W / H is the screen resolution, and C is the calibration offset.

[0152] 3. Dynamic calibration mechanism

[0153] Initial calibration: When using the remote for the first time, the user needs to align the pointing remote control with the center of the screen to complete the coordinate system origin calibration.

[0154] Real-time compensation: Corrects gyroscope accumulated errors through magnetometers or optical reference points to ensure long-term pointing stability.

[0155] In combination with the above-mentioned principle description, an embodiment of the present application provides a display device, including:

[0156] a display configured to display images and / or a user interface;

[0157] The controller is configured as:

[0158] Get the cursor coordinates of the remote control and the screen range of the display;

[0159] If a coordinate component of the cursor coordinates exceeds the screen range, the coordinate component exceeding the screen range is modified to be within the screen range, and the cursor of the remote control is displayed according to the modified cursor coordinates;

[0160] A pop-up prompt is displayed to indicate that the cursor coordinates before the modification exceed the screen range, and a visual effect prompt is displayed at the screen boundary corresponding to the cursor coordinates before the modification to indicate that the cursor coordinates before the modification exceed the screen range.

[0161] Here, taking a pointing remote control as an example, the cursor coordinates of the pointing remote control refer to the position of the cursor of the pointing remote control within the screen range. The screen range of the display can be represented by physical size or resolution, which is not specifically limited in the embodiments of the present application.

[0162] The screen range is represented by the resolution, such as 1920*1080, and the cursor coordinates are represented by (x, y). The upper left corner of the screen is the coordinate origin, the right is the positive direction of the x-axis, and the downward is the positive direction of the y-axis. For example, the value range of the horizontal coordinate component within the screen range is 0 to 1920, and the value range of the vertical coordinate component is 0 to 1080. For a certain cursor coordinate, if x is less than 0 or x is greater than 1920, the horizontal coordinate component of the cursor coordinate exceeds the screen range; if y is less than 0 or y is greater than 1080, the vertical coordinate component of the cursor coordinate exceeds the screen range.

[0163] Through the above process, it is possible to determine whether there are coordinate components in the cursor coordinates that exceed the screen range. On this basis, the coordinate components that exceed the screen range can be modified. Specifically, the modification method can modify the cursor coordinates to be within the screen range, such as the center position, which is not specifically limited in the embodiment of the present application. After the modification is completed, the cursor pointing to the remote control can be further displayed according to the modified cursor coordinates. Thus, when the user moves the cursor pointing to the remote control, even if the cursor exceeds the screen range during the movement, the cursor can still be maintained displayed within the screen range and remain visible.

[0164] In addition to keeping the cursor displayed within the screen range, the user can also be further made aware that the cursor is moved outside the screen range during the process of moving the cursor. Thus, a pop-up prompt can be displayed to indicate that the cursor coordinates before the modification exceed the screen range. Among them, the pop-up prompt can be a toast pop-up window, and the pop-up window can contain text prompts, images or video prompts such as "The pointing cursor has currently left the TV screen range", and the embodiments of the present application do not make specific restrictions on this. The display position of the pop-up window can be the middle bottom of the screen, the upper right corner, or the position of the cursor coordinates before the modification, and the present application does not make specific restrictions on this.

[0165] In addition to displaying pop-up prompts, visual effect prompts can also be displayed at the screen edges corresponding to the cursor coordinates before the modification. The screen edges corresponding to the cursor coordinates before the modification can refer to the entire screen edges, that is, the upper, lower, left, and right edges simultaneously display visual effect prompts; or they can refer to the edges adjacent to the cursor coordinates before the modification, which is not specifically limited in the embodiments of the present application. For example, if the cursor coordinates before the modification are outside the screen range and located in the upper left corner area, visual effect prompts can be displayed at both the left and upper edges.

[0166] It should be noted that when displaying visual effect prompts, the visual effect may not fill the entire edge. For example, when the cursor coordinates before modification are outside the screen range and are located in the upper left corner area, the visual effect prompt can be displayed simultaneously only on the portion of the left edge and the portion of the upper edge close to the upper left corner vertex; of course, the visual effect prompt can also be displayed simultaneously on the entire left edge and the entire upper edge, and this embodiment of the application does not specifically limit this. In addition, the specific visual effect to be used can be set according to actual needs. For example, a red stroke with a shadow effect can be displayed to prompt the user, and this embodiment of the application does not specifically limit this.

[0167] It should also be noted that, in actual implementation, only one of the two prompt methods, pop-up prompt and visual effect prompt, can be used for prompting; or when the user moves the remote control, both prompt methods can be used at the same time when the user exceeds the screen range for the first time, and the visual effect prompt method can be used for prompting when the user exceeds the screen range subsequently. The embodiments of the present application do not make specific limitations on this.

[0168] The above-mentioned prompt method for notifying the user that the cursor has exceeded the screen range is mainly a visual prompt. In actual implementation, the prompt can also be provided through auditory or tactile means, such as a voice prompt announcing that the cursor has exceeded the screen range, or a tactile prompt such as a vibration on the remote control itself. Visual, auditory, and tactile prompts can also be used simultaneously, and this embodiment of the application is not specifically limited to this.

[0169] The above-mentioned display device can automatically modify the cursor coordinates to be within the screen range when the cursor exceeds the screen range. This allows the cursor to be displayed within the screen range even if the user moves the cursor pointing to the remote control beyond the screen range, and the cursor remains visible, so that the user can understand the cursor position in real time, which provides convenience for user operation. On this basis, although the cursor coordinates are modified to be within the screen range, the user may not realize that the previous operation has moved the cursor outside the screen range, so that the visual feedback to the user can be enhanced through pop-ups and visual effects at the edge of the screen, making the user more intuitive about the interaction status between the pointing remote control and the display device. Overall, the user's operating experience on the display device can be improved. In addition, the above functions can all be implemented through software without the need for additional hardware support, thereby reducing costs.

[0170] In some embodiments, the area beyond the screen range is divided into an upper left corner area, a left edge area, a lower left corner area, a lower edge area, a lower right corner area, a right edge area, an upper right corner area, and an upper edge area; the controller modifies the coordinate components beyond the screen range to within the screen range, and is configured to:

[0171] When the cursor coordinates are in the upper left corner area, the horizontal coordinate component of the cursor coordinates is modified to the horizontal coordinate component of the upper left vertex, and the vertical coordinate component of the cursor coordinates is modified to the vertical coordinate component of the upper left vertex;

[0172] When the cursor coordinates are in the left edge area, the abscissa component of the cursor coordinates is modified to the abscissa component of the point on the left edge;

[0173] When the cursor coordinates are in the lower left corner area, the horizontal coordinate component of the cursor coordinates is modified to the horizontal coordinate component of the lower left vertex, and the vertical coordinate component of the cursor coordinates is modified to the vertical coordinate component of the lower left vertex;

[0174] When the cursor coordinates are in the lower edge area, the ordinate component of the cursor coordinates is modified to the ordinate component of the point on the lower edge;

[0175] When the cursor coordinates are in the lower right corner area, the abscissa component of the cursor coordinates is modified to the abscissa component of the lower right corner vertex, and the ordinate component of the cursor coordinates is modified to the ordinate component of the lower right corner vertex;

[0176] When the cursor coordinates are in the right edge area, the horizontal coordinate component of the cursor coordinates is modified to the vertical coordinate component of the point on the right edge;

[0177] When the cursor coordinates are in the upper right corner area, the abscissa component of the cursor coordinates is modified to the abscissa component of the upper right corner vertex, and the ordinate component of the cursor coordinates is modified to the ordinate component of the upper right corner vertex;

[0178] When the cursor coordinates are in the upper edge area, the ordinate component of the cursor coordinates is modified to the ordinate component of the point on the upper edge.

[0179] Specifically, refer to Figure 5 .exist Figure 5 In the figure, area A can correspond to the upper left corner area, area A can correspond to the upper left corner area, area B corresponds to the left edge area, area C corresponds to the lower left corner area, area D corresponds to the lower edge area, area E corresponds to the lower right corner area, area F corresponds to the right edge area, area G corresponds to the upper right corner area, and area H corresponds to the upper edge area.

[0180] For example, if the screen range is represented by a resolution of 1920*1080, and the cursor coordinates are represented by (x, y), with the upper left corner of the screen as the coordinate origin, the rightward direction being the positive x-axis, and the downward direction being the positive y-axis, then the range of the horizontal coordinate component within the screen range is 0 to 1920, and the range of the vertical coordinate component is 0 to 1080. Therefore, if X<0, Y<0, the cursor is outside the screen at the upper left corner, that is, in area A. In this case, the X coordinate value can be modified to 0, and the Y coordinate value can be modified to 0, that is, the cursor is displayed at the upper left corner.

[0181] If X < 0, 1080 > Y > 0, the cursor is off the left edge of the screen, meaning it is in area B. In this case, you can modify the X coordinate to 0, leaving the Y coordinate unchanged, to position the cursor at the left edge. At this point, the cursor can move up and down along the left edge based on the Y value.

[0182] If X<0, Y>1080, the cursor is out of the screen at the lower left corner, that is, it is located in area C. At this time, you can modify the X coordinate value to 0 and the Y coordinate value to 1080, that is, the cursor is displayed in the lower left corner.

[0183] If Y>1080, 1920>X>0, the cursor will be outside the screen at the bottom edge, that is, in area D. In this case, you can change the Y coordinate value to 1080, while keeping the X coordinate value unchanged, so that the cursor is displayed at the bottom edge. At this time, the cursor can be moved left and right along the bottom edge according to the X value.

[0184] If X>1920, Y>1080, the cursor is outside the screen at the lower right corner, that is, it is located in area E. In this case, you can modify the X coordinate value to 1920 and the Y coordinate value to 1080, that is, the cursor is displayed in the lower right corner.

[0185] If X > 1920 and 1080 > Y > 0, the cursor will be off the right edge of the screen, meaning it will be in area F. In this case, you can modify the X coordinate to 1920, leaving the Y coordinate unchanged, to position the cursor at the right edge. At this point, the cursor can be moved up and down along the right edge based on the Y value.

[0186] If X>1920, Y<0, the cursor is outside the screen at the upper right corner, that is, it is located in area F. In this case, you can modify the X coordinate value to 1920 and the Y coordinate value to 0, that is, the cursor is displayed in the upper right corner.

[0187] If Y < 0, 1920 > X > 0, the cursor is outside the screen at the top edge, meaning it is in zone H. In this case, you can modify the Y coordinate to 0, leaving the X coordinate unchanged, to position the cursor at the top edge. At this point, the cursor can move up and down along the top edge based on the X value.

[0188] In this embodiment, since the area beyond the screen range can be divided into multiple areas, the cursor coordinates are automatically modified to within the screen range according to the area in which the cursor coordinates fall. This allows the cursor to be maintained within the screen range and remain visible even if the user moves the cursor pointing to the remote control beyond the screen range, thereby enabling the user to understand the cursor position in real time, providing convenience for user operation.

[0189] In some embodiments, the controller is further configured to:

[0190] When it is determined according to the cursor coordinates of the remote controller that the cursor is hovering over the control, description information for describing the control in the properties of the control is displayed through a floating window.

[0191] Among them, the interface layout of the display device is usually implemented by the control layout, and each control has a display range. Therefore, it is possible to determine whether the cursor coordinates pointing to the remote control fall within the corresponding display range of the control and whether the cursor coordinates are hovering over the control and which control it is hovering over. The description information of the control is contained in the properties of the control, such as the view property. The view property can specifically include the control ID, name, and type, etc., which is not specifically limited in the embodiments of the present application. The description information can be used as extended information contained in the view property to introduce and explain the control.

[0192] Specifically, corresponding description information can be added to each control in the control layout of the display device in advance. After the cursor coordinates are sent to the display device by pointing the remote control, the display device can determine whether the cursor is hovering over a certain control based on the cursor coordinates. If it is determined that the cursor is hovering over a certain control, the control properties can be obtained through the OnHoverListener mentioned in the previous content, and the description information therein can be passed to the floating window module. After the floating window module parses the data, the description information can be displayed. For example, if the control is a play button, the description information can be "play / pause"; if the control is a volume adjustment slider, the description information can be "volume adjustment". Among them, the style of the floating window can be customized, such as setting the font, color, background, etc., and the embodiment of the present application does not specifically limit this.

[0193] In this embodiment, when the cursor hovers over a control, description information for describing the control can be displayed according to the control attributes, thereby providing a prompt for key content of the control where the cursor is hovering, thereby enhancing the intuitiveness of the interaction.

[0194] In some embodiments, the controller is further configured to:

[0195] When it is determined based on the cursor coordinates of the remote controller that the cursor is hovering over a control, a corresponding matching cursor shape rule is determined based on the associated information of the control;

[0196] Modifies the cursor shape when the cursor hovers over a control, according to the cursor shape rules.

[0197] Among them, the method of judging whether the cursor is hovering over a control can refer to the previous content and will not be repeated here. The associated information may refer to the type of content displayed by the control, or the location of the control, etc., and the embodiment of the present application does not specifically limit this. The associated information can be obtained through OnHoverListener. By pre-establishing a mapping relationship between the associated information and the cursor shape rules, the corresponding matching cursor shape rules can be determined based on the associated information actually obtained. The cursor shape rules can be set according to actual needs. For example, if the associated information is used to represent the heat of the content displayed by the control, then when the heat represented by the associated information reaches a preset level, the cursor shape can be a flame shape. If the associated information is used to represent the type of content displayed by the control, then when the type is an action movie, the cursor shape can be an action element.

[0198] In this embodiment, when the cursor hovers over a control, the cursor shape can be changed according to the associated information of the control, thereby prompting the user through the cursor shape, enhancing the visual feedback to the user, and increasing interest and recognition.

[0199] In some embodiments, the controller is further configured to:

[0200] When it is detected that the cursor of the remote control has been staying for a predetermined time, a visual effect prompt indicating that the cursor will be hidden is displayed;

[0201] When the cursor is hidden, display a visual effect prompt indicating that the cursor is hidden;

[0202] When the cursor of the remote controller is detected to be moved, the cursor is restored to display.

[0203] Among them, the visual effect prompt can be to display an animation or display text; of course, instead of a visual effect prompt, an auditory prompt such as voice can be used, and the embodiment of the present application does not specifically limit this. When it is detected that the cursor has stayed for a preset time, it means that the user may be watching a movie or TV program, and the user continues to not manipulate the cursor. In this way, the cursor can be hidden. Before hiding, a visual effect prompt can be displayed to the user to remind the user that the cursor is about to be hidden. When the cursor is hidden, a visual effect prompt can also be displayed to remind the user that the cursor has been hidden. When it is detected that the cursor starts to move again, it means that the user needs to manipulate the cursor, and the cursor can be restored.

[0204] For example, a timer can be set on the pointing remote control end or the display device end. When it is detected that the user does not move the cursor of the pointing remote control at all within a preset time (such as 5 seconds), the cursor fade-out countdown can be started. During the fade-out process, a countdown number and a text prompt indicating that the cursor is about to be hidden can be displayed in the middle of the screen, that is, a visual effect prompt indicating that the cursor is about to be hidden. When the cursor is completely hidden, text is displayed in the middle of the display device to prompt "The current cursor is hidden. The cursor will be displayed when the pointing remote control is moved again", which is a visual effect prompt indicating that the cursor has been hidden. The text displayed in the middle can last for 5 seconds and then disappear automatically. When the user moves the pointing remote control again, the cursor display can be restored.

[0205] In this embodiment, when the cursor remains stationary for a long time, the cursor can be gradually hidden and a visual effect can be used to prompt the user, thereby reducing distractions when the user is viewing the display device and improving the viewing experience. In addition, when the cursor pointing to the remote control is needed for control, the hidden cursor can be moved to restore the cursor, thereby simultaneously meeting the user's control needs.

[0206] In some embodiments, the visual effect prompt used to indicate that the cursor is about to be hidden is a countdown number displayed on the cursor; the visual effect prompt used to indicate that the cursor has been hidden is a pop-up prompt.

[0207] Among them, the pop-up prompt can be a toast pop-up window, which can be displayed in a position such as the upper right corner that does not cover the displayed content. The embodiment of the present application does not make specific limitations on this.

[0208] In this embodiment, since the visual effect prompt used to indicate that the cursor is about to be hidden is displayed on the cursor, it is more closely related to the hidden object and is more directly and effectively perceived by the user. The pop-up prompt can clearly inform the user that the cursor has been hidden, allowing the user to effectively understand the cursor status.

[0209] In some embodiments, the controller performs a process of detecting movement of a cursor of a remote controller and is configured to:

[0210] When the changes in the coordinate components of the detected cursor coordinates all exceed the preset changes, it is determined that the cursor movement is detected; when the changes in the coordinate components of the detected cursor coordinates do not exceed the preset changes, it is determined that the cursor movement is not detected.

[0211] Considering that the user may mistakenly trigger the movement of the cursor, causing the cursor to resume display, the change in the cursor coordinates can be judged to determine whether the cursor movement is a false trigger. Specifically, if the change in the coordinate components of the cursor coordinates, such as the change in the X change and the change in the Y change, exceeds the preset change, it indicates that the cursor movement is large and unlikely to be caused by a false trigger, and thus it can be determined that the cursor movement has been detected. If the change in the coordinate components of the cursor coordinates does not exceed the preset change, it indicates that the cursor movement is small and may be the result of a false trigger, and thus it can be determined that the cursor movement has not been detected.

[0212] Among them, a possible scenario for false triggering is that the user places the pointing remote control on the sofa, and a slight vibration may cause the pointing remote control to move, thereby causing the coordinate components of the cursor coordinates to change. It should be noted that the above judgment condition is only one possible implementation method. In actual implementation, different judgment conditions can be set based on the change in the coordinate components. This embodiment of the application does not specifically limit this.

[0213] In this embodiment, since it is possible to determine whether the cursor is falsely triggered when the cursor movement is detected after the cursor is hidden, it is possible to reduce the situation where the cursor is restored due to false triggering, thereby reducing interference when the user views the display device.

[0214] In some embodiments, the controller is further configured to:

[0215] In response to a function setting operation on the remote controller, setting at least one of an on / off state or a prompt style of a prompt function to update a local configuration file;

[0216] Synchronize local profiles to the cloud so that profiles can be synchronized when logging in on other devices.

[0217] Specifically, each of the functions mentioned above can be turned on or off accordingly through function settings. For example, the cursor coordinate modification function when the cursor exceeds the screen range, the pop-up prompt function, the visual effect prompt function, the floating window prompt function when the cursor is suspended on the control, the function of changing the cursor shape, the cursor hiding function, etc. In addition, some functions can also have some custom settings, such as the floating window size, font, color, etc. of the floating window prompt function. The setting information of the above functions can be stored in the local configuration file of the display device. At the same time, the local configuration file can also be synchronized to the cloud, and the configuration file can be synchronized when logging in to other devices using the same account; or, if the local configuration file is cleared when the computer is shut down, the local configuration file can be downloaded from the cloud the next time it is turned on.

[0218] In this embodiment, since the above functions can be set, it can meet customized needs and the display is more flexible.

[0219] The above content mainly describes the display device. In an exemplary embodiment, a cursor-based interaction method is also provided, which is applied to the display device mentioned above. Figure 6 , the method comprising:

[0220] Step 602: Obtain the cursor coordinates of the remote controller and the screen range of the display.

[0221] Step 604: If there are coordinate components in the cursor coordinates that exceed the screen range, modify the coordinate components that exceed the screen range, and display the cursor of the remote controller according to the modified cursor coordinates.

[0222] Step 606: Display a pop-up prompt indicating that the cursor coordinates before modification are beyond the screen range based on the cursor coordinates before modification, and display a visual effect prompt indicating that the cursor coordinates before modification are beyond the screen range at the screen boundary corresponding to the cursor coordinates before modification.

[0223] Before executing the above method, you can refer to the previous content to establish a link between the display device and the pointing remote control. Figure 7 ,exist Figure 7 The process mainly starts from the user turning on the device to the completion of Bluetooth pairing connection between the display device and the pointing remote control. After the Bluetooth pairing connection is completed, if the user opens the default page of the device or an application, the display device can obtain the page update content and control properties from the cloud for display and filling, which is a continuation of the previous Figure 7 , the specific process can be referred to Figure 8 .

[0224] The above-mentioned cursor-based interaction method can automatically modify the cursor coordinates to within the screen range when the cursor exceeds the screen range. This means that even if the user moves the cursor pointing to the remote control beyond the screen range, the cursor can still be displayed within the screen range and remain visible, allowing the user to understand the cursor position in real time, providing convenience for user operations. On this basis, although the cursor coordinates are modified to within the screen range, the user may not realize that the previous operation has moved the cursor outside the screen range, so that the visual feedback to the user can be enhanced through pop-ups and visual effects at the edge of the screen, making the user more intuitive about the interaction status between the pointing remote control and the display device. Overall, the user's operating experience on the display device can be improved. In addition, the above functions can all be implemented through software without the need for additional hardware support, thereby reducing costs.

[0225] In some embodiments, the area beyond the screen range is divided into an upper left corner area, a left edge area, a lower left corner area, a lower edge area, a lower right corner area, a right edge area, an upper right corner area, and an upper edge area; and modifying the coordinate components beyond the screen range to within the screen range includes:

[0226] When the cursor coordinates are in the upper left corner area, the horizontal coordinate component of the cursor coordinates is modified to the horizontal coordinate component of the upper left vertex, and the vertical coordinate component of the cursor coordinates is modified to the vertical coordinate component of the upper left vertex;

[0227] When the cursor coordinates are in the left edge area, the abscissa component of the cursor coordinates is modified to the abscissa component of the point on the left edge;

[0228] When the cursor coordinates are in the lower left corner area, the horizontal coordinate component of the cursor coordinates is modified to the horizontal coordinate component of the lower left vertex, and the vertical coordinate component of the cursor coordinates is modified to the vertical coordinate component of the lower left vertex;

[0229] When the cursor coordinates are in the lower edge area, the ordinate component of the cursor coordinates is modified to the ordinate component of the point on the lower edge;

[0230] When the cursor coordinates are in the lower right corner area, the abscissa component of the cursor coordinates is modified to the abscissa component of the lower right corner vertex, and the ordinate component of the cursor coordinates is modified to the ordinate component of the lower right corner vertex;

[0231] When the cursor coordinates are in the right edge area, the horizontal coordinate component of the cursor coordinates is modified to the vertical coordinate component of the point on the right edge;

[0232] When the cursor coordinates are in the upper right corner area, the abscissa component of the cursor coordinates is modified to the abscissa component of the upper right corner vertex, and the ordinate component of the cursor coordinates is modified to the ordinate component of the upper right corner vertex;

[0233] When the cursor coordinates are in the upper edge area, the ordinate component of the cursor coordinates is modified to the ordinate component of the point on the upper edge.

[0234] The process of modifying the cursor coordinates that are out of the screen range to be within the screen range, displaying a pop-up window prompt indicating that the cursor coordinates before the modification are out of the screen range, and displaying a visual effect prompt indicating that the cursor coordinates before the modification are out of the screen range can be referred to. Figure 9 The interaction diagram shown.

[0235] In this embodiment, since the area beyond the screen range can be divided into multiple areas, the cursor coordinates are automatically modified to within the screen range according to the area in which the cursor coordinates fall. This allows the cursor to be maintained within the screen range and remain visible even if the user moves the cursor pointing to the remote control beyond the screen range, thereby enabling the user to understand the cursor position in real time, providing convenience for user operation.

[0236] In some embodiments, the method further comprises:

[0237] When it is determined according to the cursor coordinates of the remote controller that the cursor is hovering over the control, description information for describing the control in the properties of the control is displayed through a floating window.

[0238] Among them, when the cursor moves and hovers over the control, the description information of the control can be displayed according to the operation content (that is, the content displayed by the control) and the function switch (that is, whether the floating window prompt function is turned on when the cursor hovers over the control). Figure 10 The interaction diagram shown.

[0239] In this embodiment, when the cursor hovers over a control, description information for describing the control can be displayed according to the control attributes, thereby providing a prompt for key content of the control where the cursor is hovering, thereby enhancing the intuitiveness of the interaction.

[0240] In some embodiments, the method further comprises:

[0241] When it is determined based on the cursor coordinates of the remote controller that the cursor is hovering over a control, a corresponding matching cursor shape rule is determined based on the associated information of the control;

[0242] Modifies the cursor shape when the cursor hovers over a control, according to the cursor shape rules.

[0243] Among them, the interactive diagram for modifying the cursor shape can be referred to Figure 11 .

[0244] In this embodiment, when the cursor hovers over a control, the cursor shape can be changed according to the associated information of the control, thereby prompting the user through the cursor shape, enhancing the visual feedback to the user, and increasing interest and recognition.

[0245] In some embodiments, the method further comprises:

[0246] When it is detected that the cursor of the remote control has been staying for a predetermined time, a visual effect prompt indicating that the cursor will be hidden is displayed;

[0247] When the cursor is hidden, display a visual effect prompt indicating that the cursor is hidden;

[0248] When movement of the cursor of the remote controller is detected, the cursor is restored to display.

[0249] The process of hiding the cursor, prompting it to be hidden, prompting it to be hidden, and restoring the cursor display can be referred to Figure 12 .

[0250] In this embodiment, when the cursor remains stationary for a long time, the cursor can be gradually hidden and a visual effect can be used to prompt the user, thereby reducing distractions when the user is viewing the display device and improving the viewing experience. In addition, when the cursor pointing to the remote control is needed for control, the hidden cursor can be moved to restore the cursor, thereby simultaneously meeting the user's control needs.

[0251] In some embodiments, the method further comprises:

[0252] The visual effect prompt used to indicate that the cursor is about to be hidden is a countdown number displayed on the cursor; the visual effect prompt used to indicate that the cursor has been hidden is a pop-up prompt.

[0253] Among them, the countdown number displayed on the cursor can be referred to Figure 13 ,exist Figure 13 The double circle pattern in the upper right corner is the cursor, and the number 5 in it represents the countdown number when the cursor is hidden. Figure 14 ,exist Figure 14 In the pop-up window, the “Cursor is hidden” pop-up window displayed in the upper right corner is the pop-up prompt.

[0254] In this embodiment, since the visual effect prompt used to indicate that the cursor is about to be hidden is displayed on the cursor, it is more closely related to the hidden object and is more directly and effectively perceived by the user. The pop-up prompt can clearly inform the user that the cursor has been hidden, allowing the user to effectively understand the cursor status.

[0255] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0256] Based on the same inventive concept, embodiments of the present application also provide an interactive device for implementing the aforementioned cursor-based interactive method. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more cursor-based interactive device embodiments provided below can be found in the limitations of the cursor-based interactive method described above and will not be further elaborated here.

[0257] In an exemplary embodiment, Figure 15 As shown, a cursor-based interaction device is provided, including: an acquisition module 1502, a modification module 1504 and a display module 1506, wherein:

[0258] An acquisition module 1502 is configured to acquire the cursor coordinates of the remote controller and the screen range of the display;

[0259] A modifying module 1504 is configured to modify the coordinate components exceeding the screen range when there are coordinate components in the cursor coordinates exceeding the screen range, and display the cursor of the remote controller according to the modified cursor coordinates;

[0260] Display module 1506 is used to display a pop-up prompt indicating that the cursor coordinates before modification are beyond the screen range based on the cursor coordinates before modification, and to display a visual effect prompt indicating that the cursor coordinates before modification are beyond the screen range at the screen boundary corresponding to the cursor coordinates before modification.

[0261] In an exemplary embodiment, the area beyond the screen range is divided into an upper left corner area, a left edge area, a lower left corner area, a lower edge area, a lower right corner area, a right edge area, an upper right corner area and an upper edge area; a modification module 1504 is used to modify the horizontal coordinate component of the cursor coordinates to the horizontal coordinate component of the upper left corner vertex and the vertical coordinate component of the cursor coordinates to the vertical coordinate component of the upper left corner vertex when the cursor coordinates are in the upper left corner area; modify the horizontal coordinate component of the cursor coordinates to the horizontal coordinate component of the point on the left edge when the cursor coordinates are in the left edge area; modify the horizontal coordinate component of the cursor coordinates to the horizontal coordinate component of the lower left corner vertex and the vertical coordinate component of the cursor coordinates to the vertical coordinate component of the lower left corner vertex when the cursor coordinates are in the lower left corner area; When the cursor coordinates are in the lower edge area, the vertical coordinate component of the cursor coordinates is modified to the vertical coordinate component of the point on the lower edge; when the cursor coordinates are in the lower right corner area, the horizontal coordinate component of the cursor coordinates is modified to the horizontal coordinate component of the lower right corner vertex, and the vertical coordinate component of the cursor coordinates is modified to the vertical coordinate component of the lower right corner vertex; when the cursor coordinates are in the right edge area, the horizontal coordinate component of the cursor coordinates is modified to the vertical coordinate component of the point on the right edge; when the cursor coordinates are in the upper right corner area, the horizontal coordinate component of the cursor coordinates is modified to the horizontal coordinate component of the upper right corner vertex, and the vertical coordinate component of the cursor coordinates is modified to the vertical coordinate component of the upper right corner vertex; when the cursor coordinates are in the upper edge area, the vertical coordinate component of the cursor coordinates is modified to the vertical coordinate component of the point on the upper edge.

[0262] In an exemplary embodiment, the display module 1506 is further configured to display description information for describing the control in the properties of the control through a floating window when it is determined according to the cursor coordinates of the remote controller that the cursor is hovering over the control.

[0263] In an exemplary embodiment, the display module 1506 is also used to determine a corresponding matching cursor shape rule based on the associated information of the control when it is determined that the cursor is hovering over a control based on the cursor coordinates of the remote control; and modify the cursor shape when the cursor is hovering over the control based on the cursor shape rule.

[0264] In an exemplary embodiment, the display module 1506 is also used to display a visual effect prompt indicating that the cursor will be hidden when it is detected that the cursor of the remote control has stayed for a preset time; display a visual effect prompt indicating that the cursor has been hidden when the cursor has been hidden; and restore the cursor when it is detected that the cursor of the remote control has moved.

[0265] In an exemplary embodiment, the visual effect prompt used to indicate that the cursor is about to be hidden is displaying a countdown number on the cursor; and the visual effect prompt used to indicate that the cursor has been hidden is a pop-up prompt.

[0266] In an exemplary embodiment, the display module 1506 is also used to determine that the cursor movement is detected when it is detected that the change amounts of the coordinate components of the cursor coordinates all exceed the preset change amounts; and to determine that the cursor movement is not detected when it is detected that the change amounts of the coordinate components of the cursor coordinates do not exceed the preset change amounts.

[0267] In an exemplary embodiment, the acquisition module 1502 is also used to set the on / off state of the prompt function or at least one of the prompt styles in response to the function setting operation of the remote control to update the local configuration file; synchronize the local configuration file to the cloud so that the configuration file can be synchronized when other devices log in.

[0268] Each module in the cursor-based interaction device described above may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0269] In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements possible implementations of the above methods when executing the computer program.

[0270] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the possible implementations provided by the above methods are realized.

[0271] In one embodiment, a computer program product is provided, including a computer program, which implements the possible implementations provided by the above methods when executed by a processor.

[0272] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile memory and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a programmable logic unit (PLC), a data processing logic unit based on quantum computing, an artificial intelligence (AI) processor, and the like.

[0273] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0274] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A display device, characterized in that: include: a display configured to display a user interface; The user interface has a cursor, and the position of the cursor on the user interface is determined by the position pointed by the remote control in three-dimensional space; The controller is configured as: Obtaining the cursor coordinates of the remote control and the screen range of the display; If a coordinate component of the cursor coordinates exceeds the screen range, modifying the coordinate component exceeding the screen range to fall within the screen range, and displaying the cursor of the remote controller according to the modified cursor coordinates; A pop-up prompt is displayed to indicate that the cursor coordinates before modification exceed the screen range, and a visual effect prompt is displayed at the screen edge corresponding to the cursor coordinates before modification to indicate that the cursor coordinates before modification exceed the screen range.

2. The display device according to claim 1, wherein The area beyond the screen range is divided into an upper left corner area, a left edge area, a lower left corner area, a lower edge area, a lower right corner area, a right edge area, an upper right corner area, and an upper edge area; the controller modifies the coordinate components beyond the screen range to within the screen range, and is configured to: When the cursor coordinates are in the upper left corner area, the abscissa component of the cursor coordinates is modified to the abscissa component of the upper left corner vertex, and the ordinate component of the cursor coordinates is modified to the ordinate component of the upper left corner vertex; When the cursor coordinates are in the left edge area, modifying the abscissa component of the cursor coordinates to the abscissa component of the point on the left edge; When the cursor coordinates are in the lower left corner area, the abscissa component of the cursor coordinates is modified to the abscissa component of the lower left corner vertex, and the ordinate component of the cursor coordinates is modified to the ordinate component of the lower left corner vertex; When the cursor coordinates are in the lower edge area, modifying the ordinate component of the cursor coordinates to the ordinate component of the point on the lower edge; When the cursor coordinates are in the lower right corner area, the abscissa component of the cursor coordinates is modified to the abscissa component of the lower right corner vertex, and the ordinate component of the cursor coordinates is modified to the ordinate component of the lower right corner vertex; When the cursor coordinates are in the right edge area, the abscissa component of the cursor coordinates is modified to the ordinate component of the point on the right edge; When the cursor coordinates are in the upper right corner area, the abscissa component of the cursor coordinates is modified to the abscissa component of the upper right corner vertex, and the ordinate component of the cursor coordinates is modified to the ordinate component of the upper right corner vertex; When the cursor coordinates are in the upper edge area, the ordinate component of the cursor coordinates is modified to the ordinate component of the point on the upper edge.

3. The display device according to claim 1, wherein The controller is further configured to: When it is determined according to the cursor coordinates of the remote controller that the cursor is hovering over a control, description information for describing the control in the properties of the control is displayed through a floating window.

4. The display device according to claim 1, wherein The controller is further configured to: In a case where it is determined according to the cursor coordinates of the remote controller that the cursor is hovering over a control, determining a corresponding matching cursor shape rule according to the associated information of the control; According to the cursor shape rule, the cursor shape when the cursor is hovering over the control is modified.

5. The display device according to claim 1, wherein The controller is further configured to: When it is detected that the cursor of the remote control has been staying for a predetermined time, a visual effect prompt indicating that the cursor will be hidden is displayed; When the cursor is hidden, displaying a visual effect prompt indicating that the cursor is hidden; When it is detected that the cursor of the remote controller moves, the cursor is restored to be displayed.

6. The display device according to claim 5, wherein: The visual effect prompt used to indicate that the cursor is about to be hidden is a countdown number displayed on the cursor; the visual effect prompt used to indicate that the cursor has been hidden is a pop-up prompt.

7. The display device according to claim 5, wherein: The controller performs a process of detecting movement of the cursor of the remote controller, and is configured to: When it is detected that the changes of the coordinate components of the cursor coordinates all exceed the preset changes, determining that the cursor movement is detected; In a case where it is detected that the change amount of the coordinate component of the cursor coordinates does not exceed the preset change amount, it is determined that no movement of the cursor is detected.

8. The display device according to claim 1, wherein The controller is further configured to: In response to a function setting operation on the remote controller, setting at least one of an on / off state or a prompt style of a prompt function to update a local configuration file; Synchronize the local configuration file to the cloud so that the configuration file can be synchronized when other devices log in.

9. A cursor-based interaction method, characterized in that: Applicable to the display device according to any one of claims 1 to 8; The method comprises: Obtaining the cursor coordinates of the remote controller and the screen range of the display; In a case where a coordinate component of the cursor coordinates exceeds the screen range, modifying the coordinate component exceeding the screen range, and displaying the cursor of the remote controller according to the modified cursor coordinates; According to the cursor coordinates before modification, a pop-up prompt is displayed to indicate that the cursor coordinates before modification exceed the screen range, and a visual effect prompt is displayed at the screen boundary corresponding to the cursor coordinates before modification to indicate that the cursor coordinates before modification exceed the screen range.

10. The method according to claim 9, characterized in that The area beyond the screen range is divided into an upper left corner area, a left edge area, a lower left corner area, a lower edge area, a lower right corner area, a right edge area, an upper right corner area and an upper edge area; The modifying of the coordinate components beyond the screen range includes: When the cursor coordinates are in the upper left corner area, the abscissa component of the cursor coordinates is modified to the abscissa component of the upper left corner vertex, and the ordinate component of the cursor coordinates is modified to the ordinate component of the upper left corner vertex; When the cursor coordinates are in the left edge area, modifying the abscissa component of the cursor coordinates to the abscissa component of the point on the left edge; When the cursor coordinates are in the lower left corner area, the abscissa component of the cursor coordinates is modified to the abscissa component of the lower left corner vertex, and the ordinate component of the cursor coordinates is modified to the ordinate component of the lower left corner vertex; When the cursor coordinates are in the lower edge area, modifying the ordinate component of the cursor coordinates to the ordinate component of the point on the lower edge; When the cursor coordinates are in the lower right corner area, the abscissa component of the cursor coordinates is modified to the abscissa component of the lower right corner vertex, and the ordinate component of the cursor coordinates is modified to the ordinate component of the lower right corner vertex; When the cursor coordinates are in the right edge area, the abscissa component of the cursor coordinates is modified to the ordinate component of the point on the right edge; When the cursor coordinates are in the upper right corner area, the abscissa component of the cursor coordinates is modified to the abscissa component of the upper right corner vertex, and the ordinate component of the cursor coordinates is modified to the ordinate component of the upper right corner vertex; When the cursor coordinates are in the upper edge area, the ordinate component of the cursor coordinates is modified to the ordinate component of the point on the upper edge.

Images