Display device and sliding control method

Abstract

The application relates to a display device and a sliding control method. In response to a sliding operation initiated by a remote controller, a corresponding sliding duration, sliding direction and initial position of a cursor are acquired; a simulated sliding event is generated and executed to enable a target display object indicated by the initial position to perform a sliding operation in the sliding direction; during the generation and execution of the simulated sliding event, if a difference between a current time when the first simulated sliding event is executed and an initial time determined based on the first simulated sliding event is greater than the sliding duration, a system load index of the display device is acquired; if the system load index is greater than a first preset threshold, a generation time interval between subsequent adjacent simulated sliding events is increased; and if the system load index is less than a second preset threshold, the generation time interval between the subsequent adjacent simulated sliding events is reduced. The display device of the application can improve user experience.

Description

Technical Field

[0001] This application relates to the field of display device technology, and in particular to a display device and a sliding control method. Background Technology

[0002] With the rapid development of display devices, when a remote control interacts with a display device, resource accumulation can occur due to system performance or processing priorities of upper-layer applications. This can easily lead to the display device not responding to the remote control's swipes, resulting in a poor user experience. Summary of the Invention

[0003] Therefore, it is necessary to provide a display device and sliding control method that can improve the user's interactive experience with the display device in response to the above-mentioned technical problems.

[0004] In a first aspect, this application provides a display device, comprising:

[0005] The display is used to show the user interface, which has a cursor. The position of the cursor on the user interface is determined by the position pointed to by the remote control in three-dimensional space.

[0006] The controller is configured as follows:

[0007] In response to a sliding operation initiated by the remote control, the corresponding sliding duration, sliding direction, and initial cursor position are obtained; the sliding duration is the time interval between the moment of touching the remote control and the moment the sliding displacement occurs on the remote control;

[0008] Generate and execute a simulated sliding event to cause the target display object, indicated by its initial position, to perform a sliding operation along the sliding direction.

[0009] During the generation and execution of simulated sliding events, if the difference between the current time after the first simulated sliding event is completed and the initial time determined based on the first simulated sliding event is greater than the sliding duration, the system load index of the display device is obtained;

[0010] If the system load index is greater than the first preset threshold, the generation time interval between subsequent adjacent simulated sliding events is increased; if the system load index is less than the second preset threshold, the generation time interval between subsequent adjacent simulated sliding events is decreased.

[0011] The above technical solution has the following advantages or beneficial effects: In the process of generating and executing simulated sliding events in response to a sliding operation initiated by a remote control, if the difference between the current time after the execution of the first simulated sliding event and the initial time determined based on the first simulated sliding event is greater than the sliding duration, the system load index of the display device is obtained, and the generation time interval between subsequent adjacent simulated sliding events is adjusted according to the system load index. Thus, when the system load is overloaded, the frequency of generating simulated sliding events on the TV can be adjusted in a timely manner, avoiding problems such as unresponsive sliding or short sliding distance caused by insufficient system resources and untimely processing of sliding events by the application. This can improve the user's interactive experience.

[0012] In some embodiments, the controller is further configured to:

[0013] During the generation and execution of simulated sliding events, the execution time of the current non-first simulated sliding events is monitored. If the execution time exceeds the third preset threshold, the cumulative time consumed from the initial time determined based on the first simulated sliding event to the current time, and the cumulative actual displacement from the initial position to the current position are obtained.

[0014] The compensation displacement is obtained based on the cumulative time consumed, the cumulative actual displacement, the preset time consumed in the simulated sliding, and the preset displacement.

[0015] Based on the compensation displacement, adjust the cursor position after the subsequent simulated sliding event is executed.

[0016] The above technical solution has the following advantages or beneficial effects: When the execution time of the current non-first simulated sliding event exceeds the third preset threshold, the compensation displacement is calculated based on the acquired cumulative time consumed, cumulative actual displacement, preset time consumed for simulated sliding, and preset displacement. The cursor position is then adjusted according to the compensation displacement after the subsequent simulated sliding event is completed. Thus, when the execution time of the current simulated sliding event is too long, the sliding displacement of the subsequent simulated sliding event can be adjusted in a timely manner, thereby ensuring that the preset displacement is completed within the preset time consumed, avoiding problems such as unresponsive sliding or short sliding distance, and thus improving the user's interactive experience.

[0017] In some embodiments, the controller executes a preset timeout and a preset displacement based on the cumulative elapsed time, the cumulative actual displacement, the simulated sliding time, and the preset displacement to obtain a compensation displacement amount, which is configured as follows:

[0018] The ratio of the cumulative actual displacement to the preset displacement is obtained as the displacement progress.

[0019] Obtain the ratio between the cumulative time elapsed and the preset time elapsed, and use it as the time progress;

[0020] Based on the lag between displacement progress and time progress, a dynamic compensation coefficient is obtained.

[0021] The compensation displacement is obtained based on the dynamic compensation coefficient and the cursor position after the current non-first simulated sliding event is executed.

[0022] The above technical solution has the following advantages or beneficial effects: by calculating the dynamic compensation coefficient based on the displacement progress and time progress, and determining the compensation displacement amount based on the dynamic compensation coefficient, the accuracy of the compensation displacement amount is effectively improved. Thus, when adjusting subsequent simulated sliding events based on the compensation displacement amount, the reliability of the execution of subsequent simulated sliding events is ensured, avoiding the problem of sliding jumps due to excessive compensation displacement amount or short sliding distances due to insufficient compensation displacement amount, thereby improving the user's interactive experience.

[0023] In some embodiments, the controller performs dynamic compensation based on the lag of displacement progress relative to time progress, and is configured to:

[0024] Based on the displacement progress, obtain the remaining displacement progress of the cumulative actual displacement compared to the preset displacement;

[0025] Based on the time progress, obtain the remaining time progress of the cumulative time spent compared to the preset time spent;

[0026] The dynamic compensation coefficient is obtained based on the remaining displacement progress and the remaining time progress.

[0027] The above technical solution has the following advantages or beneficial effects: by calculating the dynamic compensation coefficient through displacement progress, remaining displacement progress, and time progress, the compensation intensity can be determined. For example, the larger the remaining displacement progress or the smaller the remaining time progress, the greater the compensation intensity. Thus, the compensation displacement is allocated to subsequent sliding events according to the compensation intensity, ensuring the reliability of the compensation displacement allocation, thereby making the compensation process natural and smooth and improving the user's interactive experience.

[0028] In some embodiments, the controller is configured to adjust the cursor position after subsequent simulated sliding events are completed based on the compensation displacement.

[0029] The compensation displacement is decomposed into multiple sub-compensation displacements;

[0030] Based on multiple sub-compensation displacements, the cursor position is adjusted after multiple simulated sliding events are executed.

[0031] The above technical solution has the following advantages or beneficial effects: by decomposing the compensation displacement into multiple sub-compensation displacements, and adjusting the cursor position after multiple simulated sliding events are executed according to the sub-compensation displacements, the problem of sliding jumps and excessive system load caused by executing all compensation displacements in a single simulated sliding event is avoided, thereby improving the user's interactive experience.

[0032] In some embodiments, the controller is further configured to:

[0033] During the generation and execution of simulated sliding events, if the difference between the current time after the execution of a non-first simulated sliding event and the initial time determined based on the non-first simulated sliding event is greater than the sliding duration, the system load index of the display device is reacquired.

[0034] If the system load index is greater than the first preset threshold, increase the execution time interval between subsequent adjacent simulated sliding events; if the system load index is less than the second preset threshold, decrease the execution time interval between subsequent adjacent simulated sliding events.

[0035] The above technical solution has the following advantages or beneficial effects: When the difference between the current moment after executing a non-first simulated sliding event and the initial moment determined based on the non-first simulated sliding event is greater than the sliding duration, the system load index of the display device is re-acquired, and the generation time interval between subsequent adjacent simulated sliding events is adjusted according to the system load index. Thus, the system load of the display device can be obtained in real time, and when the system load is overloaded, the frequency of subsequent simulated sliding events generated on the TV can be adjusted in a timely manner. This avoids problems such as unresponsive sliding or short sliding distance caused by insufficient memory resources leading to untimely processing of sliding events by the application, thereby improving the user's interactive experience.

[0036] In some embodiments, the initial time is the moment when the first simulated sliding event is sent to the application after the first simulated sliding event is generated; the application is used to execute the simulated sliding event.

[0037] The above technical solution has the following advantages or beneficial effects: by determining the initial moment when the TV sends the first simulated sliding event, a benchmark can be provided for the subsequent calculation of the sliding duration and sliding displacement of the simulated sliding event, thereby ensuring the accuracy of the cursor position after the application executes the simulated sliding event, and thus improving the user's interactive experience.

[0038] Secondly, this application provides a sliding control method, the method comprising:

[0039] In response to a sliding operation initiated by the remote control, the corresponding sliding duration, sliding direction, and initial cursor position are obtained; the sliding duration is the time interval between the moment of touching the remote control and the moment the sliding displacement occurs on the remote control;

[0040] Generate and execute a simulated sliding event to cause the target display object, indicated by its initial position, to perform a sliding operation along the sliding direction.

[0041] During the generation and execution of simulated sliding events, if the difference between the current time after the first simulated sliding event is completed and the initial time determined based on the first simulated sliding event is greater than the sliding duration, the system load index of the display device is obtained;

[0042] If the system load index is greater than the first preset threshold, the generation time interval between subsequent adjacent simulated sliding events is increased; if the system load index is less than the second preset threshold, the generation time interval between subsequent adjacent simulated sliding events is decreased.

[0043] The above technical solution has the following advantages or beneficial effects: In the process of generating and executing simulated sliding events in response to a sliding operation initiated by a remote control, if the difference between the current time after the execution of the first simulated sliding event and the initial time determined based on the first simulated sliding event is greater than the sliding duration, the system load index of the display device is obtained, and the generation time interval between subsequent adjacent simulated sliding events is adjusted according to the system load index. Thus, when the system load is overloaded, the frequency of generating simulated sliding events on the TV can be adjusted in a timely manner, avoiding problems such as unresponsive sliding or short sliding distance caused by insufficient system resources and untimely processing of sliding events by the application. This can improve the user's interactive experience.

[0044] In some embodiments, the method further includes:

[0045] During the generation and execution of simulated sliding events, the execution time of the current non-first simulated sliding events is monitored. If the execution time exceeds the third preset threshold, the cumulative time consumed from the initial time determined based on the first simulated sliding event to the current time, and the cumulative actual displacement from the initial position to the current position are obtained.

[0046] The compensation displacement is obtained based on the cumulative time consumed, the cumulative actual displacement, the preset time consumed in the simulated sliding, and the preset displacement.

[0047] Based on the compensation displacement, adjust the cursor position after the subsequent simulated sliding event is executed.

[0048] The above technical solution has the following advantages or beneficial effects: When the execution time of the current non-first simulated sliding event exceeds the third preset threshold, the compensation displacement is calculated based on the acquired cumulative time consumed, cumulative actual displacement, preset time consumed for simulated sliding, and preset displacement. The cursor position is then adjusted according to the compensation displacement after the subsequent simulated sliding event is completed. Thus, when the execution time of the current simulated sliding event is too long, the sliding displacement of the subsequent simulated sliding event can be adjusted in a timely manner, thereby ensuring that the preset displacement is completed within the preset time consumed, avoiding problems such as unresponsive sliding or short sliding distance, and thus improving the user's interactive experience.

[0049] In some embodiments, obtaining the compensation displacement based on the cumulative elapsed time, the cumulative actual displacement, the preset elapsed time of the simulated sliding, and the preset displacement includes:

[0050] The ratio of the cumulative actual displacement to the preset displacement is obtained as the displacement progress.

[0051] Obtain the ratio between the cumulative time elapsed and the preset time elapsed, and use it as the time progress;

[0052] Based on the lag between displacement progress and time progress, a dynamic compensation coefficient is obtained.

[0053] The compensation displacement is obtained based on the dynamic compensation coefficient and the cursor position after the current non-first simulated sliding event is executed.

[0054] The above technical solution has the following advantages or beneficial effects: by calculating the dynamic compensation coefficient based on the displacement progress and time progress, and determining the compensation displacement amount based on the dynamic compensation coefficient, the accuracy of the compensation displacement amount is effectively improved. Thus, when adjusting subsequent simulated sliding events based on the compensation displacement amount, the reliability of the execution of subsequent simulated sliding events is ensured, avoiding the problem of sliding jumps due to excessive compensation displacement amount or short sliding distances due to insufficient compensation displacement amount, thereby improving the user's interactive experience. Attached Figure Description

[0055] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a schematic diagram illustrating an operational scenario between a display device and a control device provided in some embodiments of this application;

[0057] Figure 2 This is a schematic diagram of the hardware configuration of a display device provided in some embodiments of this application;

[0058] Figure 3 This is a schematic diagram of the hardware configuration of the control device provided in some embodiments of this application;

[0059] Figure 4 This is a schematic diagram of the software configuration of a display device provided in some embodiments of this application;

[0060] Figure 5 This application provides schematic diagrams of cursor hovering for some embodiments.

[0061] Figure 6 A tapping diagram provided for some embodiments of this application;

[0062] Figure 7 Selection diagrams provided for some embodiments of this application;

[0063] Figure 8 This is a drag-and-drop multi-select illustration provided for some embodiments of this application;

[0064] Figure 9 This application provides drag-and-drop illustrations for some embodiments;

[0065] Figure 10 A schematic diagram of sliding provided for some embodiments of this application;

[0066] Figure 11 Schematic diagram of continuous sliding provided for some embodiments of this application;

[0067] Figure 12 This is a schematic diagram of continuous reverse sliding provided for some embodiments of this application;

[0068] Figure 13 This is a framework diagram of a processor for a display device in the related art;

[0069] Figure 14 This is a schematic diagram illustrating signal source switching provided in some embodiments of this application;

[0070] Figure 15 A frame diagram of a processor for a display device provided in some embodiments of this application;

[0071] Figure 16 A flowchart illustrating the execution of sliding events by a simulated sliding module, provided in some embodiments of this application;

[0072] Figure 17 Here is a flowchart of a sliding control method in one embodiment;

[0073] Figure 18This is a schematic diagram illustrating the interaction between a user, a pointing remote control, and a display device in one embodiment. Detailed Implementation

[0074] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application as detailed in the claims.

[0075] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0076] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0077] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0078] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0079] In this embodiment, the display device 200 generally refers to a device with screen display and data processing capabilities. For example, the display device 200 includes, but is not limited to, smart TVs, mobile terminals, computers, monitors, advertising screens, wearable devices, virtual reality devices, augmented reality devices, etc.

[0080] Figure 1 This is a schematic diagram illustrating an operational scenario between a display device and a control device provided in some embodiments of this application. For example... Figure 1 As shown, users can operate the display device 200 via touch operation, mobile terminal 300, and control device 100. For example, control device 100 can be a remote control, stylus, gamepad, etc.

[0081] The mobile terminal 300 can function as a control device for human-computer interaction between the user and the display device 200. It can also function as a communication device for establishing a communication connection with the display device 200 and exchanging data. In some embodiments, the mobile terminal 300 can have software applications installed on it and communicate with the display device 200 via network communication protocols to achieve one-to-one control and data communication. Furthermore, it can transmit audio and video content displayed on the mobile terminal 300 to the display device 200 for synchronized display.

[0082] like Figure 1 The diagram also shows that the display device 200 communicates with the server 400 via various communication methods. This allows the display device 200 to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0083] Display device 200 can provide broadcast television reception function, and can also be equipped with intelligent network television function that provides computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), etc.

[0084] Figure 2 Provided for some embodiments of this application Figure 1 Hardware configuration block diagram of display device 200.

[0085] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0086] In some embodiments, detector 230 is used to acquire signals from the external environment or to interact with the outside world. For example, detector 230 includes a light receiver, a sensor for acquiring ambient light intensity; or, detector 230 includes an image acquisition device, such as a camera, which can be used to acquire external environmental scenes, user attributes, or user interaction gestures; or, detector 230 includes a sound acquisition device, such as a microphone, for receiving external sounds.

[0087] In some embodiments, the display 260 includes display function components for presenting images and driving components for driving image display. The display 260 is used to receive and display image signals output from the controller 250. For example, the display 260 can be used to display video content, image content, menu control interface components, and user control UI interfaces, etc.

[0088] In some embodiments, the communication device 220 is a component used to communicate with external devices or the server 400 according to various communication protocol types. The display device 200 may have multiple communication devices 220 depending on the supported communication methods. For example, when the display device 200 supports wireless network communication, it may have a communication device 220 with WiFi functionality. When the display device 200 supports Bluetooth connectivity, it needs to have a communication device 220 with Bluetooth functionality.

[0089] The communication device 220 enables the display device 200 to communicate with external devices or the server 400 via wireless or wired connections. Wired connections utilize data cables, interfaces, or other components to connect the display device 200 to external devices. Wireless connections utilize wireless signals or wireless networks. The display device 200 can directly establish a connection with external devices or indirectly through gateways, routers, or other connection devices.

[0090] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in memory. The controller 250 controls the overall operation of the display device 200.

[0091] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0092] In some embodiments, a user can input user commands through a graphical user interface (GUI) displayed on a display 260, and the user input interface receives user input commands through the graphical user interface (GUI).

[0093] In some embodiments, the audio output device 270 can be a built-in speaker of the display device 200 or an external audio output device connected to the display device 200. For the external audio output device connected to the display device 200, the display device 200 may also be provided with an external audio output terminal, through which the audio output device can be connected to the display device 200 to output sound from the display device 200.

[0094] In some embodiments, the user input interface 280 can be used to receive instructions from user input.

[0095] Figure 3 Provided for some embodiments of this application Figure 1 Hardware configuration block diagram of the central control device. (Example) Figure 3 As shown, the control device 100 may include: a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0096] The control device 100 is configured to control the display device 200, and to receive user input operation commands and convert the operation commands into commands that the display device 200 can recognize and respond to, thus acting as an intermediary for interaction between the user and the display device 200.

[0097] In some embodiments, the control device 100 may be an intelligent device. For example, the control device 100 may be equipped with various applications for controlling the display device 200 according to user needs.

[0098] In some embodiments, such as Figure 1 As shown, the mobile terminal 300 or other smart electronic devices can perform similar functions to the control device 100 after installing the application of the control display device 200.

[0099] The controller 110 includes a processor 112, RAM 113, ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation of the control device 100, as well as the communication and cooperation between internal components and the external and internal data processing functions.

[0100] Under the control of the controller 110, the communication interface 130 enables communication of control signals and data signals with the display device 200. The communication interface 130 may include at least one of other near-field communication modules such as WiFi chip 131, Bluetooth module 132, and NFC module 133.

[0101] User input / output interface 140, wherein the input interface includes at least one of other input interfaces such as microphone 141, touchpad 142, sensor 143, and button 144.

[0102] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with the communication interface 130, such as a WiFi, Bluetooth, or NFC module, which can encode user input commands via WiFi, Bluetooth, or NFC protocols and send them to the display device 200.

[0103] The memory 190 is used to store various operating programs, data, and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can also store various control signal instructions input by the user.

[0104] The power supply 180 is used to provide operating power support for the various components of the control device 100 under the control of the controller.

[0105] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program used to manage and control the hardware and software resources in the display device 200. The operating system can (control the display device) provide a user interface, allowing users to interact with the display device 200 and supporting the running of various applications.

[0106] It should be noted that the operating system can be a native operating system based on a specific operating platform, a third-party operating system that is deeply customized based on a specific operating platform, or an independent operating system specifically developed for display devices.

[0107] An operating system can be divided into different modules or levels based on the functions it implements, for example... Figure 4 As shown, in some embodiments, the system is divided into four layers, from top to bottom: the Applications layer (referred to as the "Application Layer"), the Application Framework layer (referred to as the "Framework Layer"), the System Library layer, and the Kernel layer.

[0108] In some embodiments, the application layer provides services and interfaces for applications, enabling the display device 200 to run applications and interact with the user based on the applications. The application layer may contain at least one application, which may be a built-in Windows program, system settings program, or clock program of the operating system; or it may be an application developed by a third-party developer. In specific implementations, the application packages in the application layer are not limited to the examples above.

[0109] The framework layer provides application programming interfaces (APIs) and a programming framework for applications. The application framework layer includes predefined functions. It acts as a central processing unit, determining the actions taken by applications within the application layer. Through the API, applications can access system resources and obtain system services during execution.

[0110] like Figure 4As shown, the application framework layer in this embodiment includes a view system, managers, and content providers. The view system designs and implements the application's interface and interactions, and includes lists, grids, text boxes, and buttons. The managers include at least one of the following modules: an activity manager for interacting with all running activities in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to application packages currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0111] In some embodiments, the Activity Manager manages the lifecycle of individual applications and common navigation and back functions, such as controlling application exit, opening, and back actions. The Window Manager manages all window programs, such as obtaining the screen size, determining if a status bar is present, locking the screen, capturing the screen, and controlling changes to the display window, such as shrinking the display window, shaking the display, or distorting the display.

[0112] In some embodiments, the system runtime library layer can provide support for the framework layer. When the framework layer is used, the operating system runs the instruction library contained in the system runtime library layer, such as the C / C++ instruction library, to implement the functions to be performed by the framework layer.

[0113] In some embodiments, the kernel layer is a functional layer situated between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. For example, ... Figure 4 As shown, hardware drivers can be configured in the kernel layer. The kernel layer can contain at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0114] It should be noted that the above examples are merely a simple division of operating system functions and do not limit the specific form of the operating system of the display device 200 in this application embodiment. Depending on the function of the display device, the type of operating system, and other factors, the number of levels and the specific level type of the operating system may be expressed in other forms.

[0115] In some embodiments, a display device is provided, the display device comprising: a display configured to: display a user interface, the user interface including at least one control; the user interface further comprising a cursor, the position of the cursor on the user interface being determined by the position pointed to by a remote controller in three-dimensional space.

[0116] Optionally, users can control the display device to start via voice, remote control, or the power switch of the TV. After the display device is started, the user interface corresponding to the homepage is displayed on the screen. Users can switch to other user interfaces via voice or remote control, such as the user interface corresponding to the TV series tag, the user interface corresponding to the movie tag, or the user interface corresponding to a certain page provided by a certain application. This application embodiment does not limit this.

[0117] Optionally, among all the user interfaces displayed on the screen, some user interfaces contain controls, while others may not contain controls. The solution provided in this application embodiment is for user interfaces that contain controls. The number of controls on a user interface that contains controls may be only one or more. This application embodiment does not limit this.

[0118] Optionally, the remote control used with the display device can be a remote control with pointing function, hereinafter referred to as a pointing remote control. When the pointing function is enabled, the user interface displayed on the screen also has a cursor. The position of the cursor on the user interface is determined by the position pointed to by the pointing remote control in three-dimensional space. The position pointed to by the pointing remote control in three-dimensional space in this embodiment is also called the position data of the pointing remote control in three-dimensional space. The specific implementation process of determining the position of the cursor on the user interface based on the position data of the pointing remote control is described below.

[0119] The position data of the pointing remote control in three-dimensional space can include: position coordinates and pointing direction. The position coordinates are the physical position of the pointing remote control relative to the display device (e.g., horizontal distance, vertical height, etc.). The pointing direction is the pointing angle of the remote control (e.g., horizontal and vertical angles relative to the display device). The pointing remote control has a built-in gyroscope to detect the rotation angle of the remote control and an accelerometer to detect the acceleration of the remote control's movement. Through these sensors, the pointing remote control can determine its own position data in three-dimensional space in real time.

[0120] In some embodiments, after acquiring its own position data in three-dimensional space, the remote controller can send the position data to the display device. The display device can perform screen coordinate transformation based on the position data, thereby converting the position data of the remote controller in three-dimensional space to the screen coordinate system. The obtained screen coordinates can be used as the position of the cursor on the user interface, and the cursor can be drawn at that position to display the cursor.

[0121] In some embodiments, the remote controller includes a coordinate transformation module. After acquiring its own position data in three-dimensional space, the remote controller can input this position data into the coordinate transformation module. The coordinate transformation module can perform screen coordinate transformation based on this position data, thereby converting the position data into the screen coordinate system to obtain screen coordinates. These screen coordinates can be used as the cursor position on the user interface, and the cursor position on the user interface is sent to the display device. After receiving the cursor position on the user interface, the display device draws the cursor at that position, thereby displaying the cursor. For example, if the cursor position on the user interface is coordinates (a, b), the display device draws the cursor at coordinates (a, b) on the user interface, thereby displaying the cursor.

[0122] For example, see Figure 5 As shown, when the position data of the remote control in three-dimensional space is A, the cursor position on the user interface is coordinates (a1, b1); when the position data of the remote control in three-dimensional space is B, the cursor position on the user interface is coordinates (a2, b2). When the position data of the remote control in three-dimensional space is C, the screen coordinates calculated in the above way exceed the screen range, the display device no longer draws the cursor, and therefore, the cursor disappears.

[0123] See Figure 6 As shown, the remote control supports tapping. When the cursor hovers over a control, the user can press the touch area (OK button) on the remote control, and the display device will respond to the operation by executing the corresponding logic. The control can be a card, button, input box, radio button, switch, dialog box, multi-select button, slider, playback progress bar, scroll bar, drop-down menu, label, list, etc. This application embodiment does not limit this.

[0124] See Figure 7As shown, the remote control supports selection. When the user interface displays multiple options and their corresponding selection boxes, if the user wants to select the first and third options, they can first control the cursor to hover over the selection box corresponding to the first option and press the touch area on the remote control. At this time, the first option is selected. Then, the user can control the cursor to hover over the selection box corresponding to the third option and press the touch area on the remote control. At this time, the third option is selected, thus completing the selection.

[0125] See Figure 8 As shown, the remote control supports drag-and-drop multi-selection. When the user interface displays multiple options and their corresponding selection boxes, if the user wants to select all options, they can hover the cursor over any position in the area above the first option, press the touch area on the remote control and drag it down until the cursor moves to the area below the last option. At this point, all options will be selected, thus completing the drag-and-drop multi-selection.

[0126] See Figure 9 As shown, the remote control supports dragging. When a video is playing on the user interface and a progress bar is displayed, the user can hover the cursor over a position on the progress bar and then press the touch area on the remote control. The video playing on the user interface will then fast forward or rewind to the corresponding frame, thus completing the dragging of the playback progress. The progress bar in this example is just one example; draggable controls also include sliders, scroll bars, labels, cards, lists, etc., and this application embodiment does not limit the scope of these controls.

[0127] See Figure 10 As shown, the remote control supports swiping. When the user interface displays a cursor, the user can swipe up, down, left, or right on the touch area of ​​the remote control, and the display device will respond to these swipes.

[0128] See Figure 11 As shown, the remote control supports continuous swiping. When the cursor is displayed on the user interface, the user can continuously swipe up, down, left, or right on the touch area of ​​the remote control. The display device will respond to these continuous swipes and display a damped swipe effect on the user interface. Figure 11 This is a schematic diagram of continuous upward sliding.

[0129] See Figure 12As shown, the remote control supports continuous reverse swipes. When the cursor is displayed on the user interface, the user can swipe in one direction and then in the opposite direction on the touch area of ​​the remote control. The display device will respond to the continuous reverse swipe and display a damped swipe effect on the user interface. Figure 12 This is a diagram illustrating the process of sliding up first and then down.

[0130] Currently, when a user touches the touch panel pointed at by the remote control, the change in capacitance is captured by the touch panel driver, and the remote control sends a data packet to the processor of the display device. Please refer to [link to relevant documentation]. Figure 13 After receiving the data packet, the processor parses it and, based on the key type and coordinates of the time the key was pressed, generates an event and writes it to ` / dev / input / event0` according to certain rules. The native layer's InputReader monitors data changes on the device node through EventHub's epoll mechanism. The InputReader converts the raw events (such as EV_KEY, EV_ABS) into standard Android events. The InputDispatcher, through WindowManagerService, determines the target window based on the Z-order and the passed key coordinates. The event is then sent to the target application's UI thread via InputChannel. The ViewRootImpl.WindowInputEventReceiver of the application's UI thread receives the event, which enters the View-level dispatch process associated with the target window (such as `dispatchTouchEvent`). Finally, the received swipe event is executed in a view on the application side, and the swipe is performed.

[0131] As described in the background section, when a user opens the remote control and points it at the display device, the user interface of the display device will show the pointing cursor. When the user interface is in the 4K UI and the pointing cursor is located in a small window where a video is playing, if the user starts to touch and slide using the remote control, the starting position of the slide is at the video playback window. Since the 4K UI display and video playback are application processing priorities and occupy most system resources, the system may experience issues such as unresponsive sliding or short sliding distances when processing the sliding event. When the user interface is in the 4K UI and an auxiliary interface (such as the signal source switching interface) is invoked, please refer to [link to relevant documentation]. Figure 14 If the user starts to use the remote control to touch and slide, the starting position of the slide is in a small window of the auxiliary interface. Since the 4K UI interface display and the auxiliary interface display are priority processing by the application and occupy most of the system resources, the system may experience problems such as unresponsive sliding or short sliding distance when processing the sliding event. Figure 14 This diagram illustrates signal source switching, including normal signal source switching and abnormal signal source switching. Normal signal source switching indicates that the system processes the sliding event normally (sliding is successful), while abnormal signal source switching indicates that the system does not process the sliding event (sliding does not respond) or only processes a portion of the sliding event (short sliding distance).

[0132] To address the aforementioned technical problems, in some exemplary embodiments, this application provides a display device, including:

[0133] The display is used to show the user interface, which has a cursor. The position of the cursor on the user interface is determined by the position pointed to by the remote control in three-dimensional space.

[0134] The controller is configured as follows:

[0135] In response to a sliding operation initiated by the remote control, the corresponding sliding duration, sliding direction, and initial cursor position are obtained; the sliding duration is the time interval between the moment of touching the remote control and the moment the sliding displacement occurs on the remote control;

[0136] Generate and execute a simulated sliding event to cause the target display object, indicated by its initial position, to perform a sliding operation along the sliding direction.

[0137] During the generation and execution of simulated sliding events, if the difference between the current time after the first simulated sliding event is completed and the initial time determined based on the first simulated sliding event is greater than the sliding duration, the system load index of the display device is obtained;

[0138] If the system load index is greater than the first preset threshold, the generation time interval between subsequent adjacent simulated sliding events is increased; if the system load index is less than the second preset threshold, the generation time interval between subsequent adjacent simulated sliding events is decreased.

[0139] For example, a swipe operation initiated by the remote control refers to a user's finger touching the touch panel of the remote control and beginning to swipe in a certain direction on the touch panel. When the remote control initiates a swipe operation, it sends the swipe duration, swipe direction, and initial cursor position to the controller of the display device, which can be a television. The swipe duration is the time interval between the start time and the initial swipe time. The start time is the moment when the user's touch on the remote control touch panel is detected, and the initial swipe time is the moment when the user begins to swipe on the remote control. The swipe direction is the direction in which the user's finger swipes on the remote control touch panel, and the swipe direction includes at least four directions: up, down, left, and right. For example, when the user's finger swipes to the left on the remote control touch panel, the swipe direction is left. The initial cursor position is the position of the cursor on the user interface when the user touches the remote control touch panel.

[0140] For example, a simulated swipe event refers to the TV simulating the user's actual swipe operation, including `action_down` and `action_move` events. The `action_down` event refers to preparatory operations before the simulated swipe, such as simulating a touch event; the `action_move` event refers to the simulated swipe execution event, and a complete simulated swipe event can be divided into multiple `action_move` events. The target display object refers to the actual operation interface displayed by the current user interface. For example, the user interface can be a 4K UI interface, where the cursor swipes; the user interface can also be a signal source switching interface, where the cursor swipes. After receiving the swipe duration, swipe direction, and initial cursor position on the TV, the TV's simulated swipe module generates a simulated swipe event and sends it to the application for execution, causing the cursor to swipe along the swipe direction on the interface where the cursor's initial position is located.

[0141] For example, the first simulated swipe event refers to the `action_down` event. The initial time determined based on the first simulated swipe event refers to the moment when the simulated swipe module on the TV generates or sends the `action_down` event (or the moment when the application on the TV receives the `action_down` event). The current time after the first simulated swipe event is completed refers to the moment when the application finishes executing the `action_down` event. During the generation and execution of the simulated swipe event, the current time (`end_downtime`) when the `action_down` event is completed and the initial time (`downtime`) when the `action_down` event is generated are collected. The difference between the current time and the initial time is used as the execution duration of the `action_down` event (`delta_downtime`), which is the execution duration of the simulated swipe event. If the execution duration of the simulated swipe event is less than the swipe duration, it means that the system load resources of the current display device can ensure that the application can normally execute subsequent `action_move` events, and subsequent `action_move` events can be generated according to the preset time interval (e.g., 25ms). If the execution duration of the simulated swipe event is greater than the swipe duration, it is necessary to determine the execution time interval of subsequent `action_move` events based on the system load index of the current display device to ensure normal cursor swipe. Among them, the system load index of the display device can be the average of memory utilization and CPU load. Memory utilization can be the ratio between the current free memory and the total system memory.

[0142] Specifically, when the system load index of the current display device exceeds a first preset threshold, the generation time interval between subsequent adjacent action_move events can be increased by a fixed duration (e.g., 5ms). For example, if the current action_move event generation time interval is 30ms, increasing it by 5ms will result in subsequent action_move events being generated at 35ms, and this time interval will not exceed the maximum preset time interval (e.g., 50ms). When the system load index of the current display device is less than a second preset threshold, the generation time interval between subsequent adjacent action_move events can be decreased by a fixed duration (e.g., 5ms). For example, if the current action_move event generation time interval is 30ms, decreasing it by 5ms will result in subsequent action_move events being generated at 25ms, and this time interval will not be less than the minimum preset time interval (e.g., 25ms).

[0143] It should be noted that when the application completes a full simulated swipe event, the cursor slides from the initial position to the ending position, and the cursor displacement is represented in pixels. The total displacement distance of the cursor is divided into multiple simulated swipe events, each corresponding to a preset number of unit pixels. For each simulated swipe event, the TV sends an `action_move` event. When each `action_move` event completes, the cursor's displacement distance may be the same or different. The time taken for the cursor to move the preset number of unit pixels can be equal to or less than the swipe duration; that is, the cursor's displacement distance must reach the preset number of pixels within the swipe duration. For example, with a preset number of 300, if the TV sends 5 `action_move` events, and the cursor's displacement distance is the same after each `action_move` event, then the displacement distance of each `action_move` event is 60 pixels.

[0144] In one example, if the currently executing simulated swipe event is the first simulated swipe event after the `action_down` event, the cursor will be at the initial position `start=(x0,y0)` when the simulated swipe event begins. If the swipe direction is downward, the cursor position should be `end=(x0,y0+step_px)` when the simulated swipe event completes. During the generation and execution of this simulated swipe event, the completion time of the `action_move` event is collected; that is, the corresponding current time (`nowtime`) is recorded after the current `action_move` event completes. The time interval between the completion time of the current `action_move` event and the completion time of the `action_down` event is the current actual swipe duration (`nowtime-end_downtime`). If the ratio (alpha) between the current actual swipe duration and the swipe duration is between 0 and 1, it indicates that the application is executing the simulated swipe event normally, and this ratio can be used as a linear transition value to calculate the actual position of the cursor. The product of the preset displacement (step_px) and the linear transition value (alpha) of the simulated sliding event is the actual sliding displacement (result) of the cursor. Based on the initial position start of the cursor and the actual sliding displacement (result) of the cursor, the actual position of the cursor at this time can be calculated as (x0, y0 + result).

[0145] In another example, if the currently executing simulated swipe event is the m-th (m ≠ 1) simulated swipe event after the action_down event, it means that the application has already executed the action_down event and the previous m-1 simulated swipe events before starting to execute this simulated swipe event. If the swipe direction is always downward, then the m-th simulated swipe event is executed, and the initial position of the cursor is start=(x0,y1), where y1=y0+(m-1)×step_px. During the generation and execution of this simulated swipe event, the completion time of each action_move event is collected. After the current action_move event is completed, its corresponding current time (nowtime) is recorded. The time interval between the completion time of the current action_move event and the completion time of the (m-1)-th simulated swipe event is the current actual swipe duration (nowtime-end_downtime-(m-1)×duration). If the ratio (alpha) between the current actual sliding duration and the actual sliding duration is between 0 and 1, it indicates that the application is executing the simulated sliding event normally, and this ratio can be used as a linear transition value to calculate the actual position of the cursor. The product of the preset displacement (step_px) of the simulated sliding event and the linear transition value (alpha) is the actual sliding displacement (result) of the cursor. Therefore, based on the initial position of the cursor (start) and the actual sliding displacement (result), the actual position of the cursor at this time can be calculated as (x0, y1 + result).

[0146] The above technical solution has the following advantages or beneficial effects: In the process of generating and executing simulated sliding events in response to a sliding operation initiated by a remote control, if the difference between the current time after the execution of the first simulated sliding event and the initial time determined based on the first simulated sliding event is greater than the sliding duration, the system load index of the display device is obtained, and the generation time interval between subsequent adjacent simulated sliding events is adjusted according to the system load index. Thus, when the system load is overloaded, the frequency of generating simulated sliding events on the TV can be adjusted in a timely manner, avoiding problems such as unresponsive sliding or short sliding distance caused by insufficient system resources and untimely processing of sliding events by the application. This can improve the user's interactive experience.

[0147] In some exemplary embodiments, the controller is also configured to:

[0148] During the generation and execution of simulated sliding events, the execution time of the current non-first simulated sliding events is monitored. If the execution time exceeds the third preset threshold, the cumulative time consumed from the initial time determined based on the first simulated sliding event to the current time, and the cumulative actual displacement from the initial position to the current position are obtained.

[0149] The compensation displacement is obtained based on the cumulative time consumed, the cumulative actual displacement, the preset time consumed in the simulated sliding, and the preset displacement.

[0150] Based on the compensation displacement, adjust the cursor position after the subsequent simulated sliding event is executed.

[0151] For example, the non-first simulated swipe event refers to the action_move event, and the execution duration of the current non-first simulated swipe event refers to the time interval between the moment when the application finishes executing the current action_move event and the moment when the TV sends the current action_move event.

[0152] Specifically, during the execution of the simulated swipe event, if the currently executing simulated swipe event is the first simulated swipe event after the completion of the `action_down` event, and the execution duration of the current `action_move` event exceeds a preset threshold, the time interval between the initial moment when the TV starts generating the `action_down` event and the moment when the application finishes executing the current `action_move` event is calculated. This time interval is the cumulative elapsed time (elapsedTime). Simultaneously, the displacement distance between the initial position of the cursor when the TV starts generating the `action_down` event and the position of the cursor when the application finishes executing the current `action_move` event is calculated. This displacement distance is the cumulative actual displacement (currentPosition). It should be noted that since the cursor remains at its initial position from the start of the `action_down` event on the TV until the completion of the `action_down` event on the application, i.e., the cursor moves 0 pixels, and thus remains at its initial position until the first `action_move` event is executed.

[0153] Specifically, the preset duration of the simulated swipe refers to the time interval between the initial moment when the action_down event is generated on the TV and the moment when the last action_move event of the simulated swipe is executed on the application. The preset displacement refers to the displacement distance between the initial position of the cursor when the action_down event is generated on the TV and the position of the cursor when the last action_move event of the simulated swipe is executed on the application.

[0154] Specifically, since the execution time of the current `action_move` event exceeds a preset threshold, the cumulative actual displacement corresponding to the accumulated time spent in executing the current `action_move` event is less than the preset displacement corresponding to the accumulated time spent. Therefore, the displacement difference between the preset displacement corresponding to the accumulated time spent and the cumulative actual displacement is the compensation displacement. To ensure that the simulated slide is completed within the preset time spent, the cursor position after subsequent simulated slide events are executed needs to be adjusted according to the compensation displacement. Generally, this position should be ahead of the preset position of the cursor after subsequent simulated slide events are executed; that is, the actual displacement of subsequent simulated slide events should be greater than the preset displacement.

[0155] The above technical solution has the following advantages or beneficial effects: When the execution time of the current non-first simulated sliding event exceeds the third preset threshold, the compensation displacement is calculated based on the acquired cumulative time consumed, cumulative actual displacement, preset time consumed for simulated sliding, and preset displacement. The cursor position is then adjusted according to the compensation displacement after the subsequent simulated sliding event is completed. Thus, when the execution time of the current simulated sliding event is too long, the sliding displacement of the subsequent simulated sliding event can be adjusted in a timely manner, thereby ensuring that the preset displacement is completed within the preset time consumed, avoiding problems such as unresponsive sliding or short sliding distance, and thus improving the user's interactive experience.

[0156] In some exemplary embodiments, the controller executes a preset timeout and preset displacement based on the cumulative elapsed time, cumulative actual displacement, and simulated sliding, to obtain the compensation displacement amount, which is configured as follows:

[0157] The ratio of the cumulative actual displacement to the preset displacement is obtained as the displacement progress.

[0158] Obtain the ratio between the cumulative time elapsed and the preset time elapsed, and use it as the time progress;

[0159] Based on the lag between displacement progress and time progress, a dynamic compensation coefficient is obtained.

[0160] The compensation displacement is obtained based on the dynamic compensation coefficient and the cursor position after the current non-first simulated sliding event is executed.

[0161] For example, displacement progress refers to the progress of the current cursor movement relative to the total sliding displacement. Time progress refers to the percentage of time taken for the cursor to move to the current position relative to the total time taken.

[0162] Specifically, the total displacement distance of the simulated sliding event is divided according to a preset unit displacement distance, thus dividing the simulated sliding event into N simulated sliding events. For each simulated sliding event, the TV sends M action_move events. For the k-th simulated sliding event, when k=1, it means that after the application executes the action_down event, the cursor needs to move a preset number of pixels (step_px) within the first sliding duration. After the p-th action_move event of this simulated sliding event is executed, the cumulative actual displacement of the current cursor and the actual time consumed after executing the action_move event are monitored. The ratio between the cumulative actual displacement and the preset number of pixels is taken as the current cursor sliding displacement progress, and the ratio between the actual time consumed and the sliding duration is taken as the current event execution time progress.

[0163] In one example, from the start of the application's execution until the completion of the `action_down` event, the cursor remains at its initial position (x0, y0). When k=1, the cursor needs to move a preset number of pixels (e.g., 300 pixels) within the first sliding duration (e.g., 300ms). If the cursor slides downwards, it means that within 300ms, the final position of the cursor in this simulated sliding event needs to be at (x0, y0+300). If the cursor is at (x0, y0+180) after the p-th `action_move` event of this simulated sliding event is completed, it means that the cumulative actual displacement (`currentPosition`) of the cursor sliding is 180 pixels, and the cumulative elapsed time (`elapsedTime`) is `nowtime-downtime`, where `downtime` is the initial time when the `action_down` event is sent, and `nowtime` is the time when the current `action_move` event is completed. Therefore, the displacement progress can be expressed as (180 pixels / 300 pixels), and the time progress can be expressed as (`nowtime-downtime`) / (300ms+delta_downtime). Delta_downtime refers to the time interval between the completion of the action_down event and the initial time when the action_down event was first sent.

[0164] Specifically, for the k-th simulated swipe event, when k≠1, it means that after the application executes the `action_down` event and the previous k-1 simulated swipe events, the cursor needs to move a preset number of pixels (step_px) within the k-th swipe duration. After the p-th `action_move` event of this simulated swipe event is executed, the cumulative actual displacement of the current cursor swipe (currentPosition) and the actual time elapsed after executing the `action_move` event (elapsedTime) are monitored. The ratio between the cumulative actual displacement and the preset number of pixels is used as the current cursor swipe displacement progress, and the ratio between the actual time elapsed and the swipe duration is used as the current event execution time progress. When the cursor slides down from its initial position (x0, y0) to its current position (x0, y1), where the initial position (x0, y0) is the cursor's position after the (k-1)th simulated sliding event, the cumulative actual displacement of the k-th simulated sliding event can be represented as (y1-y0). The actual time taken after executing the current `action_move` event can be represented as (nowtime-end_time), where `end_time` refers to the moment the (k-1)th simulated sliding event was completed. Therefore, the current cursor displacement can be represented as [(y1-y0)] / step_px, and the current event execution time can be represented as [(nowtime-end_time)] / duration.

[0165] For example, when the application layer executes a sliding event normally, the corresponding time progress should equal the displacement progress after the current action_move event is completed. The lag of the displacement progress relative to the time progress refers to the fact that when the execution duration of the current action_move event exceeds a third preset threshold, the corresponding time progress should be greater than the displacement progress. The theoretical displacement progress can be derived from the current time progress, and then the lag of the actual displacement progress relative to the theoretical displacement progress can be compared to obtain the corresponding compensation coefficient. Further, based on this compensation coefficient, and the actual sliding displacement obtained from the position of the current action_move event, and a distance of a preset number of pixels, the corresponding compensation displacement is calculated.

[0166] The above technical solution has the following advantages or beneficial effects: by calculating the dynamic compensation coefficient based on the displacement progress and time progress, and determining the compensation displacement amount based on the dynamic compensation coefficient, the accuracy of the compensation displacement amount is effectively improved. Thus, when adjusting subsequent simulated sliding events based on the compensation displacement amount, the reliability of the execution of subsequent simulated sliding events is ensured, avoiding the problem of sliding jumps due to excessive compensation displacement amount or short sliding distances due to insufficient compensation displacement amount, thereby improving the user's interactive experience.

[0167] In some exemplary embodiments, the controller performs dynamic compensation based on the lag of displacement progress relative to time progress, and is configured as follows:

[0168] Based on the displacement progress, obtain the remaining displacement progress of the cumulative actual displacement compared to the preset displacement;

[0169] Based on the time progress, obtain the remaining time progress of the cumulative time spent compared to the preset time spent;

[0170] The dynamic compensation coefficient is obtained based on the remaining displacement progress and the remaining time progress.

[0171] For example, the remaining displacement progress refers to the progress that the current cursor needs to continue sliding relative to the total sliding displacement. Here, the displacement progress corresponding to the preset displacement can be set to 1, and the corresponding displacement progress λ can be calculated based on the actual displacement of the cursor. Then, the remaining displacement progress can be expressed as (1-λ).

[0172] The remaining time progress refers to the remaining time that the sliding event can continue to execute relative to the total preset time. Here, the time progress corresponding to the total preset time can be set to 1, and the corresponding time progress μ can be calculated based on the actual time of the sliding event. Then, the remaining time progress can be expressed as (1-μ).

[0173] It should be noted that when the displacement progress lags behind the time progress, the theoretical displacement progress is derived from the current time progress, and the theoretical displacement progress is equal to the current time progress. That is, when the cumulative time consumption is the same, the theoretical pixel displacement of the cursor should be greater than the actual pixel displacement of the cursor.

[0174] For example, if the current time progress is μ, then the theoretical displacement progress is also μ. Therefore, based on the displacement progress λ and the theoretical displacement progress μ, the corresponding compensation coefficient can be expressed as μ-λ. Furthermore, based on this compensation coefficient, the corresponding compensation displacement can be calculated, specifically expressed as (μ-λ)×step_px.

[0175] Furthermore, if the current remaining time progress is 1-μ, then the theoretical remaining displacement progress is also 1-μ. Therefore, based on the remaining displacement progress 1-λ and the theoretical remaining displacement progress 1-μ, the corresponding compensation coefficient can be expressed as (1-λ)-(1-μ). Further, based on this compensation coefficient, the corresponding compensation displacement can be calculated, specifically expressed as [(1-λ)-(1-μ)]×step_px.

[0176] The above technical solution has the following advantages or beneficial effects: by calculating the dynamic compensation coefficient through displacement progress, remaining displacement progress, and time progress, the compensation intensity can be determined. For example, the larger the remaining displacement progress or the smaller the remaining time progress, the greater the compensation intensity. Thus, the compensation displacement is allocated to subsequent sliding events according to the compensation intensity, ensuring the reliability of the compensation displacement allocation, thereby making the compensation process natural and smooth and improving the user's interactive experience.

[0177] In some exemplary embodiments, the controller is configured to adjust the cursor position after subsequent simulated sliding events are completed, based on the compensation displacement.

[0178] The compensation displacement is decomposed into multiple sub-compensation displacements;

[0179] Based on multiple sub-compensation displacements, the cursor position is adjusted after multiple simulated sliding events are executed.

[0180] For example, decomposing the compensation displacement into multiple sub-compensation displacements means that the compensation displacement can be divided according to the number of subsequent action_move events, in which case the number of sub-compensation displacements is equal to the number of subsequent action_move events; or the compensation displacement can be divided without regard to the number of subsequent action_move events, in which case the number of sub-compensation displacements is less than the number of subsequent action_move events. The compensation displacement can be divided evenly or unevenly.

[0181] For example, if multiple sub-compensation displacements are allocated to a corresponding number of subsequent action_move events, then after each subsequent action_move event is executed, the actual position of the cursor should be ahead of the theoretical position of the cursor. That is, within the same time period, after each subsequent action_move event is executed, the actual pixel displacement of the cursor should be greater than the theoretical pixel displacement.

[0182] The above technical solution has the following advantages or beneficial effects: by decomposing the compensation displacement into multiple sub-compensation displacements, and adjusting the cursor position after multiple simulated sliding events are executed according to the sub-compensation displacements, the problem of sliding jumps and excessive system load caused by executing all compensation displacements in a single simulated sliding event is avoided, thereby improving the user's interactive experience.

[0183] In some exemplary embodiments, the controller is also configured to:

[0184] During the generation and execution of simulated sliding events, if the difference between the current time after the execution of a non-first simulated sliding event and the initial time determined based on the non-first simulated sliding event is greater than the sliding duration, the system load index of the display device is reacquired.

[0185] If the system load index is greater than the first preset threshold, increase the execution time interval between subsequent adjacent simulated sliding events; if the system load index is less than the second preset threshold, decrease the execution time interval between subsequent adjacent simulated sliding events.

[0186] For example, after executing multiple simulated sliding events, when the application is executing the simulated sliding events normally, each simulated sliding event needs to be completed within its sliding duration. During the generation and execution of simulated sliding events, the initial time when the TV starts sending the simulated sliding event and the current time when the application finishes executing the simulated sliding event are collected. The difference between the current time and the initial time is taken as the execution duration of the simulated sliding event. If the execution duration is less than the sliding duration, it indicates that the system load resources of the current display device can guarantee the application's normal execution of subsequent simulated sliding events, thus allowing the generation of multiple `action_move` events for the next simulated sliding event at preset time intervals. If the execution duration is greater than the sliding duration, the execution time interval of each `action_move` event for the next simulated sliding event needs to be determined based on the system load indicators of the current display device to ensure normal cursor movement.

[0187] Specifically, when the system load index of the current display device exceeds a first preset threshold, the generation time interval between adjacent action_move events of the next simulated sliding event can be increased by a fixed duration (e.g., 5ms). For example, if the generation time interval of the current action_move event is 30ms, increasing it by 5ms will result in a subsequent action_move event being generated at 35ms, and this time interval will not exceed the maximum preset time interval (e.g., 50ms). When the system load index of the current display device is less than a second preset threshold, the generation time interval between action_move events of the next simulated sliding event can be decreased by a fixed duration (e.g., 5ms). For example, if the generation time interval of the current action_move event is 30ms, decreasing it by 5ms will result in a subsequent action_move event being generated at 25ms, and this time interval will not be less than the minimum preset time interval (e.g., 25ms).

[0188] The above technical solution has the following advantages or beneficial effects: When the difference between the current moment after executing a non-first simulated sliding event and the initial moment determined based on the non-first simulated sliding event is greater than the sliding duration, the system load index of the display device is re-acquired, and the generation time interval between subsequent adjacent simulated sliding events is adjusted according to the system load index. Thus, the system load of the display device can be obtained in real time, and when the system load is overloaded, the frequency of subsequent simulated sliding events generated on the TV can be adjusted in a timely manner. This avoids problems such as unresponsive sliding or short sliding distance caused by insufficient memory resources leading to untimely processing of sliding events by the application, thereby improving the user's interactive experience.

[0189] In some exemplary embodiments, the initial time is the moment when the first simulated sliding event is sent to the application after the first simulated sliding event is generated; the application is used to execute the simulated sliding event.

[0190] The first simulated swipe event refers to the `action_down` event generated on the TV. After the `action_down` event is generated on the TV, the moment when the TV sends the `action_down` event to the application is recorded as the initial moment. The application is used to execute the `action_down` and `action_move` events.

[0191] The above technical solution has the following advantages or beneficial effects: by determining the initial moment when the TV sends the first simulated sliding event, a benchmark can be provided for the subsequent calculation of the sliding duration and sliding displacement of the simulated sliding event, thereby ensuring the accuracy of the cursor position after the application executes the simulated sliding event, and thus improving the user's interactive experience.

[0192] In one specific embodiment, please refer to Figure 15 When the remote control sends a swipe event to the display device's controller, the processor's Linux Kernel receives and parses the swipe event, writing it to ` / dev / input / event0`. The simulated swipe module listens for the touch swipe on the device node and generates one `action_down` event and multiple `action_move` events sequentially, sending them to the `InputReader`. The `InputDispatcher` determines the target window via `WindowManagerService` (WMS) and sends the events sequentially to the target application's UI thread via `InputChannel`. The application's UI thread's `ViewRootImpl.WindowInputEventReceiver` receives the event, which enters the view-level dispatch process (such as `dispatchTouchEvent`), ultimately executing the received swipe event in a view on the application side, thus performing the swipe.

[0193] For details, please refer to Figure 16 When a user begins to touch and swipe using the remote control, the TV receives the data packet sent by the remote control through a listening node. After parsing the packet header and confirming it's a touch event, a touch event thread is established. The data packet is then parsed to obtain the swipe direction (left, right, up, or down) and the duration of the user's touch swipe. First, the simulated swipe module sends an `action_down` event to the application. Then, based on the duration from sending the `action_down` event to the application's completion of the `action_down` event, and combined with the duration sent from the remote control, the simulated swipe module flexibly calculates the next `action_move` event sent by the TV to the application—the distance the application will move the cursor. Upon receiving this movement distance, the application begins controlling the cursor to move across the current page.

[0194] When the duration of the simulated swipe module sending an `action_down` event to the application exceeds the user's touch swipe duration, the scheduler is activated to perform system load monitoring. Under low load, the time interval between subsequent `action_move` events is reduced; under both high and low load, the time interval is increased. After the current `action_move` event is executed, a position deviation is detected based on the cursor's actual position and its theoretical position. If the deviation is small, subsequent `action_move` events are sent normally; if the deviation is large, a compensation event is sent to subsequent `action_move` events. Based on the execution of subsequent `action_move` events, the cursor position is continuously updated, and a decision is made on whether to continue swiping. If the TV receives a restore event from the remote control, the simulated swipe module sends one `action_up` event to the application and then stops sending events to the application, i.e., swiping stops.

[0195] In some exemplary embodiments, please refer to Figure 17 This application provides a sliding control method, the method comprising:

[0196] Step 1702: In response to a sliding operation initiated by the remote control, obtain the corresponding sliding duration, sliding direction and initial position of the cursor; the sliding duration is the time interval between the moment of touching on the remote control and the moment of generating a sliding displacement on the remote control.

[0197] Step 1704: Generate and execute a simulated sliding event to cause the target display object, indicated by the initial position, to perform a sliding operation along the sliding direction.

[0198] Step 1706: During the generation and execution of simulated sliding events, if the difference between the current time after the first simulated sliding event is completed and the initial time determined based on the first simulated sliding event is greater than the sliding duration, obtain the system load index of the display device.

[0199] Step 1708: If the system load index is greater than the first preset threshold, increase the generation time interval between subsequent adjacent simulated sliding events; if the system load index is less than the second preset threshold, decrease the generation time interval between subsequent adjacent simulated sliding events.

[0200] The interaction process between the user, the remote control, and the display device during the execution of the above method can be found in the following reference: Figure 18 ,exist Figure 18In this process, the user initiates a swipe operation on the remote control. In response, the remote control sends the swipe duration, swipe direction, and initial cursor position to the display device system. The swipe duration is the time interval between the moment the touch occurs on the remote control and the moment the swipe displacement occurs. Upon receiving the swipe duration, swipe direction, and initial cursor position from the remote control, the display device system generates and executes a simulated swipe event, causing the target display object indicated by the initial position to perform a swipe operation along the swipe direction. During the generation and execution of the simulated swipe event, if the difference between the current time after the execution of the first simulated swipe event and the initial time determined based on the first simulated swipe event is greater than the swipe duration, the display device system obtains a system load index. If the system load index is greater than a first preset threshold, the display device system increases the generation time interval between subsequent adjacent simulated swipe events; if the system load index is less than a second preset threshold, the display device system decreases the generation time interval between subsequent adjacent simulated swipe events.

[0201] The above technical solution has the following advantages or beneficial effects: In the process of generating and executing simulated sliding events in response to a sliding operation initiated by a remote control, if the difference between the current time after the execution of the first simulated sliding event and the initial time determined based on the first simulated sliding event is greater than the sliding duration, the system load index of the display device is obtained, and the generation time interval between subsequent adjacent simulated sliding events is adjusted according to the system load index. Thus, when the system load is overloaded, the frequency of generating simulated sliding events on the TV can be adjusted in a timely manner, avoiding problems such as unresponsive sliding or short sliding distance caused by insufficient system resources and untimely processing of sliding events by the application. This can improve the user's interactive experience.

[0202] In some exemplary embodiments, the method further includes:

[0203] During the generation and execution of simulated sliding events, the execution time of the current non-first simulated sliding events is monitored. If the execution time exceeds the third preset threshold, the cumulative time consumed from the initial time determined based on the first simulated sliding event to the current time, and the cumulative actual displacement from the initial position to the current position are obtained.

[0204] The compensation displacement is obtained based on the cumulative time consumed, the cumulative actual displacement, the preset time consumed in the simulated sliding, and the preset displacement.

[0205] Based on the compensation displacement, adjust the cursor position after the subsequent simulated sliding event is executed.

[0206] The above technical solution has the following advantages or beneficial effects: When the execution time of the current non-first simulated sliding event exceeds the third preset threshold, the compensation displacement is calculated based on the acquired cumulative time consumed, cumulative actual displacement, preset time consumed for simulated sliding, and preset displacement. The cursor position is then adjusted according to the compensation displacement after the subsequent simulated sliding event is completed. Thus, when the execution time of the current simulated sliding event is too long, the sliding displacement of the subsequent simulated sliding event can be adjusted in a timely manner, thereby ensuring that the preset displacement is completed within the preset time consumed, avoiding problems such as unresponsive sliding or short sliding distance, and thus improving the user's interactive experience.

[0207] In some exemplary embodiments, the compensation displacement is obtained based on the cumulative elapsed time, the cumulative actual displacement, the preset elapsed time of the simulated sliding, and the preset displacement, including:

[0208] The ratio of the cumulative actual displacement to the preset displacement is obtained as the displacement progress.

[0209] Obtain the ratio between the cumulative time elapsed and the preset time elapsed, and use it as the time progress;

[0210] Based on the lag between displacement progress and time progress, a dynamic compensation coefficient is obtained.

[0211] The compensation displacement is obtained based on the dynamic compensation coefficient and the cursor position after the current non-first simulated sliding event is executed.

[0212] The above technical solution has the following advantages or beneficial effects: by calculating the dynamic compensation coefficient based on the displacement progress and time progress, and determining the compensation displacement amount based on the dynamic compensation coefficient, the accuracy of the compensation displacement amount is effectively improved. Thus, when adjusting subsequent simulated sliding events based on the compensation displacement amount, the reliability of the execution of subsequent simulated sliding events is ensured, avoiding the problem of sliding jumps due to excessive compensation displacement amount or short sliding distances due to insufficient compensation displacement amount, thereby improving the user's interactive experience.

[0213] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0214] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0215] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0216] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0217] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A display device, characterized in that, include: A display for showing a user interface, the user interface having a cursor whose position on the user interface is determined by the position pointed to by the remote control in three-dimensional space; The controller is configured to: In response to a sliding operation initiated by the remote control, the corresponding sliding duration, sliding direction, and the initial position of the cursor are obtained; the sliding duration is the time interval between the moment of touching the remote control and the moment when a sliding displacement occurs on the remote control. A simulated sliding event is generated and executed to cause the target display object indicated by the initial position to perform a sliding operation along the sliding direction; During the generation and execution of simulated sliding events, if the difference between the current time after the first simulated sliding event is completed and the initial time determined based on the first simulated sliding event is greater than the sliding duration, the system load index of the display device is obtained. If the system load index is greater than a first preset threshold, the generation time interval between subsequent adjacent simulated sliding events is increased; if the system load index is less than a second preset threshold, the generation time interval between subsequent adjacent simulated sliding events is decreased.

2. The display device according to claim 1, characterized in that, The controller is also configured to: During the generation and execution of simulated sliding events, the execution time of the current non-first simulated sliding events is monitored. If the execution time exceeds a third preset threshold, the cumulative time consumed from the initial time determined based on the first simulated sliding event to the current time and the cumulative actual displacement from the initial position to the current position are obtained. Based on the cumulative time consumed, the cumulative actual displacement, the preset time consumed during simulated sliding, and the preset displacement, the compensation displacement is obtained; Based on the compensated displacement, the cursor position is adjusted after the subsequent simulated sliding event is executed.

3. The display device according to claim 2, characterized in that, The controller executes a calculation based on the cumulative elapsed time, the cumulative actual displacement, the preset elapsed time of the simulated sliding, and the preset displacement to obtain the compensation displacement amount, which is configured as follows: The ratio of the cumulative actual displacement to the preset displacement is obtained as the displacement progress. The ratio between the cumulative time consumed and the preset time consumed is obtained as the time progress; Based on the degree of lag between the displacement progress and the time progress, a dynamic compensation coefficient is obtained; Based on the dynamic compensation coefficient and the cursor position after the current non-first simulated sliding event is executed, the compensation displacement is obtained.

4. The display device according to claim 3, characterized in that, The controller performs dynamic compensation based on the lag of the displacement progress relative to the time progress, and is configured as follows: Based on the displacement progress, obtain the remaining displacement progress of the cumulative actual displacement relative to the preset displacement; Based on the time progress, obtain the remaining time progress of the cumulative time consumption compared to the preset time consumption; Based on the remaining displacement progress and the remaining time progress, a dynamic compensation coefficient is obtained.

5. The display device according to claim 2, characterized in that, The controller, based on the compensated displacement, adjusts the cursor position after subsequent simulated sliding events are completed, and is configured as follows: The compensation displacement is decomposed into multiple sub-compensation displacements; Based on the multiple sub-compensation displacements, the position of the cursor is adjusted after each of the subsequent simulated sliding events is executed.

6. The display device according to claim 1, characterized in that, The controller is also configured to: During the generation and execution of simulated sliding events, if the difference between the current time after the execution of a non-first simulated sliding event and the initial time determined based on the non-first simulated sliding event is greater than the sliding duration, the system load index of the display device is reacquired. If the system load index is greater than a first preset threshold, the execution time interval between subsequent adjacent simulated sliding events is increased; if the system load index is less than a second preset threshold, the execution time interval between subsequent adjacent simulated sliding events is decreased.

7. The display device according to any one of claims 1 to 6, characterized in that, The initial time is the moment when the first simulated sliding event is sent to the application after the first simulated sliding event is generated; the application is used to execute the simulated sliding event.

8. A sliding control method, characterized in that, The method includes: In response to a sliding operation initiated by the remote control, the corresponding sliding duration, sliding direction, and initial position of the cursor are obtained; the sliding duration is the time interval between the moment of touching the remote control and the moment when a sliding displacement occurs on the remote control; A simulated sliding event is generated and executed to cause the target display object indicated by the initial position to perform a sliding operation along the sliding direction; During the generation and execution of simulated sliding events, if the difference between the current time after the first simulated sliding event is completed and the initial time determined based on the first simulated sliding event is greater than the sliding duration, the system load index of the display device is obtained. If the system load index is greater than a first preset threshold, the generation time interval between subsequent adjacent simulated sliding events is increased; if the system load index is less than a second preset threshold, the generation time interval between subsequent adjacent simulated sliding events is decreased.

9. The sliding control method according to claim 8, characterized in that, The method further includes: During the generation and execution of simulated sliding events, the execution time of the current non-first simulated sliding events is monitored. If the execution time exceeds a third preset threshold, the cumulative time consumed from the initial time determined based on the first simulated sliding event to the current time and the cumulative actual displacement from the initial position to the current position are obtained. Based on the cumulative time consumed, the cumulative actual displacement, the preset time consumed during simulated sliding, and the preset displacement, the compensation displacement is obtained; Based on the compensated displacement, the cursor position is adjusted after the subsequent simulated sliding event is executed.

10. The sliding control method according to claim 9, characterized in that, The step of obtaining the compensation displacement based on the cumulative elapsed time, the cumulative actual displacement, the preset elapsed time of the simulated sliding, and the preset displacement includes: The ratio of the cumulative actual displacement to the preset displacement is obtained as the displacement progress. The ratio between the cumulative time consumed and the preset time consumed is obtained as the time progress; Based on the degree of lag between the displacement progress and the time progress, a dynamic compensation coefficient is obtained; Based on the dynamic compensation coefficient and the cursor position after the current non-first simulated sliding event is executed, the compensation displacement is obtained.

Images