Display device and method of processing input event

Abstract

The application discloses a display device and a processing method of an input event. The display device comprises a sending controller and a receiving controller, which are communicatively connected through an internal network transmission channel. The sending controller receives an interactive instruction input by a user, and converts the interactive instruction into at least one of a touch event or a key event. According to an event type and a priority mapping rule, the input event is determined as a high-priority event and a low-priority event. The high-priority event has a higher real-time response requirement. The high-priority event is sent to the receiving controller in an asynchronous transmission mode, and the low-priority event is sent in a synchronous transmission mode. After receiving the input event, the receiving controller distributes or responds according to a deployment position of a target application corresponding to the input event. The application can solve the problem that a high-priority key event is blocked by a low-priority frequent event under a multi-chip architecture, guarantee that the input event is accurately processed by a corresponding controller, and thus improve the interactive experience.

Description

Technical Field

[0001] This application relates to the field of display device technology, and in particular to a display device and a method for processing input events. Background Technology

[0002] Display devices are intelligent devices capable of presenting a user interface and supporting user interaction. Taking smart TVs as an example, smart TVs are television products based on Internet application technology, equipped with open operating systems and chips, possessing open application platforms, enabling two-way human-computer interaction, and integrating multiple functions such as audio-visual, entertainment, and data to meet diverse and personalized user needs.

[0003] Display devices can employ a multi-chip architecture, where different chips collaborate to meet performance requirements in various scenarios. A multi-chip architecture may include a master control chip responsible for running the system, applications, and interaction logic, and slave chips responsible for processing complex image quality algorithms. In this architecture, different types of input devices can be physically connected to their respective chips and driven by those chips. For example, a pointing remote control can be connected to the master control chip, while peripherals such as Bluetooth remote controls, keyboards, and mice can be connected to the image quality chip. Input events are communicated between different chips via internal network transmission channels. All input events collected by a slave chip must be transmitted synchronously to the master control chip in sequence. Only after the input events have been transmitted can they be stored in the slave chip's local queue for distribution.

[0004] However, this method of transmitting all input events synchronously in sequence can lead to slow operation response or abnormal interaction state, affecting the user's interactive experience, because different operation events correspond to different importance and frequency of occurrence. Highly important critical operation events (such as key press and release events) will be blocked by less important and frequently occurring non-critical operation events (such as movement and hover movement events). Summary of the Invention

[0005] This application provides a display device and a method for processing input events to solve the problems of slow operation response and abnormal interaction state of multi-chip architecture display devices.

[0006] In a first aspect, this application provides a display device, comprising: monitor; First controller; The second controller is electrically connected to the display and establishes a communication connection with the first controller through an internal network transmission channel; the second controller is also configured with a user input interface, which is configured to receive interactive commands input by the user; The second controller is also configured to: The interactive command is converted into a corresponding input event, wherein the input event is at least one of a touch event or a key event; According to the mapping rule between the event type and priority of the input events, the first type of input event is determined as the first priority event, and the second type of input event is determined as the second priority event; the priority of the first priority event is higher than the priority of the second priority event, and the real-time response requirement of the first priority event is higher than the real-time response requirement of the second priority event. For the first priority event, it is sent to the first controller via the internal network transmission channel in an asynchronous manner; For the second priority event, it is transmitted to the first controller via the internal network transmission channel in a synchronous transmission manner; The first controller is configured as follows: After receiving the input event through the internal network transmission channel, the system distributes or responds to the target application corresponding to the input event based on its deployment location, wherein the target application is deployed on the first controller or the second controller.

[0007] The above technical solution has the following beneficial effects or advantages: By employing a differentiated mechanism of high-priority asynchronous transmission and low-priority synchronous transmission, timely responses to critical operations affecting the correctness of the interaction state are ensured, resolving the issue of high-priority critical events being blocked by frequent low-priority events during synchronous transmission of input events in multi-chip architectures. Simultaneously, a distribution strategy based on the target application's deployment location adapts to scenarios where applications run on any controller in a dual-chip architecture, ensuring that the application accurately receives and responds to interaction commands.

[0008] In some embodiments of this application, the first type of input event includes at least the key press event and key release event in the key events, and the press event, release event, hover enter event, hover exit event, and operation cancel event in the touch events; The second type of input event includes at least the movement event and hover movement event in the touch event.

[0009] The above technical solution has the following beneficial effects or advantages: Input events that change the interaction state are classified as first priority, while input events that maintain the interaction state are classified as low priority. This ensures that critical operations such as button press, release, and touch start and end are not blocked. At the same time, motion events that need to be rate-limited are locked to avoid unnecessary event dropping that affects the user experience.

[0010] In some embodiments of this application, the second controller is further configured to: Maintain the transmission count of the second priority events; When a second priority event is sent, the send count value is incremented by a preset amount; The second controller executes synchronous transmission by sending the second priority event to the first controller through the internal network transmission channel, specifically configured as follows: Upon detecting a target second-priority event, read the count value of the transmission count; If the count value is the first value, then the target second priority event is sent, and the sending time of the target second priority event is recorded as the historical sending time; If the count value is the second value, then the target second priority event is sent or discarded according to the preset synchronization transmission rules; wherein, the first value indicates that the second priority event has not been sent, and the second value indicates that at least one second priority event has been sent; Before the second controller executes asynchronous transmission to send the first priority event to the first controller via the internal network transmission channel, it is further configured to: Upon detecting the first priority event, the transmission count is reset to the first value, and the historical transmission time is cleared.

[0011] The above technical solution has the following beneficial effects or advantages: A dynamic frequency reduction sending mechanism for second-priority events is established. This mechanism distinguishes between first-time and non-first-time sending scenarios by using a sending count, ensuring timely transmission of the first second-priority event in each interaction cycle and avoiding initial delays when the user begins moving the cursor or touching the screen. Resetting the count and historical time when a first-priority event is triggered ensures the continuity and independence of the interaction.

[0012] In some embodiments of this application, the second controller executes the sending or discarding of the target second priority event according to a preset synchronization transmission rule, specifically configured as follows: Read the historical transmission time, which is the transmission time corresponding to the previous historical second priority event; Based on the historical transmission time, calculate the transmission time interval between the target second priority event and the historical second priority event; If the sending time interval is greater than the preset time interval, then the target second priority event is sent, and the historical sending time is updated according to the sending time of the target second priority event; If the transmission time interval is less than or equal to the preset time interval, the target second priority event is discarded.

[0013] The above technical solution has the following beneficial effects or advantages: By controlling the transmission time interval between adjacent events, the number of redundant second-priority events (such as movement events) can be reduced without affecting user perception, and the possibility of high-priority events being blocked can also be reduced.

[0014] In some embodiments of this application, the first controller performs distribution or response based on the deployment location of the target application corresponding to the input event, specifically configured as follows: Perform a consistency check on the touch event, and convert the consistency-checked touch event into a distributable event; Add the distributable event to the first distribution queue; If the target application is deployed on the first controller, the distributable event is distributed to the target application based on the first distribution queue in response to the interaction command; If the target application is deployed on the second controller, the second controller is further configured as follows: Convert the touch event into a dispatchable event; Add the distributable event to the second distribution queue; Based on the second distribution queue, the distributable event is distributed to the target application in response to the interaction command.

[0015] The above technical solution has the following beneficial effects or advantages: A cross-chip event distribution system is constructed. The first controller performs consistency verification on touch events, filtering out issues such as inappropriate timestamps or abnormal omissions during transmission, thus ensuring the correctness of distributed events. Furthermore, after converting input events into standard distributable events, each controller detects the target application affected by the current input event, and the controller currently running the target application distributes the distributable event.

[0016] In some embodiments of this application, the touch event includes a state start event and a corresponding state end event. The state start event is a press event or a hover enter event in the touch event. The state end event corresponding to the press event is a release event in the touch event. The state end event corresponding to the hover enter event is a hover exit event. The state start event is used to start a target state, and the state end event is used to end the target state. The first controller performs a consistency check on the touch event, specifically configured as follows: In response to receiving the state start event, the timestamp of the state start event is recorded as the latest occurrence time of the target state; If the target state in the system is enabled, then an end event for the state is generated, and the target state is terminated. The state end event is converted into a first distributable event, and the first distributable event is added to the first distribution queue; Generate the state start event and activate the target state; The state start event is converted into a second distributable event, and the second distributable event is added to the first distribution queue.

[0017] The above technical solution has the following beneficial effects or advantages: The system ensures the integrity of interactive states by automatically completing the previous state's end event upon receiving a new state start event. In the event of a lost lift event or hover exit event, the system can automatically restore to the correct state, preventing malfunctions such as the interface remaining in a pressed or hovered state and the interface becoming unresponsive.

[0018] In some embodiments of this application, the first controller is further configured to: If the target state in the system is "end", then generate the "start state" event and start the target state. The state start event is converted into a third distributable event, and the third distributable event is added to the first distribution queue.

[0019] The above technical solution has the following beneficial effects or advantages: When the current target state is already in the end state, the new state start information is recorded directly and the corresponding dispatchable event is generated to ensure that the interaction process can enter the correct state normally and will not encounter the problem of state deadlock and inability to proceed.

[0020] In some embodiments of this application, the touch event further includes a state intermediate event, which is a second priority event, and the state intermediate event is used to maintain the target state; the first controller performs a consistency check on the touch event, specifically configured as follows: In response to receiving the intermediate state event, the timestamp of the intermediate state event is detected; If the timestamp of the intermediate state event is earlier than the latest occurrence time of the target state, then the intermediate state event is discarded. If the timestamp of the intermediate state event is later than the latest occurrence time of the target state, the intermediate state event is converted into a fourth distributable event and the fourth distributable event is added to the first distribution queue.

[0021] The above technical solution has the following beneficial effects or advantages: By comparing the timestamps of intermediate state events with the latest occurrence time of the target state, outdated events that occur earlier than the latest state change are filtered out. This solves the problem of abnormal timing of low-priority intermediate state events caused by asynchronous transmission, avoids cursor position jumps and touch trajectory errors caused by the reversed event transmission order, and ensures the correctness and smoothness of dynamic parameter updates during the interaction process.

[0022] Secondly, this application also discloses a display device, comprising: monitor; The first controller is configured with a user input interface, which is configured to receive interactive commands input by the user. The second controller is electrically connected to the display and establishes a communication connection with the first controller through an internal network transmission channel; The first controller is also configured to: The interactive command is converted into a corresponding input event, wherein the input event is at least one of a touch event or a key event; According to the mapping rule between the event type and priority of the input events, the first type of input event is determined as the first priority event, and the second type of input event is determined as the second priority event; the priority of the first priority event is higher than the priority of the second priority event, and the real-time response requirement of the first priority event is higher than the real-time response requirement of the second priority event. For the first priority event, it is sent to the second controller via the internal network transmission channel in an asynchronous manner; For the second priority event, it is transmitted to the second controller via the internal network transmission channel in a synchronous transmission manner; The second controller is configured as follows: After receiving the input event through the internal network transmission channel, the system distributes or responds to the target application corresponding to the input event based on its deployment location, wherein the target application is deployed on the first controller or the second controller.

[0023] The above technical solution has the following beneficial effects or advantages: Similar to the display device in the first aspect, when the input interface is mounted on the first controller, the differentiated mechanism of first-priority asynchronous transmission and low-priority synchronous transmission can still ensure timely response to critical operations affecting the correctness of the interaction state. This solves the problem of first-priority critical events being blocked by frequent low-priority events when synchronously transmitting input events in a multi-chip architecture. Simultaneously, a distribution strategy based on the target application deployment location adapts to scenarios where the application runs on any controller in a dual-chip architecture, ensuring that the application accurately receives and responds to interaction commands.

[0024] Thirdly, this application also provides a method for processing input events, including: The system receives interactive commands input by the user and converts the interactive commands into corresponding input events; the input events are at least one of touch events or key events. According to the event type and priority mapping rule of the input events, the first type of input event is determined as the first priority event, and the second type of input event is determined as the second priority event; the real-time response requirement of the first priority event is higher than that of the second priority event. For the first priority event, it is sent to the receiving controller via the internal network transmission channel in an asynchronous transmission manner; For the second priority event, it is transmitted to the receiving controller via the internal network transmission channel in a synchronous transmission mode.

[0025] The above technical solution has the following beneficial effects or advantages: By employing a differentiated transmission mechanism that prioritizes asynchronous transmission over low-priority synchronous transmission, timely responses to critical operations are ensured, resolving the issue of high-priority critical events being blocked by frequent low-priority events during synchronous transmission of input events in multi-chip architectures. Furthermore, a distribution strategy based on the target application's deployment location adapts to scenarios where applications run on any controller in a multi-chip architecture, ensuring that applications accurately receive and respond to user-issued interactive commands, thereby improving the smoothness and responsiveness of multi-chip architecture display devices. Attached Figure Description

[0026] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a schematic diagram illustrating the operation scenarios of a display device provided in some embodiments of this application; Figure 2 This is a schematic diagram of the hardware configuration of a display device provided in some embodiments of this application; Figure 3 This is a schematic diagram of the software configuration of a display device provided in some embodiments of this application; Figure 4 Input system architecture diagrams provided for some embodiments of this application; Figure 5 A schematic diagram illustrating the principle of cross-chip synchronous transmission provided for some embodiments of this application; Figure 6Interaction diagrams illustrating optimized front-end cross-chip transmission provided in some embodiments of this application; Figure 7 This is a connection diagram of a dual-chip architecture provided in some embodiments of this application; Figure 8 An interactive diagram illustrating the optimized cross-chip transmission provided in some embodiments of this application; Figure 9 This application provides a schematic diagram of a complete event handling process for some embodiments; Figure 10 This is a schematic diagram of the transmission processing flow provided for some embodiments of this application; Figure 11 This application provides another complete event handling flowchart for some embodiments; Figure 12 A flowchart illustrating an input event processing method provided in some embodiments of this application; Figure 13 This is a timing interaction diagram of the input event processing method provided in some embodiments of this application. Detailed Implementation

[0028] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application as detailed in the claims.

[0029] In this embodiment, display device 200 generally refers to a device with screen display and data processing capabilities. For example, display device 200 includes, but is not limited to, smart TVs, mobile terminals, computers, monitors, advertising screens, wearable devices, virtual reality devices, augmented reality devices, etc.

[0030] Figure 1 This is a schematic diagram illustrating an operational scenario between a display device and a control device provided in some embodiments of this application. For example... Figure 1 As shown, users can operate the display device 200 via touch operation, mobile terminal 300, and control device 100. For example, control device 100 can be a remote control, stylus, gamepad, etc.

[0031] The mobile terminal 300 can function as a control device for human-computer interaction between the user and the display device 200. It can also function as a communication device for establishing a communication connection with the display device 200 and exchanging data. In some embodiments, the mobile terminal 300 can have software applications installed on it and communicate with the display device 200 via network communication protocols to achieve one-to-one control and data communication. Furthermore, it can transmit audio and video content displayed on the mobile terminal 300 to the display device 200 for synchronized display.

[0032] like Figure 1 The diagram also shows that the display device 200 communicates with the server 400 via various communication methods. This allows the display device 200 to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0033] Display device 200 can provide broadcast television reception function, and can also be equipped with intelligent network television function that provides computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), etc.

[0034] Figure 2 Provided for some embodiments of this application Figure 1 Hardware configuration block diagram of display device 200.

[0035] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface 280.

[0036] In some embodiments, detector 230 is used to collect signals from the external environment or signals interacting with the outside world. For example, detector 230 may include millimeter-wave radar, which can be used to detect whether a user is present within a preset range. Detector 230 may also include a voice acquisition unit to collect voice commands input by the user.

[0037] In some embodiments, the display 260 includes display function components for presenting images and driving components for driving image display. The display 260 is used to receive and display image signals output from the controller 250. For example, the display 260 can be used to display video content, image content, menu control interface components, and user control UI interfaces, etc.

[0038] In some embodiments, the communication device 220 is a component used to communicate with external devices or the server 400 according to various communication protocol types. The display device 200 may have multiple communication devices 220 depending on the supported communication methods. For example, when the display device 200 supports wireless network communication, it may have a communication device 220 with WiFi functionality. When the display device 200 supports Bluetooth connectivity, it needs to have a communication device 220 with Bluetooth functionality.

[0039] The communication device 220 enables the display device 200 to communicate with external devices or the server 400 via wireless or wired connections. Wired connections utilize data cables, interfaces, or other components to connect the display device 200 to external devices. Wireless connections utilize wireless signals or wireless networks. The display device 200 can directly establish a connection with external devices or indirectly through gateways, routers, or other connection devices.

[0040] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in memory. The controller 250 controls the overall operation of the display device 200.

[0041] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0042] In some embodiments, a user can input user commands through a graphical user interface (GUI) displayed on a display 260, and the user input interface receives user input commands through the graphical user interface (GUI).

[0043] In some embodiments, the audio output device 270 can be a built-in speaker of the display device 200 or an external audio output device connected to the display device 200. For the external audio output device connected to the display device 200, the display device 200 may also be provided with an external audio output terminal, through which the audio output device can be connected to the display device 200 to output sound from the display device 200.

[0044] In some embodiments, the user input interface 280 can be used to receive instructions input by a user. The user input interface 280 may include at least one of a microphone, touchpad, sensor, remote control, etc. The display device 200 can then receive user-input instructions based on the user input interface 280 to perform interactive functions with the user.

[0045] In some embodiments, to enable user interaction, the display device 200 may run an operating system. An operating system is a computer program that manages and controls the hardware and software resources of the display device 200. The operating system can control the display device to provide a user interface; for example, the operating system can directly control the display device to provide a user interface, or it can provide a user interface by running applications. The operating system also allows users to interact with the display device 200.

[0046] It should be noted that the operating system can be a native operating system based on a specific operating platform, a third-party operating system that is deeply customized based on a specific operating platform, or an independent operating system specifically developed for display devices.

[0047] An operating system can be divided into different modules or levels based on the functions it implements, for example... Figure 3 As shown, in some embodiments, the system is divided into four layers, from top to bottom: the Applications layer (referred to as the "Application Layer"), the Application Framework layer (referred to as the "Framework Layer"), the System Library layer, and the Kernel layer.

[0048] In some embodiments, the application layer provides services and interfaces for applications, enabling the display device 200 to run applications and interact with the user based on the applications. The application layer may contain at least one application, which may be a built-in Windows program, system settings program, or clock program of the operating system; or it may be an application developed by a third-party developer. In specific implementations, the application packages in the application layer are not limited to the examples above.

[0049] The framework layer provides application programming interfaces (APIs) and a programming framework for applications. The application framework layer includes predefined functions. It acts as a central processing unit, determining the actions taken by applications within the application layer. Through the API, applications can access system resources and obtain system services during execution.

[0050] like Figure 3As shown, the application framework layer in this embodiment includes a view system, managers, and content providers. The view system designs and implements the application's interface and interactions, and includes lists, grids, text boxes, and buttons. The managers include at least one of the following modules: an activity manager for interacting with all running activities in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to application packages currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0051] In some embodiments, the Activity Manager manages the lifecycle of individual applications and common navigation and back functions, such as controlling application exit, opening, and back actions. The Window Manager manages all window programs, such as obtaining the screen size, determining if a status bar is present, locking the screen, capturing the screen, and controlling changes to the display window, such as shrinking the display window, shaking the display, or distorting the display.

[0052] In some embodiments, the system runtime library layer can provide support for the framework layer. When the framework layer is used, the operating system runs the instruction library contained in the system runtime library layer, such as the C / C++ instruction library, to implement the functions to be performed by the framework layer.

[0053] In some embodiments, the kernel layer is a functional layer situated between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. For example, ... Figure 3 As shown, hardware drivers can be configured in the kernel layer. The kernel layer can contain at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0054] In some embodiments, the display device 200 adopts a single-chip architecture, that is, the display device 200 contains only one controller 250.

[0055] For a single-chip architecture, in some embodiments, the controller 250 of the display device 200 is also configured with a user input interface 280. The display device 200 is connected to at least one input device through the user input interface 280 and receives interactive commands input by the user based on the input device. For example, the input device includes external input devices such as an infrared remote control, a Bluetooth remote control, a pointing remote control, a keyboard, a mouse, and a touch screen, and the connection method can include at least one of wired or wireless connection. The display device 200 can distribute and respond to input events through the input system.

[0056] Figure 4 The diagram illustrates the input system architecture provided for some embodiments of this application. For example... Figure 4 As shown, in some embodiments, the display device 200 translates interactive instructions into standard input events and processes them. Figure 3 The software configuration shown illustrates how device 200 processes input events, divided into three layers from bottom to top: the hardware device layer, the kernel space layer, and the user space layer. The hardware device layer, as the physical source of input events, is responsible for converting user physical operations (such as key presses, finger swipes, and mouse movements) into raw electrical signals as interactive commands. The kernel space layer, through drivers such as RC-5, Bluetooth, and HID (Human Interface Device), receives the raw electrical signals generated by the hardware devices and converts them into a standard input event format (including fields such as event type, timestamp, key value / coordinates, and action status). It also creates a corresponding device node for each input device, serving as the interface for user space to access kernel events.

[0057] In some embodiments, the core processing logic of the user space layer runs in the system server process, with input events managed uniformly by the InputManagerService (IMS). IMS contains two independent threads: the InputReader thread, responsible for event reading and conversion, and the InputDispatcher thread, responsible for event distribution. The InputReader thread interacts directly with the kernel through the EventHub submodule, listening to events from all input device nodes and handling hot-plug detection. The InputManager submodule converts the raw kernel events read from EventHub into specific event types recognizable by the upper layer; key events are converted to KeyEvents (containing key values, press / release states, and function key states, i.e., key value events), and touch or pointer events are converted to MotionEvents (containing coordinates, multi-touch information, and move / press / release states, i.e., touch events). The InputReader thread also has a PointerController submodule, specifically for handling pointer-based devices (mouse, pointing remote), responsible for calculating cursor position and managing cursor display state. After receiving the processed input event from the InputReader, the InputDispatcher thread queries the Window ManagerService (WMS) for information about the currently focused window. It then sends the event to the corresponding application process through a socketpair inter-process communication channel and manages the event timeout mechanism (such as key press judgment, ANR detection, etc.).

[0058] In some embodiments, WMS and IMS belong to the same system server process, serving as a bridge between the input and display systems. WMS manages the hierarchy, size, position, and focus state of all windows in the system, provides the InputDispatcher with a query service for the currently focused window, and requests the corresponding drawing surface (Surface) and cursor-specific sprite layer from the image compositing service (SurfaceFlinger). SurfaceFlinger, as the image compositing service, is responsible for compositing all application window layers onto the screen. When the PointerController updates the cursor position, SurfaceFlinger, upon the arrival of the next vertical sync signal (VSYNC), composites the cursor layer with all application window layers, ultimately sending the composited image to the screen via the hardware compositor (HWComposer). The application process receives KeyEvents or MotionEvents sent by the InputDispatcher via socketpair, performs event dispatching in the application's main thread, passes the events to the view hierarchy (View tree) of the currently focused window, calls callback methods such as onKeyDown and onTouchEvent layer by layer, triggers View redrawing after processing, and submits the new image content to SurfaceFlinger for compositing and display via the Surface.

[0059] In the aforementioned single-chip architecture, all input events in the display device 200 are processed within the same system server process, resulting in low processing latency and high state consistency. However, with the development of intelligent display devices, a single-chip architecture can no longer simultaneously meet the multiple demands of high-quality image processing, complex application ecosystems, and low-latency interaction. Therefore, in some embodiments, the display device 200 may adopt a multi-chip architecture, where different chips work together to meet the performance requirements of various scenarios.

[0060] In some implementations, the display device 200 includes a main SoC and an image quality chip, with the main SoC responsible for system operations and application logic, and the image quality chip responsible for image quality algorithm processing.

[0061] In other implementations, the display device 200 includes a main SoC, an image quality chip, and an ultra-wide screen chip. For example, in a scenario where the image quality chip only supports 16:9 resolution output while the ultra-wide screen chip supports 21:9 resolution output, an additional chip is added to be responsible for the final display output and image composition.

[0062] For a multi-chip display device 200, the physical connections and drivers of different input devices may be distributed across different chips. For example, a pointing remote control shares a hardware module with the camera and is physically mounted on the circuit board corresponding to the main chip; while other peripherals such as ordinary infrared remote controls, Bluetooth remote controls, and USB keyboards and mice are connected to the image quality chip through their respective interfaces. This distributed connection pattern requires the display device 200 to synchronously transmit input events collected from different chips across chips.

[0063] In some embodiments, the cross-chip synchronous transmission of input events in the display device 200 is achieved through a local area network (LAN). Specifically, an internal network transmission channel (e.g., a socket-based HSPcore component) is established via a private LAN or private USB protocol between chips, and input events are synchronously transmitted between different chips through this internal network transmission channel. This internal network transmission channel has low latency, high stability, and can respond quickly to input events.

[0064] Figure 5 This is a schematic diagram illustrating the principle of cross-chip synchronous transmission provided in some embodiments of this application. For example... Figure 5 As shown, in some embodiments, the user input interface 280 of the main chip establishes a communication connection with the pointing remote control and receives interactive commands sent by the pointing remote control; the user input interface 280 of the image quality chip establishes a communication connection with a regular infrared remote control, Bluetooth remote control, and USB keyboard and mouse, and receives interactive commands sent by the regular infrared remote control, Bluetooth remote control, and USB keyboard and mouse. Unlike the single-chip architecture described above, the InputDispatcher of the display device 200 also includes an hspcore component and an input receiving (DualInputReceiver) module. The input receiving module is responsible for receiving input events transmitted from the InputReader thread. Different chips synchronize input events through the hspcore component. It should be noted that for the remaining processing flow of input events, please refer to the processing flow of the single-chip architecture in the above embodiments, which will not be elaborated here.

[0065] Figure 6 These are schematic diagrams illustrating optimized front-end cross-chip transmission interactions provided in some embodiments of this application. For example... Figure 6As shown, in some embodiments, raw events generated by peripherals (such as keyboards and mice) are first read and processed into NotifyArgs events (including NotifyKeyArgs, NotifyMotionArgs, etc.) by the InputReader thread of the chip (such as the image processing chip), and then stored in NotifyArgsQueue. In this embodiment, NotifyArgs events are represented as input events. Subsequently, the events in this queue are preferentially forwarded synchronously to the network input event receiving thread of the main chip through an internal network transmission channel (such as a socket), and stored in the InboundQueue (EventEntryQueue) on the main chip side; only after the network transmission is completed will the events be stored in the InboundQueue of the slave chip itself to await local distribution. In this embodiment, EventEntryQueue is represented as a distributable event.

[0066] Each event must wait for the network transmission to complete before the next event can be processed. When the InputReader reads events from multiple devices at once, or multiple events (e.g., Android's default maximum of 16 events per read), and the frequency of read events exceeds the network transmission speed or a network timeout occurs, events will accumulate in the queue or be abnormally lost. For example, events with high real-time response requirements, such as button presses and releases, touch presses and releases, may be blocked by events with low real-time response requirements, such as movement events and hover movement events, resulting in severe delays in critical operations. On the UI displayed on the monitor 260, this may manifest as short presses being misidentified as long presses, button ANRs, and lag in interaction.

[0067] It should be noted that the aforementioned main chip, slave chip, and image quality chip are chips responsible for different functions. In this application, different chips are distinguished by the description of a first controller and a second controller. The first controller may be a main chip or a slave chip, and the second controller may correspond to a slave chip or a main chip. This application does not limit the specific division of labor between the first controller and the second controller.

[0068] To address the aforementioned issues, this application provides a display device and a method for processing input events, specifically for a multi-chip architecture display device 200, to improve problems such as latency, blocking, timing errors, and abnormal states during cross-network transmission of input events. For example... Figure 7 As shown, in some embodiments, the display device 200 includes a first controller 251 and a second controller 252. The first controller 251 and the second controller 252 establish a communication connection with each other through an internal network transmission channel, and the second controller 252 is electrically connected to the display 260. For example, the first controller 251 may be a main SoC chip, and the second controller 252 may be an image quality chip.

[0069] In an exemplary scenario, the second controller 252 (such as an image quality chip) is responsible for handling complex computational tasks such as image quality algorithms, while the first controller 251 (such as the main SoC) is responsible for running system applications and interaction logic, and undertaking global event decision-making functions.

[0070] In some implementations, the internal network transmission channel is a dedicated inter-chip communication link established based on a private local area network or a private USB protocol, without passing through an external network.

[0071] In some embodiments, the first controller 251 and the second controller 252 can also be interconnected through other device interfaces 240, such as HDMI interfaces, USB interfaces, etc. For example, peripherals such as pointing remote controls, Bluetooth remote controls, keyboards, mice, and touch screens can be physically connected to the user input interfaces of the first controller 251 or the second controller 252, and the input subsystem InputReader running on the corresponding controller is responsible for collecting and processing the raw input events.

[0072] When performing synchronous transmission of input events across chips, both the first controller 251 and the second controller 252 can be transmitting controllers, that is, controllers that send input events to another controller; correspondingly, the first controller 251 and the second controller 252 can also be receiving controllers, that is, controllers that receive input events.

[0073] Compared to Figure 6 In some embodiments of the synchronous transmission shown, corresponding modules are added to the transmitting end controller and the receiving end controller, namely a transmitting end processing module and a receiving end processing module, respectively. For example... Figure 8 As shown, the transmitting end processing module is deployed on the transmitting end controller, and the receiving end processing module is deployed on the receiving end controller. The transmitting end processing module includes a priority marking module 801, a dynamic frequency reduction module 802, an asynchronous transmission module 803, and a synchronous transmission module 804; the receiving end processing module includes a timestamp correction module 805 and a state machine management module 806. These modules are inserted into the transmission link after the transmitting end InputReader and before the receiving end InputDispatcher, ensuring compatibility of synchronous transmission of input events across chips.

[0074] It should be noted that the transmitting and receiving controllers described in this application are not fixed roles; both the first controller 251 and the second controller 252 can act as either the transmitting or receiving end. That is, when an input device is connected to the second controller 252 and needs to send input events to the first controller 251, the second controller 252 acts as the transmitting controller, and the first controller 251 acts as the receiving controller; conversely, when an input device is connected to the first controller 251 and needs to send input events to the second controller 252, the first controller 251 acts as the transmitting controller, and the second controller 252 acts as the receiving controller. If a chip both transmits and receives input events via the network, both the transmitting and receiving processing modules will be deployed on that chip. Furthermore, this application is not limited to a dual-chip architecture but is also applicable to multi-chip architectures containing three or more controllers. Each pair of controllers communicating with each other through an internal network transmission channel can have its transmitting and receiving processing modules deployed independently, achieving optimized transmission of input events across multiple chips.

[0075] Figure 9 This is a schematic diagram illustrating a complete event handling process provided for some embodiments of this application. When the second controller 252 acts as the sending end and the first controller 251 acts as the receiving end, the second controller 252 is configured with a user input interface, which is configured to receive interactive commands input by the user. In some embodiments, such as... Figure 9 As shown, the second controller 252 is configured to perform the following steps: S901 receives interactive instructions and converts them into input events.

[0076] The InputReader thread of the second controller 252 (sender controller) continuously collects raw user input events from user input interfaces (such as device nodes under the / dev / input directory). After being read by EventHub and processed by InputMapper, these events are converted into standard NotifyArgs events (including NotifyKeyArgs, NotifyMotionArgs, etc.) and stored in NotifyArgsQueue. At this point, the events are no longer directly forwarded via synchronous network, but instead first enter the sender processing module for priority classification and differential processing.

[0077] S902. According to the mapping rules between the event type and priority of the input event, the first type of input event is determined as the first priority event, and the second type of input event is determined as the second priority event.

[0078] Priority is classified according to the event type and priority mapping rules. The second controller 252 divides the input event into first priority event and second priority event by obtaining the event type identifier corresponding to the input event.

[0079] In other words, the first priority event has a higher priority than the second priority event; the first priority event is a high priority event, and the second priority event is a low priority event.

[0080] In some implementations, if the event type identifier belongs to a preset set of high-priority event types, the input event is determined as a first-priority event; if the event type identifier does not belong to the preset set of high-priority event types, the input event is determined as a second-priority event. The real-time response requirement for the first-priority event is higher than that for the second-priority event.

[0081] In some embodiments, the first type of input event (first priority event) includes at least: key press event (KEY_DOWN) and key release event (KEY_UP) in key events, and press event (ACTION_DOWN), release event (ACTION_UP), hover enter event (HOVER_ENTER), hover exit event (HOVER_EXIT), and operation cancel event (ACTION_CANCEL) in touch events. The second type of input event (second priority event) includes at least: touch event (ACTION_MOVE) and hover move event (HOVER_MOVE).

[0082] In other words, the first priority events are all state transition events, namely state start events (such as pressing or hovering to enter) and state end events (such as releasing, hovering to exit, or canceling the operation). If these events are delayed, blocked, or lost, the system cannot enter or exit the interactive state normally, directly causing the operation to fail or the interface to freeze, requiring extremely high real-time performance. The second priority events are all intermediate state events (such as moving or hovering to move), used to continuously update position parameters within the already opened target state. They have strong redundancy, and a small delay or loss will not affect the overall interaction logic, and the user will not be aware of it.

[0083] S903. For the first priority event, send it to the first controller in an asynchronous manner through the internal network transmission channel.

[0084] Asynchronous transmission is performed for first-priority events. When a first-priority event is detected, it is sent immediately to ensure that the first-priority event can reach the receiving end as quickly as possible and is not blocked by the second-priority events that have accumulated in the queue.

[0085] In this application, asynchronous transmission refers to a non-blocking, instantaneous transmission mechanism for high-priority events. Whether the internal network transmission channel is idle or busy, it does not wait for the preceding data transmission to complete; upon detecting a high-priority event, it is immediately encapsulated and sent independently, without time interval restrictions or drop / rate limiting. This asynchronous transmission method does not need to be bound to the transmission timing of second-priority events, independently preempting transmission resources to ensure that highly real-time interactive events are delivered to the receiving end first and immediately, avoiding blockage, delay, or discarding by consecutive second-priority events.

[0086] S904. For second priority events, the data is transmitted to the first controller via the internal network transmission channel in a synchronous transmission mode.

[0087] Synchronous transmission is performed for second-priority events. In some implementations, the second controller 252 performs synchronous transmission and dynamic frequency reduction for second-priority events. When a second-priority event is detected, the transmission frequency of the second-priority event is reduced by a preset event interval. If the transmission interval is less than or equal to the preset interval, the current target second-priority event is discarded without transmission, thereby reducing the amount of data transmitted per unit time, releasing network transmission resources, and preventing second-priority events from consuming too much bandwidth and blocking first-priority events. Simultaneously, the dynamic frequency reduction mechanism does not affect the interaction effect. Since the second-priority event itself is a redundant update in the middle of the state, discarding redundant events with too close an interval will not change the final position trend, and the user will not perceive any loss of interaction accuracy.

[0088] In this application, synchronous transmission means that second priority events are sent sequentially in an orderly manner, and the transmission of second priority events is limited by the historical transmission sequence. They need to be sent in an orderly manner according to the preset time interval rules, and there are interval rate limiting and timeout discard logic. Instant preemptive transmission is not supported.

[0089] The first controller 251 is also configured to perform the following steps: S905. After receiving an input event through the internal network transmission channel, the system distributes or responds to the target application corresponding to the input event based on its deployment location. The target application is deployed on either the first controller 251 or the second controller 252.

[0090] The network input event receiving thread of the first controller 251 (receiving controller) receives NotifyArgs events from the second controller 252 through an internal network transmission channel (such as a socket-based HSPcore component). For example, the received events include first-priority events (transmitted asynchronously) and second-priority events (transmitted synchronously) processed by the sending end processing module. The event data structure contains complete information such as event type identifier, timestamp, coordinates / key value, and action status. After the event arrives at the first controller 251, it first enters the receiving end processing module, rather than being directly stored in the distribution queue.

[0091] The first controller 251 detects the target application affected by the input event. If the target application runs on the first controller 251 itself, the first controller 251 stores the input event in a dispatch queue for processing; or, if the input event is a global event requiring a response from the first controller 251, it stores the input event in the dispatch queue for processing; otherwise, the input event is discarded. For example, if the target application is deployed on the first controller 251, the first controller 251 directly dispatches dispatchable events to the target application based on the first dispatch queue, and the target application's View tree processes the events and responds to interactive commands.

[0092] In this embodiment of the disclosure, by prioritizing and classifying the transmission, the first priority state switching events are sent asynchronously and in real time to avoid being blocked by low-priority redundant movement events, thus ensuring the real-time performance of core interactive events. At the same time, by dynamically reducing the frequency of second priority events and discarding redundant data, the amount of data transmitted across chips can be reduced without affecting the user's perception, thereby reducing network congestion and improving the problems of event accumulation and high-priority operation response delay during synchronous transmission.

[0093] In a scenario where the second controller 252 acts as the transmitter and the first controller 251 acts as the receiver, such as Figure 10 As shown, in some embodiments, the second controller 252 also maintains a transmission count for second priority events (S1001). Each time a second priority event is transmitted, the transmission count is incremented by a preset amount (S1002).

[0094] That is, the second controller 252 maintains a send count variable mSendCount for second-priority events in memory, with an initial value of 0. After each successful send of a second-priority event, the send count is incremented by a preset amount (e.g., by 1). At the same time, the sending controller maintains a historical send time variable mlasendTime to record the timestamp of the most recent successful send of a second-priority event.

[0095] When the second controller 252 detects a target second priority event, it reads the transmission count value (S1003). If the count value is the first value, it transmits the target second priority event and records the transmission time of the target second priority event as the historical transmission time (S1004). If the count value is the second value, it transmits or discards the target second priority event according to the preset synchronization transmission rules (S1005). Here, the first value (e.g., 0) indicates that no second priority event has been transmitted, and the second value (e.g., an integer greater than 0) indicates that at least one second priority event has been transmitted.

[0096] In other words, the first second-priority event is sent directly. When the second controller detects a second-priority event and the current send count mSendCount is 0 (indicating that no second-priority event has been sent in this interaction cycle), it sends this second-priority event directly, records the current system timestamp as mlestSendTime, and increments mSendCount to 1. This ensures that the first second-priority event of each interaction cycle (such as the first Move event when the cursor starts moving) can be transmitted in a timely manner, avoiding initial delays when the user starts moving the cursor or touching, and guaranteeing an instant response experience.

[0097] For non-first-time second-priority events, interval-based rate limiting is applied. When the second controller detects a second-priority event and the current send count mSendCount is not 0 (indicating that at least one second-priority event has been sent in this interaction cycle), the difference between the current event timestamp and mLestSendTime (the send time corresponding to the historical second-priority event) is calculated as the send interval. If the send interval is greater than a preset interval (e.g., 12ms), the second-priority event is allowed to be sent, mLestSendTime is updated to the current timestamp, and mSendCount is incremented; if the send interval is less than or equal to the preset interval, the second-priority event is discarded. In this way, the sending frequency of second-priority events is reduced, achieving effective frequency reduction.

[0098] When the second controller 252 detects a first-priority event, it resets the transmission count to the first value and clears the historical transmission time. That is, it resets the frequency reduction state when the first-priority event is triggered. When the second controller 252 detects a first-priority event at any time, it resets mSendCount to 0 and clears mlestSendTime. Since the occurrence of a first-priority event (such as a button press or touch release) signifies a fundamental change in the current interaction state, the previously accumulated transmission history and count of second-priority events are no longer valid. After the reset, the second-priority events in the next interaction cycle start counting and frequency reduction from the new state, ensuring independence between different interaction cycles.

[0099] In some embodiments, the preset time interval can be flexibly configured according to the hardware performance of the display device 200, network transmission conditions, and user interaction scenarios. For example, a shorter preset time interval (e.g., 5ms) can be set in game scenarios with low latency requirements, while a longer preset time interval (e.g., 16ms, corresponding to a refresh rate of approximately 60Hz) can be set in ordinary interaction scenarios. It can also be adaptively and dynamically adjusted according to the real-time load of the internal network transmission channel.

[0100] In some embodiments, the first controller 251 further performs a consistency check on the received touch events, converts the consistency-checked touch events into distributable events, and adds the distributable events to a first distribution queue. If the target application is deployed on the first controller, the distributable events are distributed to the target application based on the first distribution queue to respond to interactive commands.

[0101] In other words, after receiving an input event, the first controller 251 determines the event type and classifies it. For key events, it skips the timestamp verification and state machine management stages, directly intercepts and judges the event, converts it into the corresponding EventEntry event (such as KeyEntry), and enqueues it into the first dispatch queue (InboundQueue) to await subsequent dispatch by the InputDispatcher. If it is a touch event, it enters the timestamp correction module and state machine management module for consistency verification and interception judgment.

[0102] For interception judgment, if the target application is deployed in the first controller 251, the verified touch event needs to be converted into a standard EventEntry (such as MotionEntry) and then enqueued into the first distribution queue (InboundQueue) to wait for subsequent InputDispatcher distribution.

[0103] If the target application is deployed on the second controller 252, the second controller 252 will also convert the touch event into a distributable event and add the distributable event to the second distribution queue. Based on the second distribution queue, the distributable event is distributed to the target application in response to the interaction command.

[0104] In other words, if the target application itself is deployed on the second controller 252, the converted EventEntry can be directly enqueued into the local distribution queue of the second controller for processing. Consistency verification is mainly used to correct timestamp offsets caused by cross-chip transmission, adjust the temporal continuity of touch event sequences, and avoid touch event sequence disorder and inconsistent action states caused by transmission delay fluctuations, thus maintaining correct response interactions.

[0105] In some embodiments, touch events include state start events (such as ACTION_DOWN or HOVER_ENTER) and corresponding state end events (such as ACTION_UP or HOVER_EXIT). The state start event is used to start the target state, and the state end event is used to end the target state. For example, the target state corresponding to ACTION_DOWN is down_state, and the target state corresponding to HOVER_ENTER is hover_enter_state.

[0106] In some embodiments, the second controller 252 maintains an interactive state machine in memory, including the following state variables: down_state (boolean, true indicates the current state is down, false indicates the current state is not down) and hover_enter_state (boolean, true indicates the current state is hover, false indicates the current state is not hover). Simultaneously, the state machine maintains the latest occurrence time variables mLestDownTime (timestamp of the most recent down event) and mLestHoverEnterTime (timestamp of the most recent hover event) for subsequent consistency comparison of intermediate event timestamps.

[0107] When the first controller 251 receives a state start event, in response to receiving the state start event, it records the timestamp of the state start event as the latest occurrence time of the target state. If the target state in the system is open, a state end event is generated, and the target state is ended; the state end event is converted into a first distributable event and added to the first distribution queue. Then, a state start event is generated again, and the target state is opened. Then, the state start event is converted into a second distributable event and added to the first distribution queue.

[0108] That is, the timestamp carried by the state start event is recorded as the latest occurrence time of the target state (mLastDownTime for the press state, and mLastHoverEnterTime for the hover enter state). Then, the current state of the target state in the system is detected: if the target state in the system is currently on (i.e., the corresponding down_state or hover_enter_state is true), it means that the previous state end event may have been lost due to network transmission anomalies (such as timeout or packet loss). At this time, the first controller 251 automatically generates a corresponding state end event (such as a lift event or hover exit event) to forcibly close the old interaction state, converts the state end event into a first distributable event (such as MotionEntry) and adds it to the first distribution queue; then, the newly received state start event is generated, the target state is set to on, the state start event is converted into a second distributable event and added to the first distribution queue.

[0109] In some embodiments, if the target state in the system is "End", a "State Start" event is generated, and the target state is enabled. The "State Start" event is converted into a third distributable event and added to the first distribution queue. That is, if the target state in the system is currently "End" (i.e., the corresponding "down_state" or "hover_enter_state" is false), a current "State Start" event is directly generated, the target state is set to "On", the "State Start" event is converted into a third distributable event and added to the first distribution queue.

[0110] In some embodiments, when the first controller 251 receives a state end event (such as ACTION_UP, HOVER_EXIT, ACTION_CANCEL), it sets the target state corresponding to the state end event to end (e.g., setting down_state or hover_enter_state to false), and then converts the state end event into a dispatchable event and adds it to the first dispatch queue. This ensures the correct reset of the state machine, preparing it for receiving subsequent state start events.

[0111] In some embodiments, the touch event further includes a state intermediate event (second priority event), which is a move event (ACTION_MOVE) or a hover move event (HOVER_MOVE) used to maintain the currently active target state. When the first controller 251 receives a state intermediate event, it detects the timestamp carried by the state intermediate event. Since the first priority event is transmitted asynchronously and may be sent in queue, a lower priority Move event that occurs earlier may arrive at the receiving end later than a first priority Down event that occurs later. That is, the Move event received by the receiving end after the Down event is actually an old event that occurred before the Down event. If it is not filtered, direct distribution will cause the cursor position to jump and the touch trajectory to be disordered. Therefore, if the timestamp of the state intermediate event is earlier than the latest occurrence time of the target state (mLastDownTime or mLastHoverEnterTime), it means that this state intermediate event is an outdated event and is directly discarded without distribution; if the timestamp of the state intermediate event is later than or equal to the latest occurrence time of the target state, the state intermediate event is a valid event, which is converted into a fourth distributable event and added to the first distribution queue.

[0112] That is, if the timestamp of the intermediate state event is earlier than the latest occurrence time of the target state, the intermediate state event is discarded; if the timestamp of the intermediate state event is later than the latest occurrence time of the target state, the intermediate state event is converted into a fourth distributable event and added to the first distribution queue.

[0113] For example, when the first controller 251 receives an ACTION_DOWN, it first records the timestamp carried by the ACTION_DOWN as the latest occurrence time of the down_state. If it is a press event, it checks the down_state. If the down_state is true, it indicates that the system is already in an interactive state that has not been properly closed, and the ACTION_UP may have been lost due to network transmission timeouts, packet loss, or other anomalies. At this time, the first controller 251 first generates a corresponding ACTION_UP event, converts it into a distributable event, and adds it to the first distribution queue to forcibly end the current abnormal interactive state and restore the system UI to normal display; then it resets the down_state to false. After forcibly closing the old state, the first controller 251 generates the currently received ACTION_DOWN, sets the down_state to true, converts the ACTION_DOWN into a distributable event, and adds it to the first distribution queue. When the first controller 251 receives an ACTION_UP again, it sets the down_state to false, converts the ACTION_UP into a distributable event, and adds it to the first distribution queue.

[0114] In some embodiments, when the display device 200 includes three or more controllers, each controller acting as a receiver independently maintains its own interaction state machine. In scenarios where different controllers carry different target applications, the state machine of each receiver is responsible for ensuring the integrity of its own interaction state.

[0115] In some embodiments, since the input event needs to be sent from the sending controller to the receiving controller, the display device 200 can also create a timestamp for the input event through the receiving controller.

[0116] Based on the above embodiments, Figure 11 This is a schematic diagram illustrating another complete event handling process provided by some embodiments of this application. In some embodiments, when the first controller 251 acts as the sending end and the second controller 252 acts as the receiving end, the first controller 251 is configured with a user input interface, which is configured to receive interactive commands input by the user. Figure 11 As shown, the first controller 251 is configured to perform the following steps: S1101. Convert the interactive command into a corresponding input event. The input event is at least one of a touch event or a button event.

[0117] S1102. Based on the mapping rule between the event type and priority of the input events, the first type of input event is determined as the first priority event, and the second type of input event is determined as the second priority event. The first priority event has a higher priority than the second priority event, and the real-time response requirement for the first priority event is higher than that for the second priority event.

[0118] S1103. For the first priority event, send it to the second controller in an asynchronous manner through the internal network transmission channel.

[0119] S1104. For the second priority event, it is sent to the second controller through the internal network transmission channel in a synchronous transmission mode.

[0120] The second controller 252 is also configured to perform the following steps: S1105. After receiving an input event through the internal network transmission channel, the system distributes or responds to the target application corresponding to the input event based on its deployment location. The target application is deployed on either the first controller or the second controller.

[0121] It is understood that in the embodiments disclosed herein, when the first controller 251 acts as the transmitting end, the embodiments corresponding to when the second controller 252 acts as the transmitting end can be executed; when the second controller 252 acts as the receiving end, the embodiments corresponding to when the first controller 251 acts as the receiving end can be executed, and this application will not elaborate further here.

[0122] Based on the above embodiments, Figure 8 The priority marking module 801 is used to mark the priority of input events according to their event type, marking input events with high real-time response requirements as first priority and touch events with low real-time response requirements as low priority. The dynamic frequency reduction module 802 is used to count and detect the transmission interval for second-priority events before transmission, filtering for packet loss of second-priority events that do not exceed a preset time interval to prevent a large number of second-priority events from consuming excessive transmission bandwidth in a short period. The asynchronous transmission module 803 is used to insert first-priority events into the asynchronous transmission queue and send them to the receiving controller preferentially through the internal network transmission channel; the synchronous transmission module 804 is used to put the second-priority events, after dynamic frequency reduction processing, into the synchronous transmission queue and send them to the receiving controller according to the transmission scheduling order.

[0123] The timestamp correction module 805 is used to correct the timestamps of input events transmitted across controllers, compensate for the time offset generated during transmission, filter out outdated intermediate events, and avoid touch trajectory errors; the state machine management module 806 is used to maintain the consistency of the interaction state and avoid abnormal interaction states by completing abnormally lost state end events.

[0124] Based on the above multi-chip architecture, some embodiments of this application also provide an input event processing method, which can be applied to the display device 200 described in the above embodiments. The display device 200 includes a transmitting end controller (a first controller 251 or a second controller 252) and a receiving end controller (corresponding to a second controller 252 or a first controller 251). The transmitting end controller and the receiving end controller establish a communication connection through an internal network transmission channel.

[0125] like Figure 12 As shown, the input event handling method may include the following steps: S1201. Receive user input interaction commands and convert the interaction commands into corresponding input events. The input event is at least one of a touch event or a button event.

[0126] In some embodiments, the InputReader thread running on the sending controller continuously collects raw events from the user input interface (device nodes under the / dev / input directory), processes them through EventHub and InputMapper, converts them into standard NotifyKeyArgs or NotifyMotionArgs events, and stores them in NotifyArgsQueue, completing the first step of the conversion from interactive commands to standardized input events. The sending controller can be either the first controller 251 or the second controller 252.

[0127] S1202. Based on the mapping rule between the event type and priority of the input events, the first type of input event is determined as the first priority event, and the second type of input event is determined as the second priority event. The real-time response requirement for the first priority event is higher than that for the second priority event.

[0128] S1203. For the first priority event, it is sent to the receiving controller through the internal network transmission channel in an asynchronous transmission mode.

[0129] S1204. For second priority events, transmit them to the receiving controller via the internal network transmission channel in a synchronous transmission mode.

[0130] Figure 13 This is a timing interaction diagram of the input event processing methods provided in some embodiments of this application. For example... Figure 13 As shown, in some embodiments, the display device 200 adopts a dual-chip hardware architecture. The first controller 251 is the main control chip and is electrically connected to the display 260. The second controller is the image quality chip. The two establish an internal network transmission channel through the socket-based hspcore component. The user input interface 280 of the second controller 252 establishes a wireless communication connection with the Bluetooth mouse. The target application is deployed on the first controller 251.

[0131] The interactive commands (raw electrical signals) generated by the user's operation of the Bluetooth mouse are first transmitted to the second controller 252. After identifying the event type of the input signal, the second controller 252 determines the priority: first-priority events such as hover entry, button press, and button release are sent to the internal network immediately through an asynchronous transmission channel; second-priority events such as cursor movement are sent through a synchronous transmission channel at preset time intervals with rate limiting (dynamic frequency reduction), discarding redundant events with too close intervals; when the first-priority input event is transmitted, the rate limiting count of low-priority transmission events is reset simultaneously. After the input event is transmitted to the first controller 251 through the internal network, the first controller 251 performs timing verification on the received input event, discarding outdated second-priority events whose timestamps are earlier than the latest state change, and automatically completing the state end event lost due to transmission anomalies, achieving integrity verification; finally, the first controller 251 distributes the verified events to the target application for response processing, such as driving the cursor to complete the corresponding movement, click, and hover interaction operations, ensuring the consistency and smoothness of user operation and interface display.

[0132] This embodiment employs a hybrid transmission method combining asynchronous and synchronous approaches for input events of different priorities. This ensures that high-priority events are transmitted first, meeting their real-time response requirements, while simultaneously reducing unnecessary bandwidth consumption by second-priority events through dynamic frequency reduction filtering. This balances the real-time performance and bandwidth resource consumption of cross-chip transmission, preventing a large number of second-priority events from blocking the transmission channel. This solves the response latency problem when transmitting input events across chips in a multi-chip architecture, improving the user's interactive experience. Furthermore, by maintaining a state machine at the receiving end and performing consistency checks and filtering for outdated events, it addresses potential state inconsistencies and trajectory deviations that may occur during cross-chip transmission, further ensuring the reliability of the interaction.

[0133] Similar parts between the embodiments provided in this application can be referred to mutually. The specific implementation methods provided above are only a few examples under the overall concept of this application and do not constitute a limitation on the scope of protection of this application. For those skilled in the art, any other implementation methods extended from the solution of this application without creative effort shall fall within the scope of protection of this application.

Claims

1. A display device, characterized in that, include: monitor; First controller; The second controller is electrically connected to the display and establishes a communication connection with the first controller through an internal network transmission channel; the second controller is also configured with a user input interface, which is configured to receive interactive commands input by the user; The second controller is also configured to: The interactive command is converted into a corresponding input event, wherein the input event is at least one of a touch event or a key event; According to the mapping rule between the event type and priority of the input events, the first type of input event is determined as the first priority event, and the second type of input event is determined as the second priority event; the priority of the first priority event is higher than the priority of the second priority event, and the real-time response requirement of the first priority event is higher than the real-time response requirement of the second priority event. For the first priority event, it is sent to the first controller via the internal network transmission channel in an asynchronous manner; For the second priority event, it is transmitted to the first controller via the internal network transmission channel in a synchronous transmission manner; The first controller is configured as follows: After receiving the input event through the internal network transmission channel, the system distributes or responds to the target application corresponding to the input event based on its deployment location, wherein the target application is deployed on the first controller or the second controller.

2. The display device according to claim 1, characterized in that, The first type of input event includes at least the key press event and key release event in the key events, and the touch events including the press event, release event, hover enter event, hover exit event, and operation cancel event; The second type of input event includes at least the movement event and hover movement event in the touch event.

3. The display device according to claim 1, characterized in that, The second controller is also configured to: Maintain the transmission count of the second priority events; When a second priority event is sent, the send count value is incremented by a preset amount; The second controller executes synchronous transmission by sending the second priority event to the first controller through the internal network transmission channel, specifically configured as follows: Upon detecting a target second-priority event, read the count value of the transmission count; If the count value is the first value, then the target second priority event is sent, and the sending time of the target second priority event is recorded as the historical sending time; If the count value is the second value, then the target second priority event is sent or discarded according to the preset synchronization transmission rules; wherein, the first value indicates that the second priority event has not been sent, and the second value indicates that at least one second priority event has been sent; Before the second controller executes asynchronous transmission to send the first priority event to the first controller via the internal network transmission channel, it is further configured to: Upon detecting the first priority event, the transmission count is reset to the first value, and the historical transmission time is cleared.

4. The display device according to claim 3, characterized in that, The second controller executes the sending or discarding of the target second priority event according to a preset synchronization transmission rule, specifically configured as follows: Read the historical transmission time, which is the transmission time corresponding to the previous historical second priority event; Based on the historical transmission time, calculate the transmission time interval between the target second priority event and the historical second priority event; If the sending time interval is greater than the preset time interval, then the target second priority event is sent, and the historical sending time is updated according to the sending time of the target second priority event; If the transmission time interval is less than or equal to the preset time interval, the target second priority event is discarded.

5. The display device according to claim 1, characterized in that, The first controller performs distribution or response based on the deployment location of the target application corresponding to the input event, specifically configured as follows: Perform a consistency check on the touch event, and convert the consistency-checked touch event into a distributable event; Add the distributable event to the first distribution queue; If the target application is deployed on the first controller, the distributable event is distributed to the target application based on the first distribution queue in response to the interaction command; If the target application is deployed on the second controller, the second controller is further configured as follows: Convert the touch event into a dispatchable event; Add the distributable event to the second distribution queue; Based on the second distribution queue, the distributable event is distributed to the target application in response to the interaction command.

6. The display device according to claim 5, characterized in that, The touch event includes a state start event and a corresponding state end event. The state start event is a press event or a hover enter event in the touch event. The state end event corresponding to the press event is a release event in the touch event. The state end event corresponding to the hover enter event is a hover exit event. The state start event is used to start the target state, and the state end event is used to end the target state. The first controller performs a consistency check on the touch event, specifically configured as follows: In response to receiving the state start event, the timestamp of the state start event is recorded as the latest occurrence time of the target state; If the target state in the system is enabled, then an end event for the state is generated, and the target state is terminated. The state end event is converted into a first distributable event, and the first distributable event is added to the first distribution queue; Generate the state start event and activate the target state; The state start event is converted into a second distributable event, and the second distributable event is added to the first distribution queue.

7. The display device according to claim 6, characterized in that, The first controller is also configured to: If the target state in the system is "end", then generate the "start state" event and start the target state. The state start event is converted into a third distributable event, and the third distributable event is added to the first distribution queue.

8. The display device according to claim 6, characterized in that, The touch event further includes a state intermediate event, which is a second priority event, and the state intermediate event is used to maintain the target state; the first controller performs a consistency check on the touch event, specifically configured as follows: In response to receiving the intermediate state event, the timestamp of the intermediate state event is detected; If the timestamp of the intermediate state event is earlier than the latest occurrence time of the target state, then the intermediate state event is discarded. If the timestamp of the intermediate state event is later than the latest occurrence time of the target state, the intermediate state event is converted into a fourth distributable event and the fourth distributable event is added to the first distribution queue.

9. A display device, characterized in that, include: monitor; The first controller is configured with a user input interface, which is configured to receive interactive commands input by the user. The second controller is electrically connected to the display and establishes a communication connection with the first controller through an internal network transmission channel; The first controller is also configured to: The interactive command is converted into a corresponding input event, wherein the input event is at least one of a touch event or a key event; According to the mapping rule between the event type and priority of the input events, the first type of input event is determined as the first priority event, and the second type of input event is determined as the second priority event; the priority of the first priority event is higher than the priority of the second priority event, and the real-time response requirement of the first priority event is higher than the real-time response requirement of the second priority event. For the first priority event, it is sent to the second controller via the internal network transmission channel in an asynchronous manner; For the second priority event, it is transmitted to the second controller via the internal network transmission channel in a synchronous transmission manner; The second controller is configured as follows: After receiving the input event through the internal network transmission channel, the system distributes or responds to the target application corresponding to the input event based on its deployment location, wherein the target application is deployed on the first controller or the second controller.

10. A method for processing input events, characterized in that, include: Receive user input interaction commands and convert the interaction commands into corresponding input events; The input event is at least one of a touch event or a key event; According to the mapping rule between the event type and priority of the input events, the first type of input event is determined as the first priority event, and the second type of input event is determined as the second priority event; the priority of the first priority event is higher than the priority of the second priority event, and the real-time response requirement of the first priority event is higher than the real-time response requirement of the second priority event. For the first priority event, it is sent to the receiving controller via the internal network transmission channel in an asynchronous transmission manner; For the second priority event, it is transmitted to the receiving controller via the internal network transmission channel in a synchronous transmission mode.

Images