Display device, processing method of display device, and event processing unit

Abstract

This application relates to a display device, a processing method for the display device, and an event processing unit. The display device includes a first processor and a second processor. The first processor is connected to a first external device and directly connected to a display, and deploys a first application and a second application. The second processor is connected to a second external device and deploys a third application. The second processor is configured to: acquire a second input event from the second external device, forward the second input event to the first processor, and if the window application currently capable of responding to the input event is the third application, send the second input event to the third application. The first processor is configured to receive the second input event forwarded by the second processor, and if the window application currently capable of responding to the input event is the third application, intercept the second input event; otherwise, send the second input event to the first application. This application enables adaptive, low-latency, and reliable transmission of input events between the two processors of the display device.

Description

Technical Field

[0001] This application relates to the field of display technology, and in particular to a display device, a method for processing the display device, and an event processing unit. Background Technology

[0002] With the rapid development of smart TVs and smart screens towards multimodal interaction, cross-device collaboration, and high-performance audio-visual experiences, single-processor architectures can no longer simultaneously meet the demands of high-quality image processing, complex application ecosystems, and low-latency interaction. Therefore, dual-processor display devices have emerged. These devices have two processors: one primarily handles image processing, while the other enables high-performance operation and provides a smooth interactive experience. For example, some smart screen products in this field include a main processor and a slave processor. The main processor handles image processing, while the slave processor enables high-performance operation and provides a smooth, smartphone-like interactive experience. However, in practical applications, some applications, such as live TV, signal source management, and channel switching, still need to run on the main processor handling image processing. When both main processors have interactive scenarios and are connected to external devices, achieving input event coordination between them becomes a key technical challenge for improving user experience. Therefore, there is a need for low-latency input event coordination between the two processors in dual-processor display devices. Summary of the Invention

[0003] This application provides a display device, a processing method for the display device, and an event processing unit to achieve low-latency coordination of input events between the two processors of a display device with dual processors.

[0004] In a first aspect, some embodiments provide a display device, which includes a first processor and a second processor, wherein the first processor is connected to a first external device, and the second processor is connected to a second external device.

[0005] The first processor is directly connected to the display and has a first application and a second application that can display the display interface content of the third application. It can send information of the window application that is currently responsive to input events to the second processor.

[0006] The second processor is connected to the first processor through a data transmission interface and has a third application deployed on it. It can send the display interface content of the third application to the second application for display.

[0007] The second processor is configured as follows:

[0008] Acquire the second input event from the second external device and forward the second input event to the first processor;

[0009] Based on the information of the window application, determine the window application that can currently respond to input events, and if the window application that can currently respond to input events is a third application, send the second input event to the third application;

[0010] The first processor is configured as follows:

[0011] Receive the second input event forwarded by the second processor;

[0012] Based on the information of the window application, determine the window application that can currently respond to input events. If the window application that can currently respond to input events is a third application, intercept the second input event. If the window application that can currently respond to input events is not a third application, send the second input event to the first application.

[0013] The display device based on the above embodiments includes a display, a first processor, and a second processor. The first processor is directly connected to the display and has a first application and a second application deployed on it. The second application displays the display interface content of a third application of the second processor. The second processor is connected to the first processor through a data transmission interface and has a third application deployed on it. The first processor can synchronize information about the window application that can currently respond to input events to the second processor. After obtaining a second input event through a second external device, the second processor can determine whether the target window application of the second input event is the third application based on the information about the window application that can currently respond to input events. If the window application that can currently respond to input events is the third application, the second input event is sent to the third application so that the third application can perform corresponding application processing based on the second input event. Simultaneously, the second input event is forwarded to the first processor. The first processor then determines, based on information about the window application that can currently respond to the input event, whether to intercept the second input event or send it to the first application. Therefore, the second input event from the second external device received by the second processor will be forwarded to the first processor at the same time as the second processor analyzes whether it needs to be sent to the third application. The first processor will then analyze whether it needs to be intercepted or sent to the first application. As a result, the second input event received by the second processor through the second external device can be promptly transmitted to the first processor. Adaptive event distribution processing can be performed in both the first and second processors, which helps to ensure adaptive low-latency and reliable transmission of the input event between the second processor and the first processor when the second processor receives the second input event through the second external device.

[0014] In some alternative embodiments, the first application includes system-level applications and non-system-level applications;

[0015] A first processor is configured to: determine the event type of a second input event; if the event type of the second input event is a non-global event and the window application currently responding to the input event is not a third application, send the second input event to the non-system-level application currently responding to the input event of the first processor; if the event type of the second input event is a global event, send the second input event to the system-level application of the first processor.

[0016] In some optional embodiments, the second processor is further configured to: determine whether a third input event paired with the first input event is received before the first input event is received; and if the third input event is not received, process the first input event based on the priority of the first input event.

[0017] In some alternative embodiments, the second processor is further configured to:

[0018] Before receiving the first input event, determine whether a third input event paired with the first input event is received by the first processor. If the third input event is not received, simulate the generation of the third input event and send the third input event and the first input event to the third application in sequence.

[0019] In some alternative embodiments, the third application includes system-level applications and non-system-level applications;

[0020] The first processor is also configured to forward the first input event to the second processor if the first input event is a global event;

[0021] The second processor is further configured to: determine the event type of the first input event forwarded by the first processor; if the event type of the first input event is a non-global event, send the first input event to a non-system-level application of the second processor that is currently capable of responding to the input event; if the event type of the first input event is a global event, send the first input event to a system-level application of the second processor.

[0022] Secondly, some embodiments provide a processing method for a display device. The display device includes a display, a first processor, and a second processor. The first processor is connected to a first external device, and the second processor is connected to a second external device. The first processor is directly connected to the display and has deployed a first application and a second application capable of displaying the display interface content of a third application. The method can send information about a window application currently responsive to input events to the second processor. The second processor is connected to the first processor via a data transmission interface and has deployed a third application. The method can send the display interface content of the third application to the second application for display. The method is executed by the first processor and includes:

[0023] Obtain information about the window application that is currently responsive to input events, and send the window application information to the second processor;

[0024] Receive the second input event forwarded by the second processor;

[0025] Determine the window application that is currently responsive to input events based on information from the window application;

[0026] If the current window application that can respond to input events is a third application, intercept the second input event;

[0027] If the first application is the window currently capable of responding to input events, the second input event is sent to the first application.

[0028] In some alternative embodiments, the first application includes system-level applications and non-system-level applications;

[0029] If the first application is the window currently capable of responding to input events, the second input event is sent to the first application, including:

[0030] If the window currently capable of responding to input events is the first application, determine the event type of the second input event;

[0031] If the second input event is a non-global event and the window application that can currently respond to the input event is not the window of the third application, the second input event is sent to the non-system application that can currently respond to the input event.

[0032] If the event type of the second input event is a global event, the second input event is sent to the system-level application.

[0033] In some optional embodiments, the method further includes:

[0034] Obtain the first input event from the first external device;

[0035] If the window application that can currently respond to input events is a third application, the first input event is forwarded to the second processor;

[0036] If the first application is the window that can currently respond to input events, the first input event is sent to the first application.

[0037] Thirdly, some embodiments provide a processing method for a display device, the display device including a display, a first processor, and a second processor. The first processor is connected to a first external device, the second processor is connected to a second external device, the first processor is directly connected to the display, and a first application and a second application capable of displaying the display interface content of a third application are deployed thereon. The method can send information about a window application that is currently responsive to input events to the second processor. The second processor is connected to the first processor through a data transmission interface, and a third application is deployed thereon. The method can send the display interface content of the third application to the second application for display. The method is executed by the second processor and includes:

[0038] Obtain information about the currently responsive window application sent by the first processor;

[0039] Obtain the second input event from the second external device and forward the second input event to the first processor;

[0040] Determine the window application that is currently responsive to input events based on information from the window application;

[0041] If the window currently capable of responding to input events is a third application, then the second input event is sent to the third application.

[0042] In some alternative embodiments, the method further includes: receiving a first input event forwarded by a first processor and sending the first input event to a third application.

[0043] In some optional embodiments, the method further includes: processing the first input event based on the priority of the first input event if no third input event paired with the first input event is received before the first input event is received by the first processor.

[0044] In some optional embodiments, the method further includes: if a third input event paired with the first input event is not received before the first input event is received, the method simulates generating a third input event and sequentially sends the third input event and the first input event to a third application.

[0045] In some optional embodiments, the third application includes system-level applications and non-system-level applications; sending the first input event to the third application includes:

[0046] Determine the event type of the first input event;

[0047] If the event type of the first input event is a non-global event, the first input event is sent to the non-system-level application of the second processor that is currently responsive to input events;

[0048] If the event type of the first input event is a global event, the first input event is sent to the system-level application of the second processor.

[0049] In some optional embodiments, the method further includes: receiving a first input event forwarded by a first processor; determining a window application that is currently responsive to the input event based on information about the window application; sending the first input event to a third application if the window application that is currently responsive to the input event is a third application; and intercepting the first input event if the window application that is currently responsive to the input event is not a third application.

[0050] In some optional embodiments, the method further includes: if the second input event is a cursor event, forwarding the second input event to a first processor to instruct the first processor to output the cursor event to a first target graphics output plane, and drawing the cursor through the first target graphics output plane.

[0051] Fourthly, some embodiments provide an event processing unit, which includes: a window management module, an event collection module, and a two-end communication module;

[0052] The window management module is configured to obtain information about the currently responsive window applications on the device where the event handling unit is located;

[0053] The event collection module is configured as follows:

[0054] Receive input events from external devices connected to the processor where the event processing unit is located;

[0055] Determine the processor category of the processor in which the event handling unit is located;

[0056] When the processor category is the primary processor, if the information of the window application determines that the window application that can currently respond to input events is an application deployed on the secondary processor, the input event is forwarded to the secondary processor connected to the two-end communication modules through the two-end communication modules; otherwise, the input event is sent to the application deployed on the processor where it is located.

[0057] When the processor type is a slave processor, the input event is forwarded to the master processor connected to the two communication modules through the two-end communication modules. If it is determined from the information of the window application that the window application that can respond to the input event is the application deployed on the slave processor, the input event is sent to the application deployed on the processor. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is a schematic diagram of the frame structure of a display device provided in some embodiments of this application;

[0060] Figure 2 A schematic diagram of the framework structure of an event processing unit provided in some embodiments of this application;

[0061] Figure 3 This is a schematic diagram of the architecture related to display devices and input event processing in related technologies;

[0062] Figure 4 This is a schematic diagram of the display device and its architecture related to the processing of input events in some embodiments of this application;

[0063] Figure 5 For an example based on Figure 4 The diagram shows the architecture of the display device and the interactive flow diagram of the display device in response to the first input event input from the first external device.

[0064] Figure 6 This is a schematic diagram illustrating the interaction flow of a device processing a first input event in response to a second external device, as shown in an example.

[0065] Figure 7 This is a schematic diagram of the display device and its architecture related to the processing of input events in other embodiments of this application;

[0066] Figure 8 For an example based on Figure 7 The diagram shows the architecture of the display device and the interactive flow diagram of the display device in response to the first input event input from the first external device.

[0067] Figure 9 This is a schematic diagram illustrating the process principle of the first processor of the display device in some embodiments of this application processing output events;

[0068] Figure 10 This is a schematic diagram illustrating the process principle of the second processor of the display device in some embodiments of this application processing output events;

[0069] Figure 11 A schematic diagram illustrating the underlying principle framework of the cursor in a display device in some embodiments of this application is provided.

[0070] Figure 12 This is a flowchart illustrating the processing method of a display device in some embodiments of this application;

[0071] Figure 13 This is a flowchart illustrating the processing method of a display device in some other embodiments of this application. Detailed Implementation

[0072] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application as detailed in the claims.

[0073] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0074] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0075] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0076] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0077] In this application, "display device" refers to any device capable of displaying images and processing data. For example, display devices include, but are not limited to, smart TVs, mobile terminals, computers, monitors, advertising screens, wearable devices, virtual reality devices, and augmented reality devices. In the following related embodiments of this application, a smart TV is used as an example for illustration.

[0078] In display devices with dual processors, there is typically a first processor and a second processor. The first processor runs a first operating system, and the second processor runs a second operating system. The first processor primarily handles image processing, while the second processor provides high-performance processing, enabling a high-performance system and a smooth interactive experience. However, in some scenarios, such as when both the first and second processors are involved in interaction, and when both are connected to external devices, achieving efficient, transparent, and low-latency collaboration of input events between the two independent systems becomes a key technical challenge for improving user experience. These challenges manifest in the following ways:

[0079] Input event routing and synchronization: The first and second processors may be connected to different external devices or simultaneously display windows on two different operating systems. Input events need to be passed between the two processors on demand, and the real-time performance and timing consistency of input event transmission must be guaranteed. For example, if the foreground application of the currently displayed interface is an application installed on the second processor, then input events received by the first processor (such as input events pointing to the remote control) need to be forwarded to the second processor. If the second processor of the display device is connected to a portable external device, and the focus window switches to the window of the first processor, then the input events need to be switched back to the first processor.

[0080] Unified management of global input strategies: In a dual-processor architecture, the capture, decision-making and execution of global input events (such as swiping back across all interfaces, pulling up to return to the home page, etc.) require the collaboration of two processors. They are uniformly managed and distributed by a logical center (such as the first processor) to avoid response conflicts or functional failures caused by system fragmentation.

[0081] Cross-processor cursor rendering: Dual-processor display devices typically display various types of external cursors, such as mice, pointing remotes, and touchpads. However, these external devices may be connected to different processors. Therefore, it is necessary to reduce latency and ensure responsiveness and smoothness, while also ensuring a consistent user experience.

[0082] Research has shown that for display devices with two processors, both of which can connect to external devices, the input events received by the first and second processors can be adaptively distributed by the two processors based on information about the windows that can currently respond to the input events. This improves the adaptive, low-latency, and reliable transmission of input events between the two processors.

[0083] Accordingly, this application provides a display device having two processors. When each of the two processors is connected to an external device, the device can adaptively transmit and distribute input events received by the first processor and the second processor respectively, thereby improving adaptive low-latency and reliable transmission between the two processors.

[0084] Some embodiments of the present application provide display devices, with reference to... Figure 1 As shown, the system includes: a display 101, a first processor 102, and a second processor 103. The first processor 102 is connected to a first external device 104, and the second processor 103 is connected to a second external device 105. The first processor 102 is directly connected to the display 101 and deploys a first application 1021 and a second application 1022 that can display the interface content of a third application 1031. It can send information about the currently responsive window application to the second processor 103. The second processor 103 transmits data via a data transmission interface (…). Figure 1 (As shown in the middle) It is connected to the first processor 102 and has a third application 1031 deployed thereon. The display interface content of the third application 1031 can be sent to the second application 1022 for display.

[0085] In some embodiments, the display 101 includes display function components for presenting an image and driving components for driving the image display. The display 101 is used to receive and display image signals output from the first processor 102. For example, the display 101 can be used to display video content, image content, menu control interface components, and user control UI interfaces, etc.

[0086] The first external device 104 can be connected to the first processor 102 via wired or wireless means, and the second external device 105 can be connected to the second processor 103 via wired or wireless means. The specific types of the first external device 104 and the second external device 105 are not limited and can be set based on the needs of the dual-processor display device.

[0087] Among them, reference Figure 1As shown, a first application 1021 and a second application 1022 are deployed in the first processor 102, and a third application 1031 is deployed in the second processor 103. The first application 1021 may include system-level applications or non-system-level applications, and the third application 1031 may include system-level applications or non-system-level applications. A system-level application is an application that runs at the operating system's lowest level, has the highest privileges, and can directly control hardware or core services. Specifically, the system-level application in the first processor 102 runs at the operating system's lowest level, has the highest privileges, and can directly control the hardware or core services of the first processor 102. Similarly, the system-level application in the second processor 103 runs at the operating system's lowest level, has the highest privileges, and can directly control the hardware or core services of the second processor 103. The non-system-level applications in the first processor 102 are applications deployed on the first processor 102 that are different from the system-level applications of the first processor 102. They are typically installed by the user or pre-installed on the first processor 102 along with the system-level applications. They are used to complete specific tasks, have restricted permissions (accessing only their own data), and can access and use sensitive functional data with user authorization. Similarly, the non-system-level applications in the second processor 103 are applications deployed on the second processor 103 that are different from the system-level applications of the second processor 103. They are typically installed by the user or pre-installed on the second processor 103 along with the system-level applications. They are used to complete specific tasks, have restricted permissions (accessing only their own data), and can access and use sensitive functional data with user authorization.

[0088] The method by which the first processor 102 sends information about the currently responsive window application to the second processor 103 is not limited. In some feasible embodiments, the first processor 102 may send identification information to the second processor 103. This identification information can be used to identify whether the currently responsive window application is an application of the second processor 103 (i.e., the third application 1031 deployed on the second processor 103), or it can be used to identify whether the signal source of the currently responsive window application originates from the second processor 103. For example, in some examples, if the identification information is a first identifier (e.g., 1), the second processor 103 can determine that the currently responsive window application is the third application 1031, that is, an application deployed on the second processor 103. If the identification information is a second identifier (e.g., 0), the second processor 103 can determine that the currently responsive window application is not the third application 1031. It is understandable that the first processor 102 can display the display interface content of the third application 1031 of the second processor 103 through the second application 1022. Therefore, when the identification information is the first identification, it means that the window application of the first processor 102 that can respond to input events is the second application 1022. That is, the signal source of the window application that can respond to input events comes from the second processor 103. Therefore, since the second application 1022 is the window application that can respond to input events, it actually needs to be processed by the second processor 103. Thus, it can be considered that the window application that can respond to input events is the third application 1031.

[0089] In other feasible embodiments, the first processor 102 may send the application identifier of the window application that is currently responsive to input events to the second processor 103. The second processor 103 may compare the application identifier with the identifier of the third application 1031 deployed on the second processor 103. If they match, the second processor 103 may determine that the window application that is currently responsive to input events is the third application 1031, that is, the application deployed on the second processor 103. If they do not match, the second processor 103 may determine that the window application that is currently responsive to input events is not the third application 1031.

[0090] In other feasible embodiments, the first processor 102 may send window-related information, such as the display coordinate range of each window application displayed on the display 101 and the hierarchical relationship of each window application, to the second processor 103. The second processor 103 may analyze and determine the window application that can currently respond to input events based on the window information received from the first processor 102, and then determine whether the window application that can currently respond to input events is a third application.

[0091] It is understood that in other feasible embodiments, the information of the window application that is currently responsive to input events sent by the first processor 102 to the second processor may also be other content and form, as long as the second processor 103 can determine whether the window application that is currently responsive to input events is a third application that does not belong to the second processor 103 based on the information of the window application that is currently responsive to input events.

[0092] Based on the display device described above, in some feasible embodiments:

[0093] The second processor 103 is configured as follows:

[0094] Acquire the second input event from the second external device 105 and forward the second input event to the first processor 102;

[0095] Based on the information of the window application, determine the window application that can currently respond to input events, and if the window application that can currently respond to input events is a third application, send the second input event to the third application 1031;

[0096] The first processor 102 is configured as follows:

[0097] Receive the second input event forwarded by the second processor 103;

[0098] Based on the information of the window application, determine the window application that can currently respond to input events. If the window application that can currently respond to input events is a third application, intercept the second input event. If the window application that can currently respond to input events is not a third application, send the second input event to the first application 1021.

[0099] The second input event refers to an input event input through the second external device 105 connected to the second processor 103. The window application that can currently respond to input events refers to the application of the window currently displayed on the display 101 that can respond to input events. The window application that can currently respond to input events can be the active window application, the foreground window application, or the window application with input focus, and it is also the window application that the system currently defaults to receiving input events. There are no restrictions on the way to determine the window application that can currently respond to input events.

[0100] When the first processor 102 receives a second input event forwarded by the second processor 103, it can determine whether the window application currently capable of responding to the input event is a third application, i.e., an application deployed on the second processor 103, based on information it maintains or confirms about the window applications currently capable of responding to input events. Based on this, it can determine whether the input event requires processing by the first processor 102. If it is a third application, it means the second input event should be processed by the third application of the second processor 103, and since the second input event was forwarded by the second processor 103, it can directly intercept the second input event without processing it. In some examples, when intercepting the second input event, it can also perform related processing such as logging. If the window application currently capable of responding to the input event is not a third application, it means the second input event needs to be processed by the first processor 102. Therefore, the first processor 102 can send the second input event to the first application deployed on the first processor 102.

[0101] The display device based on the above embodiments includes a display, a first processor, and a second processor. The first processor is directly connected to the display and has a first application and a second application deployed on it. The second application displays the display interface content of a third application of the second processor. The second processor is connected to the first processor through a data transmission interface and has a third application deployed on it. The first processor can synchronize information about the window application that can currently respond to input events to the second processor. After obtaining a second input event through a second external device, the second processor can determine whether the target window application of the second input event is the third application based on the information about the window application that can currently respond to input events. If the window application that can currently respond to input events is the third application, the second input event is sent to the third application so that the third application can perform corresponding application processing based on the second input event. Simultaneously, the second input event is forwarded to the first processor. The first processor then determines, based on information about the window application that can currently respond to the input event, whether to intercept the second input event or send it to the first application. Therefore, the second input event from the second external device received by the second processor will be forwarded to the first processor at the same time as the second processor analyzes whether it needs to be sent to the third application. The first processor will then analyze whether it needs to be intercepted or sent to the first application. As a result, the second input event received by the second processor through the second external device can be promptly transmitted to the first processor. Adaptive event distribution processing can be performed in both the first and second processors, which helps to ensure adaptive low-latency and reliable transmission of the input event between the second processor and the first processor when the second processor receives the second input event through the second external device.

[0102] In some optional embodiments, the first application may include multiple applications, including system-level applications and non-system-level applications. The first processor 102 is configured to: determine the event type of the second input event; if the event type of the second input event is a non-global event and the currently responding window application is not a third application, send the second input event to the currently responding non-system-level application of the first processor; if the event type of the second input event is a global event, send the second input event to the system-level application of the first processor.

[0103] Non-global events, also known as local / window-level input events, are typically limited to a process or window. They are usually only received when the window is active or has focus. Global events, on the other hand, are input events that can cover the entire operating system. They can be received regardless of which window has focus, whether it is in the background, or even when there is no window. Global events are usually for system-level global processing. Some possible global events may include system-level shortcut key input events (such as screenshot input events, global voice wake-up, etc.), but are not limited to these.

[0104] When the first processor 102 receives a second input event forwarded by the second processor 103, it can first determine the event type of the second input event. If the event type of the second input event is a global event, it can directly send the second input event to the system-level application deployed on the first processor 102 for system-level input event processing. If the event type of the second input event is a non-global event, it can further determine whether the window application currently capable of responding to the input event is a third application deployed on the second processor 103. If it is not a third application deployed on the second processor 103, it sends the second input event to the non-system-level application currently capable of responding to the input event on the first processor 102 for processing by the non-system-level application of the first processor 102. If it is a third application deployed on the second processor 103, it can intercept the second input event and not process it, or simply record the second input event.

[0105] Accordingly, after receiving the second input event forwarded by the second processor, the first processor determines the event type of the second input event. If the event type of the second input event is a non-global event and the window application currently responding to the input event is not a third application, it indicates that the target window of the second input event is a non-system-level application deployed on the first processor. Therefore, the second input event is sent to the non-system-level application currently responding to the input event deployed on the first processor so that the non-system-level application of the first processor can process the second input event accordingly. If the event type of the second input event is a global event, it indicates that the second input event requires system-level global processing. Therefore, the second input event is sent to the system-level component application of the first processor for system-level processing. This achieves adaptive distribution of system-level and non-system-level second input events.

[0106] In some optional embodiments, the first processor 102 is further configured to: acquire a first input event from the first external device 104, and if the window application currently capable of responding to the input event is the third application 1031, forward the first input event to the second processor 103, and if the window application currently capable of responding to the input event is not the third application, send the first input event to the first application 1021.

[0107] The second processor 103 is also configured to receive a first input event forwarded by the first processor 102 and send the first input event to the third application 1031.

[0108] The first input event refers to the input event obtained through the first external device 104 connected to the first processor 102.

[0109] Upon receiving a first input event, the first processor 102 can first determine whether the window application currently capable of responding to the input event is a third application 1031 deployed on the second processor 103, i.e., whether the input event requires processing by the second processor 103. If the window application currently capable of responding to the input event is a third application 1031 deployed on the second processor 103, then the first input event should be processed by the third application 1031 of the second processor 103. Since the first input event was received by the first processor 102 through the first external device 104 connected to it, the first input event can be forwarded to the second processor 103 for processing. If the window application currently capable of responding to the input event is not a third application 1031 deployed on the second processor 103, then the first input event requires processing by the first processor. Therefore, the first input event can be sent to the first application 1021. In related examples, when sending the first input event to the first application 1021, the first input event can also be recorded or otherwise processed.

[0110] When the second processor 103 receives the first input event forwarded by the first processor 102, it indicates that the first input event is an input event that the first processor 102 has already confirmed and should be processed by the second processor 103. Therefore, the first input event can be directly sent to the third application 1031 without the need to determine whether the window application that can currently respond to the input event is the third application 1031 deployed on the second processor 103.

[0111] Accordingly, the first processor also acquires the first input event from the first external device connected to the first processor, and only forwards the first input event to the second processor when it is determined that the window application that can respond to the input event is the third application. Therefore, for the second processor, the first input event received from the first processor is an input event that the first processor has determined needs to be processed by the second processor. Therefore, the received first input event can be directly sent to the third application, which helps to further improve the transmission and processing efficiency of the first input event.

[0112] In some optional embodiments, the second processor 103 is further configured to: determine whether a third input event paired with the first input event is received by the first processor 102 before the first input event is received; and if the third input event is not received, process the first input event based on the priority of the first input event.

[0113] The third input event paired with the first input event refers to the input event that usually appears in pairs with the first input event and corresponds to each other. For example, if the first input event is releasing the keyboard, then its paired third input event is pressing the keyboard. If the first input event is releasing the mouse button, then its paired third input event is pressing the mouse button.

[0114] After receiving the first input event, the second processor 103 also determines whether a third input event paired with the first input event was received before receiving the first input event. If no third input event paired with it is received, it indicates that packet loss may have occurred. Therefore, the first input event can be processed based on its priority.

[0115] The priority of input events refers to the order in which responses are given to each input event and the rules governing which input events can be ignored. The priority of each input event can be set based on actual needs. In the absence of a third input event paired with the first input event, processing the first input event based on its priority can help recover input event processing and reduce system lag caused by lost input events. The specific priorities of different input events and the corresponding processing mechanisms under different priorities can be set based on actual technical needs; this application does not impose specific limitations on this.

[0116] Accordingly, after receiving the first input event forwarded by the first processor, the second processor also determines whether a third input event paired with the first input event was received before receiving the first input event. If no third input event is received, the first input event is processed based on the priority of the first input event. Thus, in the absence of a third input event paired with the first input event, processing is based on the priority of the first input event, which helps to enhance the system stability of the display device and improve the user experience.

[0117] In some optional embodiments, the second processor 103 is further configured to: determine whether a third input event paired with the first input event is received before the first input event is received, and if the third input event is not received, simulate the generation of the third input event and send the third input event and the first input event to the third application in sequence.

[0118] Accordingly, if the second processor has not received a third input event paired with the first input event before determining that the first input event has been received, it can generate a third input event by simulation and send the third input event and the first input event to the third application. This helps to improve the integrity of the sent input events, maintain the continuity of the processing logic, improve the fault tolerance of the processing device, reduce the possibility of failure or abnormality due to event loss, and further enhance the user experience.

[0119] In some alternative embodiments, the third application includes system-level applications and non-system-level applications; the first processor 102 is also configured to forward the first input event to the second processor 103 if the first input event is a global event;

[0120] The second processor 103 is further configured to: determine the event type of the first input event forwarded by the first processor; if the event type of the first input event is a non-global event, send the first input event to a non-system-level application of the second processor that is currently capable of responding to the input event; if the event type of the first input event is a global event, send the first input event to a system-level application of the second processor.

[0121] After receiving the first input event forwarded by the first processor 102, the second processor 103 can determine whether the event type of the first input event is a global event in the same way as the first processor 102 determines whether the event type of the second input event is a global event, which will not be elaborated here.

[0122] Accordingly, if the first input event received is a global event, the first processor will forward the first input event to the second processor. After receiving the first input event forwarded by the first processor, the second processor will determine the event type of the first input event. If the event type of the first input event is a non-global event, it means that the first input event forwarded by the first processor needs to be processed by the non-system-level application of the second processor. Therefore, the first input event is sent to the non-system-level application of the second processor for corresponding processing. If the event type of the first input event is a global event, it means that the first input event is a system-level input event. Therefore, the first input event is distributed to the system-level application of the second processor for system-level processing. This realizes the adaptive distribution of the second processor to system-level and non-system-level first input events.

[0123] In some optional embodiments, the first processor 102 has a first computer peripheral interface, the second processor 103 has a second computer peripheral interface, and the first computer peripheral interface and the second computer peripheral interface are connected by a cable.

[0124] The first processor 102 is also configured to forward a first input event to the second processor 103 via a first computer peripheral interface, and to receive a second input event forwarded by the second processor 103 via the first computer peripheral interface.

[0125] The second processor 103 is also configured to forward a second input event to the first processor 102 via a second computer peripheral interface, and to receive a first input event forwarded by the first processor 102 via the second computer peripheral interface.

[0126] A computer peripheral interface is a hardware port that connects to external devices. It is a connection circuit and physical interface between the host and the external device for transmitting data, control signals, and power supply signals. It is a bridge for communication between the host and the peripheral device and is responsible for data exchange, device identification, and command transmission. The specific type of computer peripheral interface is not specifically limited in the embodiments of this application. Some examples of computer peripheral interfaces may include Ethernet interfaces, but are not limited to this.

[0127] Accordingly, by deploying a first computer peripheral interface on the first processor and a second computer peripheral interface on the second processor, and connecting the first and second computer peripheral interfaces via cables, the first processor forwards a first input event to the second processor and receives a second input event from the second processor through the first computer peripheral interface, and the second processor forwards a second input event to the first processor and receives a first input event from the first processor through the second computer peripheral interface. This allows the first input event forwarded by the first processor to be sequentially forwarded to the kernel layer of the second processor via the first computer peripheral interface, the cable, and the second computer peripheral interface. Similarly, the second input event forwarded by the second processor can be sequentially forwarded to the kernel layer of the first processor via the second computer peripheral interface, the cable, and the first computer peripheral interface. Since the kernel layer is at the lowest level of the system and interacts directly with the hardware, input events directly reach the kernel, minimizing latency in the processing chain and enabling the system to respond to input events extremely quickly, thus further reducing the latency of the display device.

[0128] In some optional embodiments, the first processor 102 is further configured to: acquire a first input event from the first external device 104, forward the first input event to the second processor 103, and determine the window application that can currently respond to the input event based on the information of the window application, and if the window application that can currently respond to the input event is the first application, send the first input event to the first application.

[0129] The second processor 103 is further configured to: receive a first input event forwarded by the first processor 102, determine the window application that can currently respond to the input event based on the information of the window application, and send the first input event to the third application if the window application that can currently respond to the input event is a third application, and intercept the first input event if the window application that can currently respond to the input event is not the third application.

[0130] Accordingly, upon receiving a first input event, the first processor forwards the first input event to the second processor. Simultaneously, based on the information of the window application, it determines which window application is currently capable of responding to the input event, and whether this window application is a third application. If the window application is not a third application of the second processor, the first input event is sent to the first application, such as a system-level application or a non-system-level application of the first processor. Upon receiving the first input event forwarded by the first processor, the second processor, based on the information of the window application, determines which window application is currently capable of responding to the input event, and whether this window application is a third application, to determine whether the first input event needs to be intercepted. The first input event is distributed in a manner that, when the first processor receives the first input event, it performs analysis on whether the window application currently capable of responding to the input event is a third application to determine whether the first input event needs to be sent to the first application. At the same time, it forwards the first input event to the second processor. The second processor also performs analysis on whether the window application currently capable of responding to the input event is a third application to determine whether the first input event needs to be intercepted or sent to the third application. Thus, the first input event received by the first processor through the first external device can be adaptively distributed in both the first and second processors, which helps to achieve adaptive low-latency and reliable transmission of the input event when the first processor receives the first input event through the first external device.

[0131] In some optional embodiments, the second processor 103 is also configured to forward the second input event to the first processor 102 if the second input event is a cursor event;

[0132] The first processor 102 is also configured to output the cursor event to the first target graphics output plane and draw the cursor through the first target graphics output plane if it is determined that the received second input event is a cursor event.

[0133] The Graphics Output Plane is a hardware-level display layer abstraction used to manage the overlay and compositing of graphics data from different sources in the final image. In this embodiment, cursor events received by the first processor and the second processor are both output to the first target graphics output plane, which then uniformly performs cursor drawing based on the cursor events.

[0134] Accordingly, when the second input event received by the second processor is a cursor event, the second input event is directly forwarded to the first processor. When the first processor determines that the received second input event is a cursor event, it outputs the cursor event to the first target image output plane and performs cursor drawing through the first target image output plane. That is, the cursor events received by the first processor and the second processor can both be uniformly processed by the first target image output plane of the first processor for cursor drawing, which helps to achieve unified rendering and display processing and further accelerates rendering efficiency.

[0135] In some optional embodiments, the first processor 102 is further configured to output interface description information to a second target graphics output plane when it is determined that user interface drawing is triggered, and to draw the user interface through the second target graphics output plane, wherein the first target graphics output plane is different from the second target graphics output plane.

[0136] Accordingly, when the first processor determines that user interface drawing is triggered, it outputs the interface description information used for user interface drawing to the second target graphics output plane, and performs user interface drawing through the second target graphics output plane. This makes cursor drawing and user interface drawing performed by different graphics output planes, that is, cursor drawing can be performed by an independent first target image output plane, which helps to further achieve unified rendering and display processing and further accelerate rendering efficiency.

[0137] In some embodiments of this application, reference is made to Figure 2 As shown, this application also provides an event processing unit 200, which can be deployed in the first processor 102 and / or the second processor 103 to process input events received by the processor.

[0138] refer to Figure 2 As shown, the event handling unit 200 may include: a window management module 201, an event collection module 202, and a two-end communication module 203; wherein:

[0139] The window management module 201 is configured to obtain information about the currently responsive window applications of the device where the event handling unit is located;

[0140] Event collection module 202 is configured as follows:

[0141] Receive input events from external devices connected to the processor where the event processing unit is located; determine the processor type of the processor where the event processing unit is located;

[0142] When the processor category is the primary processor, if the information of the window application determines that the window application that can currently respond to input events is an application deployed on the secondary processor, the input event is forwarded to the secondary processor connected to the two-end communication modules through the two-end communication modules; otherwise, the input event is sent to the application deployed on the processor where it is located.

[0143] When the processor type is a slave processor, the input event is forwarded to the master processor connected to the two communication modules through the two-end communication modules. If it is determined from the information of the window application that the window application that can respond to the input event is the application deployed on the slave processor, the input event is sent to the application deployed on the processor.

[0144] Therefore, taking the aforementioned display device as an example, when the event processing unit 200 is deployed in the first processor 102, the processing method of the event processing unit 200 is as follows:

[0145] The window management module 201 obtains information about the window applications currently capable of responding to input events on the device where the event handling unit is located (i.e., the aforementioned display device);

[0146] The event collection module 202 receives a first input event from a first external device 104 connected to the processor (i.e., the first processor 102) where the event processing unit 200 is located, and determines that the processor where the event processing unit 200 is located is a master processor. If the window application that can respond to the input event is determined to be an application deployed on a slave processor based on the information of the window application, the first input event is forwarded to the second processor connected to the two-end communication module 203 through the two-end communication module 203. Otherwise, the input event is sent to the first application deployed on the processor (i.e., the first processor 102).

[0147] When the event handling unit 200 is deployed on the second processor, the processing method of the event handling unit 200 is as follows:

[0148] The window management module 201 obtains information about the window applications that can currently respond to input events in the device where the event processing unit is located (i.e., the display device mentioned above). As described above, the window management module 201 can obtain information about the window applications that can currently respond to input events transmitted from the first processor 102, or obtain display window information transmitted from the first processor 102, and obtain information about the window applications that can currently respond to input events based on the display window information.

[0149] The event collection module 202 receives a second input event from a second external device 105 connected to the processor (i.e., the second processor 103) where the event processing unit 200 is located, and determines that the processor where the event processing unit 200 is located is a slave processor. Then, it forwards the second input event to the first processor 102 connected to the two-end communication module 203 through the two-end communication module 203. At the same time, if it is determined from the information of the window application that the window application that can respond to the input event is an application deployed on the slave processor, the second input event is sent to a third application that does not belong to the second processor 103.

[0150] Based on the embodiments described above, the following detailed examples of several applications based on the embodiments are provided.

[0151] For a display device with dual processors, each processor connected to an external device, the architecture diagram related to input event processing is as follows: Figure 3 As shown, combined with Figure 3 As can be seen, in this architecture, the display device includes a monitor 101, a first processor 102, and a second processor 103. The first processor 102 is connected to a first external device 104, and the second processor 103 is connected to a second external device 105. The first processor 102, as the main processor, is responsible for image quality processing, cursor drawing, and global event decision-making. The second processor 103 serves as an input channel (e.g., an HDMI (High-Definition Multimedia Interface) input channel) for the first processor 102, providing data. The first processor 102 deploys a first kernel layer, a first core service process, a first application, a second application, and an image compositing service. The first core service process may include a first event collection module and a first event distribution module. The second processor 103 deploys a second kernel layer, a second core service process, and a third application. The second core service process may include a second event collection module and a second event distribution module.

[0152] In the case where the window application currently capable of responding to input events is a third application deployed on the second processor 103, if an input event is received through the first external device 104 of the first processor 102, the input event needs to be transmitted from the first processor 102 to the second processor 103. Conversely, in the case where the window application currently capable of responding to input events is a first application deployed on the first processor 102, if an input event is received through the second external device 105 of the second processor 103, the input event needs to be transmitted from the second processor 103 to the first processor 102.

[0153] In the first processor 102, input events can be collected through a first event collection module, such as collecting input events obtained through the first kernel layer (including input events from the first external device 104) and input events forwarded by the second processor 103. The first event distribution module can also distribute the input events, for example, sending them to a first application. If the input event is a cursor drawing event, it is processed through a graphics compositing service. In the second processor 103, input events can be collected through a second event collection module, such as collecting input events obtained through the second kernel layer (including input events from the second external device 105) and input events forwarded by the first processor 102. The second event module can also distribute the input events, for example, sending them to a third application.

[0154] based on Figure 3 The architecture shown, in order to achieve efficient distribution of input events, is based on the display device of the embodiments of this application as described above. In some example architectures of this application, it combines... Figure 4 As shown, a first dual-input control module can be deployed in the first processor 102 and a second dual-input control module can be deployed in the second processor 103. The first dual-input control module can be used as a virtual external device of the first processor 102 and the second dual-input control module can be used as a virtual external device of the second processor 103, thereby realizing the transmission of input events between the first processor 102 and the second processor 103.

[0155] The first dual-input control module can be deployed in the kernel layer of the first processor 102 and implement the transmission of input events in the user layer. The second dual-input control module can be deployed in the kernel layer of the second processor 103 and implement the transmission of input events in the user layer. The first processor 102 and the second processor can be connected via an interface such as a USB (Universal Serial Bus) interface or a network interface to implement the transmission of input events through the interface.

[0156] based on Figure 4In the illustrated framework, the first input events received by the first processor 102 through the first external device 104 are forwarded to the second processor 103 via the first dual-input control module. In a specific example, the first dual-input control module may include a first event reading module and a first event forwarding module. The first dual-input control module can read the input events received by the first kernel layer through the first event reading module, and the first event forwarding module forwards the input events read by the first event reading module to the second processor 103. Similarly, the input events received by the second processor 103 through the second external device 105 are forwarded to the first processor 102 via the second dual-input control module. In a specific example, the second dual-input control module may include a second event reading module and a second event forwarding module. The second dual-input control module can read the input events received by the second kernel layer through the second event reading module, and the second event forwarding module forwards the input events read by the second event reading module to the first processor 102. The first event distribution module of the first processor 102 can synchronously transmit information about the currently responsive window applications displayed on the maintained display to the second processor 103.

[0157] Accordingly, after obtaining an input event (including input events read from the second kernel layer and input events forwarded by the first processor 102), the second event collection module of the second processor 103 first determines whether the currently respondable window application is a third application based on the information of the window application currently responding to the input event synchronized from the first processor 102. If so, the input event is distributed to the third application; otherwise, the input event is blocked. After obtaining an input event (including input events read from the first kernel layer and input events forwarded by the second processor 103), the first event collection module of the first processor 102 first determines whether the currently respondable window application is a third application based on the window information maintained by the first event distribution module. If so, the input event is blocked; otherwise, the input event is sent to the first application. At the same time, the first event distribution module also determines whether the input event is a global event. If it is a global event, global event distribution is performed.

[0158] based on Figure 4 The architecture shown only requires deploying a first dual-input control service on the first processor and a second dual-input control service on the second processor to send all received input events to the other end. The second core service process of the first processor only synchronizes the information of the window application that can currently respond to input events with the second processor. The logic changes compared to the existing display device are small, and the cursor drawing can continue to use the original logic.

[0159] Therefore, based on Figure 4The interactive flowchart of the display device in the architecture shown, when the display device distributes a first input event received from a first external device and a second input event received from a second external device, can be shown as follows: Figure 5 , Figure 6 As shown.

[0160] refer to Figure 5 As shown, based on Figure 4 The architecture of the display device shown, and the interaction flow for the display device to handle the first input event input from the first external device, can be described as follows:

[0161] When the first processor 102 receives a first input event from the first external device 104, it can send the first input event to the first processor management component of the first processor 102. The first processor management component can manage the first input event, such as recording the information of the received first input event.

[0162] For the first input event obtained, the first processor 102 forwards the first input event to the second processor. At the same time, it determines the window application that can currently respond to the input event based on the information of the window application that can currently respond to the input event. If the window application that can currently respond to the input event is the first application, the first input event is sent to the first application. If the window application that can currently respond to the input event is the third application, no other processing is performed on the first input event.

[0163] When the second processor 103 receives the first input event forwarded by the first processor 102, it determines the window application that can currently respond to the input event based on the information of the window application that can currently respond to the input event, and determines whether the window application that can currently respond to the input event is a third application deployed on 103. If it is a third application, it sends the first input event to the third application; otherwise, it intercepts the first input event, that is, it does not perform any other processing on the first input event.

[0164] refer to Figure 6 As shown, based on Figure 4 The architecture of the display device shown, and the interaction flow for the display device to handle the second input event from the second external device, can be described as follows:

[0165] When the second processor 103 receives the second input event from the second external device 105, it can directly forward the second input event to the first processor 102. At the same time, the second processor 103 determines the window application that can currently respond to the input event based on the information of the window application that can currently respond to the input event, and determines whether the window application that can currently respond to the input event is a third application deployed on 103. If it is a third application, the second input event is sent to the third application; otherwise, the second input event is intercepted, that is, no other processing is performed on the second input event.

[0166] When the first processor 102 receives a second input event forwarded by the second processor 103, on the one hand, it sends the second input event to the first processor management component of the first processor 102. The first processor management component can manage the second input event, such as recording the information of the received second input event. On the other hand, the first processor 102 determines the window application that can currently respond to the input event based on the information of the window application that can currently respond to the input event. If the window application that can currently respond to the input event is the first application, the second input event is sent to the first application. If the window application that can currently respond to the input event is the third application, the second input event is intercepted and no other processing is performed on the second input event.

[0167] The architecture of the display device related to input event processing provided in other examples of this application is referenced. Figure 7 As shown, the input management service (i.e., ...) of the first processor 102 can be used. Figure 7 The first event processing unit is deployed in the first input management service shown in the second processor 103 (i.e., the input management service of the second processor 103). Figure 7 The second event processing unit is deployed in the second input management service shown in the figure. The architecture of the first and second event processing units can be the same as that of the event processing unit 200 mentioned in the above embodiments. The first event processing unit includes a first event collection module, a first window management module, and a first two-end communication module. The first event collection module is communicatively connected to the first event reading module, and the first event reading module is communicatively connected to the cursor drawing-related module. The first event processing unit can be deployed in the first event distribution module (e.g., the inputdispatcher module of the first processor 102), and the first event distribution module can be deployed in the first input management engine (DualInputManager) of the first input management service (i.e., the IMS (InputManagerService) of the first processor 102).

[0168] The second event processing unit includes a second event collection module, a second window management module, and a second-end communication module. The second event collection module is communicatively connected to the second event reading module. The first-end communication module and the second-end communication module of the first processor 102 can be connected via a network cable or other cable. The second event processing unit can be deployed in the second event distribution module (e.g., the inputdispatcher module of the second processor 103), and the second event distribution module can be deployed in the second input management service (i.e., the second input management engine (DualInputManager) of the IMS (InputManagerService) of the second processor 103).

[0169] Based on the architecture in this example, a dual-processor, dual-input adaptive scheme based on a private LAN communication protocol can be implemented. The first and second processors only need to be connected via a cable (such as a network cable) to form a LAN. Input events are not forwarded through virtual devices but are directly passed to the other processor's event dispatch module (inputdispatcher), and then dispatched based on the window state—that is, based on information about the currently responding window application. Furthermore, cursor drawing can be separated and drawn using a dedicated graphics output plane (GOP), reducing drawing latency.

[0170] The system comprises a first input management engine and a second input management engine, forming a dual-input system. These engines act as relay stations for input events. The first input management engine reads input events through the first event reading module (the inputReader module of the first processor), or through first virtual input events injected via WMS (WindowManagerService) or APP (Application), and transmits these events to the first event dispatch module after processing. Similarly, the second input management engine reads input events through the second event reading module (the inputReader module of the second processor), or through second virtual input events injected via WMS (WindowManagerService) or APP (Application), and transmits these events to the second event dispatch module after processing.

[0171] The first and second end communication modules are communication modules capable of bidirectional communication. In the relevant examples, they can be implemented based on a private communication protocol, but are not limited to this. They can provide low-latency and stable communication transmission, thereby enabling low-latency and stable bidirectional transmission between the first and second processors.

[0172] The first window management module and the second window management module are used to manage windows. The first window management module of the first processor 102 records all window information transmitted from the graphics compositing service (SurfaceFlinger). When the window information changes, it forwards the changed window information or the changed window information to the second processor 103 through the first two-end communication module. The second processor 103 can obtain window information through the second two-end communication module. Both the first window management module and the second window management module can determine the window application that can currently respond to input events based on the window information, and then determine whether the input event needs to be distributed, that is, whether the input event needs to be forwarded to the application of the processor.

[0173] The input events acquired by the first event collection module include physical input events read by the first event reading module and trusted first simulated input events injected via IMS or APP. Unlike related technologies that enqueue all received input events and then read events from the queue to forward to the application, in the relevant examples of this application, the decision to forward or distribute the input event is based on the window state. Specifically, the first event collection module determines whether to forward or distribute the input event based on the window state. If it is not a global event, it distributes it to the first processor application or forwards it to the second processor via the second dual-end communication module. If it is a global event, it performs both forwarding and distribution simultaneously.

[0174] The second event collection module forwards all acquired input events to the first processor. The first event collection module of the first processor receives the forwarded input events via the first two-way communication module and determines the event type. If it is a global event, it distributes it to the application that handles global events (i.e., a system-level application). Otherwise, it determines whether to distribute it to a non-system-level first application based on the window state. Simultaneously, the second event collection module also determines, based on the displayed window information, whether the window application currently capable of responding to the input event is a third application of the second processor (i.e., whether the second processor needs to process it). If so, it simultaneously distributes the acquired input event to the third application; otherwise, it intercepts the input event.

[0175] Therefore, based on Figure 7 The interactive flowchart of the display device in the architecture shown, when the display device distributes a first input event received from a first external device and a second input event received from a second external device, can be shown as follows: Figure 8 , Figure 6 As shown.

[0176] refer to Figure 8 As shown, based on Figure 7The architecture of the display device shown, and the interaction flow for the display device to handle the first input event input from the first external device, can be described as follows:

[0177] When the first processor 102 receives a first input event from the first external device 104, it can send the first input event to the first processor management component of the first processor 102. The first processor management component can manage the first input event, such as recording the information of the received first input event.

[0178] For the first input event obtained, the first processor 102 determines the window application that can currently respond to the input event based on the information of the window application that can currently respond to the input event. If the window application that can currently respond to the input event is the first application, the first input event is sent to the first application. If the window application that can currently respond to the input event is the third application, the first input event is forwarded to the second processor.

[0179] When the second processor 103 receives the first input event forwarded by the first processor 102, it determines the window application that can currently respond to the input event based on the information of the window application that can currently respond to the input event, and determines whether the window application that can currently respond to the input event is a third application deployed on 103. If it is a third application, it sends the first input event to the third application; otherwise, it intercepts the first input event, that is, it does not perform any other processing on the first input event.

[0180] refer to Figure 6 As shown, based on Figure 7 The architecture of the display device shown, and the interaction flow for the display device to handle the second input event input from the second external device, can be described as follows:

[0181] When the second processor 103 receives the second input event from the second external device 105, it can directly forward the second input event to the first processor 102. At the same time, the second processor 103 determines the window application that can currently respond to the input event based on the information of the window application that can currently respond to the input event, and determines whether the window application that can currently respond to the input event is a third application deployed on 103. If it is a third application, the second input event is sent to the third application; otherwise, the second input event is intercepted, that is, no other processing is performed on the second input event.

[0182] When the first processor 102 receives a second input event forwarded by the second processor 103, on the one hand, it sends the second input event to the first processor management component of the first processor 102. The first processor management component can manage the second input event, such as recording the information of the received second input event. On the other hand, the first processor 102 determines the window application that can currently respond to the input event based on the information of the window application that can currently respond to the input event. If the window application that can currently respond to the input event is the first application, the second input event is sent to the first application. If the window application that can currently respond to the input event is the third application, the second input event is intercepted and no other processing is performed on the second input event.

[0183] It can be seen that, based on Figure 7 In the architecture shown, all second input events received by the second processor are sent to the first processor, while only the first input events received by the first processor that require processing by the second processor are sent to the second processor.

[0184] Accordingly, based on the embodiments described above, the schematic diagram of the processing mechanism of the first processor of the display device for input events can be shown as follows: Figure 9 As shown, the schematic diagram of the processing mechanism of the second processor of the display device for input events can be illustrated as follows: Figure 10 As shown.

[0185] Among them, reference Figure 9As shown, the principle flow of the input event processing mechanism of the first processor of the display device can be described as follows: The first processor can read the input events of the first external device connected to it (i.e., the aforementioned first input event); the first processor can also receive the input events transmitted by the second processor (i.e., the aforementioned second input event). For the input event transmitted by the second processor, it is determined whether the input event is a cursor event. If it is a cursor event, it is transmitted to the first GOP interface for cursor drawing. For the received input event transmitted by the second processor, the event time of the input event is also recorded, and the input event is marked as a second processor event. For the obtained input event from the first external device or the input event transmitted by the second processor, the first processor can determine whether the window application that can currently respond to the input event is the first application of the first processor, that is, whether the distribution window of the input event is a window deployed in the first application of the first processor. If so, the input event is distributed, that is, the input event is sent to the corresponding first application. If the window application that can currently respond to the input event is not the first application of the first processor, and it is determined based on the mark of the input event that it is not an input event forwarded by the second processor, then the input event is forwarded to the second processor. If the window application that can currently respond to the input event is not the first application of the first processor, nor is the input event forwarded by the second processor, then it is further determined whether the input event is a global event or a touch event. If so, it is distributed to the input event.

[0186] Among them, reference Figure 10 As shown, the principle flow of the input event processing mechanism of the second processor of the display device can be described as follows:

[0187] The second processor can receive input events from the second external device connected to it (i.e., the aforementioned second input event). If the input event is a cursor event, the cursor display is masked and the input event is forwarded to the first processor. At the same time, based on the window information obtained from the first processor, it determines whether the window application that can currently respond to the input event is a third application, that is, whether the target window corresponding to the input event is a window of the third application. If so, the input event is distributed to the third application.

[0188] The second processor can also receive input events transmitted by the first processor, record the time of the input events, and, for the input events forwarded by the first processor, configure the display device as follows: Figure 4 In the framework shown, if the first processor does not determine the input event, it determines whether the window application currently capable of responding to the input event is a third application based on the window information obtained from the first processor. If so, the input event is distributed to the third application. (The last sentence appears to be incomplete and possibly refers to a different context.) Figure 7In the framework shown, if the first processor only forwards input events that have been determined and need to be processed by the second processor to the second processor, then the second processor can directly send the input event to the third application.

[0189] As described above, both cursor display and drawing can be handled by the first processor. If the input event received by the second processor is a cursor drawing input event, then the cursor display is disabled, and the cursor drawing input event is forwarded to the first processor. Upon receiving the input event forwarded by the second processor and determining that the cursor needs to be displayed based on the input event, the first processor calls the following... Figure 11 The cursor drawing module sets the cursor position, cursor type, and cursor style, and calls cursor resources. Based on the received cursor drawing synchronization signal, it positions the cursor through a specially set first target graphics output plane for cursor drawing.

[0190] Based on the display device described above, the relevant embodiments of this application also provide a processing method for the display device.

[0191] In some examples, the processing methods of the display device can be executed by the main processor of a dual-processor display device (e.g., the first processor mentioned above). The display device includes a monitor, a first processor, and a second processor. The first processor is connected to a first external device, and the second processor is connected to a second external device. The first processor is directly connected to the monitor and deploys a first application and a second application that can display the interface content of a third application. It can send information about the currently responsive window application to the second processor. The second processor is connected to the first processor via a data transmission interface and deploys a third application. It can send the display interface content of the third application to the second application for display. (See reference) Figure 12 As shown, the method includes:

[0192] Step 1201: Obtain information about the window application that is currently responsive to input events, and send the window application information to the second processor;

[0193] Step 1202: Receive the second input event forwarded by the second processor, and determine the window application that can currently respond to the input event based on the information of the window application;

[0194] Step 1203: If the window application that can currently respond to input events is a third application, intercept the second input event;

[0195] Step 1204: If the first application is the window application that can currently respond to input events, send the second input event to the first application.

[0196] In some optional embodiments, the first application includes system-level applications and non-system-level applications; step 1204, which involves sending the second input event to the first application when the currently responsive window application is the first application, may include:

[0197] If the window currently capable of responding to input events is the first application, determine the event type of the second input event;

[0198] If the second input event is a non-global event and the window application that can currently respond to the input event is not the window of the third application, the second input event is sent to the non-system application that can currently respond to the input event.

[0199] If the event type of the second input event is a global event, the second input event is sent to the system-level application.

[0200] In some optional embodiments, the method further includes: acquiring a first input event from a first external device; forwarding the first input event to a second processor if the currently responsive window application is a third application; and sending the first input event to a first application if the currently responsive window application is a first application.

[0201] In some optional embodiments, the method further includes: if it is determined that the received second input event is a cursor event, outputting the cursor event to a first target graphics output plane, and drawing the cursor through the first target graphics output plane.

[0202] In some optional embodiments, the method further includes: when it is determined that user interface drawing is triggered, outputting interface description information to a second target graphics output plane, and drawing the user interface through the second target graphics output plane, wherein the first target graphics output plane is different from the second target graphics output plane.

[0203] In some examples, the processing methods of the display device can be executed by the slave processor (e.g., the second processor mentioned above) of a dual-processor display device. The display device includes a monitor, a first processor, and a second processor. The first processor is connected to a first external device, and the second processor is connected to a second external device. The first processor is directly connected to the monitor and deploys a first application and a second application capable of displaying the interface content of a third application. It can send information about the currently responsive window application to the second processor. The second processor is connected to the first processor via a data transmission interface and deploys a third application. It can send the display interface content of the third application to the second application for display. (See reference) Figure 13 As shown, the method includes:

[0204] Step 1301: Obtain information about the window application currently responsive to input events, sent by the first processor;

[0205] Step 1302: Obtain the second input event from the second external device and forward the second input event to the first processor;

[0206] Step 1303: Determine the window application that is currently responsive to input events based on information from the window application;

[0207] Step 1304: If the window application that can currently respond to input events is the third application, send the second input event to the third application.

[0208] In some alternative embodiments, the method further includes: receiving a first input event forwarded by a first processor and sending the first input event to a third application.

[0209] In some optional embodiments, the method further includes: processing the first input event based on the priority of the first input event if no third input event paired with the first input event is received before the first input event is received by the first processor.

[0210] In some optional embodiments, the method further includes: if a third input event paired with the first input event is not received before the first input event is received, the method simulates generating a third input event and sequentially sends the third input event and the first input event to a third application.

[0211] In some optional embodiments, the third application includes system-level applications and non-system-level applications; sending the first input event to the third application includes: determining the event type of the first input event; if the event type of the first input event is a non-global event, sending the first input event to a non-system-level application of the second processor that is currently responsive to input events; if the event type of the first input event is a global event, sending the first input event to a system-level application of the second processor.

[0212] In some optional embodiments, the method further includes: receiving a first input event forwarded by a first processor; determining a window application that is currently responsive to the input event based on information about the window application; sending the first input event to a third application if the window application that is currently responsive to the input event is a third application; and intercepting the first input event if the window application that is currently responsive to the input event is not a third application.

[0213] In some optional embodiments, the method further includes: if the second input event is a cursor event, forwarding the second input event to a first processor to instruct the first processor to output the cursor event to a first target graphics output plane, and drawing the cursor through the first target graphics output plane.

[0214] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. For a specific implementation of the chip event processing method, please refer to the above description of the display device and event processing unit. Unless otherwise explicitly stated herein, there is no strict order restriction on the execution of these steps; they can be executed in other orders. Furthermore, at least some steps in the flowcharts of the embodiments described above may include multiple steps or stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the steps or stages in other steps.

[0215] In some embodiments, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the processing method of the display device as described in any of the embodiments above.

[0216] In some embodiments, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the processing method of the display device as described in any of the embodiments above.

[0217] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0218] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0219] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A display device, characterized in that, The display device includes a display, a first processor, and a second processor. The first processor is connected to a first external device, and the second processor is connected to a second external device. The first processor is directly connected to the display and has a first application and a second application that can display the display interface content of a third application. It can send information of the window application that can currently respond to input events to the second processor. The second processor is connected to the first processor through a data transmission interface and has the third application deployed on it. It can send the display interface content of the third application to the second application for display. The second processor is configured as follows: Acquire the second input event from the second external device and forward the second input event to the first processor; Based on the information of the window application, determine the window application that can currently respond to input events, and if the window application that can currently respond to input events is a third application, send the second input event to the third application; The first processor is configured as follows: Receive the second input event forwarded by the second processor; Based on the information of the window application, determine the window application that can currently respond to input events. If the window application that can currently respond to input events is a third application, intercept the second input event. If the window application that can currently respond to input events is not a third application, send the second input event to the first application.

2. The display device according to claim 1, characterized in that: The first processor is also configured to: The system acquires a first input event from the first external device, and if the window application that can currently respond to the input event is a third application, it forwards the first input event to the second processor; if the window application that can currently respond to the input event is not a third application, it sends the first input event to the first application. The second processor is also configured to: The system receives the first input event forwarded by the first processor and sends the first input event to the third application.

3. The display device according to claim 2, characterized in that: The first processor has a first computer peripheral interface, and the second processor has a second computer peripheral interface. The first computer peripheral interface and the second computer peripheral interface are connected by a cable. The first processor is further configured to forward the first input event to the second processor via the first computer peripheral interface, and to receive the second input event forwarded by the second processor via the first computer peripheral interface; The second processor is further configured to forward the second input event to the first processor via the second computer peripheral interface, and to receive the first input event forwarded by the first processor via the second computer peripheral interface.

4. The display device according to claim 1, characterized in that: The first processor is also configured to: The system acquires a first input event from the first external device, forwards the first input event to the second processor, and determines the window application that can currently respond to the input event based on the information of the window application. If the window application that can currently respond to the input event is the first application, the system sends the first input event to the first application. The second processor is also configured to: The system receives the first input event forwarded by the first processor, determines the window application that can currently respond to the input event based on the information of the window application, and sends the first input event to the third application if the window application that can currently respond to the input event is a third application, and intercepts the first input event if the window application that can currently respond to the input event is not a third application.

5. The display device according to any one of claims 1 to 4, characterized in that: The second processor is also configured to: If the second input event is a cursor event, the second input event is forwarded to the first processor; The first processor is also configured to: If the received second input event is determined to be a cursor event, the cursor event is output to the first target graphics output plane, and the cursor is drawn through the first target graphics output plane.

6. The display device according to claim 5, characterized in that, The first processor is also configured to: When it is determined that user interface drawing is triggered, the interface description information is output to the second target graphics output plane, and the user interface is drawn through the second target graphics output plane. The first target graphics output plane is different from the second target graphics output plane.

7. A processing method for a display device, characterized in that, The display device includes a display, a first processor, and a second processor. The first processor is connected to a first external device, and the second processor is connected to a second external device. The first processor is directly connected to the display and has a first application and a second application that can display the display interface content of a third application deployed thereon. It can send information of the window application that is currently responsive to input events to the second processor. The second processor is connected to the first processor through a data transmission interface and has the third application deployed on it. It can send the display interface content of the third application to the second application for display. The method is executed by the first processor, and the method includes: Obtain information about the window application that is currently responsive to input events, and send the information about the window application to the second processor; Receive the second input event forwarded by the second processor, and determine the window application that is currently responsive to the input event based on the information of the window application; If the current window application that can respond to input events is a third application, intercept the second input event; If the first application is the window application that can currently respond to input events, the second input event is sent to the first application.

8. The method according to claim 7, characterized in that, The method further includes: If it is determined that the received second input event is a cursor event, the cursor event is output to the first target graphics output plane, and the cursor is drawn through the first target graphics output plane; When it is determined that user interface drawing is triggered, the interface description information is output to the second target graphics output plane, and the user interface is drawn through the second target graphics output plane. The first target graphics output plane is different from the second target graphics output plane.

9. A processing method for a display device, characterized in that, The display device includes a display, a first processor, and a second processor. The first processor is connected to a first external device, and the second processor is connected to a second external device. The first processor is directly connected to the display and has a first application and a second application that can display the display interface content of a third application deployed thereon. It can send information of the window application that is currently responsive to input events to the second processor. The second processor is connected to the first processor through a data transmission interface and has the third application deployed on it. It can send the display interface content of the third application to the second application for display. The method is executed by the second processor, and the method includes: Obtain information about the currently responsive window application sent by the first processor; Obtain the second input event from the second external device and forward the second input event to the first processor; The window application that is currently responsive to input events is determined based on the information of the window application. If the window application that can currently respond to input events is a third application, the second input event is sent to the third application.

10. An event processing unit, characterized in that, The event handling unit includes: a window management module, an event collection module, and a two-end communication module; The window management module is configured to obtain information about the currently responsive window applications of the device where the event handling unit is located; The event collection module is configured as follows: Receive input events from external devices connected to the processor where the event processing unit is located; Determine the processor category of the processor in which the event processing unit is located; In the case where the processor category is the primary processor, if the information of the window application determines that the window application currently capable of responding to input events is an application deployed on a secondary processor, the input event is forwarded to the secondary processor connected to the two-end communication module through the two-end communication module; otherwise, the input event is sent to the application deployed on the processor in which it resides. When the processor category is a slave processor, the input event is forwarded to the master processor connected to the two-end communication module through the two-end communication module, and if it is determined based on the information of the window application that the window application that can currently respond to the input event is an application deployed on the slave processor, the input event is sent to the application deployed on the processor where it is located.

Images