Multi-screen touch association method and device based on openharmoney system

Abstract

The application discloses a multi-screen touch association method and device based on an openharmoney system. The method comprises the following steps: listening to all touch screen hardware events and all input hardware events; obtaining unbound touch hardware events in all touch hardware events and unbound input hardware events in all input hardware events, and reading the discovery sequence of each unbound touch hardware event and each unbound input hardware event; and associating each unbound touch hardware event and each unbound input hardware event according to the discovery sequence, so as to realize multi-screen touch association. The application determines each unbound touch hardware event and each unbound input hardware event by adopting an automatic listening mode, and then dynamically matches each unbound touch hardware event and each unbound input hardware event based on the discovery sequence, so as to realize automatic association of a USB access touch ID and HDMI input, and realize touch control capability of a double-screen different display.

Description

Technical Field

[0001] This application relates to the field of video display technology, and in particular to a multi-screen touch association method and device based on the OpenHarmony system. Background Technology

[0002] With the rapid development of industries such as video media, dual-screen video players have emerged. Customers can create content for two screens according to their needs, playing content suitable for the screen display method on each screen, such as playing videos on the top screen and images on the bottom screen.

[0003] For multiple HDMI screens with touch capabilities, the USB touch ID and HDMI input of the HDMI screen are two separate hardware ports. Therefore, when the USB PID and VID are the same, the USB touch ID cannot be associated with the HDMI input, resulting in the inability to achieve touch control capabilities for dual-screen simultaneous display.

[0004] Therefore, existing technologies still need to be improved and enhanced. Summary of the Invention

[0005] The technical problem to be solved by this application is to provide a multi-screen touch association method and device based on the openharmoney system, which addresses the shortcomings of the existing technology.

[0006] To address the aforementioned technical problems, the first aspect of this application provides a multi-screen touch association method based on the OpenHarmony system, wherein the multi-screen touch association method based on the OpenHarmony system specifically includes:

[0007] Listen for all touch screen hardware events and all input hardware events;

[0008] Retrieve unbound touch hardware events from all touch hardware events and unbound input hardware events from all input hardware events, and read the discovery order of each unbound touch hardware event and each unbound input hardware event;

[0009] The unbound touch hardware events and unbound input hardware events are associated in the order of discovery to achieve multi-screen touch association.

[0010] The multi-screen touch association method based on the OpenHarmony system, before listening to all touch screen hardware events and all input hardware events, further includes:

[0011] Check whether a USB device is plugged into the touch hardware interface of each touch screen;

[0012] When a USB device is detected to be plugged into the touch hardware interface, listen for all touch screen hardware events and all input hardware events.

[0013] The multi-screen touch association method based on the OpenHarmony system, wherein the discovery order of reading each unbound touch hardware event and each unbound input hardware event specifically includes:

[0014] Receive user input for multi-screen touch-related operations;

[0015] If a multi-screen touch association operation is received within a preset time, then multi-screen touch association is performed according to the multi-screen touch association operation, wherein the multi-screen touch association operation includes the association relationship between touch hardware events and input hardware events;

[0016] If no multi-screen touch association operation is received within the preset time, the discovery order of each unbound touch hardware event and each unbound input hardware event is read.

[0017] The multi-screen touch association method based on the OpenHarmony system, wherein the monitoring of all touch screen hardware events and all input hardware events specifically refers to:

[0018] The multi-mode component in the framework layer sends touch screen listening instructions to the touch screen driver in the driver layer, wherein the touch screen listening instructions are used to listen for all touch screen hardware events.

[0019] The multi-mode component in the framework layer sends input listening instructions to the input driver in the driver layer, wherein the input listening instructions are used to listen for all input hardware events.

[0020] The multi-screen touch association method based on the OpenHarmony system, wherein the discovery order is the recognition time order of touch hardware events and input hardware events.

[0021] The multi-screen touch association method based on the OpenHarmony system, wherein associating each unbound touch hardware event and each unbound input hardware event according to the discovery order specifically includes:

[0022] The unbound touch hardware events and unbound input hardware events are sorted according to the order of discovery to form a hardware event sequence;

[0023] Select the first target hardware event in the hardware event sequence;

[0024] Select the target associated hardware event corresponding to the target hardware event from the hardware event sequence, wherein when the target hardware event is a touch hardware event, the target associated hardware event is an input hardware event, and when the target hardware event is an input hardware event, the target associated hardware event is a touch hardware event;

[0025] Associate the target hardware event with the target associated hardware event, and remove the target hardware event and the target associated hardware event from the hardware event sequence to update the hardware event sequence;

[0026] The updated hardware event sequence is used as the hardware event sequence, and the target hardware event that is the first in the hardware event sequence is re-executed until the updated hardware event sequence is empty.

[0027] The multi-screen touch association method based on the OpenHarmony system, wherein after associating each unbound touch hardware event and each unbound input hardware event according to the discovery order to achieve multi-screen touch association, the method further includes:

[0028] When it is detected that the access device has been unplugged, the touch screen hardware events and / or input hardware events corresponding to the access device are unbound.

[0029] The second aspect of this application provides a multi-screen touch association device based on the OpenHarmony system, wherein the multi-screen touch association device based on the OpenHarmony system specifically includes:

[0030] The listening module is used to listen for all touch screen hardware events and all input hardware events.

[0031] The acquisition module is used to acquire unbound touch hardware events from all touch hardware events and unbound input hardware events from all input hardware events, and read the discovery order of each unbound touch hardware event and each unbound input hardware event;

[0032] The association module is used to associate each unbound touch hardware event and each unbound input hardware event according to the discovery order, so as to realize multi-screen touch association.

[0033] A third aspect of this application provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps in the multi-screen touch association method based on the openharmoney system as described above.

[0034] A fourth aspect of this application provides a terminal device, which includes: a processor and a memory;

[0035] The memory stores a computer-readable program that can be executed by the processor;

[0036] When the processor executes the computer-readable program, it implements the steps in any of the above-described multi-screen touch association methods based on the OpenHarmony system.

[0037] Beneficial Effects: Compared with existing technologies, this application provides a multi-screen touch association method and device based on the OpenHarmony system. The method includes listening to all touch screen hardware events and all input hardware events; obtaining unbound touch hardware events from all touch hardware events and unbound input hardware events from all input hardware events, and reading the discovery order of each unbound touch hardware event and each unbound input hardware event; associating each unbound touch hardware event and each unbound input hardware event according to the discovery order to achieve multi-screen touch association. This application uses an automatic listening method to determine each unbound touch hardware event and each unbound input hardware event, and then dynamically matches each unbound touch hardware event and each unbound input hardware event based on the discovery order, realizing automatic association between USB access touchID and HDMI input, and realizing touch capability for dual-screen display. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a schematic diagram of an application framework provided in an embodiment of this application.

[0040] Figure 2 A flowchart illustrating a multi-screen touch association method based on the OpenHarmony system provided in this application embodiment.

[0041] Figure 3 This is a timing diagram of a multi-screen touch association method based on the OpenHarmony system provided in an embodiment of this application.

[0042] Figure 4 This is a diagram illustrating the relationships between different relationships.

[0043] Figure 5 This is an illustration to prompt the user to confirm the association relationship.

[0044] Figure 6This is a schematic diagram of a multi-screen touch association device based on the OpenHarmony system provided in the embodiments of this application.

[0045] Figure 7 A schematic block diagram of the terminal device provided in the embodiments of this application. Detailed Implementation

[0046] This application provides a multi-screen touch association method and device based on the OpenHarmony system. To make the purpose, technical solution, and effects of this application clearer and more explicit, the following describes this application in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining this application and are not intended to limit this application.

[0047] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0048] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0049] It should be understood that the sequence number and size of each step in this embodiment do not imply the order of execution. The execution order of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of this application embodiment.

[0050] Research has revealed that with the rapid development of industries such as video media, dual-screen video players have emerged. Customers can create content for two screens according to their needs, playing content suitable for each screen's display mode, such as playing video on the top and images on the bottom. Currently, for multi-HDMI screens with touch capabilities, the USB touch ID and HDMI input of the HDMI screen are two separate hardware ports. Therefore, when the USB PID and VID are the same, the USB touch ID cannot be associated with the HDMI input, preventing the implementation of touch control capabilities for dual-screen display.

[0051] To address the aforementioned issues, this application embodiment monitors all touch screen hardware events and all input hardware events; acquires unbound touch hardware events from all touch hardware events and unbound input hardware events from all input hardware events, and reads the discovery order of each unbound touch hardware event and each unbound input hardware event; and associates each unbound touch hardware event and each unbound input hardware event according to the discovery order to achieve multi-screen touch association. This application uses an automatic monitoring method to determine each unbound touch hardware event and each unbound input hardware event, and then dynamically matches each unbound touch hardware event and each unbound input hardware event based on the discovery order, realizing automatic association between USB access touch ID and HDMI input, and achieving touch capability for dual-screen display.

[0052] The application content will be further explained below with reference to the accompanying drawings and the description of the embodiments.

[0053] This embodiment provides a multi-screen touch association method based on the OpenHarmony system, such as... Figure 1 As shown, the application framework of the multi-screen touch association method can include an application layer, a framework layer, and a driver layer. The application layer is used to connect with screen interaction applications, the framework layer is used to provide interaction interfaces for the application layer, and the driver layer is used to drive touch hardware events and input hardware events to access the touch screen hardware. The application layer deploys multiple interactive applications, each corresponding to a different touch screen. The framework layer deploys a multi-mode component, a window component, and a USB component. The multi-mode component is used to implement multi-screen display, the window component is used to access input hardware events, and the USB component is used to access touch hardware events. The driver layer includes an input driver component and a USB driver component. The input driver component is used to drive input hardware events to access the touch screen, and the USB driver component is used to drive touch hardware events to access the touch screen.

[0054] Multi-mode components are applied to standard systems to provide devices with single-finger touch input capabilities. Specifically, the multi-mode component reports hardware events generated by touch input (including touch hardware events and input hardware events) to the JS UI framework or user application framework. The JS UI framework then re-encapsulates the reported events and provides interfaces to the interactive application. Furthermore, when the application framework connects to multiple touch screen devices, the multi-mode component interacts with the driver layer using a listening mechanism. This mechanism enables automatic recognition and dynamic binding of USB-connected devices to the connected input screen.

[0055] Specifically, such as Figure 2 and Figure 3 As shown, the multi-screen touch association method based on the OpenHarmony system specifically includes:

[0056] S10: Listen for all touch screen hardware events and all input hardware events.

[0057] Specifically, the touch screen hardware events are events triggered by the touch screen's touch hardware interface, and the input hardware events are events triggered by the touch screen's input interface. The touch hardware interface can be a USB interface, and the input interface can be an input interface. In other words, the touch screen hardware events are events triggered by the USB interface, and the input hardware events are events triggered by the input interface.

[0058] Furthermore, the listening operation for all touch screen hardware events and all input hardware events can be continuous, or it can be performed at preset time intervals, or it can be triggered based on preset triggering conditions. In this embodiment, the listening operation is triggered based on the touch hardware interface being occupied. Therefore, before listening to all touch screen hardware events and all input hardware events, the method further includes:

[0059] Check whether a USB device is plugged into the touch hardware interface of each touch screen;

[0060] When a USB device is detected to be plugged into the touch hardware interface, listen for all touch screen hardware events and all input hardware events.

[0061] Specifically, the USB driver can be used to identify whether a USB device is plugged in. In other words, the USB driver can detect whether a USB device is plugged into the touch hardware interface. When a USB device is detected, it means that multi-screen touch association is needed. To do this, a listening operation will be performed, which listens for all touch screen hardware events and all input hardware events. Conversely, when no USB device is detected, it means that multi-screen touch association is not needed. To do this, the driver can continue to detect whether a USB device is plugged into the touch hardware interface of each touch screen.

[0062] Furthermore, touch screen hardware events and input hardware events can be handled through multi-modal components in the framework layer. In other words, listening to all touch screen hardware events and all input hardware events specifically involves:

[0063] The multi-mode component in the framework layer sends touch screen listening instructions to the touch screen driver in the driver layer, wherein the touch screen listening instructions are used to listen for all touch screen hardware events.

[0064] The multi-mode component in the framework layer sends input listening instructions to the input driver in the driver layer, wherein the input listening instructions are used to listen for all input hardware events.

[0065] Specifically, the driver layer includes a USB driver component and an input driver component. The multi-mode component interacts with both the USB driver component and the input driver component, sending listening commands to each. Specifically, it sends a touchscreen listening command to the USB driver component and an input listening command to the input driver component. The touchscreen listening command listens for all touchscreen hardware events, and the input listening command listens for all input hardware events. In other words, the multi-mode component sends a touchscreen listening command to the USB driver component to receive feedback on all touchscreen hardware events, and sends an input listening command to the input driver component to receive feedback on all input hardware events.

[0066] Based on this, such as Figure 3 As shown, the multi-mode component listens to all touch screen hardware events connected to the USB driver component and simultaneously listens to all input hardware events connected to the input driver component. The USB driver component reports all touch screen hardware events to the multi-mode component, and each touch screen hardware event carries a touch IP (i.e., touch ID). At the same time, the input driver component also reports input hardware events to the multi-mode component. This embodiment of the application obtains all touch screen hardware events and all input hardware events through the interaction between the multi-mode component and the USB driver component and the input driver component in the driver layer, thus realizing the listening of touch screen hardware events and input hardware events.

[0067] S20. Obtain unbound touch hardware events from all touch hardware events and unbound input hardware events from all input hardware events, and read the discovery order of each unbound touch hardware event and each unbound input hardware event.

[0068] Specifically, unbound touch hardware events refer to touch hardware events that are not bound to input hardware events, and unbound input hardware events refer to input hardware events that are not bound to touch hardware events. In other words, both unbound touch hardware events and unbound input hardware events are events that have been connected but not bound.

[0069] Furthermore, upon acquiring all unbound touch hardware events and all unbound input hardware events, the system can detect whether the PID (device identifier determined by the vendor) and VID (device identifier applied for by the vendor from the USB Execution Forum; each vendor's VID is unique) are the same. When the PID and VID are different, the unbound touch hardware events and unbound input hardware events are directly associated based on the PID and VID. When the PID and VID are the same, the system cannot associate the unbound touch hardware events and unbound input hardware events based on the PID and VID. Instead, it reads the discovery order of each unbound touch hardware event and each unbound input hardware event and associates them based on the discovery order.

[0070] Before reading the discovery order of each unbound touch hardware event and each unbound input hardware event, it is also possible to detect whether the user has specified the association relationship between the unbound touch hardware events and each unbound input hardware event. If specified, the unbound touch hardware events and unbound input hardware events are associated according to the specified association relationship, for example, such as... Figure 5 As shown, when multiple touch devices are detected, the interface displays "Multiple touch devices detected, please click the OK button to associate them." Then, upon receiving a confirmation command triggered by the "OK" button, the system receives user input based on that command. Figure 4 The preset association relationship shown is the user setting association relationship selected, and the unbound touch hardware events and unbound input hardware events are associated according to the association relationship.

[0071] If not specified, the discovery order of each unbound touch hardware event and each unbound input hardware event is read, and the unbound touch hardware events and unbound input hardware events are associated based on the discovery order. Furthermore, in practical applications, to improve association efficiency and avoid association failures due to users' lack of understanding of how to associate them, interaction with the user can be used to directly associate the unbound touch hardware events and unbound input hardware events according to the discovery order.

[0072] For example, the discovery order of reading each unbound touch hardware event and each unbound input hardware event specifically includes:

[0073] Receive user input for multi-screen touch-related operations;

[0074] If a multi-screen touch association operation is received within a preset time, then multi-screen touch association is performed according to the multi-screen touch association operation;

[0075] If no multi-screen touch association operation is received within the preset time, the discovery order of each unbound touch hardware event and each unbound input hardware event is read.

[0076] Specifically, multi-screen touch association operations are used to control multi-screen touch associations. These operations include the association relationships between touch hardware events and input hardware events. In other words, users specify the association relationships between unbound touch hardware events and various unbound input hardware events by inputting multi-screen touch association operations. These operations can be triggered based on the user-inputted association relationships, or they can be pre-displayed to the user, and then the user's selection is received to form the multi-screen touch association operation. For example, as... Figure 4 As shown, several connection relationships can be displayed to the user, such as connecting HDMI screen 1 first, connecting HDMI screen 2 first, and cross-connection. Connecting HDMI screen 1 first includes connecting input1 cable - connecting USB1 cable, connecting input2 cable - connecting USB2 cable; connecting HDMI screen 2 first includes connecting input2 cable - connecting USB2 cable, connecting input2 cable - connecting USB2 cable; and cross-connection includes connecting USB1 cable of HDMI screen 1 - connecting input2 cable of HDMI screen 2, connecting input1 cable of HDMI screen 1 - connecting USB2 cable of HDMI screen 2, connecting USB2 cable of HDMI screen 2 - connecting input1 cable of HDMI screen 1, and connecting input2 cable of HDMI screen 2 - connecting USB1 cable of HDMI screen 1.

[0077] Furthermore, the discovery order refers to the order in which touch hardware events and input hardware events are identified, that is, the connection time of the touch screen's USB interface and input interface to the device. Of course, in practical applications, the discovery order can be other orders, such as the listening order of the multi-mode component detecting touch hardware events and input hardware events. In this embodiment, the discovery order is the order in which touch hardware events and input hardware events are identified. That is, when the USB driver component and the input driver component feed back touch hardware events and input hardware events to the multi-mode component, they configure an identification time for each touch hardware event and input hardware event, so that the multi-mode component can determine the discovery order of each unbound touch hardware event and each unbound input hardware event based on the configured identification time.

[0078] S30. Associate each unbound touch hardware event and each unbound input hardware event according to the discovery order to achieve multi-screen touch association.

[0079] Specifically, associating each unbound touch hardware event and each unbound input hardware event according to the discovery order refers to associating the access devices of the USB interface of each touch screen (such as an HDMI screen) with the access devices of the input interface, and in accordance with the access time of the access devices. The associated USB interface and input interface can belong to the same touch screen or be on different touch screens. For example, the USB interface of HDMI screen 1 is associated with the input interface of HDMI screen 1, or the USB interface of HDMI screen 1 is associated with the input interface of HDMI screen 2, etc.

[0080] For example, associating each unbound touch hardware event and each unbound input hardware event according to the discovery order specifically includes:

[0081] The unbound touch hardware events and unbound input hardware events are sorted according to the order of discovery to form a hardware event sequence;

[0082] Select the first target hardware event in the hardware event sequence;

[0083] Select the target associated hardware event corresponding to the target hardware event from the hardware event sequence;

[0084] Associate the target hardware event with the target associated hardware event, and remove the target hardware event and the target associated hardware event from the hardware event sequence to update the hardware event sequence;

[0085] The updated hardware event sequence is used as the hardware event sequence, and the target hardware event that is the first in the hardware event sequence is re-executed until the updated hardware event sequence is empty.

[0086] Specifically, the hardware event sequence is ordered from front to back according to the discovery order. That is, for two adjacent hardware events in the hardware event sequence, the hardware event that comes first was discovered earlier than the hardware event that comes later. The target hardware event in the first position refers to the earliest discovered hardware event, which can be a touch hardware event or an input access event.

[0087] The target associated hardware event is a hardware event used to associate with a target hardware event. The event type of the target hardware event differs from the event type of the target managed hardware event. Specifically, when the target hardware event is a touch hardware event, the target associated hardware event is an input hardware event; conversely, when the target hardware event is an input hardware event, the target associated hardware event is a touch hardware event. This achieves the association between touch hardware events and input hardware events. Furthermore, in practical applications, when selecting the target associated hardware event corresponding to the target hardware event from the hardware event sequence, the association type of the hardware event can be determined first, and then the target associated hardware event can be selected based on the association type.

[0088] The association type can be pre-set or obtained through user interaction. The association type can be same-screen association, cross-association, or arbitrary association. Specifically, when the association type is same-screen association, after obtaining the target hardware event, the touchscreen corresponding to the target hardware event is first determined. Then, the first hardware event of that touchscreen is selected from the hardware event sequence in descending order. The event type of the first hardware event is different from the event type of the target hardware event. When the first hardware event is selected, it is used as the target associated hardware event of the target hardware event. Conversely, when the first hardware event is not selected, the target hardware event is moved to the waiting queue. Then, when the hardware event sequence is empty, the waiting queue is used as the hardware event sequence to re-execute the steps of associating each unbound touch hardware event and each unbound input hardware event in the discovery order, until the number of executions reaches a preset threshold. Finally, the hardware events in the waiting queue are fed back to the user with an association failure message.

[0089] When the association type is cross association, after obtaining the target hardware event, the touch screen corresponding to the target hardware event will be determined first. Then, other touch screen second hardware events will be selected in the hardware event sequence in order from first to last. The event type of the second hardware event is different from that of the target hardware event. Then, the subsequent processing will be carried out according to the above-mentioned processing methods for selected and unselected events, which will not be elaborated here.

[0090] When the association type is cross-association, after obtaining the target hardware event, a third hardware event is selected from the hardware event sequence in descending order. The event type of the third hardware event is different from that of the target hardware event. Then, subsequent processing is performed according to the methods described above for selected and unselected events, which will not be elaborated here. It should be noted that the third hardware event may or may not belong to the same touchscreen as the target hardware event.

[0091] In one implementation, after associating each unbound touch hardware event and each unbound input hardware event in the discovery order to achieve multi-screen touch association, the access status of each hardware interface and input interface can be monitored. When the access device connected to a hardware interface or input interface is detected to be disconnected, the touch hardware events and / or input hardware events corresponding to that access device will be unbound. Therefore, after associating each unbound touch hardware event and each unbound input hardware event in the discovery order to achieve multi-screen touch association, the method further includes:

[0092] When it is detected that the access device has been unplugged, the touch screen hardware events and / or input hardware events corresponding to the access device are unbound.

[0093] Specifically, unbinding the touch screen hardware event and / or input hardware event corresponding to the access device means unbinding the hardware event corresponding to the access device and also unbinding the associated hardware events. For example, when the hardware event corresponding to the access device is a touch screen hardware event, the touch screen hardware event and the input hardware event associated with it are unbound simultaneously; when the hardware event corresponding to the access device is an input hardware event, the input hardware event and the touch screen hardware event associated with it are unbound simultaneously.

[0094] In summary, this embodiment provides a multi-screen touch association method and device based on the OpenHarmony system. The method includes listening to all touch screen hardware events and all input hardware events; obtaining unbound touch hardware events from all touch hardware events and unbound input hardware events from all input hardware events, and reading the discovery order of each unbound touch hardware event and each unbound input hardware event; associating each unbound touch hardware event and each unbound input hardware event according to the discovery order to achieve multi-screen touch association. This application uses an automatic listening method to determine each unbound touch hardware event and each unbound input hardware event, and then dynamically matches each unbound touch hardware event and each unbound input hardware event based on the discovery order, realizing automatic association between USB access touch ID and HDMI input, and realizing touch capability for dual-screen display.

[0095] Based on the above-described multi-screen touch association method using the OpenHarmony system, this embodiment provides a multi-screen touch association device using the OpenHarmony system, such as... Figure 6 As shown, the multi-screen touch association device based on the OpenHarmony system specifically includes:

[0096] The listening module is used to listen for all touch screen hardware events and all input hardware events.

[0097] The acquisition module is used to acquire unbound touch hardware events from all touch hardware events and unbound input hardware events from all input hardware events, and read the discovery order of each unbound touch hardware event and each unbound input hardware event;

[0098] The association module is used to associate each unbound touch hardware event and each unbound input hardware event according to the discovery order, so as to realize multi-screen touch association.

[0099] The multi-screen touch association device based on the OpenHarmony system further includes:

[0100] The detection module is used to detect whether a USB device is plugged into the touch hardware interface of each touch screen;

[0101] The execution module is used to call the listening module to listen for all touch screen hardware events and all input hardware events when a USB device is detected inserted into the touch hardware interface.

[0102] The multi-screen touch association device based on the OpenHarmony system, wherein the acquisition module includes:

[0103] The listening unit is used to receive user input for multi-screen touch-related operations.

[0104] The first processing unit is configured to perform multi-screen touch association according to the multi-screen touch association operation if a multi-screen touch association operation is received within a preset time. The multi-screen touch association operation includes the association relationship between touch hardware events and input hardware events.

[0105] The second processing unit is used to read the discovery order of each unbound touch hardware event and each unbound input hardware event if no multi-screen touch association operation is received within a preset time.

[0106] The aforementioned multi-screen touch association device based on the OpenHarmony system, wherein the monitoring module is specifically used for:

[0107] The multi-mode component in the framework layer sends touch screen listening instructions to the touch screen driver in the driver layer, wherein the touch screen listening instructions are used to listen for all touch screen hardware events.

[0108] The multi-mode component in the framework layer sends input listening instructions to the input driver in the driver layer, wherein the input listening instructions are used to listen for all input hardware events.

[0109] The multi-screen touch association device based on the OpenHarmony system, wherein the discovery order is the recognition time order of touch hardware events and input hardware events.

[0110] The multi-screen touch association device based on the OpenHarmony system, wherein the association module specifically includes:

[0111] The sorting unit is used to sort each unbound touch hardware event and each unbound input hardware event according to the discovery order to form a hardware event sequence;

[0112] The first selection unit is used to select the first target hardware event in the hardware event sequence;

[0113] The second selection unit is used to select a target associated hardware event corresponding to the target hardware event from the hardware event sequence, wherein when the target hardware event is a touch hardware event, the target associated hardware event is an input hardware event, and when the target hardware event is an input hardware event, the target associated hardware event is a touch hardware event;

[0114] An association unit is configured to associate the target hardware event with the target associated hardware event, and remove the target hardware event and the target associated hardware event from the hardware event sequence to update the hardware event sequence;

[0115] The execution unit is used to take the updated hardware event sequence as the hardware event sequence and re-execute the target hardware event that is the first one in the hardware event sequence until the updated hardware event sequence is empty.

[0116] The multi-screen touch association device based on the OpenHarmony system further includes:

[0117] The unbinding module is used to unbind the touch screen hardware events and / or input hardware events corresponding to the access device when it is detected that the access device has been unplugged.

[0118] Based on the above-described multi-screen touch association method based on the OpenHarmony system, this embodiment provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the steps in the multi-screen touch association method based on the OpenHarmony system as described in the above embodiment.

[0119] Based on the above-described multi-screen touch association method using the OpenHarmony system, this application also provides a terminal device, such as... Figure 7 As shown, it includes at least one processor 20; a display screen 21; and a memory 22, and may also include a communication interface 23 and a bus 24. The processor 20, display screen 21, memory 22, and communication interface 23 can communicate with each other via the bus 24. The display screen 21 is configured to display a preset user guide interface in the initial setup mode. The communication interface 23 can transmit information. The processor 20 can invoke logical instructions in the memory 22 to execute the methods described in the above embodiments.

[0120] Furthermore, the logical instructions in the aforementioned memory 22 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0121] The memory 22, as a computer-readable storage medium, can be configured to store software programs, computer-executable programs, such as program instructions or modules corresponding to the methods in the embodiments of this disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions, or modules stored in the memory 22, thereby implementing the methods in the above embodiments.

[0122] The memory 22 may include a program storage area and a data storage area. The program storage area may store the operating system and application programs required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 22 may include high-speed random access memory (RAM) and non-volatile memory. Examples include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, as well as transient storage media.

[0123] Furthermore, the specific process of loading and executing multiple instruction processors in the aforementioned storage medium and terminal device has been described in detail in the above method, and will not be repeated here.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A multi-screen touch association method based on an openharmony system, characterized in that, The multi-screen touch association method based on the openharmony system specifically comprises the following steps: All touch screen hardware events and all input hardware events are listened to; Unbound touch hardware events in all touch hardware events and unbound input hardware events in all input hardware events are obtained, and the discovery order of each unbound touch hardware event and each unbound input hardware event is read; According to the discovery order, each unbound touch hardware event and each unbound input hardware event are associated to realize multi-screen touch association; According to the discovery order, each unbound touch hardware event and each unbound input hardware event are sorted to form a hardware event sequence; A target hardware event located at the first position in the hardware event sequence is selected; In the hardware event sequence, a target associated hardware event corresponding to the target hardware event is selected, wherein when the target hardware event is a touch hardware event, the target associated hardware event is an input hardware event, and when the target hardware event is an input hardware event, the target associated hardware event is a touch hardware event; The target hardware event and the target associated hardware event are associated, and the target hardware event and the target associated hardware event are removed from the hardware event sequence to update the hardware event sequence; The updated hardware event sequence is taken as a hardware event sequence, and the selection of the target hardware event located at the first position in the hardware event sequence is re-executed until the updated hardware event sequence is empty; After the association of each unbound touch hardware event and each unbound input hardware event according to the discovery order to realize multi-screen touch association, the method further comprises: When it is identified that the access device is unplugged, the touch screen hardware event and / or the input hardware event corresponding to the access device are unbound. Before listening to all touch screen hardware events and all input hardware events, the method further comprises:

2. The multi-screen touch association method based on the openharmony system according to claim 1, characterized in that, Detecting whether a touch hardware interface of each touch screen is inserted into a USB device; When it is detected that the touch hardware interface is inserted into the USB device, all touch screen hardware events and all input hardware events are listened to. The reading of the discovery order of each unbound touch hardware event and each unbound input hardware event specifically comprises:

3. The method of claim 1, wherein the openharmony system is used. Receiving a multi-screen touch association operation input by a user; If the multi-screen touch association operation is received within a preset time, the multi-screen touch association is performed according to the multi-screen touch association operation, wherein the multi-screen touch association operation comprises an association relationship between a touch hardware event and an input hardware event; If the multi-screen touch association operation is not received within a preset time, the discovery order of each unbound touch hardware event and each unbound input hardware event is read. The listening to all touch screen hardware events and all input hardware events specifically comprises:

4. The method of claim 1, wherein the openharmony system is used. Sending a touch screen listening instruction to a touch screen driver of a driver layer through a multi-module component of a framework layer, wherein the touch screen listening instruction is used to listen to all touch screen hardware events; Sending an input listening instruction to an input driver of the driver layer through the multi-module component of the framework layer, wherein the input listening instruction is used to listen to all input hardware events. ​ 5. The method according to claim 1, wherein, The discovery sequence is the identification time sequence in which the touch hardware event and the input hardware event are identified.

6. A multi-screen touch association device based on an openharmony system, characterized in that, The multi-screen touch association device based on the openharmony system comprises: A listening module is configured to listen to all touch screen hardware events and all input hardware events. An acquisition module is configured to acquire unbound touch hardware events from all touch hardware events and unbound input hardware events from all input hardware events, and read the discovery sequence of each unbound touch hardware event and each unbound input hardware event. An association module is configured to associate each unbound touch hardware event and each unbound input hardware event according to the discovery sequence to realize multi-screen touch association. The association module is configured to associate each unbound touch hardware event and each unbound input hardware event according to the discovery sequence to realize multi-screen touch association. The association module is configured to associate each unbound touch hardware event and each unbound input hardware event according to the discovery sequence to realize multi-screen touch association. The association module is configured to associate each unbound touch hardware event and each unbound input hardware event according to the discovery sequence to realize multi-screen touch association. The association module is configured to associate each unbound touch hardware event and each unbound input hardware event according to the discovery sequence to realize multi-screen touch association. The association module is configured to associate each unbound touch hardware event and each unbound input hardware event according to the discovery sequence to realize multi-screen touch association. The association module is configured to associate each unbound touch hardware event and each unbound input hardware event according to the discovery sequence to realize multi-screen touch association. The association module is configured to associate each unbound touch hardware event and each unbound input hardware event according to the discovery sequence to realize multi-screen touch association.

7. A computer-readable storage medium, characterized in that, The association module is configured to associate each unbound touch hardware event and each unbound input hardware event according to the discovery sequence to realize multi-screen touch association.

8. A terminal device, comprising: The computer readable storage medium stores one or more programs that can be executed by one or more processors to implement the steps of the multi-screen touch association method based on the openharmony system according to any one of claims 1-5. It comprises: A processor and a memory; The memory stores a computer readable program that can be executed by the processor; The processor executes the computer readable program to implement the steps of the multi-screen touch association method based on the openharmony system according to any one of claims 1-5.

Images