Touch event processing method, related apparatus, and vehicle

Abstract

The application relates to a touch event processing method, related devices and a vehicle. The method runs on a target terminal, the target terminal is configured with a first operating system and a second operating system; in response to a touch event on the target terminal, the scene mode of the target terminal is determined; and based on the scene mode, at least one of the first operating system and the second operating system to which the touch event is forwarded is selected. The method enables the touch event to be directly distributed and processed in the respective operating system without passing through a centralized event manager, reduces the delay of touch event processing, improves the response speed, significantly improves the user's interactive experience, especially in a multitask concurrent environment, and meanwhile, the distributed design also reduces the risk of single-point failure and enhances the reliability of the system.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, and in particular to a touch event processing method, related device and vehicle. Background Technology

[0002] In existing smart cockpits, two different operating systems are used for the central control and instrument panel. The central control display device needs to display multiple operating systems simultaneously to save hardware costs. In related technologies, display data from one operating system needs to be transmitted to the other operating system for composite display. At the same time, touch events are sent back to the first operating system through the second. This method results in a long transmission link for touch events and a large delay, leading to untimely touch event processing and affecting the user experience. Summary of the Invention

[0003] This application provides a touch event handling method, related device, and vehicle to solve the above-mentioned problems.

[0004] To achieve the above objectives, according to a first aspect of this application, a touch event processing method is provided, the method running on a target terminal, the target terminal being configured with a first operating system and a second operating system;

[0005] In response to a touch event on the target terminal, the scene mode of the target terminal is determined;

[0006] Based on the scenario mode, the touch event is forwarded to at least one of the first operating system and the second operating system.

[0007] Optionally, the step of selecting to forward the touch event to at least one of the first operating system and the second operating system based on the scene mode includes:

[0008] In response to the scene mode being the target scene mode, the touch event is forwarded to the operating system corresponding to the target scene mode.

[0009] Optionally, the step of forwarding the touch event to the operating system corresponding to the target scene mode in response to the scene mode being a target scene mode includes:

[0010] According to the preset mapping relationship, the touch event is forwarded to the operating system corresponding to the target scene mode.

[0011] The preset mapping relationship indicates the preset operating system corresponding to each of the multiple preset target scene modes.

[0012] Optionally, the target scene mode includes a low-power scene mode and / or a game scene mode.

[0013] In response to the target scene mode being a low-power scene mode, the touch event is forwarded to the first operating system;

[0014] In response to the target scene mode being a game scene mode, the touch event is forwarded to the second operating system.

[0015] Optionally, the method further includes:

[0016] In response to the scenario mode being a non-target scenario mode, the touch event is forwarded to at least one of the first operating system and the second operating system based on the display window on the target terminal.

[0017] Optionally, multiple display windows are displayed on the same display device of the target terminal.

[0018] The step of forwarding the touch event to at least one of the first operating system and the second operating system based on the display window on the target terminal includes:

[0019] Based on the touch location of the touch event on the target terminal, determine the target display window where the touch event is located;

[0020] In response to the target display window being a display window under the first operating system, the touch event is forwarded to the first operating system;

[0021] In response to the target display window being a display window under the second operating system, the touch event is forwarded to the second operating system.

[0022] Optionally, the multiple display windows are displayed on different display devices of the target terminal.

[0023] The step of forwarding the touch event to at least one of the first operating system and the second operating system based on the display window on the target terminal includes:

[0024] Based on the touch location of the touch event on the target terminal, the target display device where the touch event occurred is determined;

[0025] In response to the target display device displaying a display window under the first operating system, the touch event is forwarded to the first operating system;

[0026] In response to the target display device displaying a display window under the second operating system, the touch event is forwarded to the second operating system.

[0027] Optionally, the method further includes:

[0028] The scene mode is determined based on the scene identification information on the target terminal.

[0029] Optionally, the method further includes:

[0030] Determine the number of operating systems currently running on the target terminal;

[0031] Based on the number of operating systems currently running, the touch event is forwarded to at least one of the first and second operating systems.

[0032] Optionally, forwarding the touch event to at least one of the first and second operating systems based on the number of running operating systems includes:

[0033] If the number is one, the touch event is forwarded to the running operating system;

[0034] When the number is two or more, the touch event is forwarded to at least one of the first operating system and the second operating system according to the touch location of the touch event.

[0035] Optionally, the touch events detected by the target terminal include touch events captured by at least one display device of the target terminal.

[0036] Optionally, the first operating system and the second operating system each create a corresponding virtual touch module;

[0037] The method further includes:

[0038] The captured touch events are sent to each of the virtual touch modules;

[0039] The configuration parameters of each virtual touch module are determined to enable or disable a preset sending function in multiple virtual touch modules, wherein the preset sending function is used to forward the touch event to the corresponding operating system.

[0040] Optionally, the touch event is the first touch event in a sequence of touch events generated by a single touch operation.

[0041] Optionally, the target terminal is an in-vehicle terminal, the first operating system is used for vehicle control, and the second operating system is used for infotainment and / or smart cockpit.

[0042] Optionally, the first operating system includes at least one of QNX, WinCE, and Linux.

[0043] Optionally, the second operating system includes at least one of Android, HarmonyOS, and iOS.

[0044] According to a second aspect of this application, embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a computer, causes the computer to implement any of the touch event processing methods described in the embodiments of this application.

[0045] According to a third aspect of this application, embodiments of this application also provide a computer program product storing instructions that, when executed by a computer, cause the computer to implement any of the touch event processing methods described in the embodiments of this application.

[0046] According to a fourth aspect of this application, embodiments of this application also provide an electronic device, comprising:

[0047] A memory on which computer programs are stored;

[0048] A processor is configured to execute the computer program in the memory to implement any of the touch event processing methods provided in the embodiments of this application.

[0049] According to a fifth aspect of this application, embodiments of this application also provide a vehicle including the aforementioned electronic equipment.

[0050] Some embodiments in this specification offer at least the following advantages: In related technologies, touch event processing often requires a centralized event manager, increasing processing latency, especially in high-concurrency or multi-tasking environments. This application, through arbitration distribution, enables touch events to be directly distributed and processed within their respective operating systems without going through a centralized manager. This reduces touch event processing latency, improves response speed, and significantly enhances the user experience, particularly in multi-tasking concurrent environments. Furthermore, the distributed design reduces the risk of single points of failure and enhances system reliability.

[0051] Other features and advantages of this application will be described in detail in the following detailed description section. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] To gain a more complete understanding of this application and its beneficial effects, the following description will be provided in conjunction with the accompanying drawings, wherein the same reference numerals in the following description denote the same parts.

[0054] Figure 1 These are application scenario diagrams illustrating the touch event handling methods according to some embodiments of this specification;

[0055] Figure 2 This is an exemplary flowchart of a touch event handling method according to some embodiments of this specification;

[0056] Figure 3 This is an exemplary schematic diagram of a touch event arbitration and distribution method according to some embodiments of this specification;

[0057] Figure 4 This is an exemplary flowchart of a touch event arbitration and distribution method according to some embodiments of this specification;

[0058] Figure 5 This is an exemplary schematic diagram illustrating touch event distribution in a Linux operating system according to some embodiments of this specification;

[0059] Figure 6 This is an exemplary schematic diagram illustrating the distribution of touch events in the Android operating system according to some embodiments of this specification;

[0060] Figure 7 This is an exemplary schematic diagram illustrating touch event arbitration and distribution according to some embodiments of this specification;

[0061] Figure 8 This is an exemplary block diagram of a target terminal shown according to some embodiments of this specification;

[0062] Figure 9 This is a schematic diagram of the structure of an electronic device according to some embodiments of this specification;

[0063] Figure 10 This is an exemplary schematic diagram of a vehicle provided in an exemplary embodiment of this application. Detailed Implementation

[0064] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the protection scope of this application.

[0065] To facilitate understanding of the implementation schemes provided in this application, the relevant application background of the touch event processing method provided in this application will be explained first.

[0066] Currently, in central control displays, different operating systems are displayed as separate layers. By overlaying and resizing these layers, the interfaces of different operating systems can be stitched together, achieving the goal of multiple operating systems sharing the same screen. However, when displaying multiple operating systems simultaneously, the instrument cluster system, which involves driving safety (such as accelerator, brake, and speed control), uses a safe, real-time system, such as a customized Linux system or QNX system. The central control system, on the other hand, requires human-machine interaction and in-vehicle entertainment, and offers a wider range of system options, such as Apple CarPlay, Android, and simplified versions of Windows. However, for driving safety, it is sometimes necessary to project navigation from the central control screen onto the instrument cluster, or to display important information from the central control screen on the instrument cluster. Implementing two independent systems and independent display devices is clearly difficult, and the display may experience delays and stuttering, impacting the user experience.

[0067] In related technologies, in cross-domain scenarios, touch devices are typically bound to a single operating system or application, requiring touch events to be managed through middleware or additional software layers, resulting in significant latency. For example, after being captured, touch events need to be transmitted back from the Linux operating system to the Android operating system, increasing the transmission link length and latency. Furthermore, when the touch screen sampling rate is high, a large amount of touch operation data needs to be transmitted back across domains per second, further increasing processing latency. Additionally, the process of capturing touch events from the Linux operating system and transmitting them to the Android operating system requires secondary distribution processing, increasing complexity and latency.

[0068] In view of this, some embodiments of this specification provide a touch event processing method. By setting up a virtual touch module for each operating system and dynamically distributing touch events across different operating systems based on the virtual touch module, the need for cross-operating system transmission is reduced, and the overhead of transmitting a large number of touch events between different operating systems is avoided, thereby improving system performance. At the same time, the virtual touch module processing simplifies cross-domain ("domain" can refer to operating system) management and improves the scalability and stability of the system.

[0069] The technical solutions disclosed in this specification can be applied to scenarios where multiple operating systems, such as Linux and Android, run simultaneously. The Android operating system runs within a Linux operating system as a container. This Linux operating system can connect to one or more input devices, allowing users to perform corresponding input operations on either the Linux or Android operating systems. These input operations can include input to physical input devices such as a mouse, keyboard, power button, or volume buttons, as well as input to non-physical input devices such as touch or voice input. By virtualizing the input devices, the Linux operating system can achieve unified management of these devices, effectively solving the compatibility issues of input devices when running Android within a Linux operating system.

[0070] Figure 1 This is an application scenario diagram of the touch event handling method shown in some embodiments of this specification.

[0071] like Figure 1 As shown, the touch event handling method may include a vehicle and an electronic device, the electronic device being mounted on the vehicle, for example, the electronic device being mounted (or fixed) to the vehicle via a bracket.

[0072] In some embodiments, the vehicle is communicatively connected to the electronic device. The vehicle can send information to or receive information from the electronic device. For example, if the vehicle and the electronic device are connected via Bluetooth, voice, music, and other data from the electronic device can be transmitted to the vehicle via Bluetooth and played through the vehicle's speakers.

[0073] The vehicle can be a gasoline-powered vehicle, a natural gas-powered vehicle, or a new energy vehicle. New energy vehicles can be pure electric vehicles, hybrid electric vehicles, or range-extended electric vehicles, etc. As an example and not a limitation, the vehicle can be a sedan, truck, motorcycle, bus, recreational vehicle, amusement park vehicle, construction equipment, tram, golf cart, train, etc., and this application does not impose any particular limitation on these categories.

[0074] The electronic device can be a mobile phone, tablet computer, wearable device, vehicle terminal, or other electronic device with display function. This application does not limit the specific type of electronic device.

[0075] It should be noted that the above description of the application scenarios of the touch event handling method is for ease of description only and should not limit this specification to the scope of the embodiments described.

[0076] Figure 2This is an exemplary flowchart of a touch event handling method according to some embodiments of this specification. In some embodiments, process 200 runs on a target terminal, which is configured with a first operating system and a second operating system. Figure 2 As shown, process 200 includes the following steps.

[0077] Step 210: In response to a touch event on the target terminal, determine the scene mode of the target terminal.

[0078] The target terminal typically refers to an electronic device that the user directly operates and interacts with. For example, the target terminal could be an in-vehicle device.

[0079] In some embodiments, the target terminal is an in-vehicle terminal. This in-vehicle terminal runs a first operating system and a second operating system. The first operating system can be a highly secure real-time operating system, such as QNX or Linux, while the second operating system can be a mobile operating system, such as Android. The first and second operating systems share the same screen to simultaneously display instrument panel information, central control information, or entertainment content. The instrument panel information relates to safe driving functions and may include, but is not limited to, speedometer, engine tachometer, fuel level, temperature, headlight indicators, engine status, mileage, and ADAS (Advanced Driving Assistance System) related information. The central control information relates to vehicle status and functions related to vehicle control. Vehicle status may include, but is not limited to, controlling the car's air conditioning and audio system. Functions related to vehicle control include, but are not limited to, navigation, driving recording, music control, air conditioning control, and Bluetooth phone control. Entertainment content is used to meet the user's entertainment needs and includes, but is not limited to, third-party applications downloaded from the internet, such as audio applications, video applications, and instant messaging applications.

[0080] In some embodiments, the first operating system and the second operating system respectively create corresponding virtual touch modules; the method further includes:

[0081] The captured touch events are sent to each virtual touch module;

[0082] Determine the configuration parameters for each virtual touch module to enable or disable the preset sending function in multiple virtual touch modules. The preset sending function is used to forward touch events to the corresponding operating system.

[0083] A virtual touch module is a software-simulated touch device used to simulate touch operations when a corresponding physical touchscreen is not available. Each operating system can create one or more virtual touch modules to handle touch events within that operating system.

[0084] In some implementations, the virtual touch module can receive events from physical touch devices and selectively forward touch events to the corresponding operating system based on configuration parameters.

[0085] In some embodiments, different operating systems create corresponding virtual touch modules in different ways. For example, in the Linux operating system, the corresponding virtual touch module can be created through uinput (user-space input subsystem). uinput allows applications to simulate the behavior of physical input devices, including keyboards, mice, and touchscreens; in the Android operating system, the corresponding virtual touch module can be created and managed by modifying the system kernel or using InputManagerService, etc.

[0086] Configuration parameters are used to control the preset sending function of the virtual touch module. The configuration parameter can be a boolean value (true or false), indicating whether the preset sending function is enabled.

[0087] In some embodiments, the virtual touch module sends the trigger event to the target terminal. The target terminal dynamically adjusts the configuration parameters of the virtual touch module of each operating system according to the scene identification information and the touch position of the touch event, and determines the forwarding path of the touch event based on the configuration parameters of different virtual touch modules.

[0088] In some embodiments, the virtual touch module can determine whether to send the corresponding touch event to the corresponding operating system based on the updated configuration parameters.

[0089] In some embodiments, the user's touch operation can be captured by the physical touch device of the display device to obtain raw touch data, including but not limited to touch point coordinates, pressure values, multi-touch information, etc. The raw touch data is encapsulated into raw events, such as by adding metadata (timestamp, type, location, etc.), and the raw events are sent to each virtual touch module. Each virtual touch module adapts and transforms the received raw events according to the preset format of its corresponding operating system, such as adding additional information or adjusting event attributes to meet the needs of specific application scenarios (e.g., adding gesture recognition results as additional information), and injects the transformed touch events into the corresponding operating system according to the updated configuration parameters, so that the upper-layer application can respond.

[0090] In related technologies, touch devices are typically bound to a single operating system or application. In cross-domain scenarios, middleware or additional software layers are needed to manage the transmission of touch events, leading to decreased system performance and increased complexity. This application virtualizes the physical touch device, enabling a single physical touch device to support multiple virtual operating systems and operate independently within each virtual operating system. The virtual touch module reduces the need for cross-domain transmission, avoiding the overhead of transferring large amounts of data between different operating systems, thereby improving system performance. Simultaneously, virtualization simplifies cross-domain management and enhances system scalability and stability.

[0091] In some embodiments of this specification, a virtual touch module can simulate touch operations when multiple operating systems share a physical touchscreen, allowing multiple operating systems to run on different hardware platforms without relying on a specific physical device. For example, in some embedded systems or virtual machine environments, multiple operating systems share a physical touchscreen, but touch operations on multiple operating systems can still be achieved through the virtual touch module.

[0092] In some embodiments, the target terminal is an in-vehicle terminal, the first operating system is used for vehicle control, and the second operating system is used for infotainment and / or smart cockpit.

[0093] The primary operating system (POS) is the operating system that assumes the main control responsibilities among multiple operating systems. The POS is used to run critical tasks, manage resource allocation, and handle important operations such as user interaction.

[0094] A second operating system is one or more auxiliary operating systems that run beneath a first operating system. The second operating system relies on the infrastructure and services provided by the first operating system and communicates with the first operating system or other second operating systems through specific interfaces.

[0095] Figure 3 This is an exemplary schematic diagram of a touch event arbitration and distribution method according to some embodiments of this specification.

[0096] In some embodiments, such as Figure 3 As shown, a virtualization module runs within the first operating system, supporting the operation of other operating systems (such as a second operating system), such as a hypervisor. A virtualization module is a virtual machine management program, an intermediate layer of software running between the physical server and the operating system, allowing multiple operating systems and applications to share a single set of underlying physical hardware.

[0097] In some embodiments, a second operating system, or other operating systems, may be run on the virtualization module.

[0098] The second operating system and the first operating system can communicate through virtual device interfaces, dedicated message queues, or channels.

[0099] In some embodiments, the second operating system may be one or more.

[0100] In some embodiments, the first operating system includes at least one of QNX, WinCE, and Linux.

[0101] In some embodiments, the second operating system includes at least one of Android, HarmonyOS, and iOS.

[0102] In some embodiments of this specification, the division into a first operating system and a second operating system helps to construct a hierarchical system architecture, where the first operating system is responsible for critical tasks and resource management, while the second operating system focuses on specific application logic. Different types of second operating systems can be optimized for specific tasks; for example, some second operating systems focus on computationally intensive tasks, while others are responsible for real-time interaction, facilitating deployment.

[0103] Touch events refer to information related to interactive actions generated when a user touches the display device of a target terminal with their finger (or other conductive object).

[0104] Touch events include touch events at any location on the screen.

[0105] In some embodiments, in response to a user's click or touch operation, the touch event of the display device can be listened to and sent to the virtual touch module.

[0106] In some implementations, the display device is a device with display and touch functions, such as a touch screen device.

[0107] In some embodiments, a touch event is the first touch event in a sequence of touch events generated by a single touch operation.

[0108] The first touch event refers to the touch event that is pressed down.

[0109] Touch events can be of several types, each indicating the type of action a user performs on the screen. For example, touch event types can include touchstart, touchmove, and touchup events. The touchstart event can also be called the touchdown event. The touchstart event is triggered when a finger presses down on the display device; the touchmove event is triggered when a finger slides across the display device; and the touchup event is triggered when a finger leaves the screen. A window refers to the window where the operating system resides.

[0110] After receiving the above touch event, the target terminal determines the touch location and type of the touch event, and then determines the window where the touch event is located based on the touch location.

[0111] In some embodiments, a user's touch operation can trigger one or a series of touch events on the display device. For example, when a user taps the screen with a single finger, the system generates two consecutive touch events: first, a touchstart event, marking the start of the touch; then, a touchend event, indicating the end of the touch.

[0112] In some embodiments, if the user performs a swipe operation, a sequence of touch events is generated that includes more steps. This sequence begins with a touchstart event, followed by a series of touchmove events that reflect the movement of the finger on the screen, and ends with a touchend event indicating that the touch operation has been completed.

[0113] Regardless of whether the user performs a single-finger or multi-finger operation, and regardless of the specific touch behavior (such as clicking, swiping, zooming, etc.), the resulting touch event will contain at least one touchstart event. In the case of multi-finger operation, there will be a corresponding number of touchstart events to identify the first touch event of each finger.

[0114] In some embodiments of this specification, by extracting and processing the first touch event, it is ensured that every touch interaction of the user can be accurately captured and responded to, thereby providing a smooth and intuitive operating experience.

[0115] A display device is a device that can detect and determine the location of touch events. The display device can convert the user's touch actions into electrical signals and send them to the target terminal for recognition and processing.

[0116] Step 220: Based on the scene mode, select to forward the touch event to at least one of the first operating system and the second operating system.

[0117] Scene mode refers to the preset or custom configuration of the hardware and software functions of the display device according to different usage scenarios, so as to meet the needs of users in specific scenarios.

[0118] In some embodiments of this specification, after detecting a touch event, the target terminal can selectively forward the touch event to at least one of multiple operating systems. This not only improves flexibility and consistency but also simplifies the complexity of cross-domain communication. It is suitable for application scenarios that require multiple operating systems to run on the same hardware platform and have high requirements for real-time performance.

[0119] In some embodiments, based on a scene mode, selecting to forward touch events to at least one of a first operating system and a second operating system includes:

[0120] In response to the target scene mode, the touch event is forwarded to the operating system corresponding to the target scene mode.

[0121] In some embodiments, different target scenario modes correspond to different operating systems.

[0122] In some embodiments, in response to a target scene mode, forwarding touch events to the operating system corresponding to the target scene mode includes:

[0123] Based on the preset mapping relationship, the touch event is forwarded to the operating system corresponding to the target scene mode.

[0124] Among them, the preset mapping relationship indicates the preset operating system corresponding to each of the multiple preset target scene modes.

[0125] The preset mapping relationship is a predefined mapping table used to map different target scene mode information to the corresponding operating system.

[0126] The vehicle can switch to different scene modes according to different driving scenarios and user needs.

[0127] In some embodiments, the target scene mode includes a low-power scene mode and / or a gaming scene mode.

[0128] In response to the target scene mode being a low-power scene mode, the touch event is forwarded to the first operating system;

[0129] In response to the target scene mode being game scene mode, the touch event is forwarded to the second operating system.

[0130] Target scenario mode refers to the scenario mode that meets the usage requirements of a specific operating system.

[0131] In some embodiments, virtual touch modules of other non-first operating systems can be disabled based on the target scenario mode to ensure that all touch events are forwarded to the first operating system; the status flag of the virtual touch module corresponding to the first operating system is updated so that the first operating system responds to the touch event.

[0132] For example, when the vehicle is in low-energy mode or energy-saving mode, by disabling the virtual touch module of the second operating system and shutting down the entertainment application, all touch events are processed in the first operating system, ensuring that critical functions operate while saving energy.

[0133] In low-power mode, all touch events are centrally processed by the primary operating system, reducing the activity of other secondary operating systems and lowering overall power consumption; even in power-saving mode, the primary operating system can independently run critical tasks, ensuring that the basic functions of the system are not affected.

[0134] In some embodiments, the corresponding operating system can be found in a preset mapping relationship according to the target scene mode; the touch event is forwarded to the corresponding operating system, which processes the touch event according to preset processing logic.

[0135] For example, when a vehicle is detected parking and it is determined to be in parking scene mode, the target terminal prioritizes forwarding touch events to the Linux operating system; when a user is detected playing games or playing music / videos on the in-vehicle terminal and it is determined to be in entertainment scene mode, the target terminal prioritizes forwarding touch events to the Android operating system; when a user is detected navigating on the in-vehicle terminal and it is determined to be in navigation scene mode, the target terminal prioritizes forwarding touch events to the Android operating system.

[0136] By distributing touch events according to the target scenario pattern and pre-defined mapping relationships, and ensuring that the events are correctly forwarded to the corresponding operating system, a more intelligent and flexible event distribution mechanism can be achieved, improving user experience and system response speed.

[0137] In some embodiments, the method further includes:

[0138] In response to a scene mode that is not the target scene mode, touch events are forwarded to at least one of the first operating system and the second operating system based on the display window on the target terminal.

[0139] Non-target scenario mode refers to a scenario mode in which multiple operating systems are displayed and operated simultaneously on the same target terminal.

[0140] In some embodiments, non-target scene modes include multi-window scene mode, screen partitioning scene mode, and input separation scene mode.

[0141] In some embodiments, in a multi-window scenario mode, when the display device simultaneously displays the display windows of a first operating system and a second operating system, for example, the display windows of the first operating system and the display windows of the second operating system, the touch event is assigned to the corresponding operating system of the display window by determining the touch position of the touch event without the user noticing, thereby achieving seamless switching of applications in different operating systems.

[0142] In some embodiments, in a screen partitioning scenario, the separated left and right screen areas can run different operating systems. One screen area is used to display entertainment applications, and the other screen area is used to display central control information, avoiding input confusion. For example, the left screen area displays central control information, and the right screen area displays entertainment applications. When the user operates the vehicle from the left, touch input is distributed to the first operating system, while when the user operates an entertainment application (such as playing music) from the right, touch input is distributed to the second operating system.

[0143] The non-target scenario mode allows multiple tasks to be handled simultaneously, improving the user experience.

[0144] In some embodiments of this specification, the screen of the display device can be divided into two or more screen areas, each of which can independently run a different operating system or display different content. In a multi-operating system environment, different operating systems (such as a first operating system and a second operating system) can handle user interaction within their respective screen areas, thereby providing a more efficient and flexible working experience.

[0145] In some embodiments, multiple display windows are displayed on the same display device of the target terminal.

[0146] Forwarding touch events to at least one of a first operating system and a second operating system based on the display window on the target terminal includes:

[0147] Based on the touch location of the touch event on the target terminal, determine the target display window where the touch event is located;

[0148] In response to the target display window being a display window under the first operating system, the touch event is forwarded to the first operating system;

[0149] In response to the target display window being a display window under the second operating system, the touch event is forwarded to the second operating system.

[0150] The target display device refers to the specific display window where the user is currently performing a touch operation.

[0151] In some embodiments, the screen of the display device can simultaneously run multiple operating system display windows or multiple screen areas can simultaneously run multiple operating systems. A unique identifier can be assigned to each display window or screen area, and a mapping relationship between each display window or screen area and the corresponding operating system can be established. Based on the coordinate values ​​(x, y) of the touch event, the display window or screen area where the touch position is located is determined, and the corresponding target operating system is identified. The touch event is forwarded to the target operating system, such as enabling the virtual touch module of the target operating system.

[0152] In some embodiments of this specification, in split-screen or multi-operating system scenarios, when the touch location of a touch event is within the window of a specific operating system (target operating system), the touch event is forwarded to the corresponding target operating system for processing. This ensures that interactive operations within each screen area are accurately processed by the operating system responsible for that area.

[0153] In some embodiments, the multiple display windows are displayed on different display devices of the target terminal.

[0154] Forwarding touch events to at least one of a first operating system and a second operating system based on the display window on the target terminal includes:

[0155] Based on the touch location of the touch event on the target terminal, determine the target display device where the touch event occurred;

[0156] In response to the target display device displaying a display window under the first operating system, the touch event is forwarded to the first operating system;

[0157] In response to the target display device displaying a display window under the second operating system, the touch event is forwarded to the second operating system.

[0158] The target display device refers to the specific display device on which the user is currently performing a touch operation.

[0159] Touch position refers to the specific coordinates of a user's touch on a display device. A touch position can be represented by a pair of coordinate values ​​(x, y), where x represents the horizontal coordinate and y represents the vertical coordinate.

[0160] In some embodiments, the top left corner of the display device's screen is taken as the origin (0, 0), and the bottom right corner is taken as the maximum value (screen width, screen height).

[0161] The target terminal is equipped with at least two independent display devices (such as monitors or touch screens), each of which can independently run display windows of different operating systems to display different content; and each display device corresponds to one or more operating systems (such as a first operating system, a second operating system A, a second operating system B, etc.), each of which can independently run its own operating system and applications in a virtualization environment.

[0162] In some embodiments, a unique identifier can be assigned to each display device, and a mapping relationship can be established between each display device and the operating system. Each display device is configured with a corresponding touch device driver to capture the user's touch operation and generate a raw event containing the coordinate values ​​(x, y) of the touch position. The display device converts the captured raw event through a virtual touch module and forwards it to the target terminal. The target terminal determines the target display device where the touch event is located based on the coordinate values ​​of the touch event, determines the corresponding target operating system, and forwards the touch event to the corresponding operating system.

[0163] In some embodiments of this specification, by identifying the target display device, it is ensured that interactive operations on each display device are accurately handled by the operating system responsible for that display device, which not only improves the responsiveness of the system but also enhances the consistency and smoothness of the user experience.

[0164] In some embodiments, at least two display devices are located on different user sides.

[0165] In some embodiments, in a multi-user environment or multi-user interactive system, each user has their own display device, and different display devices are located in different physical locations or face different users.

[0166] The vehicle features multiple display devices, including a central control screen, a head-up display (HUD), a rear entertainment screen, and a smart social display device. The driver can control vehicle functions through the central control screen, while passengers can watch videos or engage in other entertainment activities through the rear entertainment screen.

[0167] In some embodiments, the method further includes:

[0168] The scene mode is determined based on the scene identification information on the target terminal.

[0169] Scene identification information refers to information used to identify the current application scene. Scene identification information can be a string, number, enumeration value, or other data type, used to distinguish different applications, functions, or user interface states. Scene identification information helps to more accurately understand and process touch events.

[0170] In some embodiments, the current scene mode (such as whether it is in a specific application interface, whether there is a pop-up menu, etc.) can be considered. Different scene modes correspond to different scene identification information, and touch events are sent to the operating system corresponding to the scene identification information. For example, in a game scene, all touch events are forwarded to the operating system where the game application is located first.

[0171] In related technologies, cross-domain touch event processing requires complex protocols and large amounts of data transmission to achieve event synchronization between domains, resulting in poor performance. This application addresses this by using intelligent forwarding to transmit cross-domain touch events only when necessary, and optimizing the transmission path and method to minimize the amount of data transmitted. Intelligent arbitration and distribution further reduces unnecessary cross-domain event transmissions, significantly reducing network bandwidth consumption, improving overall system processing efficiency, and minimizing user operation delays caused by network latency, thus further optimizing the user experience.

[0172] In some embodiments of this specification, by routing touch events, it is possible to avoid sending touch events to unrelated operating systems, thereby reducing unnecessary processing and resource consumption, and significantly improving overall efficiency and response speed.

[0173] In some embodiments, if scene identification information is available, touch events are forwarded to at least one of multiple operating systems based on the scene identification information;

[0174] In the absence of scene identification information, touch events are forwarded to at least one of multiple operating systems based on the touch location.

[0175] In some embodiments, if scene identification information exists, the scene identification information is obtained and a judgment is made. For example, in game mode, touch events are forwarded to the operating system where the game is located first, such as enabling the virtual touch module of the operating system where the game is located.

[0176] In some embodiments, if scene identification information is not available, the touch location of the touch event is determined, and the touch event is forwarded to the operating system corresponding to the window where the touch location is located.

[0177] In some embodiments, the method further includes:

[0178] Determine the number of operating systems currently running on the target terminal;

[0179] Based on the number of operating systems currently running, touch events are forwarded to at least one of the first and second operating systems.

[0180] In some embodiments, forwarding touch events to at least one of a first operating system and a second operating system based on the number of running operating systems includes:

[0181] If there is only one touch event, forward the touch event to the running operating system.

[0182] In the case of two or more touch events, the touch event is forwarded to at least one of the first operating system and the second operating system based on the touch location of the touch event.

[0183] It should be noted that the above description of the process is for illustrative purposes only and does not limit the scope of this specification. Those skilled in the art can make various modifications and changes to the process under the guidance of this specification. However, these modifications and changes remain within the scope of this specification.

[0184] Figure 4 This is an exemplary flowchart of a touch event arbitration and distribution method according to some embodiments of this specification.

[0185] In some embodiments, such as Figure 4 As shown, a touch event arbitration and distribution method specifically includes the following steps:

[0186] S401: Dual operating system sharing physical touch device. In the vehicle, when a user operates the central control screen, a touch event occurs. The central control screen sends the touch event to the virtual touch module connected to it. Touch events can include three main types of events: ACTION_DOWN, ACTION_MOVE, and ACTION_UP, which represent touch operations such as pressing, moving, and lifting the finger, respectively.

[0187] S402: Create a virtual touch module in the Linux operating system. This virtual touch module receives touch events from physical touch devices and distributes them to the upper-layer arbitration and distribution module.

[0188] S403: Create a virtual touch module in the Android operating system. This virtual touch module receives touch events from physical touch devices and distributes them to the upper-layer arbitration and distribution module.

[0189] S404: Arbitration and Distribution Module: After receiving touch events from the virtual touch module of the Linux or Android operating system, it selects and distributes the touch events to different operating systems according to a preset mechanism, such as making a decision based on the touch position of the touch event ACTION_DOWN.

[0190] S405: The input backend of the Linux operating system, used to enable communication between virtual hardware devices and applications. The Linux operating system's input backend converts touch events into the format corresponding to the application, thereby distributing them to the applications within the Linux operating system.

[0191] S406: The input manager of the Android operating system, used to enable communication between virtual hardware devices and applications. The input manager of the Android operating system distributes touch events to the corresponding applications.

[0192] Figure 5 This is an exemplary schematic diagram illustrating the distribution of touch events in the Linux operating system according to some embodiments of this specification.

[0193] In some embodiments, such as Figure 5 As shown, touch event distribution in the Linux operating system specifically includes the following steps:

[0194] S501: Input backend capture event.

[0195] In Linux operating systems, the driver passes captured touch events to an input backend (e.g., Weston's input backend). The input backend is responsible for managing input devices and passing touch events to a pre-configured server (e.g., Wayland server).

[0196] S502: Convert to a preset event.

[0197] The input backend will convert future touch events into preset events (such as Wayland events). Preset events are event types defined in the Wayland protocol, used to transmit touch events between preset servers and clients.

[0198] S503: Pass the preset event to the preset server.

[0199] The input backend will pass the converted preset event to the preset server.

[0200] S504: The default server will pass the default event to the corresponding client.

[0201] The default server will pass the event to the corresponding client based on the type of touch event and the target window.

[0202] S505: Client-side event handling.

[0203] After receiving a preset event, the client will process it accordingly based on the event type.

[0204] Figure 6This is an exemplary schematic diagram illustrating the distribution of touch events in the Android operating system according to some embodiments of this specification.

[0205] In some embodiments, such as Figure 6 As shown, touch event distribution in the Android operating system specifically includes the following steps:

[0206] S601: Read events from the event center.

[0207] In the Android operating system, the event hub (e.g., EventHub) monitors the virtual touch module. When a new touch event occurs, it reads the touch event into user space. User space then reads the touch event from the event hub and parses it into the format corresponding to the Android operating system.

[0208] S602: Input dispatcher dispatches events.

[0209] An input dispatcher (such as InputDispatcher) determines the target window based on the focus of the currently active window, the touch area operating system of the currently active window, and other window properties.

[0210] S603: Window management service receives events.

[0211] Window management services (such as WindowManagerService, WMS) further distribute touch events to target windows (such as Activities) based on the window's focus state and the operating system of the input area.

[0212] S604: View root object handles events.

[0213] The root view object (e.g., ViewRootImpl) passes the touch event to the root view of the window (e.g., Root View) after receiving the touch event.

[0214] S605: View layer event handling.

[0215] The view layer responds to user input from touch devices by overriding event handling methods (such as onTouchEvent, onKeyEvent, etc.).

[0216] Figure 7 This is an exemplary schematic diagram illustrating touch event arbitration distribution according to some embodiments of this specification.

[0217] In some embodiments, such as Figure 7 As shown, the touch event arbitration and distribution process includes the following steps:

[0218] Touch events are generated when a user interacts with a physical touch device (such as a touchscreen). The physical touch device forwards these touch events to an arbitration mechanism through the virtual device layer.

[0219] The arbitration mechanism determines whether scene identification information exists. Different scene identification information corresponds to different operating systems. Scene identification information may include, but is not limited to, energy-saving scenes, screen partitioning scenes, functional scenes, etc.

[0220] The scene identification information indicates a low-power scene mode, and touch events are distributed to the Linux operating system based on the scene identification information.

[0221] The scene identification information is a game scene mode, and touch events are distributed to the Android operating system based on the scene identification information.

[0222] If no scene identifier information is available, determine whether a window for the Linux operating system exists on the display device at the current moment. Based on the determination result, distribute the application to either the Linux or Android operating system.

[0223] If there is no Linux operating system display window on the display device, the virtual touch module of the Linux operating system is directly disabled, and all touch events are distributed by the Android operating system.

[0224] When a Linux operating system display window exists on the display device, touch events are distributed to either the Linux or Android operating system based on the touch location.

[0225] If the touch event occurs within the display window of the Linux operating system, the virtual touch module of the Android operating system is directly disabled, and all touch events are distributed to the Linux operating system.

[0226] If the touch event is not located within the display window of the Linux operating system, the virtual touch module of the Linux operating system is directly disabled, and all touch events are distributed to the Android operating system.

[0227] Through the above mechanism, touch events can be dynamically arbitrated and distributed across different operating systems.

[0228] Figure 8 This is an exemplary block diagram of a target terminal shown according to some embodiments of this specification.

[0229] like Figure 8 As shown, one or more embodiments of this specification also provide a target terminal, which is configured with multiple operating systems and includes an arbitration distribution module.

[0230] The arbitration distribution module can be used to execute the steps in the embodiments corresponding to the above-mentioned touch event handling method. For the specific implementation of these modules and more details, please refer to the corresponding method section, which will not be elaborated here.

[0231] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0232] Figure 9 This is a schematic diagram of the structure of an electronic device according to some embodiments of this specification. For example... Figure 9 As shown, the electronic device 900 may include a processor 901 and a memory 902. The electronic device 900 may also include one or more of a multimedia component 903, an input / output (I / O) component 904, and a communication component 905. In this embodiment, the electronic device 900 may be a device that implements the touch event processing method provided in this embodiment.

[0233] The processor 901 controls the overall operation of the electronic device 900 to complete all or part of the steps in the touch event handling method described above. The memory 902 stores various types of data to support the operation of the auxiliary electronic device 900. This data may include, for example, instructions for any application or method operating on the auxiliary electronic device 900, and application-related data such as contact data, sent and received messages, pictures, audio, video, etc. The memory 902 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 903 may include a screen and audio components. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 902 or transmitted via communication component 905. The audio component also includes at least one speaker for outputting audio signals. I / O component 904 provides an interface between processor 901 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 905 is used for wired or wireless communication between the electronic device 900 and other devices. Wireless communication includes, for example, Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, Narrow Band Internet of Things (NB-IoT), Enhanced Machine Type Communication (eMTC), or other 5G technologies, or combinations thereof, without limitation. Therefore, the corresponding communication component 905 may include: a Wi-Fi module, a Bluetooth module, an NFC module, etc.

[0234] In one exemplary embodiment, the electronic device 900 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the touch event processing method described above.

[0235] In another exemplary embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a computer, implements the steps of the touch event handling method described above.

[0236] In another exemplary embodiment, a computer program product is also provided, the computer program product storing instructions that, when executed by a computer, cause the computer to perform the touch event processing method described above.

[0237] Figure 10 This is an exemplary schematic diagram of a vehicle provided in an exemplary embodiment of this application.

[0238] like Figure 10 As shown, this application also provides a vehicle equipped with the electronic device provided in any of the above embodiments, the electronic device being used to execute the touch event processing method provided in any of the above embodiments. The vehicle may be a gasoline-powered vehicle, a plug-in hybrid electric vehicle, or a new energy vehicle, etc., and this specification does not specifically limit it.

[0239] In one embodiment, the vehicle can be configured for fully or partially autonomous driving. For example, the vehicle can control itself while in autonomous driving mode, and can determine the current state of the vehicle and its surrounding environment through human intervention, determine the possible behaviors of at least one other vehicle in the surrounding environment, and determine the confidence level corresponding to the probability of that other vehicle performing a possible behavior, and control the vehicle based on the determined information. When the vehicle is in autonomous driving mode, it can be configured to operate without human interaction.

[0240] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0241] The embodiments, implementation methods, and related technical features of this application can be combined and substituted for each other without conflict.

[0242] The above are merely preferred embodiments of this application and are not intended to limit this application in any way. Although the descriptions of each embodiment in this application have different focuses, and the parts not described in detail in a certain embodiment can be referred to the relevant embodiments of other embodiments, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of this application without departing from the content of the technical solution of this application shall still fall within the scope of the technical solution of this application.

Claims

1. A touch event handling method, characterized in that, The method runs on a target terminal, which is configured with a first operating system and a second operating system. In response to a touch event on the target terminal, the scene mode of the target terminal is determined; Based on the scenario mode, the touch event is forwarded to at least one of the first operating system and the second operating system.

2. The method according to claim 1, characterized in that, The step of forwarding the touch event to at least one of the first operating system and the second operating system based on the scene mode includes: In response to the scene mode being the target scene mode, the touch event is forwarded to the operating system corresponding to the target scene mode.

3. The method according to claim 2, characterized in that, The step of forwarding the touch event to the operating system corresponding to the target scene mode in response to the scene mode being the target scene mode includes: According to the preset mapping relationship, the touch event is forwarded to the operating system corresponding to the target scene mode. The preset mapping relationship indicates the preset operating system corresponding to each of the multiple preset target scene modes.

4. The method according to claim 2, characterized in that, The target scene modes include low-power scene modes and / or game scene modes. In response to the target scene mode being a low-power scene mode, the touch event is forwarded to the first operating system; In response to the target scene mode being a game scene mode, the touch event is forwarded to the second operating system.

5. The method according to claim 2, characterized in that, The method further includes: In response to the scenario mode being a non-target scenario mode, the touch event is forwarded to at least one of the first operating system and the second operating system based on the display window on the target terminal.

6. The method according to claim 5, characterized in that, Multiple display windows are displayed on the same display device of the target terminal. The step of forwarding the touch event to at least one of the first operating system and the second operating system based on the display window on the target terminal includes: Based on the touch location of the touch event on the target terminal, determine the target display window where the touch event is located; In response to the target display window being a display window under the first operating system, the touch event is forwarded to the first operating system; In response to the target display window being a display window under the second operating system, the touch event is forwarded to the second operating system.

7. The method according to claim 5, characterized in that, The multiple display windows are displayed on different display devices of the target terminal, respectively. The step of forwarding the touch event to at least one of the first operating system and the second operating system based on the display window on the target terminal includes: Based on the touch location of the touch event on the target terminal, the target display device where the touch event occurred is determined; In response to the target display device displaying a display window under the first operating system, the touch event is forwarded to the first operating system; In response to the target display device displaying a display window under the second operating system, the touch event is forwarded to the second operating system.

8. The method according to claim 1, characterized in that, The method further includes: The scene mode is determined based on the scene identification information on the target terminal.

9. The method according to claim 1, characterized in that, The method further includes: Determine the number of operating systems currently running on the target terminal; Based on the number of operating systems currently running, the touch event is forwarded to at least one of the first and second operating systems.

10. The method according to claim 9, characterized in that, The step of forwarding the touch event to at least one of the first operating system and the second operating system based on the number of running operating systems includes: If the number is one, the touch event is forwarded to the running operating system; When the number is two or more, the touch event is forwarded to at least one of the first operating system and the second operating system according to the touch location of the touch event.

11. The method according to claim 1, characterized in that, The touch events detected by the target terminal include touch events captured by at least one display device of the target terminal.

12. The method according to claim 11, characterized in that, The first and second operating systems each create corresponding virtual touch modules; The method further includes: The captured touch events are sent to each of the virtual touch modules; The configuration parameters of each virtual touch module are determined to enable or disable a preset sending function in multiple virtual touch modules, wherein the preset sending function is used to forward the touch event to the corresponding operating system.

13. The method according to any one of claims 1-12, characterized in that, The touch event is the first touch event in the sequence of touch events generated by a single touch operation.

14. The method according to any one of claims 1-12, characterized in that, The target terminal is an in-vehicle terminal, the first operating system is used for vehicle control, and the second operating system is used for infotainment and / or smart cockpit.

15. The method according to claim 14, characterized in that, The first operating system includes at least one of QNX, WinCE, and Linux.

16. The method according to claim 14, characterized in that, The second operating system includes at least one of Android, HarmonyOS, and iOS.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a computer, the computer program causes the computer to implement the touch event handling method according to any one of claims 1 to 16.

18. A computer program product, characterized in that, The computer program product stores instructions that, when executed by a computer, cause the computer to perform the touch event handling method according to any one of claims 1 to 16.

19. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the touch event handling method according to any one of claims 1 to 16.

20. A vehicle, characterized in that, Includes the electronic device as described in claim 19.

Images