Multi-modal input conflict resolution method and system for vehicle-mounted multimedia system

Abstract

The invention discloses a multi-mode input conflict resolution method and system for a vehicle-mounted multimedia system, and relates to the field of vehicle control, and the method comprises the steps: collecting an input event composed of at least two input sources from touch input, voice input, knob input and key input; the input event is subjected to packaging standardization processing, and the packaging standardization processing comprises an input source, an action type, a timestamp and context information corresponding to the input event; acquiring context information corresponding to the input event and a preset priority strategy for processing the input event; performing arbitration according to the context information of the corresponding input event and a preset priority strategy for processing the input event, and determining an execution sequence of the input event; obtaining a current input event according to the determined execution sequence of the input events; and updating and synchronizing the global state machine according to the current input event.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of vehicle control, in particular to a multi-modal input conflict resolution method for a vehicle multimedia system, a multi-modal input conflict resolution system for a vehicle multimedia system, an electronic device, a storage medium and a vehicle. BACKGROUND

[0002] With the development of vehicle multimedia systems, users can operate through various ways such as touch screen, voice command, hardware knob, steering wheel key, etc. However, the prior art has the following problems when dealing with multi-input scenarios:

[0003] Input conflict: when the user uses two input methods almost at the same time (such as voice playing instruction + touch song switching), the system lacks a unified arbitration mechanism, resulting in instruction coverage or function confusion.

[0004] State inconsistency: when multiple inputs are triggered at the same time, the UI display state is inconsistent with the background playing state, such as the UI still showing "playing" after the voice "pause".

[0005] Lack of dynamic priority management: current solutions usually use fixed priorities, but do not consider the context. For example, in the navigation scenario, voice should be higher than touch, while in the music browsing scenario, the priority strategy may need to be adjusted.

[0006] Concurrency risk: multiple input signals directly modify the playing logic, which can easily cause thread competition and deadlock. SUMMARY

[0007] The purpose of the present application is to provide a multi-modal input conflict resolution method for a vehicle multimedia system, a multi-modal input conflict resolution system for a vehicle multimedia system, an electronic device, a storage medium and a vehicle, which at least solves one of the technical problems.

[0008] 1. The problem of invalid or incorrect operation caused by input conflict; 2. The problem of poor user experience caused by fixed priority; 3. The problem of asynchronous UI state and playing state; 4. The problem of multi-thread concurrency risk.

[0009] The present application provides the following solutions:

[0010] According to a first aspect of the present application, a multi-modal input conflict resolution method for a vehicle multimedia system is provided, comprising:

[0011] Collecting input events from at least two input sources of touch input, voice input, knob input and key input;

[0012] Standardizing the input events, which includes the input source, action type, timestamp and context information corresponding to the input events.

[0013] acquire context information corresponding to the input event and a preset priority policy for processing the input event;

[0014] arbitrate according to the context information corresponding to the input event and the preset priority policy for processing the input event, to determine an execution order of the input event;

[0015] acquire a current input event according to the determined execution order of the input event;

[0016] update synchronization according to a global state machine of the current input event.

[0017] According to a second aspect of the present application, a multi-modal input conflict resolution system for a vehicle-mounted multimedia system is provided, comprising:

[0018] an input event acquisition module configured to acquire an input event from at least two input sources including touch input, voice input, knob input, and key input;

[0019] an input event standardization module configured to encapsulate and standardize the input event, including input source, action type, timestamp, and context information corresponding to the input event;

[0020] a context management module configured to acquire context information corresponding to the input event and / or a preset priority policy for processing the input event;

[0021] an input event arbitration module configured to arbitrate according to the context information corresponding to the input event and / or the preset priority policy for processing the input event, to determine an execution order of the input event;

[0022] acquire a current input event according to the determined execution order of the input event;

[0023] a global state machine module configured to update synchronization according to a global state machine of the current input event.

[0024] Further, the priority policy comprises:

[0025] setting a priority of the input event based on driving safety and input source type;

[0026] wherein, according to the context information, it is determined whether the input event is a driving safety related input event;

[0027] if yes, the input event has the highest priority, otherwise, it is determined whether the input source is a voice input event;

[0028] if yes, the input event has the highest priority, otherwise, it is determined whether the input source is a knob input event or a key input event;

[0029] If yes, the input event has the highest priority, otherwise, determine whether the input source is a touch input event;

[0030] If yes, the input event has the highest priority.

[0031] Further, the context information includes:

[0032] Display based on state information of navigation interface, instrument interface or multimedia interface;

[0033] Determine whether it involves driving safety according to the state information display;

[0034] If yes, the input source data of driving or interaction is associated with the input event related to driving safety to generate a strong instruction;

[0035] If no, context awareness is performed according to the input source type.

[0036] Further, the context awareness according to the input source type includes:

[0037] If the current interface is a navigation interface, voice input is preferred to knob input or key input to form an input event in the navigation interface;

[0038] If the current interface is a multimedia interface, knob input or key input is preferred to touch input to form an input event in the multimedia interface.

[0039] Further, the context awareness according to the input source type includes:

[0040] Within a preset time window, determine whether different input source data based on the same input event are received;

[0041] If yes, determine whether the input source action directions are opposite;

[0042] If opposite, detect input conflict, and determine whether it is a conflict between voice input and knob or key input;

[0043] If yes, determine whether the voice input event involves a preset strong instruction;

[0044] If yes, the input event formed by voice input is preferentially executed, and knob input event is ignored within a preset time;

[0045] If it is a normal instruction, the input event formed by the nearest input source is executed according to the distance of time;

[0046] If the input event formed by continuous action is detected in the nearest input source, the input event formed by continuous action is preferentially executed.

[0047] Further, the abnormality rollback module is further included.

[0048] The abnormality rollback module is configured to rollback the in-vehicle multimedia system to the last stable state when the input event arbitration fails or a timeout occurs.

[0049] According to a third aspect of the present application, an electronic device is provided, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are in communication with each other through the communication bus.

[0050] The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the method for solving multi-modal input conflict of an in-vehicle multimedia system.

[0051] According to a fourth aspect of the present application, a computer readable storage medium is provided, which stores a computer program executable by an electronic device, and when the computer program is run on the electronic device, the electronic device performs the steps of the method for solving multi-modal input conflict of an in-vehicle multimedia system.

[0052] According to a fifth aspect of the present application, a vehicle is provided, comprising:

[0053] The electronic device is configured to implement the steps of the method for solving multi-modal input conflict of an in-vehicle multimedia system.

[0054] The processor runs a program, and when the program is run, the data output from the electronic device performs the steps of the method for solving multi-modal input conflict of an in-vehicle multimedia system.

[0055] The storage medium is configured to store a program, and when the program is run, the data output from the electronic device performs the steps of the method for solving multi-modal input conflict of an in-vehicle multimedia system.

[0056] Through the above scheme, the following beneficial technical effects are obtained:

[0057] The present application introduces an input event arbitrator + context awareness mechanism + global state machine, and has the following beneficial effects:

[0058] 1. Input conflict is significantly reduced: all input events are centrally arbitrated to prevent concurrent modification of the state.

[0059] 2. Better user experience: dynamically adjust input priority according to the scene to meet the real needs of users.

[0060] 3. Enhanced state consistency: global state machine management, UI and logic remain synchronized.

[0061] 4. System stability is improved: unified interface is used to execute control logic, and direct thread competition is avoided. BRIEF DESCRIPTION OF DRAWINGS

[0062] Figure 1 is a flowchart of a multi-modal input conflict resolution method for a vehicle-mounted multimedia system provided by one or more embodiments of the present application.

[0063] Figure 2 is a structural diagram of a multi-modal input conflict resolution system for a vehicle-mounted multimedia system provided by one or more embodiments of the present application.

[0064] Figure 3 is a schematic diagram of a system architecture provided by one specific embodiment of the present application.

[0065] Figure 4 is a schematic diagram of an arbitration process provided by one specific embodiment of the present application.

[0066] Figure 5 is a structural block diagram of an electronic device for a multi-modal input conflict resolution method for a vehicle-mounted multimedia system provided by one or more embodiments of the present application. DETAILED DESCRIPTION

[0067] The technical solutions of the present application will be described clearly and completely below in conjunction with the drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0068] Figure 1 is a flowchart of a multi-modal input conflict resolution method for a vehicle-mounted multimedia system provided by one or more embodiments of the present application.

[0069] As shown in the multi-modal input conflict resolution method for a vehicle-mounted multimedia system, the method comprises: Figure 1

[0070] Step S1, collecting input events from at least two input sources among touch input, voice input, knob input, and key input;

[0071] Step S2, performing standardization processing on the input events, which includes input source, action type, timestamp, and context information corresponding to the input events;

[0072] Step S3, obtaining context information corresponding to the input events and a preset priority strategy for processing the input events;

[0073] ​Step S4, arbitration is performed according to the context information corresponding to the input event and the preset priority strategy for processing the input event, and the execution order of the input event is determined;

[0074] Step S5, the current input event is acquired according to the determined execution order of the input event;

[0075] Step S6, the global state machine is updated according to the current input event.

[0076] Figure 2 is a structural diagram of a multi-modal input conflict resolution system for a vehicle-mounted multimedia system provided by one or more embodiments of the present application.

[0077] As shown in Figure 2 the multi-modal input conflict resolution system for the vehicle-mounted multimedia system, comprising:

[0078] An input event collection module is configured to collect input events from at least two input sources including touch input, voice input, knob input, and key input;

[0079] An input event standardization module is configured to encapsulate and standardize the input events, and the encapsulation and standardization process includes the input source, action type, timestamp, and context information corresponding to the input event;

[0080] A context management module is configured to acquire the context information corresponding to the input event or / and the preset priority strategy for processing the input event;

[0081] An input event arbitration module is configured to perform arbitration according to the context information corresponding to the input event or / and the preset priority strategy for processing the input event, and determine the execution order of the input event;

[0082] According to the determined execution order of the input event, the current input event is acquired;

[0083] A global state machine module is configured to update the global state machine according to the current input event.

[0084] In this embodiment, the priority strategy includes:

[0085] The priority of the input event is set based on driving safety and input source type;

[0086] Wherein, the context information is used to determine whether the input event is related to driving safety;

[0087] If yes, the priority of the input event is the highest, otherwise, it is determined whether the input source is a voice input event;

[0088] If yes, the input event has the highest priority, otherwise, it is judged whether the input event is a touch input event or not;

[0089] If yes, the input event has the highest priority, otherwise, it is judged whether the input event is a touch input event or not;

[0090] If yes, the input event has the highest priority.

[0091] In the embodiment, the context information includes:

[0092] Displaying based on state information of a navigation interface, an instrument interface or a multimedia interface;

[0093] Judging whether it involves driving safety according to the state information display;

[0094] If yes, it involves, the input source data of driving or interaction is associated to the input event related to driving safety to generate a strong instruction;

[0095] If no, it involves, context awareness is performed according to the input source type.

[0096] In the embodiment, the context awareness according to the input source type includes:

[0097] If the current interface is a navigation interface, voice input is prior to knob input or key input to form an input event in the navigation interface;

[0098] If the current interface is a multimedia interface, knob input or key input is prior to touch input to form an input event in the multimedia interface.

[0099] In the embodiment, the context awareness according to the input source type includes:

[0100] In a preset time window, it is judged whether different input source data based on the same input event is received or not;

[0101] If yes, it is judged whether the input source action directions are opposite or not;

[0102] If yes, the input conflict is detected, and it is judged whether it is a conflict of voice input and knob or key input;

[0103] If yes, it is judged whether the voice input event involves a preset strong instruction or not;

[0104] If yes, the input event formed by voice input is executed preferentially, and knob input events are ignored in a preset time;

[0105] If it is a normal instruction, the input event formed by the nearest input source is executed according to the distance of time;

[0106] If a continuous action input event is detected in the recently input sources, the continuous action input event is executed preferentially.

[0107] In the embodiment, the system further comprises an abnormal fallback module.

[0108] The abnormal fallback module is configured to fallback the vehicle-mounted multimedia system to a last stable state when the input event arbitration fails or a timeout occurs.

[0109] It is worth noting that, although the system / device only discloses the above-mentioned modules / units, it does not mean that the system / device is limited to the above-mentioned basic functional modules / units. On the contrary, the meaning expressed by the present application is that, on the basis of the above-mentioned basic functional modules, a person skilled in the art can add one or more functional modules to form infinite embodiments or technical solutions in combination with the prior art. That is to say, the system is open rather than closed, and it cannot be considered that the protection scope of the present application is limited to the above-mentioned basic functional modules only because the present embodiment discloses individual basic functional modules.

[0110] In one specific embodiment, a multi-modal input conflict resolution system for a vehicle-mounted multimedia system is disclosed, as shown in Figure 3 The system comprises:

[0111] An input event collection module is configured to receive input events from multiple input sources, including but not limited to touch input, voice input, knob input, and key signal.

[0112] An input event standardization module is configured to convert the multiple input events into a unified data format.

[0113] A context management module is configured to obtain current system context information, including a current running interface, a driving state, a focus module, and the like.

[0114] An input event arbitration module is configured to select one or more input events to execute from the multiple input events based on a preset priority strategy and the context information.

[0115] A global state machine module is configured to synchronize the input execution result to a system playback state, a volume state, and a UI interface state.

[0116] Further, the embodiment also discloses a vehicle-mounted multimedia input arbitration method, as shown in Figure 4 The method comprises the following steps:

[0117] Collecting input events of multiple input sources (touch, voice, knob, and key).

[0118] Standardize all input events into uniform format, generate event objects containing input source, action type, timestamp;

[0119] Dynamically determine the priority of each input event according to the current system context;

[0120] Execute the input event with the highest priority, ignore or cache other events;

[0121] Update the execution result to the global state machine, and drive the UI interface to refresh.

[0122] In another specific embodiment, a multi-modal input conflict resolution method and system based on input event arbitration and context awareness is disclosed, comprising the following steps:

[0123] 1. Input event standardization processing

[0124] data class InputEvent(

[0125] val source: InputSource, / / TOUCH, VOICE, KNOB, BUTTON

[0126] val action: ActionType, / / PLAY, PAUSE, NEXT, VOLUME_UP

[0127] val timestamp: Long )

[0129] All inputs (touch, voice, knob, button) are packaged into a uniform structure InputEvent, including: input source, action type, timestamp, context information.

[0130] 2. Input event arbitrator

[0131] All input events first enter the arbitrator, and the arbitrator decides the execution order according to the priority strategy + context information.

[0132] Priority strategy example:

[0133] Driving safety related > voice > hardware knob > touch operation

[0134] Context awareness example:

[0135] Currently in the navigation interface, voice navigation commands are given priority over music control.

[0136] Currently in the lyrics interface, the knob adjusts the lyrics scrolling, not the volume.

[0137] 3. Arbitration algorithm (pseudo code)

[0138] fun arbitrate(events: List <inputevent>, context: SystemContext):InputEvent? {

[0139] return events.maxByOrNull { getPriority(it, context)}

[0140] }

[0141] fun getPriority(event: InputEvent, context: SystemContext): Int {

[0142] return when {

[0143] context.isDrivingSafetyCritical() -> 100

[0144] event.source == InputSource.VOICE -> 90

[0145] event.source == InputSource.KNOB -> 80

[0146] else -> 70

[0147] }

[0148] }

[0149] 4. Global state machine synchronization

[0150] The final event executed by the arbitrator is sent to the state machine, which saves the event for arbitration in subsequent operations.

[0151] Update playback status, volume status, and UI status to ensure consistency.

[0152] 5. Abnormal rollback mechanism

[0153] When arbitration fails or execution times out, the system rolls back to the most recent stable state to prevent deadlock.

[0154] The beneficial effects are derived from the above embodiments:

[0155] 1. Conflict prevention: The arbitrator centrally manages inputs, resolving the risk of multiple inputs being written to the logic simultaneously.

[0156] 2. Intelligent Response: Adjust priorities through context awareness to reduce user errors.

[0157] 3. Consistency: Global state machine management, avoiding UI and logic disconnection.

[0158] 4. High Reliability: Exception rollback mechanism ensures stable operation, avoiding deadlock and lag.

[0159] Example 1: Workflow of Multi-Input Arbitrator

[0160] 1) User issues two operations almost simultaneously:

[0161] Voice Command: "Play Next Song"

[0162] Knob Operation: Increase Volume

[0163] 2) System collects two InputEvents:

[0164] { "source": "VOICE", "action": "NEXT", "timestamp": 1000}

[0165] { "source": "KNOB", "action": "VOLUME_UP", "timestamp": 1010}

[0166] 3) Arbitrator determines the current context:

[0167] Current scenario is "Music Playback", driving state, not in navigation interface.

[0168] 4) Arbitrator adjusts priority based on strategy:

[0169] VOICE (90 points) > KNOB (80 points), execute "Play Next Song" first, then adjust volume.

[0170] 5) Update global state machine after execution:

[0171] Playback state: Track #2 → Track #3

[0172] UI refresh: Display new song and update volume.

[0173] 6) Result: Both operations are effective, and the system has no conflict.

[0174] Example 2: Context-aware Priority Adjustment

[0175] 1) Current vehicle enters navigation interface, user performs the following operations:

[0176] Voice Command: "Open navigation to company"

[0177] Touch operation: Click Next Song

[0178] 2) Arbitrator adjusts priority:

[0179] Navigation command (driving safety) -> priority 100

[0180] Touch operation -> priority 70

[0181] 3) Result: Navigation command is executed first, music playback remains current state.

[0182] Example 3: Exception rollback mechanism

[0183] 1) User executes voice "pause", but voice recognition times out.

[0184] 2) Arbitrator fails to execute the command, exception rollback module restores the previous stable state (keep playing).

[0185] 3) UI is not changed, avoiding the false pause state.

[0186] Example 4: Volume up / down conflict handling

[0187] When the system receives volume adjustment commands from different input sources (e.g. voice input "turn up volume" and knob input "turn down volume") within a short time window, the application implements arbitration through conflict detection and merging logic to avoid frequent playback state jitter.

[0188] 1) Conflict detection: The system records the recent voice and knob input events. If the time difference between the two is less than a preset window (e.g. 500ms) and the action direction is opposite, it is considered a conflict.

[0189] 2) Arbitration rules:

[0190] If the voice command is a strong command (e.g. "turn to maximum"), execute the voice command first and ignore the knob input within 1 second.

[0191] If both voice and knob are normal adjustment commands, execute the last-in logic.

[0192] If continuous knob operation is detected (multiple events triggered continuously), execute the knob operation first as it is usually more explicit.

[0193] if (isConflict(voiceEvent, knobEvent)) {

[0194] if (voiceEvent.isStrongCommand()) {

[0195] execute(voiceEvent)

[0196] blockKnobFor(1000)

[0197] } else if (knobEvent.isContinuous()) {

[0198] execute(knobEvent)

[0199] } else {

[0200] execute(mostRecentEvent())

[0201] }

[0202] }

[0203] 3) The execution logic pseudo code:

[0204] 4) Effects:

[0205] Avoid UI state jitter.

[0206] Ensure that the user's true intention is prioritized.

[0207] System stability is improved.

[0208] Figure 5 It is an electronic device structure block diagram provided by one or more embodiments of the application for a multi-modal input conflict resolution method for a vehicle-mounted multimedia system.

[0209] As Figure 5 shown, the present application provides an electronic device, comprising: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus.

[0210] The memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of a multi-modal input conflict resolution method for a vehicle-mounted multimedia system.

[0211] The present application also provides a computer readable storage medium storing a computer program executable by an electronic device, which makes the electronic device execute the steps of a multi-modal input conflict resolution method for a vehicle-mounted multimedia system when the computer program runs on the electronic device.

[0212] The present application also provides a vehicle cabin, comprising:

[0213] An electronic device for implementing the steps of a multi-modal input conflict resolution method for a vehicle-mounted multimedia system.

[0214] A processor, the processor running a program, the program performing steps of the multi-modal input conflict resolution method for a vehicle multimedia system when running on data output from the electronic device;

[0215] A storage medium, for storing a program, the program performing steps of the multi-modal input conflict resolution method for a vehicle multimedia system when running on data output from the electronic device.

[0216] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0217] The electronic device includes a hardware layer, an operating system layer running on the hardware layer, and an application layer running on the operating system. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and a memory. The operating system can be any one or more computer operating systems that implement electronic device control through processes, such as a Linux operating system, a Unix operating system, an Android operating system, an iOS operating system, or a windows operating system, etc. In the embodiments of the present application, the electronic device can be a handheld device such as a smartphone or a tablet computer, or an electronic device such as a desktop computer or a portable computer, which is not particularly limited in the embodiments of the present application.

[0218] The execution subject of the electronic device control in the embodiments of the present application can be an electronic device, or a functional module in the electronic device capable of calling and executing a program. The electronic device can obtain a firmware corresponding to the storage medium, the firmware corresponding to the storage medium being provided by a supplier, and the firmware corresponding to different storage media can be the same or different, which is not limited herein. After the electronic device obtains the firmware corresponding to the storage medium, the electronic device can write the firmware corresponding to the storage medium into the storage medium, specifically, burn the firmware corresponding to the storage medium into the storage medium. The process of burning the firmware into the storage medium can be implemented by using the prior art, which is not described in detail in the embodiments of the present application.

[0219] The electronic device can also obtain a reset command corresponding to the storage medium, the reset command corresponding to the storage medium being provided by a supplier, and the reset command corresponding to different storage media can be the same or different, which is not limited herein.

[0220] At this time, the storage medium of the electronic device is the storage medium in which the corresponding firmware is written, and the electronic device can respond to the reset command corresponding to the storage medium in the storage medium in which the corresponding firmware is written, so that the electronic device resets the storage medium in which the corresponding firmware is written according to the reset command corresponding to the storage medium. The process of resetting the storage medium according to the reset command can be implemented in the prior art, and will not be described in detail in the embodiments of the present application.

[0221] For the convenience of description, the above device is described as various units and modules in terms of functions. Of course, the functions of the units and modules can be implemented in the same or multiple software and / or hardware in the implementation of the present application.

[0222] Those skilled in the art can understand that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which the present application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood in the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined.

[0223] For the convenience of description, the above device is described as various units and modules in terms of functions. Of course, the functions of the units and modules can be implemented in the same or multiple software and / or hardware in the implementation of the present application.

[0224] From the above description of the embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software and the necessary general hardware platform. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product, which can be stored in a storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments of the present application.

[0225] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions recorded in the above embodiments can be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.< / inputevent>

Claims

1. A multi-modal input conflict resolution method for in-vehicle multimedia systems, characterized in that, The multi-modal input conflict resolution method for the vehicle-mounted multimedia system comprises: collecting input events from at least two input sources of touch input, voice input, knob input and key input; standardizing the input events, which comprises input source, action type, timestamp and context information corresponding to the input events; obtaining context information corresponding to the input events and a preset priority strategy for processing the input events; arbitrating according to the context information corresponding to the input events and the preset priority strategy for processing the input events to determine the execution order of the input events; obtaining the current input event according to the execution order of the input events; updating the global state machine according to the current input event.

2. A multi-modal input conflict resolution system for an in-vehicle multimedia system, characterized by, The multi-modal input conflict resolution system for the vehicle-mounted multimedia system comprises: an input event collection module for collecting input events from at least two input sources of touch input, voice input, knob input and key input; an input event standardization module for standardizing the input events, which comprises input source, action type, timestamp and context information corresponding to the input events; a context management module for obtaining context information corresponding to the input events or / and a preset priority strategy for processing the input events; an input event arbitration module for arbitrating according to the context information corresponding to the input events or / and the preset priority strategy for processing the input events to determine the execution order of the input events; obtaining the current input event according to the execution order of the input events; a global state machine module for updating the global state machine according to the current input event.

3. The multi-modal input conflict resolution system for in-vehicle multimedia systems of claim 2, wherein, The priority strategy comprises: setting the priority of the input events based on driving safety and input source type; wherein, according to the context information, it is determined whether the input event is related to driving safety; if yes, the input event has the highest priority, otherwise, it is determined whether the input event is a voice input event; if yes, the input event has the highest priority, otherwise, it is determined whether the input event is a knob input event or a key input event; if yes, the input event has the highest priority, otherwise, it is determined whether the input event is a touch input event; if yes, the input event has the highest priority.

4. The multi-modal input conflict resolution system for in-vehicle multimedia systems of claim 2, wherein, The context information comprises: displaying state information based on navigation interface, instrument interface or multimedia interface; determining whether it involves driving safety according to the state information display; if yes, associating the input source data of driving or interaction to the input event related to driving safety to generate a strong instruction; if no, context awareness according to input source type.

5. The multi-modal input conflict resolution system for in-vehicle multimedia systems of claim 4, wherein, The context awareness according to input source type comprises: if the current interface is a navigation interface, voice input is preferred to knob input or key input to form an input event in the navigation interface; if the current interface is a multimedia interface, knob input or key input is preferred to touch input to form an input event in the multimedia interface.

6. The multi-modal input conflict resolution system for in-vehicle multimedia systems of claim 4, wherein, The context awareness according to input source type comprises: within a preset time window, it is determined whether different input source data based on the same input event is received. If yes, it is determined whether the input source action direction is opposite; If yes, it is determined whether it is a conflict between voice input and knob or button input; If yes, it is determined whether the voice input event involves a preset strong instruction; If yes, the input event constituted by voice input is executed preferentially, and knob input events are ignored within a preset time; If it is a normal instruction, the input event constituted by the nearest input source is executed according to the distance of time; If the input event constituted by continuous actions is detected in the nearest input source, the input event constituted by continuous actions is executed preferentially.

7. The multi-modal input conflict resolution system for in-vehicle multimedia systems of any one of claims 1 to 6, wherein, Further comprising: An abnormality rollback module; The abnormality rollback module is configured to rollback the vehicle-mounted multimedia system to a last stable state when input event arbitration fails or execution times out.

8. An electronic device, comprising: Comprising: A processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory complete mutual communication through the communication bus; The memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the multi-modal input conflict resolution method for a vehicle-mounted multimedia system according to claim 1.

9. A computer-readable storage medium, characterized in that, The memory stores a computer program executable by the electronic device, and when the computer program runs on the electronic device, the electronic device executes the steps of the multi-modal input conflict resolution method for a vehicle-mounted multimedia system according to claim 1.

10. A vehicle characterized by comprising: Comprising: The electronic device is configured to execute the steps of the multi-modal input conflict resolution method for a vehicle-mounted multimedia system according to claim 1; The processor runs a program, and when the program runs, the data output from the electronic device executes the steps of the multi-modal input conflict resolution method for a vehicle-mounted multimedia system according to claim 1; The storage medium is configured to store a program, and when the program runs, the data output from the electronic device executes the steps of the multi-modal input conflict resolution method for a vehicle-mounted multimedia system according to claim 1.

Images