A method and system for parallel racing and preemption decision of in-vehicle speech heterogeneous classifier

Abstract

The application relates to the technical field of vehicle-mounted man-machine voice interaction, and discloses a vehicle-mounted voice heterogeneous classifier parallel racing preemption decision method and system. After receiving the final text of ASR, the first classifier based on rule matching and the second classifier based on neural network inference are simultaneously started for parallel racing. An atomic operation is used to maintain a racing state machine with an idle state, a racing state, a first classifier winning state, a second classifier releasing state and a resolution state. A nested first time window T1 and a second time window T2 are introduced to drive state migration. If the first classifier returns a high-priority intention before the expiration of T1, the first classifier directly wins; if the first classifier does not return an intention at the expiration of T1, the second classifier is released. When an offline state is detected, the second classifier is skipped, and a bottom time window T3 is registered, so that the system can still give a response within a limited time in a weak network scenario.

Description

Technical Field

[0001] This invention relates to the field of vehicle-mounted human-machine voice interaction technology, specifically to a parallel racing preemption decision-making method and system for vehicle-mounted voice heterogeneous classifiers. Background Technology

[0002] In in-vehicle voice interaction scenarios, user voice commands can be broadly categorized into two types: one is structured system commands such as "turn on the air conditioning" and "next song," with clear semantics and requiring quick execution; the other is open-ended natural language commands such as "find me a nearby gas station," with complex semantics and requiring deep understanding. How to handle both types of commands within the same system is a major challenge in this field.

[0003] Currently, common approaches in the industry include: first calling the local rule engine, and then calling the remote large model if the match is not found. This serial approach results in the system command response delay being equal to the sum of the time spent on the two paths. Alternatively, multiple similar models can be started simultaneously and the result can be selected based on confidence level. However, this approach fails to reflect the applicability differences of different types of engines in different scenarios and is difficult to effectively cover weak offline networks. Another approach is to predict and cache the intent in advance during the speech recognition process and verify it after the final text arrives. However, the verification method is relatively simple and it is difficult to flexibly deal with problems such as misidentification of near-homophones.

[0004] In addition, existing solutions generally lack the ability to reverse the process: once a path is opened and starts outputting, another path cannot be reintegrated even if it returns a more suitable result later; in the case of vehicles, this means that some instructions that require immediate response may not be executed in time because the processing path is occupied.

[0005] In summary, existing technologies have not yet provided a complete solution that can simultaneously ensure parallel collaboration of heterogeneous engines, priority guarantee of system commands, flexible correction of misidentification, and effective backup for offline scenarios. Summary of the Invention

[0006] To achieve the above objectives, the present invention provides the following technical solution: a parallel racing preemptive decision-making method for vehicle-mounted heterogeneous voice classifiers, comprising the following steps:

[0007] Receive the final text of Automatic Speech Recognition (ASR) and simultaneously activate the first classifier and the second classifier. The first classifier is a rule-matching engine, and the second classifier is a neural network reasoning engine.

[0008] A race state machine is maintained through atomic operations. The race state machine includes an idle state, a race state, a first classifier winning state, a second classifier releasing state, and a resolution state. Nested first time window T1 and second time window T2 are started, where T1 is less than T2. ​​The timeout events of the first time window T1 and the second time window T2 respectively trigger the state transition of the race state machine.

[0009] If the first classifier returns a high-priority intent before T1 expires, the racing state machine is switched from the racing state atom to the first classifier winning state, the output of the second classifier is canceled, and the high-priority intent is distributed to the vehicle actuator through the vehicle control bus.

[0010] If the racing state machine is still in racing state when T1 is cut off, then the atom switches to the second classifier release state, allowing the output of the second classifier to be released;

[0011] If the first classifier returns a high-priority intent while the second classifier is in the released state and T2 has not expired, then reverse preemption is performed: the race state machine is atomically converted to the winning state of the first classifier, the output of the second classifier and the unsubmitted side effects are canceled, and the high-priority intent is distributed.

[0012] If the racing state machine has not entered the decision state by the time T2 expires, then the atom transitions to the decision state, and the second classifier exclusively completes the intention decision.

[0013] When an offline state is detected, offline mode is enabled, the startup of the second classifier is skipped, only the first classifier is started, and a fallback time window T3 longer than T2 is registered. If the first classifier does not return an intention within T3, the race is forcibly ended and a failure feedback is returned.

[0014] Preferably, the cancellation of the output of the second classifier and the side effect of not submitting include:

[0015] Obtain the instructions issued by the second classifier in the second classifier release state, and the execution status of those instructions;

[0016] When the execution status indicates that the instruction has not yet entered the vehicle control bus transmission queue, the instruction is discarded directly;

[0017] When the execution status indicates that the instruction has entered the vehicle control bus transmission queue and no confirmation signal has been received from the vehicle actuator, the instruction is canceled by sending an abort frame to the vehicle control bus.

[0018] When the execution status indicates that the instruction has received a confirmation signal from the vehicle actuator and has triggered a physical action, the physical action is retained, and subsequent instructions associated with the instruction are prevented from being issued.

[0019] Preferably, before receiving the final ASR text, the first classifier receives audio frames in parallel during the ASR streaming recognition process, generating an early preemption intent and a confidence level for that intent; the method further includes:

[0020] When the early grabbing intention belongs to a preset high-priority intention set and the confidence level of the early grabbing intention is not lower than the early grabbing threshold, the early grabbing intention together with the current timestamp is treated as a cache item and atomically written into an early grabbing cache slot.

[0021] After receiving the final ASR text, the cached item is atomically retrieved from the early snatch cache slot, and the early snatch intent contained in the cached item is subjected to three-stage verification in sequence:

[0022] The first step is to intercept negative keywords. Check whether the final text of the ASR contains the preset negative keywords corresponding to the early snatching intent in the cached item. If it does, reject the early snatching intent in the cached item.

[0023] The second step is positive keyword matching. It checks whether the final text of the ASR contains the preset positive keyword corresponding to the early snatching intention in the cache item. If it does, the verification in this step is passed and the next verification step is performed.

[0024] The third step is to ensure text consistency between the two ASR engines. After Chinese normalization processing, the first text output by the first ASR engine and the second text output by the second ASR engine are calculated to achieve character-level similarity. If the character-level similarity is not lower than the preset similarity threshold, this step is verified.

[0025] If all three verifications pass, the early preemption intent in the cached item is adopted, and the early preemption intent is mapped to a command object and then distributed to the vehicle actuator.

[0026] If any verification step fails, the positive keyword table is queried in reverse. Only if the matched candidate intent is unique and not blocked by the corresponding negative keyword, the unique candidate intent is executed. Otherwise, the cached item is discarded and the output of the second classifier on the final text of the ASR is used as the final intent decision result.

[0027] Preferably, the negative keyword blocking further includes:

[0028] For each preset negative keyword, automatically generate near-phonetic extended words with a pinyin editing distance within a preset range to form an extended negative word set;

[0029] If the final text of the ASR matches the preset negative keyword, then the early snatching intent in the cached item is directly rejected;

[0030] If the final ASR text only matches the near-phonetic extended words, then a joint determination is made by combining the confidence of the early snatching intention. When the confidence is lower than the extended determination threshold, the early snatching intention in the cached item is rejected; otherwise, the positive keyword matching is allowed.

[0031] Preferably, the preset similarity threshold is dynamically adjusted based on the real-time recognition status and historical performance of the first ASR engine and the second ASR engine, respectively.

[0032] When the confidence level of the first ASR engine in recognizing the current output text is higher than the first confidence level benchmark, and the consistency rate of the recognition results of the first ASR engine and the second ASR engine for the same speech within a preset historical time window is higher than the first consistency rate benchmark, the preset similarity threshold is reduced.

[0033] When the confidence level of the first ASR engine in recognizing the current output text is lower than the second confidence benchmark, or when the noise estimate of the environment in which the first ASR engine is located is higher than the noise threshold, the preset similarity threshold is increased.

[0034] When the conditions for lowering the preset similarity threshold and the conditions for raising the preset similarity threshold are met simultaneously, the preset similarity threshold is raised first.

[0035] Preferably, when atomically retrieving the cached item from the early-grabbing cache slot, the difference between the current time and the timestamp in the cached item is also compared. If the difference exceeds the preset maximum lifespan, the cached item is determined to be expired and discarded.

[0036] When a vehicle-mounted system wake-up session reset is detected, the early preemption cache slot is emptied via an atomic operation.

[0037] Preferably, before starting the first classifier and the second classifier, the final ASR text is further subjected to long text determination and compound instruction detection;

[0038] If the text length exceeds a preset length threshold, or is determined to be a compound instruction by the compound instruction detector, the racing state machine is kept in an idle state, and the final ASR text is directly sent to the second classifier for processing.

[0039] Preferably, at the same time the racing state machine atom transitions to the second classifier release state, the current session is marked as a system command restricted mode;

[0040] In the system command restricted mode, in subsequent rounds, the first classifier only allows intents that match the preset system command whitelist and filters out other intents; wherein, the preset system command whitelist contains all categories of the high-priority intents, and the system command restricted mode is cleared when the vehicle wake-up session is reset.

[0041] Preferably, the high-priority intent consists of at least two types of system commands from vehicle control, media control, and navigation, and each type of system command constituting the high-priority intent can cause a change in the physical state of the corresponding vehicle component, media player, or navigation actuator.

[0042] A parallel racing preemptive decision-making system for heterogeneous vehicle voice classifiers includes:

[0043] The ASR controller is used to output the first text of the first ASR engine and the second text of the second ASR engine, and to provide the first text and the second text to the editor.

[0044] The first classifier is a rule-matching engine used to receive audio frames in parallel during ASR streaming recognition to generate early snatching intent and the confidence level of the early snatching intent, and to perform rule matching on the final ASR text after receiving the final ASR text to output the intent.

[0045] The second classifier is a neural network reasoning engine used to perform neural network reasoning on the final ASR text to output the intent.

[0046] The parallel scheduler maintains a race state machine through atomic operations. The race state machine includes an idle state, a race state, a first classifier winning state, a second classifier releasing state, and a resolution state. It also starts nested first time window T1 and second time window T2, where T1 is less than T2.

[0047] An orchestrator, connected to the ASR controller, the first classifier, the second classifier, and the parallel scheduler, is used to simultaneously drive the first classifier and the second classifier after receiving the final ASR text, and to execute the above method according to the state transition and time window events of the parallel scheduler; wherein, the orchestrator is also used to obtain the first text and the second text from the ASR controller for use when performing the three-segment verification.

[0048] Compared with the prior art, the beneficial effects of the present invention are:

[0049] This invention achieves parallel racing by simultaneously launching a first classifier based on rule matching and a second classifier based on neural network inference, and introduces an atomic state machine driven by nested time windows to coordinate the results of the two paths. This breaks the limitation of traditional serial cascading, which requires waiting for one path to complete before starting another. In high-priority system command scenarios, end-to-end response latency is significantly reduced. By setting a reverse preemption mechanism in the release state of the second classifier in the racing state machine, even if the first classifier fails to win within the priority period, it can still regain control within the decision window, fundamentally guaranteeing the highest priority of system commands and solving the problem in existing solutions where one path cannot intervene in reverse after another path is allowed. Through early preemption caching combined with three-segment short-circuit verification, and especially by introducing a pre-set negative keyword table for near-sounding words... Explicit interception of misidentification significantly reduces the false execution rate of system commands. Meanwhile, the negative keyword list, as a purely configurable item, can be directly maintained by product personnel, shortening the repair process from model retraining to configuration distribution, thus significantly improving system maintainability. Dual ASR engine text consistency fallback verification utilizes the self-alignment similarity of the heterogeneous recognition results from the offline and cloud engines as the final judgment criterion, improving verification effectiveness and effectively covering word order variation scenarios not fully covered by the keyword list. By setting an offline mode, the startup of the second classifier is skipped when network unavailability is detected, relying solely on the local first classifier and registering a fallback time window T3, ensuring that the system can still respond within a limited time in weak network or offline scenarios, avoiding prolonged unresponsiveness due to waiting for remote services. Attached Figure Description

[0050] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a system framework diagram of the present invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of protection of this invention.

[0052] Please see Figure 1 and Figure 2 This invention provides an in-vehicle voice interaction system, which is deployed on the vehicle's main unit and coupled with the vehicle control bus, media player, and navigation actuator. The system includes modules such as an ASR controller, a first classifier, a second classifier, a parallel scheduler, and an orchestrator.

[0053] The ASR controller comprises a first ASR engine and a second ASR engine. The first ASR engine is an offline speech recognition engine that runs locally on the vehicle's infotainment system and features low latency. The second ASR engine is a cloud-based, high-accuracy speech recognition engine that provides more precise recognition results via a network connection. The ASR controller outputs the first text from the first ASR engine and the second text from the second ASR engine, and provides both texts to the orchestrator for use in subsequent three-segment verification.

[0054] The first classifier is a rule-matching engine that uses keyword matching and slot filling to identify intent. During the ASR streaming recognition process, the first classifier starts working by receiving audio frames in parallel, generating early snooping intents and their confidence levels; after receiving the final ASR text, it then performs a complete rule match on the final text.

[0055] The second classifier is a large language model engine based on neural network inference, deployed in the cloud or on the edge, with natural language understanding and tool invocation capabilities, used to process open, complex or compound natural language commands.

[0056] The parallel scheduler is the core scheduling module of this invention. It maintains a race state machine through atomic operations to manage the parallel race process between the first classifier and the second classifier. The race state machine includes five states: idle state, race state, first classifier winning state, second classifier releasing state, and resolution state. All state transitions are completed through atomic operations to ensure no race state in a multi-threaded callback environment. The parallel scheduler is also responsible for starting the nested first time window T1 and second time window T2, where T1 is less than T2. ​​The timeout events of the first time window T1 and the second time window T2 trigger the state transitions of the race state machine, respectively.

[0057] The orchestrator is the system's central control module, connected to the ASR controller, the first classifier, the second classifier, and the parallel scheduler, coordinating the data and control flows between these modules. After receiving the final ASR text, the orchestrator simultaneously drives the first and second classifiers and executes race decision-making based on the state transitions and time window events of the parallel scheduler.

[0058] The vehicle control bus maintains an internal transmission queue to temporarily store instructions to be sent to the vehicle actuators.

[0059] Based on the above system architecture, the intent decision-making method of the present invention will be described in detail below through specific embodiments. The steps described in each embodiment can be implemented on the above system architecture.

[0060] Example 1

[0061] This embodiment elaborates in detail on the core racing logic, state machine transition, time window mechanism, and offline fallback scheme of this method.

[0062] In in-vehicle voice interaction scenarios, user commands can be broadly categorized into two types: one is structured system commands such as "turn on the air conditioning" or "next song," which require immediate physical action from the vehicle; the other is open-ended natural language commands such as "find me a nearby gas station." Traditional methods either sequentially call both types of engines, resulting in cumulative delays in system command response; or they call them in parallel and only select the result based on confidence level, failing to reflect the highest priority that system commands should have.

[0063] To address the aforementioned issues, this method proposes an atomic racing mechanism driven by nested time windows. Upon receiving the final ASR text, the editor simultaneously activates both the first and second classifiers, allowing the two engines to operate in parallel and compete at the same time.

[0064] Furthermore, to precisely manage this race process, the parallel scheduler initiates and maintains a race state machine through atomic operations. This state machine is thread-safe and defines five states: idle, race, first classifier winner, second classifier release, and resolution. Simultaneously with the transition from the idle to the race state, the system initiates nested first and second time windows T1 and T2, with T1's duration strictly less than T2. ​​Timeout events for T1 and T2 serve as trigger signals for state machine transitions.

[0065] Furthermore, for in-vehicle scenarios where system commands require rapid response, this method sets a priority period, T1, for the rule engine. If the first classifier returns a high-priority intent within the T1 time window, the racing state machine atomically transitions from the racing state to the first classifier's winning state. For example, the system determines "turn on the air conditioner" as a high-priority instruction that can cause changes in the physical state of vehicle components based on a preset whitelist. Once this state transition occurs, it indicates that the race has ended prematurely. This method will immediately cancel the output being performed by the second classifier and distribute the high-priority intent to the corresponding actuator through the vehicle control bus, which is the fastest response path.

[0066] Furthermore, if the first classifier fails to win within the T1 priority period, for example due to recognition delay caused by in-vehicle noise, the T1 timeout event will trigger a state transition, atomically changing the racing state machine from the racing state to the second classifier release state. This state indicates that the first classifier did not return a high-quality result within the priority period, and the system begins to allow the output of the second classifier, allowing it to stream text to the user or perform operations; however, the racing mechanism of this method has a unique design here: even if the second classifier has been released, the racing is not completely over.

[0067] Before the T2 window closes, the rule engine retains the right to overtake. If the first classifier delays returning a high-priority intent within the T2 window due to reasons such as secondary recognition, this method will execute a reverse preemption step. At this time, the race state machine is forcibly pulled from the second classifier's released state back to the first classifier's winning state, and the system will immediately cancel the second classifier's active output and unsubmitted side effects, and instead distribute the first classifier's instructions. This mechanism fundamentally ensures that at any time, the high-priority system command can win the final competition.

[0068] Furthermore, if T2 also times out and the first classifier still does not return a high-quality result, the racing state machine will atomize into the decision state, whereby the second classifier will exclusively complete the intention decision and output it, and the current round of racing will end.

[0069] In addition to online racing, this method also considers vehicle availability in weak network or offline scenarios such as underground parking garages. When the orchestrator detects an offline state, it will activate offline mode. This mode directly skips the activation of the second classifier and relies solely on the local first classifier. Simultaneously, to avoid prolonged unresponsiveness of the local engine in offline mode, the system registers a fallback time window T3 much longer than T2, for example, 3000 milliseconds; if the first classifier still does not return any intent after T3 expires, the system will forcibly end the race and report the request failure to the user, thus avoiding continuous waiting.

[0070] Example 2

[0071] This embodiment further refines how to safely cancel the output of the second classifier and the unsubmitted side effects when reverse preemption occurs. This process is not a simple clearing, but rather employs a hierarchical processing mechanism to adapt to the irreversible nature of physical operations in vehicle-mounted scenarios.

[0072] When reverse preemption is triggered, the second classifier may have already issued some instructions. If all of them are revoked indiscriminately, it may cause dangerous actions by vehicle components. The "uncommitted side effects" referred to here are instructions and their associated operations that have been generated and started outputting by the second classifier, but have not yet produced irreversible physical consequences at the vehicle actuator end; typical states include software instructions that have not yet entered the vehicle bus transmission queue, and instructions that have entered the queue but have not yet been confirmed for execution by the actuator; to solve this safety problem, this method proposes a hierarchical processing approach based on instruction execution status.

[0073] Specifically, the orchestrator first obtains the instructions issued by the second classifier in its released state and queries its current execution status. Based on the different processing stages of the instructions in the system, it takes differentiated measures:

[0074] Furthermore, if the instruction is still at the software level, such as a piece of streaming text that is about to be pushed to the screen but has not yet entered the vehicle control bus transmission queue, the instruction has no physical side effects. This method directly discards it and clears the residual display content of the user interface.

[0075] Furthermore, if a command has been packaged into the vehicle control bus transmission queue, but the vehicle actuator (such as the air conditioning or window controller) has not yet acknowledged receipt, the command is in a pending transmission state. In this case, this method immediately sends a stop frame to the actuator via the bus to actively cancel the command.

[0076] Furthermore, and most critically, if the command has not only been sent but also received a confirmation signal from the vehicle's actuator and actually triggered a physical action—for example, the air conditioner has started blowing air—this method will preserve this physical action to avoid dangerous reversals. However, at the same time, the system will precisely cut off a series of subsequent operations associated with the command, such as preventing the automatic temperature adjustment program from being executed after the air conditioner is turned on, thus completing the response to the new command while ensuring safety.

[0077] Example 3

[0078] This embodiment explains the early snatching mechanism of the first classifier before receiving the final ASR text, and how to perform rigorous cross-validation on the early snatching results after receiving the final text.

[0079] To leverage the low-latency potential of the first classifier, this method does not require it to wait for the final ASR result. Before receiving the final ASR text, the first classifier receives audio frames in parallel during the ASR streaming process and generates early intent prediction and its confidence level. This is equivalent to using the user's speaking gaps to predict intent in advance.

[0080] For example, when a user says "open empty", the first classifier may have already generated an early grabbing intent with a confidence level of 0.78. At this point, the orchestrator initiates the first screening: determining whether the early grabbing intent belongs to a preset high-priority intent set and whether its confidence level is not lower than a preset early grabbing threshold. Assuming the early grabbing threshold is set to 0.7, after meeting the above conditions, the early grabbing intent and the current precise timestamp will be atomically written to the early grabbing cache slot as a cache item.

[0081] Once the complete ASR final text (such as "turn on the air conditioner") arrives, this method immediately and atomically retrieves the cached item from the early-response cache slot. However, the system does not directly adopt it because early-response prediction is based on partial information and carries the risk of misidentification, especially for homophones. Therefore, it must undergo strict three-stage sequential verification.

[0082] The first step is to intercept negative keywords. Check whether the early bird intent is associated with a set of pre-set negative keywords. For example, the negative keyword list for the intent to "cut song" may include "place order" and "download"; if the final text contains these negative words, it indicates that the early bird may misidentify near-but irrelevant instructions, and the system will directly reject the cached item.

[0083] If the first step passes, proceed to the second step: positive keyword matching. Check if there are any pre-defined positive keywords corresponding to the intent in the final text; for example, positive keywords for the intent "change song" might include "change song", "change to another song", and "next song"; if a match is found, the verification is successful.

[0084] If the first two steps fail to provide a clear assessment, the system proceeds to the third step, a fallback step involving dual ASR text consistency. Leveraging the system's heterogeneous dual ASR capabilities, the system retrieves the first text output by the first ASR engine and the second text output by the second ASR engine. The system performs Chinese normalization processing on both texts (e.g., removing modal particles and punctuation) and calculates character-level edit distance similarity. If the recognition results from the two independent engines are highly consistent (e.g., both identify as "cancel mute"), even if the positive keyword in the previous step did not match, the intent can be considered credible and the verification is successful.

[0085] Only after all three verification steps pass will the orchestrator adopt and execute the early-snatch intent. If any verification step fails, this method initiates a fallback text reverse lookup mechanism: it searches the positive keyword table for intents that match the current text, and executes the unique intent only if the matched candidate intent is unique and not blocked by its corresponding negative keyword; if the candidate intent is empty or there are multiple ambiguous options, to ensure absolute safety, this method directly discards the cached item and instead uses the output of the second classifier on the final ASR text as the final intent decision result.

[0086] Example 4

[0087] This embodiment is an enhancement and supplement to the first step of negative keyword interception. In practical applications, a fixed negative keyword list is difficult to exhaustively cover all error cases, especially near-homophones caused by dialects or accents. For example, if a user mispronounces "next song" as "next move," it may not be able to hit the precise negative keyword "place an order."

[0088] To address the issue of insufficient coverage of the fixed vocabulary, this method introduces a near-phonetic expansion and joint judgment mechanism. For each preset negative keyword, the system automatically generates its near-phonetic expansion words. The generation rule is that words based on pinyin and with edit distance within a preset range are all included in the expanded negative word set.

[0089] When the final ASR text arrives, the interception logic is as follows: if the text matches a precise pre-defined negative keyword, the cached item is directly rejected; if the text only matches near-homophones in the extended word set, this method does not perform a blanket rejection, but rather combines the original confidence level of the early grabbing intent for joint judgment; the system will evaluate whether the confidence level of the early grabbing intent is lower than an extended judgment threshold that is more stringent than the early grabbing threshold. If the confidence level is low, it indicates that the reliability of this early grabbing itself is questionable, and coupled with the match of near-homophone negative words, the probability of misidentification is high, so it is rejected; conversely, if the confidence level is extremely high, it indicates that the model's judgment is very certain, and even if near-homophone extended words are matched, it can be allowed to proceed to the second step of positive keyword matching. This method achieves a balance between preventing misidentification and avoiding excessive interception.

[0090] Example 5

[0091] This embodiment illustrates that the preset similarity threshold in the three-stage verification is not a fixed value, but is dynamically adjusted based on the real-time and historical performance of the two ASR engines.

[0092] When a vehicle is in motion, the speech recognition environment changes dynamically, making it difficult for a fixed similarity threshold to perform optimally in all scenarios. The solution proposed in this method is to enable the threshold to adaptively adjust based on the engine's recognition confidence and historical reputation.

[0093] Furthermore, the adjustment logic is divided into two directions: the first is to lower the threshold, that is, to relax the verification threshold. When the first ASR engine has a very high confidence in recognizing the current speech (e.g., reaching 0.92, far higher than the benchmark value of 0.85), and the system statistics show that its recognition result is consistent with the second ASR engine's recognition result in the past historical time window (e.g., 24 hours) as high as 95%, it indicates that the offline engine is in excellent condition and highly reliable. At this time, this method actively lowers the similarity threshold (e.g., from 0.7 to 0.6), so that more early-responding results can quickly pass verification, further reducing response latency.

[0094] Secondly, the threshold is raised, i.e., verification is tightened. When the recognition confidence of the first ASR engine is very low, or when the vehicle audio acquisition module detects a surge in the estimated environmental noise value, it indicates that the current recognition conditions are poor and the results are unreliable. At this time, this method actively raises the similarity threshold (e.g., from 0.7 to 0.8) to strictly control and prevent erroneous command execution. An important safety design is that if the conditions for relaxing and tightening the threshold are met simultaneously, the system unconditionally prioritizes the threshold-raising operation, with safety as the primary principle.

[0095] Example 6

[0096] This embodiment explains two protection mechanisms established to ensure the effectiveness of the early-grabbing cache slot in the time dimension and the cleanliness in the session dimension.

[0097] The essence of caching is trading space for time, but cached data is subject to the risk of invalidation and corruption. To address this, this method designs two lines of defense for early-preemption cache slots.

[0098] The first line of defense is timestamp expiration protection. The instant a cached item is retrieved from the early-grabbing cache slot, this method first calculates the difference between the current time and the timestamp generated by the cached item. If this difference exceeds the preset maximum lifespan (e.g., 2000 milliseconds), it indicates that the cached item has expired, most likely generated by the previous round of dialogue or an earlier voice recording, and is unrelated to the current instruction. In this case, the system directly determines it to be expired and discards it, without further verification.

[0099] The second line of defense is session reset and clearing. The most common interaction interruption in the vehicle environment is voice interruption or the user directly waking up a new session by voice. Once the orchestrator detects the vehicle wake-up session reset event, it clears the entire early-preemption cache slot through atomic operations. This operation eliminates the old intent of the previous round of dialogue from the source and avoids the problem of accidental triggering across rounds.

[0100] Example 7

[0101] This embodiment defines the pre-filtering conditions for the racing process, clarifying under what circumstances racing should not be initiated, and the task should be directly handed over to the second classifier for processing.

[0102] The speed-up mechanism is designed to handle short, clear, high-priority instructions. However, in actual conversations, users often speak complex, long sentences containing multiple intentions. To avoid false hits caused by extracting isolated short keywords from long sentences, this method sets up two pre-filtering measures before starting the two classifiers.

[0103] The first step is long text judgment. If the length of the final text received by ASR exceeds a preset threshold (e.g., 30 characters), such as "Help me find a route home that is not congested and does not take the highway", this method determines that it is obviously not a vehicle control command that the rule engine can process. At this time, the racing state machine remains in the idle state, and the system directly sends the text to the second classifier for processing, and the racing process is completely skipped.

[0104] The second step is compound instruction detection. A dedicated compound instruction detector analyzes the text, and if it determines that the text contains multiple independent operation intentions (such as "turn on the air conditioner and set the temperature to 26 degrees"), this method also bypasses them and hands them over to a second classifier that is better at understanding and decomposing complex instructions. Through these two pre-filters, the speed-up mechanism is confined to a clear application boundary and only processes the high-priority simple instructions that it is best at.

[0105] Example 8

[0106] This embodiment explains an important session-level policy adjustment triggered by the system when the racing state machine transitions to the second classifier release state.

[0107] When T1 times out, the race state machine atomically transitions to the second classifier release state. This state transition itself reflects the fact that, in this round of dialogue, the first classifier failed to give a high-confidence judgment on the user's intention within the priority period. This usually means that the user is engaging in an open dialogue rather than issuing a brief control command.

[0108] Based on this judgment, this method marks the entire current session as "system command restricted mode" while performing state transition. This mode is an adaptive degradation strategy. During its effective period, the behavior of the first classifier in subsequent dialogue rounds is precisely limited: only intents that hit the preset system command whitelist (such as "turn on the air conditioner") are allowed, while all other non-core vehicle control intents such as casual conversation and queries are actively filtered out and no longer participate in the race.

[0109] The advantage of this design is that it avoids the rule engine mistakenly intercepting user statements that coincidentally contain several keywords during complex dialogues dominated by the second classifier. At the same time, it ensures that once a user switches to issuing system commands, the rule engine can immediately capture and respond quickly. It's worth noting that this preset system command whitelist includes all high-priority intent categories used to trigger race-based preemption, ensuring that even in restricted mode, the fast track for critical commands remains open. This mode will persist until the user makes the next voice wake-up or the system resets the session, at which point it will be completely cleared.

[0110] Example 9

[0111] This embodiment precisely defines the core concept "high-priority intent" that runs through the entire solution and clarifies its scope of protection.

[0112] In all the racing, preemptive, early-bird, and restricted modes mentioned above, the "high-priority intent" is the core basis for decision-making. This concept is not vaguely defined in this method, but has clear and objectively measurable boundaries; it is defined as a set consisting of at least two of the three categories of system commands closely related to driving safety: vehicle control, media control, and navigation.

[0113] More importantly, each type of command constituting this set must possess a common physical characteristic: the ability to cause a change in physical state. Specifically, vehicle control commands (such as turning the air conditioner, windows, and lights on / off) cause the corresponding vehicle components to move; media control commands (such as changing tracks, pausing, and adjusting volume) cause a change in the playback state of the media player; and navigation commands (such as starting navigation and canceling navigation) cause a change in the working state of the navigation actuator. The ability to cause a change in physical state is the fundamental criterion for strictly distinguishing high-priority commands from simple information queries, weather reports, and other commands without physical side effects.

[0114] Example 10

[0115] This embodiment takes a complete system workflow perspective, connecting all modules and clearly demonstrating the process by which each module works collaboratively to execute all the aforementioned method steps.

[0116] The in-vehicle voice interaction system on which this embodiment is based includes key components such as an ASR controller, a first classifier, a second classifier, a parallel scheduler, and an orchestrator. These components each have their own functions and are closely interconnected.

[0117] First, the ASR controller manages the first ASR engine and the second ASR engine, and is responsible for outputting the first and second texts generated by these two heterogeneous engines, and providing the two texts to the system's master control module—the orchestrator—in real time.

[0118] The first classifier is a rule matching engine with dual functions: during the ASR streaming recognition process, it works in advance by receiving audio frames in parallel to generate an early sniffing intent with confidence; after receiving the final ASR text, it immediately performs rule matching on the complete text.

[0119] The second classifier focuses on processing the final ASR text, utilizing its neural network reasoning capabilities for deep semantic understanding.

[0120] The core scheduling of the system is handled by the parallel scheduler. It maintains a five-state race state machine containing idle state, race state, first classifier winner state, second classifier release state, and resolution state through low-level atomic operations, and is responsible for starting and managing nested T1 and T2 time windows.

[0121] Finally, the orchestrator, acting as the central controller connecting all modules, simultaneously drives the first and second classifiers to race after receiving the final ASR text. Throughout the decision-making process, it makes its final decision based on the state transitions of the racing state machine in the parallel scheduler and time window timeout events. In particular, when three-stage verification is required, the orchestrator retrieves the first and second texts obtained from the ASR controller to complete a fallback check for consistency between the two ASR texts. It is through this architecture, with clearly defined responsibilities among modules and a clear data flow, that all the aforementioned methods and steps can operate reliably within a complete physical system.

[0122] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A parallel racing preemptive decision-making method for a vehicle-mounted heterogeneous voice classifier, characterized in that, Includes the following steps: Receive the final text of Automatic Speech Recognition (ASR) and simultaneously activate the first classifier and the second classifier. The first classifier is a rule-matching engine, and the second classifier is a neural network reasoning engine. A race state machine is maintained through atomic operations. The race state machine includes an idle state, a race state, a first classifier winning state, a second classifier releasing state, and a resolution state. Nested first time window T1 and second time window T2 are started, where T1 is less than T2. ​​The timeout events of the first time window T1 and the second time window T2 respectively trigger the state transition of the race state machine. If the first classifier returns a high-priority intent before T1 expires, the racing state machine is switched from the racing state atom to the first classifier winning state, the output of the second classifier is canceled, and the high-priority intent is distributed to the vehicle actuator through the vehicle control bus. If the racing state machine is still in racing state when T1 is cut off, then the atom switches to the second classifier release state, allowing the output of the second classifier to be released; If the first classifier returns a high-priority intent while the second classifier is in the released state and T2 has not expired, then reverse preemption is performed: the race state machine is atomically converted to the winning state of the first classifier, the output of the second classifier and the unsubmitted side effects are canceled, and the high-priority intent is distributed. If the racing state machine has not entered the decision state by the time T2 expires, then the atom transitions to the decision state, and the second classifier exclusively completes the intention decision. When an offline state is detected, offline mode is enabled, the startup of the second classifier is skipped, only the first classifier is started, and a fallback time window T3 longer than T2 is registered. If the first classifier does not return an intention within T3, the race is forcibly ended and a failure feedback is returned.

2. The method according to claim 1, characterized in that, The side effects of canceling the output of the second classifier and not submitting include: Obtain the instructions issued by the second classifier in the second classifier release state, and the execution status of those instructions; When the execution status indicates that the instruction has not yet entered the vehicle control bus transmission queue, the instruction is discarded directly; When the execution status indicates that the instruction has entered the vehicle control bus transmission queue and no confirmation signal has been received from the vehicle actuator, the instruction is canceled by sending an abort frame to the vehicle control bus. When the execution status indicates that the instruction has received a confirmation signal from the vehicle actuator and has triggered a physical action, the physical action is retained, and subsequent instructions associated with the instruction are prevented from being issued.

3. The method according to claim 1, characterized in that, Before receiving the final ASR text, the first classifier receives audio frames in parallel during the ASR streaming recognition process, generating an early preemption intent and a confidence level for that intent; the method further includes: When the early grabbing intention belongs to a preset high-priority intention set and the confidence level of the early grabbing intention is not lower than the early grabbing threshold, the early grabbing intention together with the current timestamp is treated as a cache item and atomically written into an early grabbing cache slot. After receiving the final ASR text, the cached item is atomically retrieved from the early snatch cache slot, and the early snatch intent contained in the cached item is subjected to three-stage verification in sequence: The first step is to intercept negative keywords. Check whether the final text of the ASR contains the preset negative keywords corresponding to the early snatching intent in the cached item. If it does, reject the early snatching intent in the cached item. The second step is positive keyword matching. It checks whether the final text of the ASR contains the preset positive keyword corresponding to the early snatching intention in the cache item. If it does, the verification in this step is passed and the next verification step is performed. The third step is to ensure text consistency between the two ASR engines. After Chinese normalization processing, the first text output by the first ASR engine and the second text output by the second ASR engine are calculated to achieve character-level similarity. If the character-level similarity is not lower than the preset similarity threshold, this step is verified. If all three verifications pass, the early preemption intent in the cached item is adopted, and the early preemption intent is mapped to a command object and then distributed to the vehicle actuator. If any verification step fails, the positive keyword table is queried in reverse. Only if the matched candidate intent is unique and not blocked by the corresponding negative keyword, the unique candidate intent is executed. Otherwise, the cached item is discarded and the output of the second classifier on the final text of the ASR is used as the final intent decision result.

4. The method according to claim 3, characterized in that, The negative keyword blocking also includes: For each preset negative keyword, automatically generate near-phonetic extended words with a pinyin editing distance within a preset range to form an extended negative word set; If the final text of the ASR matches the preset negative keyword, then the early snatching intent in the cached item is directly rejected; If the final ASR text only matches the near-phonetic extended words, then a joint determination is made by combining the confidence of the early snatching intention. When the confidence is lower than the extended determination threshold, the early snatching intention in the cached item is rejected; otherwise, the positive keyword matching is allowed.

5. The method according to claim 3, characterized in that, The preset similarity threshold is dynamically adjusted based on the real-time recognition status and historical performance of the first ASR engine and the second ASR engine, respectively. When the confidence level of the first ASR engine in recognizing the current output text is higher than the first confidence level benchmark, and the consistency rate of the recognition results of the first ASR engine and the second ASR engine for the same speech within a preset historical time window is higher than the first consistency rate benchmark, the preset similarity threshold is reduced. When the confidence level of the first ASR engine in recognizing the current output text is lower than the second confidence benchmark, or when the noise estimate of the environment in which the first ASR engine is located is higher than the noise threshold, the preset similarity threshold is increased. When the conditions for lowering the preset similarity threshold and the conditions for raising the preset similarity threshold are met simultaneously, the preset similarity threshold is raised first.

6. The method according to claim 3, characterized in that, When the cached item is atomically retrieved from the early-grabbing cache slot, the difference between the current time and the timestamp in the cached item is also compared. If the difference exceeds the preset maximum lifespan, the cached item is determined to be expired and discarded. When a vehicle-mounted system wake-up session reset is detected, the early preemption cache slot is emptied via an atomic operation.

7. The method according to claim 1, characterized in that, Before starting the first and second classifiers, long text determination and compound instruction detection are performed on the final ASR text. If the text length exceeds a preset length threshold, or is determined to be a compound instruction by the compound instruction detector, the racing state machine is kept in an idle state, and the final ASR text is directly sent to the second classifier for processing.

8. The method according to claim 1, characterized in that, While the racing state machine atom transitions to the second classifier release state, the current session is marked as system command restricted mode; In the system command restricted mode, in subsequent rounds, the first classifier only allows intents that match the preset system command whitelist and filters out other intents; wherein, the preset system command whitelist contains all categories of the high-priority intents, and the system command restricted mode is cleared when the vehicle wake-up session is reset.

9. The method according to claim 1, characterized in that, The high-priority intent consists of at least two types of system commands from vehicle control, media control, and navigation, and each type of system command constituting the high-priority intent can cause a change in the physical state of the corresponding vehicle component, media player, or navigation actuator.

10. A parallel racing preemptive decision-making system for vehicle-mounted heterogeneous voice classifiers, characterized in that, include: The ASR controller is used to output the first text of the first ASR engine and the second text of the second ASR engine, and to provide the first text and the second text to the editor. The first classifier is a rule-matching engine used to receive audio frames in parallel during ASR streaming recognition to generate early snatching intent and the confidence level of the early snatching intent, and to perform rule matching on the final ASR text after receiving the final ASR text to output the intent. The second classifier is a neural network reasoning engine used to perform neural network reasoning on the final ASR text to output the intent. The parallel scheduler maintains a race state machine through atomic operations. The race state machine includes an idle state, a race state, a first classifier winning state, a second classifier releasing state, and a resolution state. It also starts nested first time window T1 and second time window T2, where T1 is less than T2. An orchestrator, connected to the ASR controller, the first classifier, the second classifier, and the parallel scheduler, is configured to simultaneously drive the first classifier and the second classifier after receiving the final ASR text, and execute the method described in any one of claims 1 to 9 based on the state transitions and time window events of the parallel scheduler; wherein the orchestrator is further configured to obtain the first text and the second text from the ASR controller for use when performing the three-segment verification.

Images