Vehicle-mounted voice intention recognition system based on adversarial reinforcement learning

Abstract

The invention provides a vehicle-mounted voice intention recognition system based on adversarial reinforcement learning, relates to the technical field of vehicle-mounted voice recognition, and realizes effective improvement of vehicle-mounted voice intention recognition robustness by constructing a dynamic adversarial closed loop of a noise generator-intention discriminator-reinforcement learning trainer. According to the system, a large language model is used as an environment, a noise generator is driven to intelligently generate an anti-noise text which is semantic-related, reasonable in grammar and highly vivid, and semantic and syntactic errors possibly generated by voice recognition in a real vehicle-mounted scene are effectively simulated; the quality and challenge of adversarial samples are ensured through a multi-dimensional reward function including intention errors, confidence decrease, semantic similarity and grammar correctness; under an alternate training mechanism, the intention discriminator is continuously exposed in a high-value confrontation sample dynamically generated by the noise generator for training, so that the recognition accuracy and stability of the intention discriminator in a complex noise environment are gradually enhanced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle-mounted voice recognition, and particularly relates to a vehicle-mounted voice intent recognition system based on adversarial reinforcement learning. BACKGROUND

[0002] As a core technology of natural human-vehicle interaction, vehicle-mounted voice recognition can significantly improve driving safety and operational convenience, and is a key link to realize intelligent cockpit and future smart travel. Existing vehicle-mounted voice recognition technology faces severe challenges in actual application. Traditional models generally lack sufficient noise resistance. Since they are mainly trained in quiet environments, when applied to real vehicle-mounted scenarios, they are easily affected by complex interference such as background music, wind noise, road noise, and multiple conversations, resulting in semantic errors or syntactic errors of the voice recognition module. The existing technology lacks an effective mechanism for robust training of such high-fidelity, context-compliant noise, resulting in a sharp decline in intent recognition performance in complex environments. SUMMARY

[0003] The present application provides a vehicle-mounted voice intent recognition system based on adversarial reinforcement learning to solve one or more technical problems existing in the prior art, at least to provide a beneficial choice or create conditions.

[0004] In one aspect, the present application provides a vehicle-mounted voice intent recognition system based on adversarial reinforcement learning, comprising: a noise generator configured to generate adversarial noise text conforming to a vehicle context based on a reinforcement learning policy network, the noise generator being trained with a large language model as an environment model, generating semantic correlation adversarial samples through output text editing actions; an intent discriminator configured to receive the adversarial noise text and the original clean text, and output intent prediction results, confidence and probability distribution, the intent discriminator being robustly trained through the adversarial samples; a reinforcement learning trainer configured to construct a dynamic adversarial closed loop: fixing the parameters of the intent discriminator, training the noise generator through a reward function fed back by the large language model; fixing the parameters of the noise generator, training the intent discriminator to improve noise resistance; wherein the reward function includes an intent prediction error reward item, a confidence drop reward item, a semantic similarity penalty item, and a syntax error penalty item.

[0005] Further, the adversarial noise text includes semantic error samples and syntactic error samples, and the difference between the semantic distribution of the adversarial noise text and the semantic distribution of the original clean text is within a preset distribution difference threshold range.

[0006] Furthermore, the noise generator includes a state coding module and a strategy and value evaluation module; The state encoding module is configured to: input raw clean text, perform context encoding through a text encoder, and output a word embedding matrix and a context representation vector for each word in the sequence; The strategy and value assessment module is configured to: output the probability distribution of text editing actions and the value estimate of the current state based on the context representation vector; and receive the reward signal fed back by the large language model, optimize the strategy parameters through the strategy gradient method, and generate adversarial examples that conform to the in-vehicle context. The probability distribution of the text editing action is determined by calculating the vocabulary-level replacement probability, deletion execution probability, or candidate word insertion probability for each token position in the original text, and the value estimation is calculated by a feedforward network for advantage function optimization.

[0007] Furthermore, the intent discriminator includes a context feature extraction module and an intent classification decision module; The context feature extraction module is configured to: receive the adversarial noise text and the original clean text, perform word segmentation processing through a pre-trained backbone network, and output a token-level context hidden state matrix containing deep semantic information of the text. The intent classification decision module is configured to: calculate the intent category probability distribution through a classification layer based on the context hidden state matrix, and output the predicted intent label and corresponding confidence level; In this context feature extraction module, the backbone network freezes the underlying parameters during the training phase of the adversarial text, and only fine-tunes the top Transformer layer to adapt to the semantics of the vehicle scene. The classification layer is optimized through the cross-entropy loss function, so that the intent discriminator can be deployed independently as a lightweight model during the inference phase.

[0008] Furthermore, the reinforcement learning trainer includes an adversarial strategy optimization module and a robust training scheduling module; The adversarial strategy optimization module is configured to: input the reward signal fed back by the large language model, the current policy parameters of the noise generator and the estimated value of the value function, calculate the policy gradient through the policy gradient method and output the parameter update instruction, and optimize the probability distribution of the text editing action of the noise generator; The robust training scheduling module is configured to: generate adversarial samples by fixing the noise generator parameters output by the adversarial strategy optimization module, mix the adversarial samples with the original clean text at a preset ratio as training data, input the data into the intent discriminator and calculate the gradient through the cross-entropy loss function, and output the parameter update instruction of the intent discriminator.

[0009] Furthermore, the reinforcement learning trainer achieves alternating training through a dynamic adversarial closed-loop mechanism: the first stage is dominated by the adversarial strategy optimization module, which iteratively optimizes the noise generator until the semantic similarity of the generated adversarial samples exceeds the preset similarity threshold and the grammatical error probability is lower than the preset error rate threshold; the second stage switches to the robust training scheduling module, which fixes the noise generator parameters to train the intent discriminator until the recognition accuracy of the intent discriminator on the adversarial sample set stabilizes and converges.

[0010] Furthermore, the intention prediction error reward item is configured as follows: input the intention prediction result of the adversarial text and the true intention of the original clean text; if the intention prediction result is inconsistent with the true intention, then output a first reward value according to a preset first weight coefficient; otherwise, output a first reward value of 0.

[0011] Furthermore, the confidence reduction reward is configured as follows: input the intent prediction confidence of the adversarial text and the intent prediction confidence of the original clean text, calculate the confidence reduction amount, and output the second reward value as the product of a preset second weight coefficient and the confidence reduction amount.

[0012] Furthermore, the semantic similarity penalty term is configured as follows: inputting adversarial noise text and original clean text, encoding them into semantic vectors through a text encoder and calculating semantic similarity, and outputting a first penalty value of 1 minus the difference in semantic similarity and a preset third weight coefficient.

[0013] Furthermore, the syntax error penalty term is configured as follows: inputting adversarial noise text, calculating the syntax error probability through a pre-trained syntax checking model, and outputting a second penalty value as the product of the syntax error probability and a preset fourth weight coefficient.

[0014] This application includes at least the following beneficial effects: It provides an in-vehicle voice intent recognition system based on adversarial reinforcement learning. By constructing a dynamic adversarial closed loop of noise generator—intent discriminator—reinforcement learning trainer, the robustness of in-vehicle voice intent recognition is effectively improved. The system utilizes a large language model as the environment, driving the noise generator to intelligently generate semantically relevant, grammatically sound, and highly realistic adversarial noise text, effectively simulating semantic and syntactic errors that may occur in real-world in-vehicle scenarios. Through a multi-dimensional reward function including intent error, confidence decrease, semantic similarity, and grammatical correctness, the quality and challenge of adversarial examples are ensured. Under an alternating training mechanism, the intent discriminator is continuously exposed to high-value adversarial examples dynamically generated by the noise generator for training, thereby gradually enhancing its recognition accuracy and stability in complex noise environments.

[0015] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the description, claims and drawings. Attached Figure Description

[0016] The accompanying drawings are provided to further understand the technical solutions of the present invention and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the technical solutions of the present invention, and do not constitute a limitation on the technical solutions of the present invention.

[0017] Figure 1 This is a structural diagram of the in-vehicle voice intent recognition system based on adversarial reinforcement learning provided in this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] The present application will be further described below with reference to the accompanying drawings and specific embodiments. The described embodiments should not be considered as limitations on the present application, and all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the present application.

[0020] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0022] With the rapid development of intelligent connected vehicle technology, in-vehicle voice interaction systems have become a core component of intelligent cockpits, undertaking various human-machine interaction functions such as navigation control, multimedia playback, air conditioning adjustment, telephone communication, and information inquiry. Users issue commands through natural language, and the system needs to accurately understand their semantic intent and execute the corresponding operation. Among these, after speech recognition (ASR) converts the speech signal into text, the Natural Language Understanding (NLU) module is the key link that determines the accuracy of the system's response. However, in real in-vehicle environments, voice input is inevitably affected by various noise interferences, such as background music, wind and road noise from outside the vehicle, conversations among passengers inside the vehicle, and the sound of the air conditioning system operating. These factors often cause deviations in the text output by ASR, thus posing a serious challenge to subsequent intent recognition.

[0023] Currently, the mainstream technologies for improving the robustness of in-vehicle voice intent recognition systems mainly rely on two major directions: data augmentation and model training optimization. Regarding data augmentation, traditional methods typically employ rule-based replacement (such as synonym replacement and near-homophone replacement), random deletion, or insertion of words, artificially adding "noise" to clean text to expand the training dataset. Another type of method relies on large-scale manual collection and annotation in real-world scenarios. This involves recording and transcribing speech under different driving conditions to create a corpus containing various types of noise for model training.

[0024] However, existing technologies still have a series of insurmountable shortcomings when dealing with complex in-vehicle environments, mainly in the following three aspects.

[0025] First, traditional intent recognition models are mostly trained on standard corpora in idealized, quiet environments, assuming that the input text is accurate and grammatically correct. However, in real-world in-vehicle scenarios, the output of the ASR module often contains semantic and syntactic errors (such as word order inversion, omissions, or repetitions). Existing models lack effective mechanisms for robust training against this type of high-fidelity, context-appropriate noise. When the input text contains these common errors found in real-world scenarios, the model's intent recognition accuracy drops sharply, leading to system malfunctions or unresponsiveness, severely impacting user experience and driving safety.

[0026] Secondly, to improve model performance in noisy environments, existing technologies heavily rely on large amounts of manually collected and labeled noisy data. This process is not only costly and time-consuming, but also struggles to cover all possible noise types and driving scenarios, resulting in incomplete dataset coverage. Furthermore, while rule-based or simple randomized data augmentation methods are low-cost, the generated "noise" often lacks semantic coherence and contextual relevance, being too random and mechanical, and unable to effectively simulate the distribution patterns of real ASR errors. For example, simply replacing "navigation" with "navigation device" or randomly deleting a word may generate grammatically incorrect or semantically deviated text. Such samples cannot realistically reflect the error characteristics of ASR systems, causing the model's learned "noise resistance" to fail in real-world scenarios, and hindering its generalization ability.

[0027] Furthermore, existing adversarial training methods are mostly static processes, meaning they pre-generate a batch of adversarial examples or use a fixed noise dataset, completing adversarial training all at once in the early stages of model training. This approach is essentially a passive defense mechanism, lacking the ability to dynamically evolve and self-improve. Once training is complete, the types of noise and attack methods the model faces are fixed, making it unable to adapt to constantly changing, new noise patterns or potential adversarial attacks. For example, with updates to ASR technology or changes in the in-vehicle environment, new error types may emerge. Static adversarial training models cannot autonomously learn and adapt to these new challenges, causing their robustness to decline over time. In addition, the process of generating adversarial examples in traditional adversarial training often lacks intelligent guidance, making it difficult to continuously generate high-value, highly challenging examples, thus limiting further improvements in the model's defense capabilities.

[0028] This application proposes an in-vehicle speech intent recognition system based on adversarial reinforcement learning. Its core lies in constructing a dynamic adversarial closed-loop training framework consisting of a noise generator, an intent discriminator, and a reinforcement learning trainer. The system innovatively utilizes a Large Language Model (LLM) as the environment model, driving the reinforcement learning-based noise generator to intelligently generate high-fidelity adversarial noise text that conforms to the in-vehicle context, is semantically relevant, and grammatically sound, effectively simulating semantic and syntactic errors in real speech recognition. By designing a multi-dimensional reward function that includes rewards for intent prediction errors, confidence decline, semantic similarity penalties, and grammatical error penalties, the system ensures that the generated adversarial examples are both challenging and semantically authentic. An alternating training mechanism is employed: first, the intent discriminator is fixed to train the noise generator to improve its "attack" capability; then, the noise generator is fixed again to train the intent discriminator to enhance its "defense" capability, forming a self-evolving and continuously optimizing closed-loop system. This approach not only significantly improves the robustness and accuracy of the intent recognition model in complex noisy environments but also completely eliminates the dependence on large amounts of manually labeled noise data, achieving efficient, intelligent, and scalable model training.

[0029] First, the in-vehicle voice intent recognition system based on adversarial reinforcement learning provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0030] Reference Figure 1 The vehicle-mounted voice intent recognition system based on adversarial reinforcement learning provided in this application includes a noise generator, an intent discriminator, and a reinforcement learning trainer.

[0031] In some embodiments of this application, the noise generator is configured to generate adversarial noise text that conforms to the in-vehicle context based on a reinforcement learning policy network. The noise generator is trained with a large language model as the environment model and generates semantically related adversarial samples by outputting text editing actions.

[0032] Reinforcement learning is a machine learning paradigm in which an agent learns optimal policies by interacting with its environment and receiving reward signals after performing actions, aiming to maximize long-term cumulative rewards. The policy network is a core component of reinforcement learning; it defines the probability distribution of the agent's actions given a state, essentially a trainable function that maps environmental states to actions. In this application, the policy network acts as a noise generator, learning to generate adversarial text that can mislead the intent recognition model. Simultaneously, it continuously optimizes its generation policy based on feedback from the reward function, achieving intelligent and high-quality automatic noise generation in a dynamic game against a large language model and an intent discriminator.

[0033] The noise generator in this application, by introducing a reinforcement learning framework, can intelligently generate high-fidelity, semantically relevant, and grammatically sound adversarial noise text. This effectively overcomes the shortcomings of traditional data augmentation methods, which rely on manual rules or random perturbations and struggle to balance challenge and linguistic logic. Its core function is to simulate "reasonable errors" that may occur in real-world in-vehicle speech recognition systems under complex noise environments, such as miswriting "adjust the fan speed" instead of "adjust the air volume," thereby providing high-quality "attack samples" for the intent recognition model to improve its robustness.

[0034] The noise generator uses a Large Language Model (LLM) as its environment model, fully leveraging the powerful semantic understanding and language generation capabilities of LLM. It not only acts as the engine for performing text editing but also plays a feedback role, dynamically evaluating the semantic consistency, grammatical correctness, and contextual relevance of generated samples to ensure the realism and effectiveness of adversarial examples. Simultaneously, the noise generator employs a generation mechanism based on text editing actions (such as replacement, deletion, and insertion), achieving precise control over the granularity and scope of perturbations. This makes each modification interpretable and allows for targeted simulation of common ASR system errors such as homonym misidentification, speech fragment loss, or background voice insertion. This fine-grained, controllable operational space, combined with the exploratory capabilities of reinforcement learning, enables the generator to continuously optimize its attack strategy. In the dynamic game with the intent discriminator, it continuously generates more challenging samples, driving the entire system to self-evolve and improve performance in the "attack-defense" process.

[0035] Furthermore, the noise generator includes a state coding module and a strategy and value evaluation module.

[0036] The state encoding module is configured to take raw clean text as input, perform context encoding through a text encoder, and output a word embedding matrix and a context representation vector for each word in the sequence.

[0037] The state encoding module is responsible for transforming the raw, clean text into machine-understandable vector representations, providing comprehensive contextual information for subsequent decision-making. By using advanced text encoders (such as BERT and RoBERTa), this module captures the semantic features of each word in the text and its contextual dependencies within the sentence, outputting a word embedding matrix and a contextual representation vector for each token. These high-dimensional vectors not only contain the semantics of the words themselves but also incorporate syntactic structure and contextual information, laying a solid foundation for the policy network to make accurate editing decisions.

[0038] The strategy and value assessment module is configured to output the probability distribution of text editing actions and the value estimate of the current state based on the context representation vector. It also receives reward signals from a large language model, optimizes policy parameters using the policy gradient method, and generates adversarial examples that fit the in-vehicle context. Specifically, the probability distribution of text editing actions is determined by calculating the vocabulary-level substitution probability, deletion execution probability, or candidate word insertion probability for each token position in the original text. The value estimate is calculated using a feedforward network for advantage function optimization.

[0039] The core function of the strategy and value assessment module is to determine the appropriate text editing action based on the contextual representation provided by the state encoding module. This module outputs two key results: first, the probability distribution of text editing actions, which calculates the vocabulary-level probability of replacement, deletion, or insertion of candidate words for each token position in the original text; and second, the value estimate of the current state, used to guide policy optimization. The generation of the action probability distribution enables the noise generator to autonomously decide whether and how to edit at each position, achieving fine-grained, adaptive noise injection. The value estimate is calculated through a feedforward neural network to construct the advantage function, measuring the long-term return of the current policy in policy gradient updates, thereby optimizing policy parameters more stably and efficiently, avoiding the high variance problem caused by solely relying on reward signals.

[0040] The strategy and value assessment module receives reward signals from the large language model and optimizes policy parameters using the policy gradient method. This is a crucial step in achieving "intelligent evolution" within the entire adversarial training loop. The reward signal is calculated using a multi-dimensional reward function, including rewards for incorrect intent prediction, confidence reduction, semantic similarity penalties, and grammatical error penalties. The noise generator aims to maximize cumulative rewards, generating adversarial examples that cause the intent discriminator to err or reduce confidence while maintaining high semantic similarity and grammatical correctness. Through the policy gradient method, the system can directly adjust the parameters of the policy network based on the reward signal feedback, allowing the generator to gradually learn which combinations of editing actions are more likely to yield high rewards, thus continuously evolving its "attack strategy." This feedback-based continuous optimization mechanism enables the noise generator to dynamically adapt to the improved defense capabilities of the intent discriminator, forming a continuously escalating adversarial game process that ultimately drives the entire system to achieve a higher level of robustness.

[0041] In some embodiments of this application, raw clean text is first input into the state coding module. Through a Transformer-based text encoder (e.g., DistilBERT) Output context representation It satisfies the following calculation formula (1): (1); In formula (1), For word embedding matrix.

[0042] Then, in the strategy and value assessment module, based on context representation... Output the probability distribution and state value estimate of the text editing action, and optimize the parameters using the policy gradient method; whereby the action probability distribution (such as replacing the token) satisfies the following calculation formula (2): (2); In formula (2), Indicates in the original clean text The Replace the token with a word from the vocabulary at each position. The probability, Indicates the first The action performed at each location; This represents the weight matrix of the policy network. This represents the bias term of the policy network. Representation of context The Middle The context representation vector of each token.

[0043] Furthermore, the value function for state value estimation satisfies the following calculation formula (3): (3); In formula (3), Indicates the current state Value estimation (a scalar measure of the quality of a state). [CLS] represents the context representation vector corresponding to the [CLS] tag output by the encoder of the original clean text; MLP represents the multilayer perceptron model.

[0044] Furthermore, the advantage function measures the advantage of the current action relative to the average level, satisfying the following calculation formula (4): (4); In formula (4), Let be the advantage function, representing the excess reward of the current action relative to the state value; Indicates the state Execute action The instant reward obtained afterward; This represents a discount factor used to control the importance of future rewards; This represents the estimated state value at the next moment.

[0045] In some embodiments of this application, the adversarial text includes semantic error samples and syntactic error samples, and the difference between the semantic distribution of the adversarial text and the semantic distribution of the original clean text is within a preset distribution difference threshold range.

[0046] In some embodiments of this application, the adversarial text includes semantic error samples and syntactic error samples. This design directly targets the two most common types of output errors in in-vehicle speech recognition systems under real-world noise environments, demonstrating high practical relevance and application value. Semantic errors refer to speech recognition results that deviate semantically but remain grammatically coherent. These errors are highly deceptive because their grammatical structure is complete, making them easy for the intent recognition model to misinterpret as the user's true intent, leading to erroneous operations. Syntactic errors refer to recognition results that exhibit grammatical disorder, such as word order reversal ("play music" becomes "music play"), word omission ("turn up volume" becomes "turn up"), or repeated insertion ("open the car window" becomes "open open the car window"). These errors disrupt the syntactic structure of natural language, potentially causing the intent recognition model to fail to correctly parse sentence components. By simultaneously generating these two types of adversarial samples, the system of this application can comprehensively cover the main error patterns of in-vehicle ASR systems, allowing the intent discriminator to be fully exposed to the most common noise types in the real world during training, thereby comprehensively improving its robustness and fault tolerance in complex environments.

[0047] The difference between the semantic distribution of the adversarial noisy text and the semantic distribution of the original clean text must be controlled within a preset distribution difference threshold. This constraint is a key guarantee for ensuring the "realism" and "effectiveness" of the adversarial examples. If the semantic deviation of the generated adversarial examples is too large, this perturbation exceeds the reasonable error range that the ASR system may produce under real noise, and belongs to "unnatural" or "unreasonable" attacks, which cannot reflect the real scene. Using such samples for training will cause the model to learn a defense capability that is detached from reality, resulting in a decrease in generalization performance in real in-vehicle environments. Therefore, by setting a threshold for semantic distribution difference, the system forces the generated adversarial examples to be sufficiently close to the original text in the semantic space, ensuring that their perturbations are subtle, gradual, and consistent with linguistic logic. This is equivalent to establishing a balance mechanism between "attackability" and "realistic rationality," making the training process closer to the physical constraints and linguistic rules of real in-vehicle voice interaction.

[0048] The measurement of semantic distribution differences is typically based on distance metrics in high-dimensional semantic embedding spaces, such as cosine distance, Wasserstein distance, or KL divergence. Pre-trained language models (such as BERT) are used to extract sentence vectors from the original and adversarial texts, and their similarity in the semantic space is calculated. The system introduces a semantic similarity penalty term into the reinforcement learning reward function. When the semantic distance between the generated adversarial sample and the original text exceeds a preset threshold, a negative reward is given, thus guiding the noise generator to avoid generating samples with excessive semantic deviation. Furthermore, controlling the semantic distribution difference within a threshold range helps prevent overfitting during adversarial training. If the semantic drift of the adversarial samples is too large, the intent discriminator may learn to recognize these extreme anomalous samples during training, neglecting its generalization ability to common, minor perturbations. By limiting semantic bias, the system ensures that the distribution of training data remains consistent with the real testing environment, thereby improving the model's generalization performance.

[0049] In some embodiments of this application, the intent discriminator is configured to receive adversarial noisy text and original clean text, and output intent prediction results, confidence levels and probability distributions. The intent discriminator is robustly trained using adversarial examples.

[0050] The core role of the intent discriminator is to act as the target model being "attacked," continuously improving its robustness against noise interference through ongoing adversarial training. Unlike traditional static intent recognition models, the intent discriminator in this application does not passively accept training on a fixed dataset, but actively participates in a dynamic adversarial loop. By continuously playing against high-fidelity adversarial examples intelligently generated by a noise generator, it gradually learns how to accurately identify the user's true intent even under the interference of semantic and syntactic errors. This discriminator not only outputs the final intent category label, but also provides the predicted confidence level and the complete probability distribution. This information is crucial for subsequent reinforcement learning reward calculations—for example, when the discriminator still gives a high confidence level to an incorrect sample, it will trigger a "confidence reduction reward," prompting the noise generator to generate more deceptive samples.

[0051] Furthermore, the intent discriminator includes a context feature extraction module and an intent classification decision module.

[0052] The context feature extraction module is configured to receive adversarial noisy text and original clean text, perform word segmentation through a pre-trained backbone network, and output a token-level context hidden state matrix containing deep semantic information of the text. Specifically, during the training phase of the adversarial noisy text, the backbone network of the context feature extraction module freezes its low-level parameters and only fine-tunes the top-level Transformer layer to adapt to the semantics of the in-vehicle scenario.

[0053] The context feature extraction module is responsible for transforming the input text (whether clean or adversarial) into a semantically rich vector representation, forming the foundation of the entire intent recognition process. This module employs a pre-trained backbone network (such as BERT, RoBERTa, or other Transformer architectures) for word segmentation and context encoding, capturing the deep semantic dependencies of each token within the sentence and outputting a token-level contextual hidden state matrix. This matrix not only contains the semantics of the words themselves but also integrates complex information such as syntactic structure, referential relationships, and contextual information, providing high-quality feature input for subsequent intent classification. By using a backbone network pre-trained on a large-scale corpus, the system can quickly acquire powerful language understanding capabilities, avoiding the high costs and long training cycles associated with starting from scratch.

[0054] The context feature extraction module employs a strategy of "freezing low-level parameters and fine-tuning only the top-level Transformer layer" during the training phase of adversarial text. This design has significant engineering and performance implications. The low-level Transformer layer typically learns general language features such as lexical, syntactic, and basic semantics. This knowledge is fully mastered during pre-training and possesses strong generalization capabilities. Freezing these parameters during fine-tuning in in-vehicle scenarios effectively prevents the "catastrophic forgetting" problem caused by small-scale domain data—that is, the model forgetting knowledge learned from general corpora. The top-level Transformer layer, on the other hand, focuses more on task-specific, high-level semantic abstractions. By fine-tuning these parameters, the model can better adapt to in-vehicle domain terminology, user expression habits, and common command structures (such as "navigate to XX" or "lower the air conditioning temperature"). This selective fine-tuning strategy maintains the model's general language capabilities while enhancing its sensitivity to in-vehicle contexts, achieving a balance between generality and specialization, and significantly improving the model's performance in specific scenarios.

[0055] The intent classification decision module is configured to: calculate the intent category probability distribution based on the context hidden state matrix through a classification layer, and output the predicted intent label and corresponding confidence score. The classification layer is optimized using the cross-entropy loss function, enabling the intent discriminator to be deployed independently as a lightweight model during the inference phase.

[0056] The intent classification decision module, based on the context hidden state matrix, calculates the intent category probability distribution through classification layers, ultimately outputting the predicted intent label and corresponding confidence score. It is the final decision-making unit for the intent recognition task. This module typically consists of one or more fully connected layers, mapping the [CLS] token or pooled vector output by the context feature extraction module to a predefined intent category space (e.g., "navigation," "music playback," "air conditioning control," etc.), and generating probability distributions for each category using the Softmax function. The output confidence score is not only used for the system's final decision but also serves as a key indicator for evaluating the model's robustness during adversarial training. For example, when faced with adversarial examples, a significant drop in the model's prediction confidence indicates sensitivity to noise; conversely, maintaining high confidence but making incorrect predictions suggests overly arbitrary judgments. By analyzing these outputs, the system can accurately assess the model's vulnerability and guide the noise generator to optimize its attack strategy.

[0057] Furthermore, the classification layer is optimized using the cross-entropy loss function to ensure that the model minimizes the difference between the predicted distribution and the true labels during training, which is a standard and efficient supervised learning objective. More importantly, this design allows the intent discriminator to be deployed independently as a lightweight model on in-vehicle terminals after adversarial training. By freezing the bottom layer of the backbone network and only fine-tuning the top layer, and by adopting a standard classification architecture, the number of parameters and computational complexity of the entire model are effectively controlled, facilitating efficient operation on resource-constrained in-vehicle computing platforms. At the same time, the lightweight design also reduces inference latency, meeting the stringent real-time requirements of in-vehicle voice interaction. In summary, the modular structure and training strategy of the intent discriminator not only ensure its robustness in adversarial environments but also take into account the feasibility of practical deployment, demonstrating the deep integration between algorithmic innovation and engineering implementation in this application.

[0058] In some embodiments of this application, the intent recognizer uses BERT or RoBERTa as a feature extractor, and outputs an intent probability distribution through a classification layer. For example, the encoder of the context feature extraction module satisfies the formula: , This represents both noise-resistant text and original clean text. Indicates input text The token-level context hidden state matrix.

[0059] Furthermore, the classification layer satisfies the following calculation formula (5): (5); In formula (5), Indicates based on The obtained probability distribution of intent categories (such as navigation, music, air conditioning, etc.); This represents the weight matrix of the classification layer. Indicates the bias term of the classification layer; express The context representation vector corresponding to the [CLS] marker.

[0060] In some embodiments of this application, the reinforcement learning trainer is configured to construct a dynamic adversarial closed loop: fixing the parameters of the intent discriminator and training the noise generator through the reward function fed back by the large language model; fixing the parameters of the noise generator and training the intent discriminator to improve noise resistance.

[0061] The reinforcement learning trainer is configured to construct a dynamic adversarial closed loop, whose core function is to realize the "attack-defense game" and co-evolution between the noise generator and the intent discriminator. This closed loop employs an alternating training mechanism: first, the parameters of the intent discriminator are fixed, and the Large Language Model (LLM) is used as the environmental feedback source. The noise generator is trained through reinforcement learning using multi-dimensional reward functions (including rewards for incorrect intent prediction, rewards for decreased confidence, penalties for semantic similarity, and penalties for grammatical errors), gradually enabling it to generate more challenging and context-appropriate adversarial examples. Subsequently, the parameters of the noise generator are fixed again, and the high-quality adversarial examples it generates are used to robustly train the intent discriminator, improving its recognition accuracy and stability under noise interference. This "freeze one, train the other" strategy avoids the training instability caused by simultaneous updates of both sides, ensuring the convergence of the learning process. More importantly, this closed loop is dynamically evolving—as the intent discriminator's defensive capabilities increase, the noise generator receives more difficult feedback signals, thus generating more complex attack examples, and vice versa. This continuous adversarial process simulates the challenge of constantly evolving noise patterns in the real world, enabling the system to self-improve and adapt over the long term.

[0062] In some embodiments of this application, the reinforcement learning trainer includes an adversarial strategy optimization module and a robust training scheduling module.

[0063] The adversarial strategy optimization module is configured to: take into account the reward signal fed back by the large language model, the current policy parameters of the noise generator and the estimated value of the value function, calculate the policy gradient through the policy gradient method and output the parameter update instruction, and optimize the probability distribution of the text editing action of the noise generator.

[0064] The adversarial strategy optimization module focuses on improving the intelligence of the "attacker"—the noise generator. This module receives reward signals from the large language model, the noise generator's current policy parameters, and a value function estimate. It calculates the policy gradient using policy gradient methods (such as REINFORCE or PPO) and generates parameter update instructions. Its core task is to optimize the probability distribution of the noise generator's text editing actions, making it more inclined to choose action combinations that maximize cumulative rewards in each editing decision. Through value function estimation, this module can also reduce the variance of policy gradient estimation using an advantage function, improving the stability and efficiency of training. Essentially, this process allows the noise generator to continuously trial and error within a reinforcement learning framework, learning "how to attack smarter," thereby generating high-value adversarial examples that are semantically reasonable, grammatically correct, but sufficient to mislead the intent discriminator.

[0065] In some embodiments of this application, the state transition depends on the text editing action (e.g., replacement, deletion, insertion of token) performed by the noise generator, and the state transition function of the reinforcement learning environment satisfies the following calculation formula (6): (6); In formula (6), This represents the state transition function of the reinforcement learning environment, i.e., the state transition function in the current state. and actions The next state generated below, This represents an environment simulator driven by a Large Language Model (LLM), based on the current state. and actions Update status; Indicates based on action For the original clean text Noise injection (such as editing operations like replacement and deletion).

[0066] Furthermore, the policy gradient method is used for updating, satisfying the following calculation formula (7): (7); In formula (7), This indicates the update direction of the policy gradient, used to optimize the parameters of the policy network (Actor). ; Indicates the state Select action The probability (output by the policy network); Indicates the parameter Find the gradient; Let be the advantage function, representing the excess reward of the current action relative to the state value.

[0067] The robust training scheduling module is configured as follows: based on the optimized noise generator parameters output by the adversarial strategy optimization module, the noise generator generates adversarial samples with a fixed noise generator, the adversarial samples are mixed with the original clean text at a preset ratio as training data, the input is input to the intent discriminator and the gradient is calculated through the cross-entropy loss function, and the parameter update instructions of the intent discriminator are output.

[0068] The robustness training scheduling module is responsible for the training scheduling and parameter updates of the "defender"—the intent discriminator—and is the key execution unit for improving model robustness in the entire closed loop. This module fixes the latest noise generator parameters output by the adversarial strategy optimization module to ensure the stable and controllable generation process of adversarial examples. Subsequently, adversarial noisy text is generated in batches using this generator and mixed with the original clean text at a preset ratio (e.g., 1:1 or dynamically adjusted according to the training phase) to construct a training dataset containing real noise challenges. This mixed dataset is input into the intent discriminator, which calculates the error gradient between the predicted intent and the real intent using standard supervised learning methods (e.g., cross-entropy loss function) and outputs parameter update instructions for the intent discriminator. This process ensures that the intent discriminator is exposed to the latest adversarial examples generated by the strongest "opponent" in each training iteration, thereby continuously strengthening its noise resistance. Through the coordination of the scheduling module, the entire training process achieves an automated, periodic "generation-training" loop, ensuring that the intent discriminator continuously evolves in the face of the latest adversarial challenges.

[0069] In some embodiments of this application, the reinforcement learning trainer implements alternating training through a dynamic adversarial closed-loop mechanism: the first stage is led by the adversarial strategy optimization module, which iteratively optimizes the noise generator until the semantic similarity of the generated adversarial examples exceeds a preset similarity threshold and the grammatical error probability is lower than a preset error rate threshold. The second stage switches to the robust training scheduling module, which trains the intent discriminator with fixed noise generator parameters until the recognition accuracy of the intent discriminator on the adversarial example set stabilizes and converges.

[0070] Specifically, the first stage is led by the adversarial strategy optimization module, whose core task is to iteratively optimize the noise generator until the generated adversarial samples simultaneously meet the preset semantic similarity threshold and grammatical error rate threshold. The key significance of this stage is ensuring the "attack quality" meets the standards—that is, the generated adversarial samples must be semantically close enough to the original text while maintaining a reasonable grammatical structure (e.g., without obvious grammatical errors or meaningless word strings). By setting semantic similarity thresholds (such as sentence vector distance calculated based on BERTS score or cosine similarity) and grammatical error probability thresholds (which can be evaluated through language model perplexity or grammar checking tools), the system sets strict admission criteria for the noise generator's output. Only when the generator can consistently produce high-fidelity, highly challenging samples is it allowed to proceed to the next stage. This step prevents low-quality, unnatural adversarial samples from contaminating subsequent training processes, ensuring that the "noise resistance" learned by the intent discriminator is built upon realistic and credible noise patterns, thereby improving the model's generalization performance in real-world in-vehicle environments.

[0071] The second phase switches to the robust training scheduling module, fixing the parameters of the noise generator and using its validated high-quality adversarial examples to perform centralized robust training on the intent discriminator. The goal of this phase is to ensure that the intent discriminator achieves stable convergence in recognition accuracy on the adversarial example set after multiple rounds of iterative training on a mixed dataset containing adversarial noisy text and original clean text. The significance of this design is that by freezing the noise generator, the interference caused by the constantly changing "attacker" during training is eliminated, allowing the intent discriminator to focus on learning effective defense strategies under stable adversarial pressure. During training, the system continuously monitors the performance of the intent discriminator on the independent validation set, including metrics such as accuracy, confidence stability, and misclassification patterns, until performance fluctuations level off, indicating that the model has fully absorbed the challenge of the current adversarial examples and reached a local optimum. This "steady-state training" mechanism avoids overfitting or training oscillations, ensuring that the model learns robust semantic understanding capabilities rather than mechanical memorization of specific noise patterns.

[0072] In summary, this alternating training mechanism achieves a step-by-step, simultaneous improvement in attack and defense capabilities through a closed-loop process of "optimizing attacks → verifying quality → fixing attacks → strengthening defenses → evaluating convergence." The first stage ensures the authenticity and challenge of adversarial examples, providing a high-quality data foundation for subsequent training; the second stage maximizes the learning efficiency and final performance of the intent discriminator under stable input conditions. The alternating execution of the two stages forms a spiraling learning curve: with the completion of each cycle, the samples generated by the noise generator become increasingly "intelligent," while the defense capabilities of the intent discriminator become increasingly stronger. This mechanism not only significantly improves the reliability of in-vehicle voice intent recognition systems in complex noisy environments but also provides a scalable technical paradigm for building intelligent human-computer interaction systems with self-evolution capabilities.

[0073] In some embodiments of this application, the reward function includes: an intention prediction error reward, a confidence decrease reward, a semantic similarity penalty, and a syntax error penalty.

[0074] The intent prediction error reward is configured as follows: if the intent prediction result of the input adversarial text is inconsistent with the true intent of the original clean text, the first reward value is output according to the preset first weight coefficient; otherwise, the first reward value is output as 0.

[0075] The purpose of the intent prediction error reward is to directly measure the "attack success rate" of the noise generator, that is, whether the adversarial noise text it generates is sufficient to cause the intent discriminator to misjudge. This reward is based on the true intent of the original clean text. If the intent discriminator's prediction of the adversarial example is inconsistent with the true intent, a positive reward value (determined by a preset first weight coefficient) is output; otherwise, the reward is zero. This mechanism embodies the "success equals reward" principle in reinforcement learning, directly incentivizing the noise generator to explore perturbation paths that can effectively disrupt the decision boundary of the intent recognition model. This reward is the core driving force behind the entire adversarial process, ensuring that the noise generator continuously evolves towards improving its attack capabilities.

[0076] The confidence reduction reward is configured as follows: input the confidence of the intent prediction of the adversarial text and the confidence of the intent prediction of the original clean text, calculate the confidence reduction amount, and output the second reward value as the product of the preset second weight coefficient and the confidence reduction amount.

[0077] The confidence decline reward term measures the "deceptiveness" and "confidence erosion" of the attack, not just the correctness of the result. Even if the intent discriminator ultimately predicts correctly, if its confidence in predicting adversarial examples is significantly lower than its confidence in predicting the original text (e.g., dropping from 95% to 60%), it indicates that the example has interfered with its judgment and shaken the model's decision confidence. This reward term calculates the difference between the predicted confidence of the adversarial example and the original text, multiplies it by a second weighting coefficient, and outputs a positive reward. The significance of this design is to encourage the noise generator to generate "marginal" adversarial examples that, while not completely misleading the model, can significantly reduce its decision stability. Such examples are extremely common in real-world scenarios; for example, when a user's speech is slightly unclear, the system can barely recognize it but responds hesitantly. By training the model to handle such high-uncertainty inputs, its fault tolerance in complex environments and user experience can be significantly improved.

[0078] The semantic similarity penalty term is configured as follows: input adversarial noise text and original clean text, encode them into semantic vectors through a text encoder and calculate semantic similarity, output the first penalty value as the product of the difference in semantic similarity and the preset third weight coefficient.

[0079] The semantic similarity penalty term constrains the semantic drift of adversarial examples, ensuring they maintain a high degree of consistency with the original text in the semantic space. This prevents the generator from generating semantically completely deviating "invalid attacks" in pursuit of attack effects. This penalty term maps the original and adversarial texts into semantic vectors using a text encoder (such as BERT), calculates their similarity (e.g., cosine similarity), and then uses "1 minus the similarity" as a semantic difference measure, multiplied by a third weighting coefficient to form the penalty value. When the generated text deviates too much semantically, the penalty value increases, and the overall reward decreases, thus suppressing such behavior. For example, changing "navigate home" to "play music" results in extremely low semantic similarity, triggering a high penalty and forcing the generator to find more subtle perturbations (e.g., "navigate back to Jia"). This mechanism ensures the "realistic plausibility" of adversarial examples, making them closer to the semantic shifts that ASR systems might produce under real noise, rather than artificially created extreme anomalies.

[0080] The syntax error penalty is configured as follows: inputting adversarial text, calculating the syntax error probability through a pre-trained syntax checking model, and outputting a second penalty value as the product of the syntax error probability and a preset fourth weight coefficient.

[0081] The purpose of the grammatical error penalty term is to ensure the grammatical correctness and naturalness of the generated text, preventing the noise generator from outputting obviously grammatically incorrect sentences or text that does not conform to language habits. This penalty term evaluates the probability of grammatical errors in the adversarial noise text using a pre-trained grammar checking model (such as Gramformer or a Transformer-based grammar correction model) and multiplies it by a fourth weight coefficient as the penalty value. The higher the probability of grammatical errors, the heavier the penalty, thus guiding the generator to avoid generating obviously grammatically disordered samples such as "turn on the air conditioner" or "volume is high". The significance of this mechanism is to improve the "readability" and "credibility" of adversarial examples, ensuring that they can be correctly parsed by the intent discriminator and avoiding ineffective training due to grammatical collapse. In the in-vehicle scenario, ASR errors usually manifest as word-level substitutions or omissions rather than overall grammatical structure destruction; therefore, this penalty term makes the generated noise more consistent with real error patterns.

[0082] In some embodiments of this application, the reward function satisfies the following calculation formula (8): (8); In formula (8), Indicating the state in a reinforcement learning task Execute action The immediate rewards obtained are used to guide the policy network in generating adversarial text; This indicates an intention to predict incorrect reward items, where This indicates the indicator function; if the attack is successful (model predicts intent) ≠ True Intention If the condition is met, return 1; otherwise, return 0. Indicates the first weighting coefficient; This indicates the reward item for decreasing confidence level, among which The confidence level for predicting intent in the original clean text. The confidence level of the intention to combat noise in the text is represented by the text's anti-noise prediction. This represents the second weighting coefficient; This represents a semantic similarity penalty term. Representing anti-noise text and original clean text semantic similarity, Indicates the third weighting coefficient; This indicates a syntax error penalty item. Representing anti-noise text The probability of grammatical errors (output by a pre-trained grammar checker model). This represents the fourth weighting coefficient.

[0083] In summary, the multi-dimensional reward function of this application constructs a refined guidance mechanism through a "dual reward + dual penalty" structure: the reward for incorrect intent prediction and the reward for decreased confidence jointly drive the improvement of attack capability, while the semantic similarity penalty and the grammatical error penalty constrain the rationality and authenticity of the generation process. The four terms achieve flexible balance through adjustable weight coefficients, enabling the noise generator to find the optimal trade-off between "aggressiveness" and "fidelity." This design not only improves the quality of adversarial examples but also ensures the scientific rigor and practicality of the entire adversarial training process, providing a crucial guarantee for the high robustness of in-vehicle voice intent recognition systems in complex real-world environments.

[0084] In summary, the in-vehicle voice intent recognition system based on adversarial reinforcement learning provided in this application has the following technical effects.

[0085] This system significantly improves the robustness and generalization ability of the model in complex noisy environments by constructing a dynamic adversarial closed loop involving a noise generator, an intent discriminator, and a reinforcement learning trainer. Utilizing a large language model as an environmental feedback mechanism, the system drives the reinforcement learning-based noise generator to intelligently generate semantically relevant, grammatically sound, and high-fidelity adversarial examples. This realistically simulates common semantic and syntactic errors in in-vehicle speech recognition, effectively overcoming the semantic fragmentation problem caused by traditional data augmentation methods that rely on manual rules or random perturbations. Guided by a multi-dimensional reward function, the system improves the attack success rate while strictly controlling semantic bias and syntactic errors, ensuring the realism and challenge of the adversarial examples, thereby providing high-quality training data for the intent discriminator.

[0086] Furthermore, the system employs an alternating training mechanism, achieving the synergistic evolution of attack and defense capabilities through a closed-loop process of "optimizing attacks → strengthening defenses." As the intent discriminator is continuously exposed to dynamically upgraded adversarial examples, it gradually learns sensitivity and fault tolerance to subtle semantic perturbations, significantly improving recognition accuracy and confidence stability in real-world in-vehicle environments. The entire training process eliminates reliance on large-scale manually labeled noisy data, possessing self-evolving and scalable characteristics. This not only reduces data collection and labeling costs but also enhances the model's adaptability to future unknown noise patterns. Ultimately, while ensuring lightweight deployment feasibility, the system achieves high reliability and practicality of in-vehicle voice interaction in high-noise, multi-interference scenarios.

[0087] It should be noted that in all specific embodiments of this application, all data processing activities related to user identity or personal characteristics, such as user information, user behavior data, historical data, and location information, will be conducted in accordance with the principles of legality, legitimacy, and necessity. All data collection, use, storage, and processing will be subject to compliance with applicable national and regional laws, regulations, and industry standards, and informed consent from users will be obtained in a clear and explicit manner before processing. For the processing of sensitive personal information, separate consent from users will be obtained through prominent means such as pop-up prompts and independent confirmation pages. If any processing conflicts with laws and regulations, the laws and regulations will prevail, and necessary data processing will only be carried out within the scope permitted by laws and regulations, ensuring that all data-based applications, analyses, and technical implementations are conducted within the scope permitted by laws and regulations.

[0088] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this application are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0089] Furthermore, although this application is described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding this application. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of ordinary skill of an engineer. Therefore, those skilled in the art can implement the application set forth in the claims using ordinary skill. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of this application, which is determined by the full scope of the appended claims and their equivalents.

[0090] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several programs to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0091] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequential list of executable programs for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, a program execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can retrieve and execute a program from or in conjunction with such a program execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can mean any means that can contain, store, communicate, propagate, or transmit a program for use by or in conjunction with a program execution system, apparatus, or device.

[0092] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Additionally, computer-readable media can even be paper or other suitable media on which programs can be printed, for example, by optically scanning the paper or other media, then editing, interpreting, or, if necessary, processing it in a suitable manner to obtain the program electronically, and then storing it in computer memory.

[0093] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable program execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0094] In the foregoing description of this specification, the reference to terms such as "one embodiment / implementation," "another embodiment / implementation," or "certain embodiments / implementations," etc., indicates that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in an embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0095] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0096] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. An on-vehicle speech intent recognition system based on adversarial reinforcement learning, characterized by, The method comprises the following steps: a noise generator configured to generate adversarial noise text conforming to a vehicle context based on a reinforcement learning policy network, the noise generator being trained with a large language model as an environment model, and generating semantic correlation adversarial samples by outputting text editing actions; an intent discriminator configured to receive the adversarial noise text and the original clean text, and output an intent prediction result, a confidence and a probability distribution, the intent discriminator being robustly trained by the adversarial samples; a reinforcement learning trainer configured to construct a dynamic adversarial closed loop: fixing the parameters of the intent discriminator, training the noise generator by a reward function fed back by the large language model; fixing the parameters of the noise generator, and training the intent discriminator to improve the noise resistance; wherein the reward function comprises an intent prediction error reward item, a confidence decrease reward item, a semantic similarity penalty item and a syntax error penalty item. 2.The on-vehicle speech intent recognition system based on adversarial reinforcement learning according to claim 1, wherein, The adversarial noise text comprises semantic error samples and syntactic error samples, and the difference between the semantic distribution of the adversarial noise text and the semantic distribution of the original clean text is within a preset distribution difference threshold range. 3.The on-vehicle speech intent recognition system based on adversarial reinforcement learning according to claim 1, wherein, The noise generator comprises a state encoding module and a policy and value evaluation module; The state encoding module is configured to input the original clean text, perform context encoding by a text encoder, and output a word embedding matrix and a context representation vector of each word in the sequence; The policy and value evaluation module is configured to output a probability distribution of text editing actions and a value estimate of the current state based on the context representation vector, and receive a reward signal fed back by the large language model, optimize the policy parameters by a policy gradient method, and generate adversarial samples conforming to the vehicle context; wherein the probability distribution of the text editing actions is determined by calculating the replacement probability, the deletion execution probability or the insertion candidate word probability of each token position in the original text at the vocabulary level, and the value estimate is calculated by a feedforward network for advantage function optimization. 4.The on-board speech intent recognition system based on adversarial reinforcement learning according to claim 1, wherein, The intent discriminator comprises a context feature extraction module and an intent classification decision module; The context feature extraction module is configured to receive the adversarial noise text and the original clean text, perform word segmentation processing by a pre-trained backbone network, and output a token-level context hidden state matrix containing deep semantic information of the text; The intent classification decision module is configured to calculate an intent category probability distribution by a classification layer based on the context hidden state matrix, and output a predicted intent label and a corresponding confidence; wherein the backbone network of the context feature extraction module freezes the bottom layer parameters in the training stage of the adversarial noise text, and only fine-tunes the top Transformer layer to adapt to the vehicle scene semantics, and the classification layer is optimized by a cross-entropy loss function, so that the intent discriminator can be independently deployed as a lightweight model in the inference stage. The reinforcement learning trainer comprises an adversarial policy optimization module and a robustness training scheduling module; 5.The on-device speech intent recognition system based on adversarial reinforcement learning according to claim 1, wherein, ​ The adversarial strategy optimization module is configured to input the reward signal fed back by the large language model, the current strategy parameter of the noise generator, and the value function estimation value, calculate a strategy gradient by a policy gradient method, and output a parameter update instruction to optimize the text editing action probability distribution of the noise generator. The robustness training scheduling module is configured to generate an adversarial sample based on the optimized noise generator parameter output by the adversarial strategy optimization module, mix the adversarial sample and the original clean text at a preset ratio as training data, input the training data into the intent discriminator, and calculate a gradient by a cross-entropy loss function to output a parameter update instruction of the intent discriminator. 6.The on-vehicle speech intent recognition system based on adversarial reinforcement learning according to claim 5, characterized in that, The reinforcement learning trainer realizes alternating training through a dynamic adversarial closed-loop mechanism: in the first stage, the adversarial strategy optimization module is dominant, and the noise generator is iteratively optimized until the semantic similarity of the generated adversarial sample exceeds a preset similarity threshold and the syntax error probability is lower than a preset error rate threshold; In the second stage, the robustness training scheduling module is switched to, the noise generator parameter is fixed, and the intent discriminator is trained until the recognition accuracy of the intent discriminator on the adversarial sample set converges stably.

7. The adversarial reinforcement learning based in-vehicle speech intent recognition system of claim 1, wherein, The intent prediction error reward term is configured to input the intent prediction result of the adversarial noise text and the true intent of the original clean text, and if the intent prediction result is inconsistent with the true intent, output a first reward value according to a preset first weight coefficient, otherwise output the first reward value as 0. 8.The on-vehicle speech intent recognition system based on adversarial reinforcement learning of claim 1, wherein, The confidence decrease reward term is configured to input the intent prediction confidence of the adversarial noise text and the intent prediction confidence of the original clean text, calculate a confidence decrease amount, and output a second reward value as a product of a preset second weight coefficient and the confidence decrease amount. 9.The on-vehicle speech intent recognition system based on adversarial reinforcement learning of claim 1, wherein, The semantic similarity penalty term is configured to input the adversarial noise text and the original clean text, encode the texts into semantic vectors by a text encoder, calculate a semantic similarity, and output a first penalty value as a product of a difference between 1 and the semantic similarity and a preset third weight coefficient.

10. The adversarial reinforcement learning based in-vehicle speech intent recognition system of claim 1, wherein, The syntax error penalty term is configured to input the adversarial noise text, calculate a syntax error probability by a pre-trained syntax checking model, and output a second penalty value as a product of the syntax error probability and a preset fourth weight coefficient.

Images