A large model-based vehicle-mounted user intention recognition method and system

Abstract

The application discloses a vehicle-mounted user intention recognition method and system based on a large model, relates to a large language model, and comprises the following steps: constructing a double-tower retrieval model DREM; constructing an intention function library containing vehicle-mounted control functions; encoding each intention function text in the intention function library by using an intention coding encoder to obtain an intention vector set and establish an index; encoding user query text into a query vector by using a query coding encoder; obtaining an intention function associated with the query vector based on a hybrid retrieval strategy; wherein the hybrid retrieval strategy comprises semantic similarity retrieval and keyword matching retrieval; constructing a prompt template according to the user query text and the obtained associated intention function; performing reasoning by using a large model according to the prompt template to obtain a vehicle-mounted user intention recognition result; and improving the fault tolerance of the system to ASR recognition errors in view of the low user intention recognition accuracy caused by ASR speech recognition noise in a vehicle-mounted environment.

Description

Technical Field

[0001] This application relates to the field of large language models, and in particular to a method and system for recognizing user intent in vehicles based on large models. Background Technology

[0002] With the rapid development of intelligent vehicle technology, in-vehicle voice interaction has become an important way for modern cars to interact with humans. Through voice commands, drivers can control in-vehicle functions such as navigation, air conditioning, and music playback while driving, greatly improving driving safety and user experience. However, the unique characteristics of the in-vehicle environment bring serious challenges to voice interaction: environmental noise such as engine noise, wind noise, and road noise, as well as factors such as the driver's accent and speech rate variations, can all seriously affect the accuracy of automatic speech recognition (ASR) systems.

[0003] Existing in-vehicle intent recognition systems generally adopt a pipeline model, which is a serial processing flow of "voice → ASR → text → intent recognition model → user intent". In this architecture, the ASR module first converts the user's speech into text, and then the downstream Natural Language Understanding (NLU) module performs intent recognition on the text. Although text intent recognition technology based on pre-trained models such as BERT is relatively mature, in real-world in-vehicle scenarios, recognition errors from the ASR module are directly passed on to downstream tasks, leading to a significant degrade in overall system performance.

[0004] Specifically, ASR systems in in-vehicle environments frequently exhibit phoneme substitution errors, such as misrecognizing "turn on the air conditioner" as "open the air purifier" or "navigate home" as "navigate the navigation door." These errors are particularly common in noisy environments, with error rates exceeding 30%. More seriously, existing intent recognition models lack robust design against ASR noise and cannot effectively handle these recognition errors. When ASR outputs erroneous text, even if the erroneous text is phonetically similar to the correct text, downstream pure text intent recognition models struggle to recover the user's true intent, resulting in severe error propagation problems.

[0005] Therefore, improving the fault tolerance of in-vehicle intention recognition systems for ASR recognition errors, enabling them to accurately understand user intentions in high-noise environments, has become an urgent technical challenge. Summary of the Invention

[0006] To address the low accuracy of user intent recognition caused by ASR speech recognition noise in in-vehicle environments, this application provides an in-vehicle user intent recognition method and system based on a large model. By constructing a noise-robust dual-tower retrieval model DREM, ASR noise simulation and alignment loss function are introduced during the training phase to enable the model to learn noise-invariant features. During the inference phase, a hybrid retrieval strategy of generating noisy text variants and multi-vector aggregation is adopted. This method can accurately recognize user intent in high-noise in-vehicle environments and significantly improve the system's fault tolerance for ASR recognition errors.

[0007] One aspect of this application provides a method for vehicle user intent recognition based on a large model, comprising: constructing a dual-tower retrieval model (DREM), the DREM including a query encoder and an intent encoder, wherein the query encoder encodes the vehicle user query text into a query vector, and the intent encoder encodes the intent function text into an intent vector; constructing an intent function library containing vehicle control functions, the intent function library containing multiple intent functions and corresponding function description text; using the intent encoder to encode the text of each intent function in the intent function library to obtain an intent vector set and establish an index; receiving user query text and using the query encoder to encode the user query text into a query vector; obtaining the intent function associated with the query vector based on a hybrid retrieval strategy; wherein the hybrid retrieval strategy calculates the correlation between the query vector and the intent vector through semantic similarity retrieval and keyword matching retrieval; constructing a prompt template based on the user query text and the obtained associated intent functions; and using the large model to perform inference based on the prompt template to obtain the vehicle user intent recognition result.

[0008] Furthermore, a dual-tower retrieval model DREM is constructed, comprising a query encoder and an intent encoder. The process includes: collecting an original training dataset containing the mapping relationship between in-vehicle user query text and corresponding intent functions; performing ASR noise simulation on the query text and generating noisy text variants using a phoneme substitution algorithm; for each intent function, constructing a set of colloquial expressions for the intent function by expanding the colloquial description text; initializing the query encoder and intent encoder using a pre-trained language model; constructing positive and negative sample pairs based on the query text, noisy text variants, intent functions, and the set of colloquial expressions; training the dual-tower retrieval model DREM using a contrastive learning loss function based on the positive and negative sample pairs; and updating the parameters of the query encoder and intent encoder using a backpropagation algorithm to maximize the vector similarity of positive sample pairs and minimize the vector similarity of negative sample pairs.

[0009] Among them, the Dual-tower Retrieval Model (DREM) employs a dual-tower structure, consisting of two independent neural network encoders that process user queries and intent functions respectively. Its core feature is that it leverages contrastive learning to make the model robust to ASR noise, enabling it to map semantically identical but superficially different texts (such as those with speech recognition errors) to similar vector spaces, thereby achieving accurate intent matching in noisy environments.

[0010] The query encoder, the first tower of the dual-tower model, is responsible for converting the user's voice query text into a fixed-dimensional vector representation. In this scheme, the encoder is initialized based on a pre-trained language model (such as BERT) and adapted to the in-vehicle scenario by adding a dedicated query projection layer.

[0011] The intent encoder, the second tower of the dual-tower model, is specifically designed to encode the functional intent of in-vehicle systems. It transforms the standard description and colloquial expression of the intent function into vector form. It uses the same infrastructure as the query encoder but with independent parameters. Through the intent projection layer, it outputs a vector of the same dimension as the query vector, allowing for similarity calculations between the two in a unified vector space.

[0012] ASR noise specifically refers to recognition errors generated by automatic voice recognition systems in an in-vehicle environment. These errors mainly include: phoneme confusion: such as recognizing "navigation" as "daohang"; homophone errors: such as recognizing "air conditioning" as "kongtiao"; tone deviation: such as recognizing "open" (dǎ kāi) as "da kāi". These noises are caused by factors such as in-vehicle ambient noise, speaker accent, and speech rate, and are the main obstacles affecting the accuracy of intent recognition.

[0013] Intent functions are standardized, executable function interfaces in an in-vehicle system, representing specific in-vehicle functions that users can control via voice. For example: `AirConditioner.TurnOn()`: turns on the air conditioning; `Navigation.SetDestination(location)`: sets the navigation destination; `Window.Open(position)`: opens the window. Each intent function has a unique identifier, function description, and required parameters, serving as a bridge connecting user natural language commands with the actions executed by the in-vehicle system.

[0014] The phoneme substitution algorithm is based on a statistically obtained phoneme confusion matrix, and substitutes phonemes in the original text with easily confused phonemes with a certain probability. The specific process is as follows: convert the text into a phoneme sequence (e.g., "打开" → "da3 kai1"); find easily confused phoneme pairs according to the confusion matrix (e.g., "da" ↔ "ta"); generate noise variants by substitution according to the probability; restore the phoneme sequence to text (e.g., "ta3 kai1" → "他开"); this algorithm can truly simulate the error patterns of the ASR system and is used in the training stage to enhance the robustness of the model.

[0015] The set of colloquial expressions is a diverse set of natural language descriptions constructed for each intent function. Different from technical function names, these expressions are closer to users' daily language habits. For example, for the "AirConditioner.TurnOn()" function, its set of colloquial expressions may include: "打开空调", "开一下冷气", "把空调打开", "空调开下", "开启制冷"; constructing such a set can help the model learn different expressions of the same function and improve the coverage and accuracy of intent recognition.

[0016] Furthermore, according to the query text, noise text variants, intent functions, and the set of colloquial expressions, positive and negative sample pairs are constructed, including: for each mapping relationship in the original training dataset, the query text and all corresponding noise text variants are used as the query set; the original text of the intent function and all corresponding colloquial expressions are used as the positive intent set; each element in the query set is paired with each element in the positive intent set to form positive sample pairs; intent functions that do not correspond to the current query text are selected from the original training dataset, and the obtained non-corresponding intent functions and corresponding colloquial expressions are used as the negative intent set; each element in the query set is paired with each element in the negative intent set to form negative sample pairs; all positive sample pairs and negative sample pairs are combined to construct a training sample set.

[0017] Furthermore, a pre-trained language model is used to initialize the query encoder and intent encoder respectively, including: the pre-trained language model can be any one of BERT, RoBERTa, or ELECTRA.

[0018] Furthermore, according to the positive and negative sample pairs, a contrastive learning loss function is used to train the two-tower retrieval model DREM, including: for each training batch, positive and negative sample pairs are sampled from the training sample set; the query text in the positive and negative sample pairs is encoded into a query vector through the query encoder; the intent function text in the positive and negative sample pairs is encoded into an intent vector through the intent encoder; a loss function L is constructed: Where λ is the weighting coefficient, λ ∈ [0.1,0.5]. For scenarios with severe ASR noise: λ=0.3-0.5 (strengthen alignment constraints); for scenarios with better ASR quality: λ=0.1-0.2 (reduce alignment constraints). This represents the contrastive learning loss function. To align the loss function, the parameters of the query encoder and intent encoder are iteratively updated using the gradient descent algorithm based on the loss function L until the loss function converges.

[0019] Furthermore, a contrastive learning loss function is constructed. This maximizes the similarity of positive sample pairs and minimizes the similarity of negative sample pairs. Where q is the query vector. This is the positive sample intent vector. τ is the negative sample intent vector, τ is the temperature coefficient, and sim(*) is the cosine similarity function;

[0020] In this context, the positive sample intent vector refers to the vector representation obtained after the intent function, which semantically matches the query text, is encoded by the intent encoder. Specifically, when a user queries "turn on the air conditioner", the corresponding positive sample intent vector is the vector encoded by the "AirConditioner.TurnOn()" function and its colloquial expression (such as "turn on the air conditioning system"). Positive samples include the vector of the original intent function and the vectors of all colloquial expressions of that function. During training, the model aims to maximize the similarity between the query vector and its corresponding positive sample intent vector. Each query may correspond to multiple positive sample intent vectors (different expressions of the same function).

[0021] Negative sample intent vectors refer to vectors encoded by other intent functions that do not semantically match the query text. In in-vehicle scenarios: for example, when a user queries "turn on the air conditioner", the vectors of other functions such as "Navigation.SetDestination()" (navigation settings) and "Radio.ChangeStation()" (switching radio stations) are all negative samples. The selection strategies for negative samples include: intra-batch negative samples: intent vectors corresponding to other queries in the same training batch; hard negative samples: intent vectors that are literally similar to the query but semantically different (such as "open the sunroof" vs. "turn on the air conditioner"). The model training objective is to minimize the similarity between the query vector and the negative sample intent vector.

[0022] The temperature coefficient is a hyperparameter controlling sample discrimination in contrastive learning. In this scheme, it plays a role in adjusting the sharpness of the similarity distribution. Due to the presence of ASR noise, an appropriate temperature coefficient allows the model to maintain a certain tolerance for noise variants while distinguishing intentions. τ ∈ [0.05, 0.5], in the early stage of training: τ=0.2-0.3 (larger value, allowing the model to explore); in the middle stage of training: τ=0.1 (standard value); in the later stage of training: τ=0.05-0.07 (smaller value, finer discrimination).

[0023] Specifically, a correct mapping between query and intent is established in the semantic space. By maximizing the similarity of positive sample pairs and minimizing the similarity of negative sample pairs, the model learns to distinguish between different user intents.

[0024] Furthermore, an alignment loss function is constructed. This ensures that the query vectors of different noisy text variants of the same query text remain similar: ;in, and Different noisy text variations of the same query text Indicates a query encoder;

[0025] Specifically, structured constraints are imposed on the output space of the query encoder, forcing different noisy variants of the same query to produce similar vector representations. This constraint gives the query encoder an inherent tolerance to ASR errors.

[0026] Furthermore, the intent function associated with the query vector is obtained based on a hybrid retrieval strategy, including: generating T noisy text variants from the received user query text using the same phoneme substitution algorithm as during training; inputting the original user query text and the T noisy text variants into the query encoder to obtain T+1 query vectors; calculating the cosine similarity between each query vector and each intent vector in the intent vector set for each query vector; for each intent function, aggregating the T+1 similarity values ​​calculated from the corresponding intent vector and the T+1 query vectors, and calculating the average as the semantic similarity score of the corresponding intent function; and selecting the top K semantic similarity scores from the intent vector set. The intent function serves as the semantic retrieval candidate set. The original user query text is segmented to extract a keyword set. Keywords are nouns, verbs, and adjectives obtained after segmentation. The keyword matching degree between the keyword set and the functional description text of each intent function in the intent function library is calculated to obtain a keyword matching score. The top M intent functions with the highest keyword matching scores are selected from the intent function library as the keyword retrieval candidate set. The semantic retrieval candidate set and the keyword retrieval candidate set are merged to obtain a mixed candidate set. For each intent function in the mixed candidate set, a comprehensive relevance score S is calculated. The intent functions in the mixed candidate set are sorted according to the comprehensive relevance score S, and the top N intent functions with the highest scores are selected as the intent functions associated with the query vector.

[0027] Calculate the overall relevance score S: ;in, The aggregated semantic similarity score. The score is the keyword matching score, and α is the weight parameter, α∈[0.6,0.9]. Standard in-vehicle scenario: α=0.7-0.8; high noise environment: α=0.8-0.9; clear voice environment: α=0.6-0.7.

[0028] In particular, traditional retrieval methods use only the single text output by the ASR for vectorization and retrieval during inference. This approach has a fundamental flaw: when the ASR produces an error, the vector representation of the erroneous text may deviate significantly from the correct semantic space; even if the model has seen similar noise during training, single-point estimation is still prone to falling into erroneous local regions.

[0029] This scheme achieves a shift from single-point estimation to distributed estimation by actively generating T noise variants during inference. It uses a phoneme substitution algorithm to simulate possible error patterns in ASR, which is equivalent to "reverse perturbation" of the ASR output. The generated T variants form a neighborhood of the original query in the noise space, increasing the probability of hitting the correct semantics. Even if the original ASR output is seriously flawed, as long as some of the generated variants are close to the real pronunciation, the correct semantics can be recovered through aggregation.

[0030] Another aspect of this application provides a vehicle-mounted user intent recognition system based on a large model, comprising: a dual-tower retrieval model construction module for constructing a dual-tower retrieval model DREM, wherein the dual-tower retrieval model DREM includes a query encoder and an intent encoder, wherein the query encoder is used to encode vehicle-mounted user query text into query vectors, and the intent encoder is used to encode intent function text into intent vectors; an intent function library construction module for constructing an intent function library containing vehicle control functions, wherein the intent function library contains multiple intent functions and corresponding function description texts; the intent function library construction module is further used to encode each intent function text in the intent function library using the intent encoder to obtain an intent vector set and establish an index; a query encoding module for receiving user query text and encoding the user query text into query vectors using the query encoder; a hybrid retrieval module for obtaining intent functions associated with query vectors based on a hybrid retrieval strategy; wherein the hybrid retrieval strategy calculates the correlation between query vectors and intent vectors through semantic similarity retrieval and keyword matching retrieval; and a large model inference module for constructing a prompt template based on the user query text and the obtained associated intent functions; and using the large model for inference based on the prompt template to obtain the vehicle-mounted user intent recognition result.

[0031] Compared to existing technologies, the advantages of this application are:

[0032] Through ASR noise simulation and alignment loss function during the training phase The joint optimization enables the dual-tower retrieval model DREM to learn noise-invariant semantic representations, meaning that different noise variants of the same query are mapped to similar vector spaces. During the inference phase, by actively generating noise text variants and aggregating the similarity of multiple query vectors, the noise robustness features learned by the model during training are fully utilized. This allows the system to accurately locate the correct intent function even when typical recognition errors such as phoneme substitution and homophone errors occur in ASR speech recognition (e.g., "turn on the air conditioner" is recognized as "open the empty strip"). Attached Figure Description

[0033] This application will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting; in these embodiments, the same reference numerals denote the same structures, wherein:

[0034] Figure 1 This is an exemplary flowchart of a vehicle-mounted user intent recognition method based on a large model, according to some embodiments of this application;

[0035] Figure 2This is an exemplary flowchart illustrating the construction of a dual-tower retrieval model (DERM) according to some embodiments of this application;

[0036] Figure 3 This is an exemplary flowchart illustrating the calculation of the comprehensive score S according to some embodiments of this application. Detailed Implementation

[0037] The methods and systems provided in the embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0038] like Figure 1 As shown, a dual-tower retrieval model DREM is constructed, comprising a query encoder and an intent encoder. The query encoder encodes the in-vehicle user query text into a query vector, while the intent encoder encodes the intent function text into an intent vector. An intent function library containing in-vehicle control functions is constructed, comprising multiple intent functions and their corresponding function description texts. The intent encoder encodes each intent function text in the intent function library to obtain a set of intent vectors and establishes an index. User query text is received, and the query encoder encodes it into a query vector. Intent functions associated with the query vectors are obtained based on a hybrid retrieval strategy. This hybrid retrieval strategy calculates the correlation between the query vector and the intent vector through semantic similarity retrieval and keyword matching retrieval. A prompt template is constructed based on the user query text and the obtained associated intent functions. Based on the prompt template, a large model is used for inference to obtain the in-vehicle user intent recognition result.

[0039] like Figure 2 As shown, construct the training dataset: collect the original dataset. ,in, For in-vehicle users to query text, For the corresponding intent function;

[0040] For each query text m noise variants are generated using a phoneme substitution algorithm. The phoneme replacement algorithm replaces the original phonemes with easily confused phonemes based on the confusion matrix; specifically, it replaces the query text with easily confused phonemes. Convert to a phoneme sequence. For example, "Turn on the air conditioner" is converted to "da3 kai1 kong1 tiao2". For each phoneme in the phoneme sequence, perform the following replacement: Find all the confusable phonemes of this phoneme according to the confusion matrix; randomly decide whether to replace according to the confusion probability (usually set the overall replacement rate to 20% - 30%); if it is decided to replace, select the target phoneme to be replaced according to the probability distribution. Convert the replaced phoneme sequence back to text form. For example: "da3 kai1 kong1 tiao2" may become "da4kai1 kong1 tiao4", corresponding to the text "大开空条". For each query text Repeat steps 2 - 3 a total of m times (usually m = 5 - 10) to generate m different noise variants. Ensure the diversity of the variants and avoid generating exactly the same variants.

[0041] For each intent function , construct a set of colloquial expressions , containing k semantically equivalent colloquial texts; specifically, decompose each intent function into two parts: action and object. For example, "AirConditioner.TurnOn" is decomposed into the action "Turn on" and the object "air conditioner". Synonym expansion: Action synonyms: Turn on → {Open, Turn on, Start, Turn on for a moment, Turn on briefly}; Object synonyms: air conditioner → {air conditioner, cold air, refrigeration, air conditioning system}. Colloquial combination: Combine different expressions of the action and the object, and add colloquial modifiers: Formal expression: "Turn on the air conditioner"; Colloquial expressions: "Turn on the air conditioner for a moment", "Turn on the cold air", "Turn on the air conditioner briefly"; Add modal particles: "Please turn on the air conditioner", "Please help me turn on the air conditioner".

[0042] Initialize the query encoder and the intent encoder respectively using a pre-trained language model; the pre-trained language model includes any one of BERT, RoBERTa or ELECTRA. Load the weight parameters of the pre-trained language model as the initialization parameters of the query encoder; load the weight parameters of the pre-trained language model as the initialization parameters of the intent encoder; add a query projection layer at the output end of the query encoder to map the hidden layer output of the pre-trained language model to a query vector with a fixed dimension; add an intent projection layer at the output end of the intent encoder to map the hidden layer output of the pre-trained language model to an intent vector with the same dimension as the query vector.

[0043] Construct training sample pairs:

[0044] Set of positive sample pairs , where represents the original query text, represents the original intent function text;

[0045] negative sample pair set ;

[0046] Define the loss function: ,in: , , Indicates query The set of all variants, This represents the query encoder, where λ is the weight coefficient.

[0047] The encoder parameters are updated using the backpropagation algorithm, and training is iterated until the loss function converges.

[0048] Build an intent function library that includes vehicle control functions:

[0049] Building an intent function library Each intent function Includes: function identifier : Uniquely identifies each intent function; function name Standardized function names, such as "AirConditioner.TurnOn"; function description text. : Describe the function's purpose in detail, such as "turn on the vehicle's air conditioning system"; parameter list The set of parameters required for this function, such as temperature value and wind speed level;

[0050] Includes all control functions supported by the in-vehicle system, including but not limited to: air conditioning controls: temperature adjustment, fan speed adjustment, and switching between internal and external air circulation; navigation controls: destination setting, route planning, and map zooming; multimedia controls: volume adjustment, song switching, and radio station selection; window controls: window opening and closing, and sunroof control; and seat controls: seat heating, position adjustment, and massage function.

[0051] For each intent function Fᵢ, generate multi-level descriptive text: Standard description: describe the function using standardized technical language; Colloquial description: add common colloquial expressions; Synonym expansion: include function-related synonyms and near-synonyms;

[0052] For each function in the intent function library : Function name and functional description text Combined to form a complete text: Use a pre-trained intent encoder right Encode: ; Obtain the set of intent vectors ,in, d is the vector dimension;

[0053] Build a vector index using an approximate nearest neighbor search algorithm (such as FAISS or Annoy): Select an index type: IVF (inverted file) index or HNSW (hierarchical navigable small world) index; Set index parameters: number of cluster centers, size of the probe list, etc.; Add the set of intent vectors V to the index and establish a mapping relationship from vector ID to function ID; Save the index file and the mapping table for online retrieval.

[0054] Obtain the query text converted by the user through the ASR system ; Record the query timestamp and session context information; For the query text Perform basic preprocessing: Remove leading and trailing spaces and extra whitespace characters; Unify full-width and half-width characters; Preserve the original text's case information.

[0055] Use the trained query encoder Encode the preprocessed text: Input the query text into the query encoder: ; Obtain the query vector through the query projection layer: ; Perform L2 normalization on the query vector: ; Among them, , which has the same dimension as the intent vector;

[0056] Apply the same phoneme replacement algorithm as in training to generate T noisy text variants: , where is the original text; Encode each variant separately to obtain a set of query vectors: ;

[0057] As Figure 3 shown, obtain the intent function associated with the query vector based on a hybrid retrieval strategy.

[0058] After receiving the user's original query text, use the same phoneme replacement algorithm as in the training phase to generate T noisy variants (usually T = 3 - 5). For example, when the user inputs "Open the sunroof", the system generates variants: Variant 1: "Greatly open the sunroof" (confusion of initial consonant d → d); Variant 2: "Open the sky bed" (confusion of final ang → uang); Variant 3: "Open the adding window" (tone change).

[0059] Input the original query text and T noisy variants into the query encoder respectively, and a total of T + 1 query vectors are obtained. The encoding process for each text includes: text tokenization and sub-word segmentation; Extract features through the multi-layer Transformer structure of the query encoder; Map to a unified dimensional space through the query projection layer; L2 normalization processing to ensure that the vector norm is 1. Store the T + 1 query vectors in a temporary vector set, denoted as , where The query vector is the original text.

[0060] right Each query vector in , with each intent vector in the intent vector set V Calculate cosine similarity: Using vectorization, calculate the similarity between a query vector and all intent vectors simultaneously. Since the vectors have already been normalized, cosine similarity simplifies to a vector dot product operation. Construct a similarity matrix. ,in, This represents the similarity value between the j-th query vector and the ith intent vector, with a value range of [-1, 1].

[0061] For each intent function Collect its intent vector Similarity values ​​with all T+1 query vectors: Similarity set: ; Calculate the average value: By aggregating multiple variants, even if the original query is misidentified by ASR, a high semantic similarity score can still be obtained as long as some noisy variants are close to the correct pronunciation.

[0062] All intent functions are scored according to semantic similarity. Sort the candidate functions in descending order; select the top K intent functions with the highest scores (usually K=10-15); record the semantic similarity score of each candidate function for subsequent comprehensive scoring. Set a minimum similarity threshold (e.g., 0.5) to filter out candidates with too low similarity to ensure the quality of the candidate set.

[0063] Perform Chinese word segmentation on the original query text: use a word segmentation tool (such as jieba) to segment the text; tag the part of speech to identify the part of speech of each word; extract nouns (such as "air conditioner" and "temperature"), verbs (such as "open" and "adjust") and adjectives (such as "maximum" and "cool"); filter out stop words and meaningless function words.

[0064] For each function in the intent function library Extract function description text The set of keywords in the query keyword set; calculate the intersection of the query keyword set and the description keyword set; match degree calculation: S_keyword[i] = |intersection| / |query keyword set|; consider synonym matching to improve the recall rate of the match.

[0065] Sort all intent functions in descending order of keyword matching score S_keyword, select the top M intent functions with the highest scores (usually M=5-10), and save the keyword matching score of each candidate.

[0066] The semantic retrieval candidate set and the keyword retrieval candidate set are combined to remove duplicate intent functions. Two scores are saved for each candidate function: semantic similarity score and keyword matching score. For functions that appear only in one candidate set: only in the semantic candidate set: keyword score is set to 0; only in the keyword candidate set: semantic score is set to 0.

[0067] Calculate a composite score for each intent function in the mixed candidate set: ;in: The semantic similarity score is normalized to [0,1]; The keyword matching score is normalized to [0,1]; α is the weight parameter, usually set to 0.7-0.8, indicating a greater emphasis on semantic similarity. All candidate functions are sorted in descending order of their comprehensive score S; the top N functions are selected as the final result (usually N=3-5); a list of functions and their corresponding scores is returned.

[0068] Build a structured prompt template, including the following key parts:

[0069] System role description: "You are an in-vehicle intelligent assistant, responsible for understanding the user's control intentions and selecting the correct in-vehicle functions."

[0070] Task Description: "The user has issued control commands via voice. Due to potential errors in voice recognition, please carefully analyze the user's true intent."

[0071] Contextual information injection: User's original query: "[Insert user query text]"; Candidate function list: List the N intent functions retrieved and their descriptions one by one; Relevance score: The overall score of each candidate function;

[0072] Decision guidance: "Analyze the degree of match between user queries and each candidate function, consider possible speech recognition errors, and select the function that best matches the user's intent."

[0073] Output format requirements:

[0074] Please output in the following format:

[0075] User intent analysis: [Analyzing what users want];

[0076] Selected function: [Function Name];

[0077] Confidence level: [High / Medium / Low];

[0078] Parameter extraction: "[Extract relevant parameters if necessary]";

[0079] Choose a suitable large language model (such as GPT-3.5, GPT-4, or other open-source models). Populate the template with the user query and candidate function information to generate complete prompt text. Input the constructed prompt into the large model and obtain the generated analysis results. Through the combination of hybrid retrieval and large model inference, the system can accurately understand the user's true intent even when ASR recognition errors occur, significantly improving the user experience of in-vehicle voice interaction.

[0080] The foregoing illustrative description of the present application and its embodiments is not restrictive and can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. The accompanying drawings are only one embodiment of the present application, and the actual structure is not limited thereto. Therefore, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the present application, such designs should fall within the scope of protection of this application. Furthermore, the word "comprising" does not exclude other elements or steps, and the word "a" preceding an element does not exclude the inclusion of "a plurality" of that element. Terms such as "first," "second," etc., are used to indicate names and do not indicate any specific order.

Claims

1. A method for recognizing in-vehicle user intent based on a large model, characterized in that, include: A dual-tower retrieval model DREM is constructed, which includes a query encoder and an intent encoder. The query encoder encodes the query text of the vehicle user into a query vector, and the intent encoder encodes the intent function text into an intent vector. The intent function text consists of the name of the intent function and the corresponding functional description text. Build an intent function library that includes vehicle control functions. The intent function library contains multiple intent functions and corresponding function description text. The intent encoder is used to encode the text of each intent function in the intent function library to obtain a set of intent vectors and create an index; Receive user query text and use a query encoder to encode the user query text into a query vector; The intent function associated with the query vector is obtained based on a hybrid retrieval strategy; wherein, the hybrid retrieval strategy calculates the degree of association between the query vector and the intent vector through semantic similarity retrieval and keyword matching retrieval. Based on the user's query text and the associated intent function, construct a prompt template; Based on the prompt template, reasoning is performed using a large model to obtain the in-vehicle user intent recognition result; Based on positive and negative sample pairs, a dual-tower retrieval model DREM is trained using a contrastive learning loss function, including: For each training batch, positive and negative sample pairs are sampled from the training sample set. The query encoder encodes the query text in the positive and negative sample pairs into query vectors. The intent function text in positive and negative sample pairs is encoded into intent vectors using an intent encoder. Construct the loss function L: , where λ is the weighting coefficient; This represents the contrastive learning loss function. Alignment loss function; Based on the loss function L, the parameters of the query encoder and intent encoder are iteratively updated using the gradient descent algorithm until the loss function converges. Constructing a contrastive learning loss function This maximizes the similarity of positive sample pairs and minimizes the similarity of negative sample pairs. Where q is the query vector. This is the positive sample intent vector. For negative sample intent vectors, Let be the temperature coefficient, and sim(*) be the cosine similarity function; where, temperature coefficient It is a hyperparameter that controls the discriminative power of samples in contrastive learning, and is used to adjust the sharpness of the similarity distribution; Constructing the alignment loss function This ensures that the query vectors of different noisy text variants of the same query text remain similar: ;in, and Different noisy text variations of the same query text. This indicates a query encoder; where, Indicates query The set of all variants; The construction of positive and negative sample pairs includes: Based on each mapping relationship in the original training dataset, the query text and all corresponding noisy text variants are used as the query set; The original text of the intent function and all its corresponding colloquial expressions are taken as the set of positive intents; Each element in the query set is paired with each element in the positive intent set to form a positive sample pair; Select intent functions that do not correspond to the current query text from the original training dataset, and use the obtained non-corresponding intent functions and their corresponding colloquial expressions as a set of negative intents; Each element in the query set is paired with each element in the negative intent set to form a negative sample pair; Combine all positive and negative sample pairs to construct a training sample set.

2. The in-vehicle user intent recognition method based on a large model according to claim 1, characterized in that: Constructing a dual-tower retrieval model DREM includes: Collect the original training dataset, which includes the mapping relationship between in-vehicle user query text and corresponding intent functions; ASR noise simulation is performed on the query text, and noisy text variants are generated through a phoneme replacement algorithm; For each intent function, a set of colloquial expressions for the intent function is constructed by expanding the colloquial description text; The query encoder and intent encoder are initialized using pre-trained language models; Construct positive and negative sample pairs based on query text, noisy text variants, intent functions, and a set of colloquial expressions; The dual-tower retrieval model DREM is trained using a contrastive learning loss function based on positive and negative sample pairs. The parameters of the query encoder and intent encoder are updated by backpropagation algorithm to maximize the vector similarity of positive sample pairs and minimize the vector similarity of negative sample pairs.

3. The in-vehicle user intent recognition method based on a large model according to claim 1, characterized in that: The query encoder and intent encoder are initialized using pre-trained language models, including: Pre-trained language models include any of BERT, RoBERTa, or ELECTRA.

4. The in-vehicle user intent recognition method based on a large model according to claim 2 or 3, characterized in that: The intent function associated with the query vector is obtained based on a hybrid retrieval strategy, including: The received user query text is augmented to generate multiple text variants; Encode the original user query text and the generated text variants into multiple query vectors; Similarity is calculated based on multiple query vectors and intent vector sets, and the semantic similarity score of each intent function is obtained by aggregating multiple similarity results; The first candidate intent function set is selected based on semantic similarity scores; Perform text analysis on the user's query text, extract text features and match them with the intent function library to obtain the text matching score for each intent function; The second set of candidate intent functions is selected based on the text matching score; By fusing the first candidate intent function set and the second candidate intent function set, a hybrid candidate set is obtained; The intent functions in the mixed candidate set are comprehensively scored S, and the intent function associated with the query vector is determined based on the comprehensive score S.

5. The in-vehicle user intent recognition method based on a large model according to claim 4, characterized in that: Calculate the overall score S: ,in, The aggregated semantic similarity score. The score represents the keyword matching score, and α is the weight parameter.

6. A vehicle-mounted user intent recognition system based on a large model, characterized in that, include: The dual-tower retrieval model construction module is used to construct a dual-tower retrieval model DREM. The dual-tower retrieval model DREM includes a query encoder and an intent encoder. The query encoder is used to encode the query text of the vehicle user into a query vector, and the intent encoder is used to encode the intent function text into an intent vector. The intent function library construction module is used to construct an intent function library containing vehicle control functions. The intent function library contains multiple intent functions and corresponding function description texts. The intent function library construction module is also used to encode each intent function text in the intent function library using an intent encoder to obtain an intent vector set and establish an index. The query encoding module is used to receive user query text and encode the user query text into a query vector using the query encoder; The hybrid retrieval module is used to obtain the intent function associated with the query vector based on a hybrid retrieval strategy. The hybrid retrieval strategy calculates the correlation between the query vector and the intent vector through semantic similarity retrieval and keyword matching retrieval. The large model inference module is used to construct a prompt template based on the user's query text and the obtained associated intent function; and to use the large model to infer based on the prompt template to obtain the in-vehicle user intent recognition result. Based on positive and negative sample pairs, a dual-tower retrieval model DREM is trained using a contrastive learning loss function, including: For each training batch, positive and negative sample pairs are sampled from the training sample set. The query encoder encodes the query text in the positive and negative sample pairs into query vectors. The intent function text in positive and negative sample pairs is encoded into intent vectors using an intent encoder. Construct the loss function L: , where λ is the weighting coefficient; This represents the contrastive learning loss function. Alignment loss function; Based on the loss function L, the parameters of the query encoder and intent encoder are iteratively updated using the gradient descent algorithm until the loss function converges. Constructing a contrastive learning loss function This maximizes the similarity of positive sample pairs and minimizes the similarity of negative sample pairs. Where q is the query vector. This is the positive sample intent vector. Let τ be the negative sample intent vector, τ be the temperature coefficient, and sim(*) be the cosine similarity function; where the temperature coefficient is... It is a hyperparameter that controls the discriminative power of samples in contrastive learning, and is used to adjust the sharpness of the similarity distribution; Constructing the alignment loss function This ensures that the query vectors of different noisy text variants of the same query text remain similar: ;in, and Different noisy text variations of the same query text. Indicates a query encoder; The construction of positive and negative sample pairs includes: Based on each mapping relationship in the original training dataset, the query text and all corresponding noisy text variants are used as the query set; The original text of the intent function and all its corresponding colloquial expressions are taken as the set of positive intents; Each element in the query set is paired with each element in the positive intent set to form a positive sample pair; Select intent functions that do not correspond to the current query text from the original training dataset, and use the obtained non-corresponding intent functions and their corresponding colloquial expressions as a set of negative intents; Each element in the query set is paired with each element in the negative intent set to form a negative sample pair; Combine all positive and negative sample pairs to construct a training sample set; The intent function text consists of the name of the intent function and its corresponding functional description text.

Images