Vehicle-mounted user intention recognition method and system based on large model

Abstract

The invention discloses a vehicle-mounted user intention recognition method and system based on a large model, and relates to a large language model. The method comprises the steps that a double-tower retrieval model DREM is constructed; constructing an intention function library containing a vehicle-mounted control function; encoding each intention function text in the intention function library by using an intention encoding encoder to obtain an intention vector set and establish an index; encoding a user query text into a query vector by using a query encoding encoder; obtaining an intention function associated with the query vector based on a hybrid retrieval strategy; wherein the mixed retrieval strategy is subjected to semantic similarity retrieval and keyword matching retrieval; constructing a prompt template according to a user query text and the obtained associated intention function; reasoning by using the large model according to the prompt template to obtain a vehicle-mounted user intention recognition result; aiming at low user intention recognition accuracy caused by ASR voice recognition noise in a vehicle-mounted environment, the fault-tolerant capability of the system for ASR recognition errors is improved.

Description

Technical Field

[0001] This application relates to the field of large language models, and particularly to a vehicle-mounted user intention recognition method and system based on a large model. Background Art

[0002] With the rapid development of intelligent vehicle technology, vehicle-mounted voice interaction has become an important way of human-vehicle interaction in modern vehicles. Through voice commands, drivers can control vehicle-mounted functions such as navigation, air conditioning, and music playback during driving, greatly improving driving safety and user experience. However, the particularity of the vehicle-mounted environment poses severe challenges to voice interaction: environmental noises such as engine noise, wind noise, and road noise, as well as factors such as the driver's dialect accent and speech rate changes, will seriously affect the accuracy of the automatic speech recognition (ASR) system.

[0003] Existing vehicle-mounted intention recognition systems generally adopt the pipeline mode, that is, the serial processing flow of "voice → ASR → text → intention recognition model → user intention". In this architecture, the ASR module first converts the user's voice into text, and then the downstream natural language understanding (NLU) module performs intention recognition on the text. Although the text intention recognition technology based on pre-trained models such as BERT has been relatively mature, in actual vehicle-mounted scenarios, the recognition errors of the ASR module will be directly transmitted to downstream tasks, resulting in a serious decline in the overall system performance.

[0004] Specifically, the ASR system in the vehicle-mounted environment often has phoneme substitution errors. For example, "turn on the air conditioner" is recognized as "turn on the empty conditioner", or "navigate home" is recognized as "navigate to home". Such errors are particularly common in noisy environments, and the error rate can reach more than 30%. More seriously, the existing intention recognition models lack a robust design for ASR noise and cannot effectively handle these recognition errors. When the ASR outputs incorrect text, even if the incorrect text is phonetically similar to the correct text, it is difficult for the downstream pure text intention recognition model to recover the user's true intention, resulting in a serious error transmission problem.

[0005] Therefore, how to improve the fault tolerance ability of the vehicle-mounted intention recognition system for ASR recognition errors and enable it to accurately understand the user's intention in a high-noise environment has become an urgent technical problem to be solved. Summary of the Invention

[0006] In response to the low accuracy of user intent recognition caused by ASR speech recognition noise in an in-vehicle environment, this application provides an in-vehicle user intent recognition method and system based on a large model. By constructing a noise-robust dual-tower retrieval model DREM, ASR noise simulation and alignment loss function are introduced in the training phase to enable the model to learn noise-invariant features. In the inference phase, a hybrid retrieval strategy of noise text variant generation and multi-vector aggregation is adopted. It can accurately recognize user intent in a high-noise environment in a vehicle and significantly improve the system's fault tolerance to ASR recognition errors.

[0007] One aspect of the present application provides a method for identifying in-vehicle user intentions based on a large model, including: constructing a dual-tower retrieval model DREM, the dual-tower retrieval model DREM including a query encoding encoder and an intention encoding encoder, wherein the query encoding encoder encodes the in-vehicle user query text into a query vector, and the intention encoding encoder encodes the intention function text into an intention vector; constructing an intention function library containing in-vehicle control functions, the intention function library containing multiple intention functions and corresponding function description texts; using the intention encoding encoder to encode each intention function text in the intention function library to obtain an intention vector set and establish an index; receiving a user query text, and using the query encoding encoder to encode the user query text into a query vector; obtaining the intention function associated with the query vector based on a hybrid retrieval strategy; wherein the hybrid retrieval strategy calculates the correlation between the query vector and the intention vector through semantic similarity retrieval and keyword matching retrieval; constructing a prompt template based on the user query text and the obtained associated intention function; and using the large model for reasoning based on the prompt template to obtain the in-vehicle user intention recognition result.

[0008] Furthermore, a dual-tower retrieval model DREM is constructed, which includes a query encoding encoder and an intent encoding encoder, including: collecting an original training data set, which includes a mapping relationship between the in-vehicle user query text and the corresponding intent function; performing ASR noise simulation on the query text, and generating a noise text variant through a phoneme replacement algorithm; for each intent function, constructing a set of spoken expressions of the intent function by expanding the spoken description text; using a pre-trained language model to initialize the query encoder and the intent encoder respectively; constructing positive and negative sample pairs based on the query text, noise text variants, intent functions, and spoken expression sets; training the dual-tower retrieval model DREM based on the positive and negative sample pairs using a contrastive learning loss function; updating the parameters of the query encoder and the intent encoder through a back-propagation algorithm to maximize the vector similarity of the positive sample pairs and minimize the vector similarity of the negative sample pairs.

[0009] The Dual-tower Retrieval Model (DREM) employs a dual-tower architecture, with two independent neural network encoders processing user queries and intent functions, respectively. Its core feature is its ability to achieve ASR noise robustness through contrastive learning. This allows it to map text with the same semantics but different surface forms (e.g., speech recognition errors) into similar vector spaces, enabling accurate intent matching in high-noise environments.

[0010] The query encoder, the first tower in the dual-tower model, is responsible for converting the user's voice query text into a fixed-dimensional vector representation. In this solution, the encoder is initialized based on a pre-trained language model (such as BERT) and adapted for in-vehicle scenarios by adding a specialized query projection layer.

[0011] The intent encoder, the second tower in the dual-tower model, is specifically responsible for encoding the functional intent of the vehicle system. It converts standard descriptions and colloquial expressions of the intent function into a vector. It shares the same basic architecture as the query encoder but with independent parameters. Through the intent projection layer, it outputs a vector of the same dimension as the query vector, enabling similarity calculations between the two in a unified vector space.

[0012] ASR noise specifically refers to recognition errors generated by automatic speech recognition systems in an in-vehicle environment. These errors primarily include: phoneme confusion, such as recognizing "navigation" as "to hang"; homophone errors, such as recognizing "air conditioning" as "empty bar"; and tone deviations, such as recognizing "open" (dǎ kāi) as "large open" (dà kāi). These errors, caused by factors such as in-vehicle ambient noise, the speaker's accent, and speech speed, are a major obstacle to accurate intent recognition.

[0013] Intent functions are standardized executable functional interfaces within the vehicle system, representing specific in-vehicle features that users can control via voice. For example: AirConditioner.TurnOn(): turns on the air conditioner; Navigation.SetDestination(location): sets the navigation destination; Window.Open(position): opens the window. Each intent function has a unique identifier, functional description, and required parameters, serving as a bridge between user natural language commands and the actions executed by the vehicle system.

[0014] The phoneme substitution algorithm is based on a statistically obtained phoneme confusion matrix, and substitutes the phonemes in the original text with easily confused phonemes with a certain probability. The specific process is as follows: convert the text into a phoneme sequence (e.g., "打开" → "da3 kai1"); find easily confused phoneme pairs according to the confusion matrix (e.g., "da" ↔ "ta"); generate noise variants by substitution according to the probability; restore the phoneme sequence to text (e.g., "ta3 kai1" → "他开"); this algorithm can truly simulate the error patterns of the ASR system and is used in the training stage to enhance the robustness of the model.

[0015] [[ID=​​​​​​​​​​, where λ is the weight coefficient, λ ∈ [0.1,0.5], scenes with severe ASR noise: λ = 0.3-0.5 (strengthening alignment constraints); scenes with good ASR quality: λ = 0.1-0.2 (reducing alignment constraints); represents the contrastive learning loss function, is the alignment loss function; according to the loss function L, the parameters of the query encoder and intent encoder are iteratively updated through the gradient descent algorithm until the loss function converges.

[0019] Furthermore, we construct a contrastive learning loss function , so that the similarity of positive sample pairs is maximized and the similarity of negative sample pairs is minimized: , where q is the query vector, is the positive sample intention vector, is the negative sample intention vector, τ is the temperature coefficient, sim(*) is the cosine similarity function;

[0020] A positive intent vector is a vector representation of an intent function that semantically matches the query text, encoded using the intent encoder. Specifically, when a user queries "turn on the air conditioner," the corresponding positive intent vector is the vector encoded with the "AirConditioner.TurnOn()" function and its colloquial expression (e.g., "turn on the air conditioning system"). Positive examples include the vector of the original intent function and the vectors of all its colloquial expressions. During training, the model aims to maximize the similarity between the query vector and its corresponding positive intent vector. Each query may correspond to multiple positive intent vectors (different expressions of the same function).

[0021] Negative intent vectors are vectors encoded from other intent functions that don't semantically match the query text. In in-car scenarios, for example, when a user queries "turn on the air conditioning," vectors for other functions like "Navigation.SetDestination()" (navigation settings) and "Radio.ChangeStation()" (changing radio stations) are all negative examples. Negative examples are selected using the following strategies: within-batch negatives: intent vectors corresponding to other queries in the same training batch; and hard negatives: intent vectors that are literal but semantically different from the query (e.g., "open the sunroof" vs. "turn on the air conditioning"). The model training goal is to minimize the similarity between the query vector and the negative intent vector.

[0022] The temperature coefficient is a hyperparameter that controls sample discrimination in contrastive learning. In this solution, it adjusts the sharpness of the similarity distribution. Due to the presence of ASR noise, an appropriate temperature coefficient allows the model to distinguish intent while maintaining a certain tolerance for noise variation. τ ∈ [0.05, 0.5]. Initial training: τ = 0.2-0.3 (a larger value to allow model exploration); mid-training: τ = 0.1 (standard value); late training: τ = 0.05-0.07 (a smaller value for finer differentiation).

[0023] Specifically, the correct mapping between query and intent is established in the semantic space. By maximizing the similarity of positive sample pairs and minimizing the similarity of negative sample pairs, the model learns to distinguish different user intents.

[0024] Furthermore, we construct the alignment loss function , so that the query vectors of different noisy text variants of the same query text remain similar: ;in, and are different noise text variants of the same query text, represents the query encoder;

[0025] Specifically, we impose structural constraints on the query encoder’s output space, forcing different noise variants of the same query to produce similar vector representations. This constraint makes the query encoder inherently resilient to ASR errors.

[0026] Furthermore, the intention function associated with the query vector is obtained based on a hybrid retrieval strategy, including: for the received user query text, T noise text variants are generated using the same phoneme replacement algorithm as used in training; the original user query text and the T noise text variants are input into the query encoder respectively to obtain T+1 query vectors; for each query vector, the cosine similarity with each intention vector in the intention vector set is calculated respectively to obtain the similarity between each query vector and each intention vector; for each intention function, the T+1 similarity values ​​calculated with the corresponding intention vector and the T+1 query vectors are aggregated, and the average value is calculated as the semantic similarity score of the corresponding intention function; the top K with the highest semantic similarity scores are selected from the intention vector set. The intent function is used as a semantic retrieval candidate set; the original user query text is segmented and a keyword set is extracted; the keywords are the nouns, verbs and adjectives obtained after segmentation; the keyword matching degree between the keyword set and the functional description text of each intent function in the intent function library is calculated to obtain the keyword matching score; the top M intent functions with the highest keyword matching scores are selected from the intent function library as the keyword retrieval candidate set; the semantic retrieval candidate set and the keyword retrieval candidate set are merged to obtain a mixed candidate set; for each intent function in the mixed candidate set, a comprehensive relevance score S is calculated; the intent functions in the mixed candidate set are sorted according to the comprehensive relevance score S, and the top N intent functions with the highest scores are selected as the intent functions associated with the query vector.

[0027] Calculate the comprehensive correlation score S: ;in, is the semantic similarity score after aggregation, is the keyword matching score, α is the weight parameter, α∈[0.6,0.9], standard vehicle scenario: α=0.7-0.8; high noise environment: α=0.8-0.9; clear speech environment: α=0.6-0.7.

[0028] In particular, traditional retrieval methods only use a single text output by ASR for vectorization and retrieval during inference. This approach has fundamental flaws: when ASR makes an error, the vector representation of the incorrect text can deviate significantly from the correct semantic space; even if the model has seen similar noise during training, single-point estimation can still easily fall into the wrong local area;

[0029] This solution achieves the transition from single-point estimation to distribution estimation by actively generating T noise variants during inference. It uses a phoneme replacement algorithm to simulate the error patterns that may be produced by ASR, which is equivalent to "reverse perturbation" of the ASR output. The T generated variants form a neighborhood of the original query in the noise space, increasing the probability of hitting the correct semantics. Even if the original ASR output is seriously erroneous, as long as some of the generated variants are close to the actual pronunciation, the correct semantics can be restored through aggregation.

[0030] Another aspect of the present application also provides a vehicle-mounted user intention recognition system based on a large model, including: a dual tower retrieval model construction module for constructing a dual tower retrieval model DREM, where the dual tower retrieval model DREM includes a query encoder and an intention encoder. Among them, the query encoder is used to encode the vehicle-mounted user query text into a query vector, and the intention encoder is used to encode the intention function text into an intention vector; an intention function library construction module for constructing an intention function library containing vehicle-mounted control functions, where the intention function library contains multiple intention functions and corresponding function description texts; the intention function library construction module is also used to encode each intention function text in the intention function library using the intention encoder, obtain an intention vector set and establish an index; a query encoding module for receiving the user query text and encoding the user query text into a query vector using the query encoder; a hybrid retrieval module for obtaining an intention function associated with the query vector based on a hybrid retrieval strategy; where the hybrid retrieval strategy calculates the correlation degree between the query vector and the intention vector through semantic similarity retrieval and keyword matching retrieval; a large model inference module for constructing a prompt template according to the user query text and the obtained associated intention function; and performing inference using the large model according to the prompt template to obtain a vehicle-mounted user intention recognition result.

[0031] Compared with the prior art, the advantages of the present application are as follows:

[0032] Through the joint optimization of ASR noise simulation and alignment loss function in the training stage the dual tower retrieval model DREM learns noise-invariant semantic representations, that is, different noise variants of the same query are mapped to a similar vector space; in the inference stage, by actively generating noise text variants and aggregating the similarities of multiple query vectors, the noise robustness features learned by the model during training are fully utilized, so that even in the case of typical recognition errors such as phoneme substitution and homophone errors in ASR speech recognition (such as "turn on the air conditioner" being recognized as "turn on the empty bar"), the system can still accurately locate the correct intention function through the multi-vector aggregation mechanism. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The present application will be further described in the form of exemplary embodiments, and these exemplary embodiments will be described in detail through the drawings. These embodiments are not restrictive. In these embodiments, the same numbers represent the same structures, where:

[0034] Figure 1 is an exemplary flowchart of a vehicle-mounted user intention recognition method according to some embodiments of the present application;

[0035] Figure 2is an exemplary flow chart of constructing a dual-tower retrieval model DERM according to some embodiments of the present application;

[0036] Figure 3 This is an exemplary flowchart of calculating the comprehensive score S according to some embodiments of the present application. DETAILED DESCRIPTION

[0037] The method and system provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0038] like Figure 1 As shown, a dual-tower retrieval model DREM is constructed, and the dual-tower retrieval model DREM includes a query encoding encoder and an intention encoding encoder, wherein the query encoding encoder encodes the vehicle user query text into a query vector, and the intention encoding encoder encodes the intention function text into an intention vector; constructs an intention function library containing vehicle control functions, and the intention function library contains multiple intention functions and corresponding function description texts; uses the intention encoding encoder to encode each intention function text in the intention function library, obtains an intention vector set and establishes an index; receives the user query text, and uses the query encoding encoder to encode the user query text into a query vector; obtains the intention function associated with the query vector based on a hybrid retrieval strategy; wherein the hybrid retrieval strategy calculates the correlation between the query vector and the intention vector through semantic similarity retrieval and keyword matching retrieval; constructs a prompt template based on the user query text and the obtained associated intention function; uses the large model for reasoning based on the prompt template to obtain the vehicle user intention recognition result;

[0039] like Figure 2 As shown, build the training data set: collect the original data set ,in, Query text for in-car users, is the corresponding intention function;

[0040] For each query text , generate m noise variants through phoneme substitution algorithm , the phoneme replacement algorithm replaces the original phoneme with the easily confused phoneme according to the confusion matrix; specifically, the query text Convert to a phoneme sequence. For example, "Turn on the air conditioner" is converted to "da3 kai1 kong1 tiao2". For each phoneme in the phoneme sequence, perform the following replacements: Look up all the confusable phonemes of this phoneme according to the confusion matrix; Randomly decide whether to replace according to the confusion probability (usually set the overall replacement rate to 20% - 30%); If the decision is to replace, select the target phoneme to be replaced according to the probability distribution. Convert the replaced phoneme sequence back to text form. For example: "da3 kai1 kong1 tiao2" may become "da4 kai1 kong1 tiao4", corresponding to the text "大开空条". For each query text Repeat steps 2 - 3 a total of m times (usually m = 5 - 10) to generate m different noise variants. Ensure the diversity of the variants and avoid generating exactly the same variants.

[0041] For each intent function , construct a set of colloquial expressions , containing k semantically equivalent colloquial texts; Specifically, decompose each intent function into two parts: action and object. For example, "AirConditioner.TurnOn" is decomposed into the action "Turn on" and the object "air conditioner". Synonym expansion: Action synonyms: Turn on → {Open, Turn on, Start, Turn on for a moment, Turn on briefly}; Object synonyms: air conditioner → {Air conditioner, Cold air, Refrigeration, Air conditioning system}. Colloquial combination: Combine different expressions of the action and the object, and add colloquial modifiers: Formal expression: "Turn on the air conditioner"; Colloquial expressions: "Turn on the air conditioner for a moment", "Turn on the cold air", "Turn on the air conditioner briefly"; Add modal particles: "Please turn on the air conditioner", "Please help me turn on the air conditioner".

[0042] Initialize the query encoder and the intent encoder respectively using a pre-trained language model; The pre-trained language model includes any one of BERT, RoBERTa, or ELECTRA. Load the weight parameters of the pre-trained language model as the initialization parameters of the query encoder; Load the weight parameters of the pre-trained language model as the initialization parameters of the intent encoder; Add a query projection layer at the output end of the query encoder to map the hidden layer output of the pre-trained language model to a query vector with a fixed dimension; Add an intent projection layer at the output end of the intent encoder to map the hidden layer output of the pre-trained language model to an intent vector with the same dimension as the query vector.

[0043] Construct training sample pairs:

[0044] Set of positive sample pairs , where represents the original query text, represents the original intent function text;

[0045] Negative sample pair set ;

[0046] Define the loss function: ,in: , , Represents a query The set of all variants of represents the query encoder, λ is the weight coefficient;

[0047] The encoder parameters are updated through the back-propagation algorithm and the training is iterated until the loss function converges.

[0048] Build an intent function library that includes vehicle control functions:

[0049] Build an intent function library , where each intent function Contains: function identifier : Uniquely identifies each intent function; function name : Standardized function name, such as "AirConditioner.TurnOn"; function description text : Describe the function in detail, such as "turn on the car's air conditioning system"; parameter list : The parameter set required by this function, such as temperature value and wind speed level;

[0050] Organize all control functions supported by the vehicle system, including but not limited to: air conditioning control: temperature adjustment, fan speed adjustment, internal and external circulation switching; navigation control: destination setting, route planning, map zooming; multimedia control: volume adjustment, song switching, radio selection; window control: window opening and closing, sunroof control; seat control: seat heating, position adjustment, massage function;

[0051] For each intent function Fᵢ, a multi-level description text is generated: standard description: uses standardized technical language to describe the function; colloquial description: adds common colloquial expressions; synonym expansion: includes synonyms and near synonyms related to the function;

[0052] For each function in the intent function library : Change the function name and functional description text Splicing to form a complete text: ; Use the trained intent encoder right To encode: ; Get the intention vector set ,in, , d is the vector dimension;

[0053] Build a vector index using an approximate nearest neighbor search algorithm (such as FAISS or Annoy): Select the index type: IVF (inverted file) index or HNSW (hierarchical navigable small world) index; Set the index parameters: the number of clustering centers, the size of the probe list, etc.; Add the set of intent vectors V to the index and establish a mapping relationship from vector ID to function ID; Save the index file and the mapping table for online retrieval;

[0054] Obtain the query text converted by the user through the ASR system ; Record the query timestamp and session context information; For the query text Perform basic preprocessing: Remove leading and trailing spaces and extra whitespace characters; Unify full-width and half-width characters; Preserve the case information of the original text;

[0055] Use the trained query encoder Encode the preprocessed text: Input the query text into the query encoder: ; Obtain the query vector through the query projection layer: ; Perform L2 normalization on the query vector: ; Among them, , which has the same dimension as the intent vector;

[0056] Apply the same phoneme replacement algorithm as in training to generate T noisy text variants: , where is the original text; Encode each variant separately to obtain a set of query vectors: ;

[0057] As Figure 3 shown, obtain the intent function associated with the query vector based on the hybrid retrieval strategy.

[0058] After receiving the user's original query text, use the same phoneme replacement algorithm as in the training stage to generate T noisy variants (usually T = 3 - 5). For example, when the user inputs "Open the sunroof", the system generates variants: Variant 1: "Greatly open the sunroof" (confusion of initial consonant d → d); Variant 2: "Open the sky bed" (confusion of final vowel ang → uang); Variant 3: "Open the adding window" (tone change).

[0059] Input the original query text and T noisy variants into the query encoder respectively, and a total of T + 1 query vectors are obtained. The encoding process for each text includes: text tokenization and sub-word segmentation; Extract features through the multi-layer Transformer structure of the query encoder; Map to a unified dimensional space through the query projection layer; L2 normalization processing to ensure that the vector norm is 1. Store the T + 1 query vectors in a temporary vector set, denoted as , where is the query vector of the original text.

[0060] right Each query vector in , and each intention vector in the intention vector set V Calculate cosine similarity: Use vectorized operations to calculate the similarity between a query vector and all intent vectors at the same time. Since the vectors have been normalized, cosine similarity is simplified to a vector dot product operation. Construct a similarity matrix ,in, Represents the similarity between the j-th query vector and the i-th intent vector, and its value range is [-1, 1].

[0061] For each intent function , collect its intent vector Similarity values ​​with all T+1 query vectors: Similarity set: ; Calculate the average: By aggregating multiple variants, even if the original query is misrecognized by ASR, as long as some noise variants are close to the correct pronunciation, a high semantic similarity score can still be obtained.

[0062] Score all intent functions according to semantic similarity Sort in descending order; select the top K intent functions with the highest scores (usually K = 10-15); record the semantic similarity score of each candidate function for subsequent comprehensive scoring. Set a minimum similarity threshold (such as 0.5) to filter out candidates with too low similarity to ensure the quality of the candidate set.

[0063] Perform Chinese word segmentation on the original query text: Use word segmentation tools (such as Jieba) to segment the text; perform part-of-speech tagging to identify the part of speech of each word; extract nouns (such as "air conditioning" and "temperature"), verbs (such as "open" and "adjust"), and adjectives (such as "maximum" and "cool"); and filter stop words and meaningless function words.

[0064] For each function in the intention function library : Extract function description text The keyword set in ; calculate the intersection of the query keyword set and the description keyword set; match calculation: S_keyword[i] = |intersection| / |query keyword set|; consider synonym matching to improve the matching recall rate.

[0065] Sort all intent functions in descending order by keyword matching score S_keyword, select the top M intent functions with the highest scores (usually M = 5-10), and save the keyword matching score of each candidate.

[0066] The semantic search candidate set and the keyword search candidate set are combined to remove duplicate intent functions. Two scores are saved for each candidate function: the semantic similarity score and the keyword matching score. For functions that appear in only one candidate set: only in the semantic candidate set: the keyword score is set to 0; only in the keyword candidate set: the semantic score is set to 0;

[0067] Calculate the comprehensive score for each intent function in the mixed candidate set: ;in: is the semantic similarity score (normalized to [0,1]); is the keyword matching score (normalized to [0,1]); α is a weight parameter, typically set to 0.7-0.8, emphasizing semantic similarity. Sort all candidate functions in descending order by their overall score S; select the top N functions as the final result (usually N = 3-5); and return a list of functions and their corresponding scores.

[0068] Build a structured prompt template that includes the following key parts:

[0069] System role description: "You are an in-vehicle intelligent assistant, responsible for understanding the user's control intent and selecting the correct in-vehicle functions."

[0070] Task description: "The user issued a control command via voice. Since voice recognition may have errors, please carefully analyze the user's true intention."

[0071] Contextual information injection: user original query: "[insert user query text]"; candidate function list: lists the N retrieved intent functions and their descriptions one by one; relevance score: the comprehensive score of each candidate function;

[0072] Decision guidance: "Analyze the match between the user query and each candidate function, consider possible speech recognition errors, and select the function that best matches the user's intent."

[0073] Output format requirements:

[0074] Please input in the following format:

[0075] User intent analysis: [analyzing what users want];

[0076] Selected function: [function name];

[0077] Confidence: [High / Medium / Low];

[0078] Parameter extraction: [Extract relevant parameters if necessary]";

[0079] Select an appropriate large language model (such as GPT-3.5, GPT-4, or other open-source models). Fill the template with the user query and candidate function information to generate a complete prompt text. Input the constructed prompt into the large model to obtain the generated analysis results. By combining this hybrid search and large-model inference, the system can accurately understand the user's true intent even when ASR recognition errors exist, significantly improving the user experience of in-vehicle voice interaction.

[0080] The invention of the present application and its implementation methods are described schematically above. This description is not restrictive. Without departing from the spirit or basic features of the present application, the present application can be implemented in other specific forms. What is shown in the accompanying drawings is only one of the implementation methods of the invention of the present application, and the actual structure is not limited to this. Therefore, if a person of ordinary skill in the art is inspired by it, without departing from the purpose of the invention, a structural method and embodiment similar to the technical solution are designed without creativity, which should all fall within the scope of protection of the present application. In addition, the word "including" does not exclude other elements or steps, and the word "one" before an element does not exclude the inclusion of "multiple" elements. Words such as first and second are used to indicate names and do not indicate any specific order.

Claims

1. A method for identifying vehicle user intention based on a large model, characterized in that: include: Construct a dual-tower retrieval model (DREM). The dual-tower retrieval model (DREM) includes a query encoder and an intent encoder. The query encoder encodes the vehicle user's query text into a query vector, and the intent encoder encodes the intent function text into an intent vector. Build an intent function library containing vehicle control functions. The intent function library contains multiple intent functions and corresponding function description texts. Use the intent encoding encoder to encode the intent function text in the intent function library, obtain the intent vector set and establish an index; Receiving a user query text, and encoding the user query text into a query vector using a query encoding encoder; Obtaining the intent function associated with the query vector based on a hybrid retrieval strategy; wherein the hybrid retrieval strategy calculates the correlation between the query vector and the intent vector through semantic similarity retrieval and keyword matching retrieval; Build a prompt template based on the user query text and the associated intent function; Based on the prompt template, the large model is used for reasoning to obtain the in-vehicle user intention recognition results.

2. The large model-based vehicle-mounted user intention recognition method according to claim 1, characterized in that: Construct a dual-tower retrieval model DREM, including: Collect the original training dataset, which includes the mapping relationship between the in-vehicle user query text and the corresponding intent function; Perform ASR noise simulation on the query text and generate noise text variants through phoneme replacement algorithm; For each intent function, we construct a set of spoken expressions of the intent function by expanding the spoken description text. Use the pre-trained language model to initialize the query encoder and intent encoder respectively; Construct positive and negative sample pairs based on the query text, noise text variants, intent functions, and colloquial expression sets; Based on the positive and negative sample pairs, the dual-tower retrieval model DREM is trained using the contrastive learning loss function; The parameters of the query encoder and intent encoder are updated through the back-propagation algorithm to maximize the vector similarity of positive sample pairs and minimize the vector similarity of negative sample pairs.

3. The large model-based vehicle-mounted user intention recognition method according to claim 2, characterized in that: Construct positive and negative sample pairs based on the query text, noise text variants, intent functions, and colloquial expression sets, including: According to each mapping relationship in the original training dataset, the query text and all corresponding noise text variants are used as the query set; The original text of the intent function and all corresponding spoken expressions are taken as the positive intent set; Pair each element in the query set with each element in the positive intent set to form a positive sample pair; Select intent functions that do not correspond to the current query text from the original training dataset, and use the obtained non-corresponding intent functions and corresponding colloquial expressions as the negative intent set; Pair each element in the query set with each element in the negative intent set to form a negative sample pair; All positive sample pairs and negative sample pairs are combined to construct a training sample set.

4. The large model-based vehicle-mounted user intention recognition method according to claim 3, characterized in that: Use the pre-trained language model to initialize the query encoder and intent encoder respectively, including: The pre-trained language model includes any of BERT, RoBERTa, or ELECTRA.

5. The large model-based vehicle-mounted user intention recognition method according to claim 3, characterized in that: According to the positive and negative sample pairs, the contrastive learning loss function is used to train the dual-tower retrieval model DREM, including: For each training batch, positive and negative sample pairs are sampled from the training sample set; Encode the query text in the positive and negative sample pairs into a query vector through the query encoder; Encode the intent function text in the positive and negative sample pairs into an intent vector through the intent encoder; Construct the loss function L: , where λ is the weight coefficient; represents the contrastive learning loss function, is the alignment loss function; According to the loss function L, the parameters of the query encoder and intent encoder are iteratively updated through the gradient descent algorithm until the loss function converges.

6. The large model-based vehicle-mounted user intention recognition method according to claim 5, characterized in that: Constructing a contrastive learning loss function , so that the similarity of positive sample pairs is maximized and the similarity of negative sample pairs is minimized: , where q is the query vector, is the positive sample intention vector, is the negative sample intention vector, τ is the temperature coefficient, and sim(*) is the cosine similarity function.

7. The large model-based vehicle-mounted user intention recognition method according to claim 6, characterized in that: Constructing alignment loss function , so that the query vectors of different noisy text variants of the same query text remain similar: ;in, and are different noise text variants of the same query text, Represents a query encoder.

8. The large model-based vehicle-mounted user intention recognition method according to any one of claims 2 to 7, characterized in that: The intent function associated with the query vector is obtained based on a hybrid retrieval strategy, including: Perform data augmentation processing on the received user query text to generate multiple text variants; Encode the original user query text and the generated text variants into multiple query vectors; Calculate similarity between multiple query vectors and intent vector sets, and obtain the semantic similarity score of each intent function by aggregating multiple similarity results; Select the first candidate intent function set based on the semantic similarity score; Perform text analysis on the user query text, extract text features and match them with the intent function library to obtain the text matching score of each intent function; Select a second candidate intent function set based on the text matching score; Fusing the first candidate intent function set and the second candidate intent function set to obtain a hybrid candidate set; Perform a comprehensive score S on the intent functions in the mixed candidate set, and determine the intent function associated with the query vector based on the comprehensive score S.

9. The large model-based vehicle-mounted user intention recognition method according to claim 8, characterized in that: Calculate the comprehensive score S: ,in, is the semantic similarity score after aggregation, is the keyword matching score, and α is the weight parameter.

10. A large model-based vehicle-mounted user intention recognition system, characterized in that: include: A dual-tower retrieval model construction module is used to construct a dual-tower retrieval model DREM, wherein the dual-tower retrieval model DREM includes a query encoder and an intent encoder, wherein the query encoder is used to encode the vehicle user query text into a query vector, and the intent encoder is used to encode the intent function text into an intent vector; An intent function library construction module is used to construct an intent function library containing vehicle control functions, wherein the intent function library includes multiple intent functions and corresponding function description texts; the intent function library construction module is also used to encode each intent function text in the intent function library using an intent encoder to obtain an intent vector set and establish an index; A query encoding module, configured to receive a user query text and encode the user query text into a query vector using a query encoder; A hybrid retrieval module, configured to obtain an intent function associated with a query vector based on a hybrid retrieval strategy, wherein the hybrid retrieval strategy calculates the degree of association between the query vector and the intent vector through semantic similarity retrieval and keyword matching retrieval; The large model inference module is used to build a prompt template based on the user query text and the obtained associated intent function; and based on the prompt template, it uses the large model to perform inference to obtain the in-vehicle user intent recognition result.

Images