Training method of retrieval enhancement model, voice interaction method, server and medium

By jointly training the preset large language model and the retrieval enhancement model, the problem of secondary encoding of the application interface by the retrieval enhancement model is solved, achieving faster voice request processing and a better user experience.

CN120783774APending Publication Date: 2025-10-14GUANGZHOU XIAOPENG MOTORS TECH CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202510954158.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-07-10
Publication Date
2025-10-14

AI Technical Summary

Technical Problem

In the existing technology, the retrieval enhancement model needs to encode the application interface twice when processing user voice requests, resulting in waste of computing resources and increased latency, affecting user experience.

Method used

Through joint training based on a preset large language model and a retrieval enhancement model, the similarity between the voice request matrix and the application interface matrix is ​​determined, and the parameters of the retrieval enhancement model are optimized according to the similarity threshold to avoid secondary encoding of the application interface.

Benefits of technology

It reduces coding redundancy, improves the speed of processing user voice requests, enhances user experience, and meets the needs of efficient and convenient interaction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120783774A_ABST
    Figure CN120783774A_ABST
Patent Text Reader

Abstract

The invention discloses a training method of a retrieval enhancement model, a voice interaction method, a server and a computer readable storage medium. The method comprises the steps that based on a preset large language model, an obtained training voice request is coded, a voice request matrix is determined, and the coding length of the voice request matrix is a preset length; then, based on the current retrieval enhancement model, application interfaces in a preset application interface database are coded, a current application interface matrix is determined, and the coding length of the current application interface matrix is a preset length; then, determining the current similarity according to the voice request matrix and the current application interface matrix; and finally, training the current retrieval enhancement model according to the current similarity and a preset similarity threshold. Thus, the application interface features retrieved by the current retrieval enhancement model can be directly used as the input of the preset large language model, and secondary coding of the application interface is avoided, so that coding redundancy is reduced, the reasoning result generation speed is increased, and the user experience is enhanced.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present application relates to the field of voice interaction, and in particular to a training method of a retrieval enhancement model, a voice interaction method, a server and a computer readable storage medium. BACKGROUND

[0002] In the related art, in the process of processing user voice requests based on a large language model and a retrieval enhancement model, the retrieval enhancement model needs to encode all application interfaces. Each time a user voice request is received, an index representation of a similar application interface is found from a vector library, and the index representation is mapped to the similar application interface. The similar application interface is concatenated with a prompt word of the large language model and then input into the large language model for re-encoding. In this way, this process includes two encodings of the similar application interface, resulting in repeated work, increased latency, and poor user experience. SUMMARY

[0003] The present application provides a training method of a retrieval enhancement model, a voice interaction method, a server and a computer readable storage medium.

[0004] The present application provides a training method of a retrieval enhancement model, a voice interaction method, a server and a computer readable storage medium.

[0005] Based on a preset large language model, the obtained training voice request is encoded and processed to determine a voice request matrix, and the encoding length of the voice request matrix is a preset length;

[0006] Based on the current retrieval enhancement model, the application interfaces in the preset application interface database are encoded and processed to determine a current application interface matrix, and the encoding length of the current application interface matrix is the preset length;

[0007] According to the voice request matrix and the current application interface matrix, a current similarity is determined;

[0008] According to the current similarity and a preset similarity threshold, the current retrieval enhancement model is trained.

[0009] Thus, based on the preset large language model, the server encodes the obtained training voice request to determine a voice request matrix, and the encoding length of the voice request matrix is the preset length. Then, based on the current retrieval enhancement model, the server encodes the application interfaces in the preset application interface database to determine a current application interface matrix, and the encoding length of the current application interface matrix is the preset length. Then, the server determines a current similarity according to the voice request matrix and the current application interface matrix. Finally, the server trains the current retrieval enhancement model according to the current similarity and a preset similarity threshold. In this way, by training the current retrieval enhancement model through the voice request matrix and the current application interface matrix with the preset length, the application interface features retrieved by the current retrieval enhancement model can be directly used as the input of the preset large language model when processing the user voice request in the future, avoiding secondary encoding of the application interface, thereby reducing encoding redundancy, accelerating the generation speed of inference results, and enhancing the user experience.

[0010] In some embodiments, the training of the current retrieval enhancement model according to the current similarity and the preset similarity threshold comprises:

[0011] During the training of the current retrieval enhancement model, the parameters of the current retrieval enhancement model are updated according to the current similarity and the preset similarity threshold, and the current parameter setting of the preset large language model is maintained.

[0012] Thus, during the training of the current retrieval enhancement model, the parameters of the current retrieval enhancement model are updated according to the current similarity and the preset similarity threshold, and the current parameter setting of the preset large language model is maintained. In this way, the current parameter setting of the preset large language model remains unchanged, which can maintain the stability of the semantic space of the encoded user voice request, so that the current retrieval enhancement model can be aligned to the semantic space, and semantic space drift caused by parameter changes of the preset large language model is avoided.

[0013] In some embodiments, the updating of the parameters of the current retrieval enhancement model according to the current similarity and the preset similarity threshold comprises:

[0014] In the case where the current similarity is less than the preset similarity threshold, a current loss value is determined according to the current similarity.

[0015] The parameters of the current retrieval enhancement model are updated according to the current loss value.

[0016] Thus, in a case where the current similarity is less than the preset similarity threshold, the server determines a current loss value according to the current similarity. Finally, the server updates the parameters of the current retrieval enhancement model according to the current loss value. In this way, by calculating the similarity of the encoded speech request matrix and the current application interface matrix, the difference between the speech request matrix and the current application interface matrix in the semantic space can be quantified. When the similarity is less than the preset similarity threshold, it indicates that the current retrieval enhancement model is insufficient in matching the semantic coding results of the application interface with the training speech request coded by the preset large language model, and the model needs to be guided for optimization through the current loss value.

[0017] In some embodiments, the method further comprises:

[0018] encoding the application interface based on the updated current retrieval enhancement model to determine a next application interface matrix of the preset length;

[0019] determining a next similarity according to the speech request matrix and the next application interface matrix;

[0020] training the current retrieval enhancement model according to the next similarity and the preset similarity threshold.

[0021] Thus, the server encodes the application interface based on the updated current retrieval enhancement model to determine a next application interface matrix of the preset length. Then, the server determines a next similarity according to the speech request matrix and the next application interface matrix. Finally, the server trains the current retrieval enhancement model according to the next similarity and the preset similarity threshold. In this way, the encoding mode of the current retrieval enhancement model for the application interface is gradually adjusted, so that the feature matrix of the application interface is closer to the speech request matrix coded by the preset large language model in the semantic space, and the precise alignment of the coding features of the retrieval enhancement model and the preset large language model is achieved.

[0022] In some embodiments, the updating the parameters of the current retrieval enhancement model according to the current similarity and the preset similarity threshold comprises:

[0023] In a case where the current similarity is greater than or equal to the preset similarity threshold, it is confirmed that the current retrieval enhancement model is trained.

[0024] Thus, in a case where the current similarity is greater than or equal to the preset similarity threshold, the server confirms that the current retrieval enhancement model training is completed. In this way, by taking the preset similarity threshold as a judgment standard, when the similarity between the application interface feature coded by the current retrieval enhancement model and the training speech request feature coded by the preset large language model reaches or exceeds the threshold, it indicates that the semantic spaces of the two have been fully aligned, and the retrieval capability of the current retrieval enhancement model meets the demand, and the training can be terminated.

[0025] The embodiments of the present application provide a voice interaction method based on the retrieval enhancement model trained by the above training method, and the method comprises:

[0026] Based on the preset large language model, voice coding information is determined according to the obtained user voice request;

[0027] Based on the retrieval enhancement model, similar application interface coding information is determined according to the voice coding information, wherein the feature dimension of the similar application interface coding information and the feature dimension of the voice coding information are consistent;

[0028] According to the preset prompt word template, the voice coding information, the pre-configured general application interface code and the similar application interface coding information, a target prompt word is determined;

[0029] According to the target prompt word, the voice interaction is performed.

[0030] Thus, based on the preset large language model, the server determines the voice coding information according to the obtained user voice request. Then, based on the retrieval enhancement model, the server determines the similar application interface coding information according to the voice coding information, wherein the feature dimension of the similar application interface coding information and the feature dimension of the voice coding information are consistent. Then, the server determines the target prompt word according to the preset prompt word template, the voice coding information, the pre-configured general application interface code and the similar application interface coding information. Finally, the server performs voice interaction according to the target prompt word. In this way, by determining the voice coding information and the similar application interface coding information with consistent dimensions through the preset large language model and the retrieval enhancement model, and then directly determining the natural language processing result according to the voice coding information and the similar application interface coding information, the repeated coding of the similar application interface is avoided, and the redundant work of twice coding of the similar application interface by the large language model and the retrieval enhancement model in the original process is eliminated.

[0031] In some embodiments, the method further comprises:

[0032] The user voice request is parsed and processed, and temporary generalization words are determined from the preset generalization word library;

[0033] The temporary generalization word is processed according to a preset rule to determine a target generalization word.

[0034] The target generalization word is used to determine a target entity.

[0035] The target entity is used to determine a similar public entity.

[0036] The similar public entity is used to determine associated vehicle knowledge information from the preset generalization word library.

[0037] In this way, the server analyzes and processes the user voice request, determines a temporary generalization word from the preset generalization word library, and then processes the temporary generalization word according to a preset rule to determine a target generalization word. Then, the server determines a target entity according to the target generalization word. Subsequently, the server determines a similar public entity according to the target entity. Finally, the server determines associated vehicle knowledge information from the preset generalization word library according to the similar public entity. In this way, the above processing can accurately extract the key semantics in the user voice request and improve the entity retrieval efficiency and accuracy, thereby optimizing the input quality of the preset large language model reasoning.

[0038] In some embodiments, the target prompt word is determined according to the preset prompt word template, the voice coding information, the pre-configured general application interface code, and the similar application interface coding information.

[0039] The voice coding information, the associated vehicle knowledge information, the pre-configured general application interface code, and the similar application interface coding information are spliced based on the preset label and the preset prompt word template to determine the target prompt word.

[0040] In this way, the server splices the voice coding information, the associated vehicle knowledge information, the pre-configured general application interface code, and the similar application interface coding information based on the preset label and the preset prompt word template to determine the target prompt word. In this way, the voice coding information, the associated vehicle knowledge information, the pre-configured general application interface code, and the similar application interface coding information are fused by splicing to provide structured input for the preset large language model, guiding the preset large language model to accurately generate an application interface and parameters.

[0041] The present application provides a server, which includes a processor and a memory, and the memory stores a computer program. When the computer program is executed by the processor, the above-mentioned method is implemented.

[0042] The present application provides a computer readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the above-mentioned method are implemented.

[0043] Additional aspects and advantages of the embodiments of the present application will be in part apparent and in part pointed out below in the description of the embodiments of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0044] The above and / or additional aspects and advantages of the present application will become apparent and be readily appreciated from the description of the embodiments, given below with reference to the following drawings, wherein:

[0045] Figure 1 is one of the flow diagrams of the training method of the search enhancement model according to some embodiments of the present application;

[0046] Figure 2 is another of the flow diagrams of the training method of the search enhancement model according to some embodiments of the present application;

[0047] Figure 3 is a third of the flow diagrams of the training method of the search enhancement model according to some embodiments of the present application;

[0048] Figure 4 is a fourth of the flow diagrams of the training method of the search enhancement model according to some embodiments of the present application;

[0049] Figure 5 is a fifth of the flow diagrams of the training method of the search enhancement model according to some embodiments of the present application;

[0050] Figure 6 is a complete flow diagram of the training method of the search enhancement model according to some embodiments of the present application;

[0051] Figure 7 is one of the flow diagrams of the voice interaction method according to some embodiments of the present application;

[0052] Figure 8 is a complete flow diagram of the voice interaction method according to some embodiments of the present application;

[0053] Figure 9 is another of the flow diagrams of the voice interaction method according to some embodiments of the present application;

[0054] Figure 10 is a third of the flow diagrams of the voice interaction method according to some embodiments of the present application. DETAILED DESCRIPTION

[0055] Embodiments of the present application are described below in detail, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are only used to explain the embodiments of the present application, and cannot be understood as a limitation on the embodiments of the present application.

[0056] In the related art, when a user voice request is processed by using a large language model (LLM) and a retrieval-augmented generation (RAG) model, there is a significant technical defect. Specifically, the retrieval-augmented generation model needs to encode all application interfaces (APIs) first, thereby constructing a corresponding vector library. Whenever a user voice request occurs, the intelligent cockpit system needs to retrieve the application interface most similar in semantics to the user voice request from the vector library. This process needs to find the index representation of the similar application interface first, and then map the specific similar application interface through the index representation.

[0057] However, the problem is that the similar application interfaces retrieved subsequently need to be spliced with the prompt of the large language model, and after splicing, they are sent into the large language model. In this link, the large language model encodes the input content again. In this way, from the initial encoding of the application interface by the retrieval-augmented generation model to the secondary encoding of the application interface by the large language model, the similar application interface is encoded twice in the whole process.

[0058] This two-time encoding process will inevitably lead to repeated work, not only wasting computing resources, but also increasing the time for processing user voice requests, and thus significantly increasing the delay in the large language model reply. In scenarios with high real-time requirements, such as vehicle cockpit voice interaction scenarios, this increase in delay will seriously affect the user experience, making the interaction process not smooth and natural, and unable to meet the user's demand for efficient and convenient interaction.

[0059] Based on the above problems, please refer to Figure 1 The embodiments of the present application provide a training method of a retrieval-augmented generation model, the method comprising:

[0060] 011: based on a preset large language model, encoding and processing the obtained training voice request to determine a voice request matrix;

[0061] 012: based on the current retrieval-augmented generation model, encoding and processing the application interfaces in the preset application interface database to determine a current application interface matrix;

[0062] 013: Determine the current similarity according to the voice request matrix and the current application interface matrix;

[0063] 014: Train the current retrieval enhancement model according to the current similarity and the preset similarity threshold.

[0064] The training method of the retrieval enhancement model provided by the embodiments of the present application can be implemented by the server of the embodiments of the present application. Specifically, the memory stores a computer program, and the processor is configured to encode the obtained training voice request based on a preset large language model to determine a voice request matrix. And encode the application interface in the preset application interface database based on the current retrieval enhancement model to determine a current application interface matrix. The processor is configured to determine the current similarity according to the voice request matrix and the current application interface matrix. And train the current retrieval enhancement model according to the current similarity and the preset similarity threshold.

[0065] The training method of the retrieval enhancement model provided by the embodiments of the present application can be implemented by the model training device of the embodiments of the present application. Specifically, the model training device comprises a determination module and a training module. The determination module is configured to encode the obtained training voice request based on a preset large language model to determine a voice request matrix. And encode the application interface in the preset application interface database based on the current retrieval enhancement model to determine a current application interface matrix. And determine the current similarity according to the voice request matrix and the current application interface matrix. The training module is configured to train the current retrieval enhancement model according to the current similarity and the preset similarity threshold.

[0066] Specifically, the preset large language model refers to a pre-trained large language model, and its parameters remain in a frozen state when training the current retrieval enhancement model, and is only used for semantic encoding of user voice requests. Utilize the semantic understanding ability of the preset large language model to convert the user voice request into a high-dimensional semantic vector, provide a stable semantic space reference for the training of the RAG model, and ensure that the API encoding of the RAG can be close to the semantic space of the LLM.

[0067] The training voice request refers to a user voice instruction used to train the current retrieval enhancement model, such as an interactive instruction in the cockpit scenario of "kinetic energy recovery standard mode" and the like. The training voice request can guide the RAG model to learn how to match the API encoding with the user semantics, and is the input data of the training process.

[0068] The encoding process refers to the process of converting natural language (user voice request or API) into a high-dimensional vector matrix with a preset length. The LLM encodes the user voice request to generate a "voice request matrix", and the RAG encodes the API to generate an "application interface matrix". In this way, the semantic information is converted into numerical features that can be calculated by the model through the encoding process, facilitating subsequent similarity calculation and model optimization, which is the core operation of semantic space alignment.

[0069] The preset length refers to the fixed dimension of the encoded vector (such as 1024 dimensions), which ensures that the voice request matrix encoded by the LLM is consistent in dimension with the application interface matrix encoded by the RAG, facilitating similarity calculation. The uniform encoding length is a prerequisite for cross-model feature alignment, avoiding calculation errors caused by dimension mismatch and ensuring the comparability of semantic space.

[0070] The voice request matrix refers to the vector matrix obtained by encoding the training voice request by the LLM, with a dimension of [user voice request token length x preset length] (for example, if the training voice request token length is M, then the dimension of the voice request matrix is M x 1024).

[0071] The current retrieval enhancement model refers to a model used to encode APIs and retrieve similar APIs, which is trained and optimized to align API features with user voice request features of the LLM. The trained current retrieval enhancement model can encode APIs into the semantic space of the LLM, reducing the encoding redundancy of subsequent LLM processing.

[0072] The preset application interface database refers to a database that stores vehicle function-related APIs, including multiple APIs (such as "Open_Or_Close", "Energy_Recycle_Value_Set", etc.) and their parameter information. The application interfaces in the preset application interface database are the encoding objects of the RAG model, and the training goal is to enable the RAG to accurately encode these APIs into vectors that match the semantics of user voice requests.

[0073] The current application interface matrix refers to the vector matrix obtained by encoding the APIs in the preset application interface database by the RAG model, with a dimension of [API number x API token length x preset length] (for example, if the number of application interfaces is N and the length of the application interface is L, then the dimension of the application interface matrix is N x L x 1024).

[0074] The current similarity refers to the semantic matching degree of the speech request matrix and the current application interface matrix calculated by the cosine distance, InfoNCE loss, etc. The current similarity reflects the proximity of the RAG model to the API coding and the semantic of the user's speech request, and is a key indicator for judging whether the model needs to be optimized.

[0075] The preset similarity threshold refers to a pre-set similarity judgment standard for measuring whether the coding effect of the RAG model meets the standard. When the current similarity is less than the preset similarity threshold, it indicates that the RAG coding is insufficiently aligned with the LLM semantic space, and the RAG model parameters need to be updated; when the current similarity is greater than or equal to the preset similarity threshold, it is considered that the RAG model optimization is completed, and the training can be stopped. The preset similarity threshold balances the precision and training efficiency of the RAG model.

[0076] First, using a pre-trained and parameter-frozen LLM, the obtained training speech request is coded to generate a speech request matrix of a preset length (such as 1024 dimensions).

[0077] Next, using the current retrieval enhancement model (RAG) to be trained, all APIs (such as “Open_Or_Close” and “Energy_Recycle_Value_Set”) in the preset application interface database are coded to generate a current application interface matrix of the same dimension as the speech request matrix.

[0078] Then, the semantic similarity between the speech request matrix and the current application interface matrix, i.e. the current similarity, is calculated by methods such as cosine distance, InfoNCE loss, or Mahalanobis distance.

[0079] Finally, the server trains the current retrieval enhancement model according to the current similarity and the preset similarity threshold.

[0080] In summary, in the voice interaction method and server provided in the embodiments of the present application, based on the preset large language model, the server encodes the obtained training voice request to determine a voice request matrix, and the encoding length of the voice request matrix is the preset length. Then, based on the current retrieval enhancement model, the server encodes the application interface in the preset application interface database to determine a current application interface matrix, and the encoding length of the current application interface matrix is the preset length. Then, the server determines the current similarity according to the voice request matrix and the current application interface matrix. Finally, the server trains the current retrieval enhancement model according to the current similarity and the preset similarity threshold. In this way, by training the current retrieval enhancement model through the voice request matrix and the current application interface matrix of the preset length, the application interface feature retrieved by the current retrieval enhancement model can be directly used as the input of the preset large language model when processing the user voice request in the future, avoiding the secondary encoding of the application interface, thereby reducing the encoding redundancy, accelerating the generation speed of the inference result, and enhancing the user experience.

[0081] In some embodiments, the step 014 (training the current retrieval enhancement model according to the current similarity and the preset similarity threshold) comprises:

[0082] 0141: In the process of training the current retrieval enhancement model, the parameters of the current retrieval enhancement model are updated according to the current similarity and the preset similarity threshold, and the current parameter setting of the preset large language model is maintained.

[0083] In some embodiments, the training module is further configured to, in the process of training the current retrieval enhancement model, update the parameters of the current retrieval enhancement model according to the current similarity and the preset similarity threshold, and maintain the current parameter setting of the preset large language model.

[0084] In some embodiments, the processor is further configured to, in the process of training the current retrieval enhancement model, update the parameters of the current retrieval enhancement model according to the current similarity and the preset similarity threshold, and maintain the current parameter setting of the preset large language model.

[0085] Specifically, the LLM has learned a large amount of general semantic knowledge in the pre-training stage, and the semantic space for encoding the user voice request has a fixed parameter setting. If the parameter setting of the LLM is updated when the RAG is trained, the semantic space will change, and the RAG cannot accurately learn the semantic mapping relationship between the API and the user voice request. By freezing the LLM parameters, it can be ensured that the encoding result (the voice request matrix) of the user voice request is always based on the same semantic space, and the RAG model can focus on optimizing the API encoding to make it closer to the stable space, thereby realizing the feature alignment across models.

[0086] And, only the parameters of the RAG model are updated during the training process, which can focus on improving the coding ability of the RAG to the API, so that it can more accurately capture the semantic association between the API and the user voice request. That is, the RAG model learns how to encode the semantic features of the API (such as the function description of "Open_Or_Close") into a vector similar to the user voice request (such as "kinetic recovery standard mode") encoded by the LLM, so as to directly match the semantic requirements of the LLM in subsequent retrieval.

[0087] In this way, when the current retrieval enhancement model is trained, the parameters of the preset large language model remain unchanged. In this way, the parameters of the preset large language model remain unchanged, which can maintain the stability of the semantic space of the encoded user voice request, so that the current retrieval enhancement model can be aligned to the semantic space, and semantic space drift caused by changes in the parameters of the preset large language model can be avoided.

[0088] Please refer to Figure 3 In some embodiments, step 0141 (updating the parameters of the current retrieval enhancement model according to the current similarity and the preset similarity threshold) comprises:

[0089] 01411: In the case where the current similarity is less than the preset similarity threshold, determining a current loss value according to the current similarity;

[0090] 01412: updating the parameters of the current retrieval enhancement model according to the current loss value.

[0091] In some embodiments, the determining module is further configured to, in the case where the current similarity is less than the preset similarity threshold, determine a current loss value according to the current similarity; and update the parameters of the current retrieval enhancement model according to the current loss value.

[0092] In some embodiments, the processor is further configured to, in the case where the current similarity is less than the preset similarity threshold, determine a current loss value according to the current similarity; and update the parameters of the current retrieval enhancement model according to the current loss value.

[0093] Specifically, the current loss value refers to a loss function value calculated according to the current similarity between the voice request matrix (the user voice request encoded by the LLM) and the current application interface matrix (the API encoded by the RAG) during the training process of the RAG model, which is used to represent the degree of semantic difference between the two. When the similarity is less than the preset similarity threshold, the similarity is converted into a loss value by a specific loss function (such as InfoNCE loss, cosine distance, Mahalanobis distance, etc.), and the greater the value, the greater the difference between the RAG encoding and the LLM semantic space, and vice versa.

[0094] The calculated current similarity is compared with a preset similarity threshold. If the current similarity is less than the preset similarity threshold, it indicates that the RAG model is insufficiently aligned with the semantic space of the LLM for the encoding of the API, and the training process needs to be started. First, the current loss value is calculated based on the current similarity. Then, the current loss value is fed back to the RAG model to update the internal parameters (such as the weights and biases of the encoding layer) of the RAG model, adjust the encoding logic of the API, and make the newly generated API feature vector closer to the semantic space of the LLM. In this way, the encoding ability of the RAG model for the API can be optimized, and the semantic matching degree between the API and the user voice request in subsequent retrieval can be improved, laying a foundation for reducing the secondary encoding redundancy of similar application interfaces.

[0095] In this way, in the case where the current similarity is less than the preset similarity threshold, the server determines the current loss value based on the current similarity. Finally, the server updates the parameters of the current retrieval enhancement model based on the current loss value. In this way, by calculating the similarity of the encoded voice request matrix and the current application interface matrix, the difference between the voice request matrix and the current application interface matrix in the semantic space can be quantified. When the similarity is less than the preset similarity threshold, it indicates that the encoding result of the current retrieval enhancement model for the application interface is insufficiently matched with the training voice request encoded by the preset large language model in terms of semantics, and the model needs to be optimized through the current loss value.

[0096] Referring to Figure 4 In some embodiments, the method further includes:

[0097] 015: performing encoding processing on the application interface based on the updated current retrieval enhancement model to determine a next application interface matrix of a preset length;

[0098] 016: determining a next similarity based on the voice request matrix and the next application interface matrix;

[0099] 017: training the current retrieval enhancement model based on the next similarity and the preset similarity threshold.

[0100] In some embodiments, the determining module is further configured to perform encoding processing on the application interface based on the updated current retrieval enhancement model to determine a next application interface matrix of a preset length, determine a next similarity based on the voice request matrix and the next application interface matrix, and train the current retrieval enhancement model based on the next similarity and the preset similarity threshold.

[0101] In some embodiments, the processor is further configured to encode the application interface based on the updated current retrieval augmentation model, determine a next application interface matrix of a preset length, determine a next similarity based on the voice request matrix and the next application interface matrix, and train the current retrieval augmentation model based on the next similarity and a preset similarity threshold.

[0102] Specifically, the updated current retrieval augmentation model refers to a current retrieval augmentation model whose parameters have been updated based on the current loss value after one or more rounds of training.

[0103] The next application interface matrix refers to a vector matrix generated by encoding the API in the preset application interface database using the updated current retrieval augmentation model, and the encoding dimension is consistent with the voice request matrix.

[0104] The next similarity refers to a numerical value obtained by calculating the semantic matching degree between the voice request matrix and the next application interface matrix, which is used to evaluate whether the encoding effect of the updated RAG model on the API is improved, and is a key indicator for determining whether to continue training.

[0105] First, the RAG model whose parameters have been updated after one round of training is used to re-encode all APIs in the preset application interface database to generate a next application interface matrix, and the encoding dimension is consistent with the voice request matrix generated by the LLM.

[0106] Next, the next similarity between the voice request matrix and the next application interface matrix (updated RAG encoded API) is calculated to quantitatively evaluate the semantic matching degree between the optimized API encoding of the RAG model and the user voice request.

[0107] Finally, the next similarity is compared with the preset similarity threshold. If the next similarity is still less than the preset similarity threshold, it means that the alignment between the RAG encoding and the LLM semantic space is insufficient, and the RAG parameters need to be updated through the loss value back propagation for the next round of optimization. If the next similarity reaches or exceeds the preset similarity threshold, it is confirmed that the RAG model optimization is complete, and the training is stopped.

[0108] Thus, the server encodes the application interface based on the current search enhancement model updated, determines the next application interface matrix of the preset length. Then, the server determines the next similarity according to the voice request matrix and the next application interface matrix. Finally, the server trains the current search enhancement model according to the next similarity and the preset similarity threshold, to optimize the current search enhancement model. In this way, the encoding mode of the current search enhancement model for the application interface is gradually adjusted, so that the feature matrix of the application interface is closer to the voice request matrix encoded by the preset large language model in the semantic space, and the precise alignment of the encoding features of the current search enhancement model and the preset large language model is realized.

[0109] Referring to Figure 5 In some embodiments, step 0141 (updating the parameters of the current search enhancement model according to the current similarity and the preset similarity threshold) comprises:

[0110] 01413: In the case where the current similarity is greater than or equal to the preset similarity threshold, it is confirmed that the optimization of the current search enhancement model is completed.

[0111] In some embodiments, the determining module is configured to, in the case where the current similarity is greater than or equal to the preset similarity threshold, confirm that the optimization of the current search enhancement model is completed.

[0112] In some embodiments, the processor is further configured to, in the case where the current similarity is greater than or equal to the preset similarity threshold, confirm that the optimization of the current search enhancement model is completed.

[0113] Specifically, when the semantic similarity between the encoding result of the RAG model for the application interface (API) and the user voice request encoded by the large language model (LLM) reaches or exceeds the preset similarity threshold, it is determined that the RAG model has been optimized and does not need to be trained continuously.

[0114] Referring to Figure 6 , Figure 6 is a complete flowchart of the training method of the search enhancement model. The application interface matrix is determined by processing the application interface in the preset application interface library by the current search enhancement model. The similarity between the application interface matrix and the voice request matrix obtained by processing the training voice request based on the preset large language model is calculated, and the loss is determined according to the current similarity calculated, so as to train the current search enhancement model and complete the optimization of the current search enhancement model.

[0115] Thus, in the case that the current similarity is greater than or equal to the preset similarity threshold, the server confirms that the current retrieval enhancement model optimization is completed. In this way, by taking the preset similarity threshold as the judgment standard, when the similarity between the application interface feature coded by the current retrieval enhancement model and the training voice request feature coded by the preset large language model reaches or exceeds the threshold, it indicates that the semantic spaces of the two have been fully aligned, and the retrieval capability of the current retrieval enhancement model meets the demand, and the training can be terminated.

[0116] Referring to Figure 7 The embodiment of the application provides a voice interaction method based on the retrieval enhancement model trained by the training method.

[0117] 021: Based on the preset large language model, determining voice coding information according to the obtained user voice request;

[0118] 022: Based on the retrieval enhancement model, determining similar application interface coding information according to the voice coding information;

[0119] 023: Determining a target prompt word according to the preset prompt word template, the voice coding information, the preconfigured general application interface coding and the similar application interface coding information;

[0120] 024: Carrying out voice interaction according to the target prompt word.

[0121] The embodiment of the application further provides a server comprising a memory and a processor. The voice interaction method of the embodiment of the application can be implemented by the server of the embodiment of the application. Specifically, the memory stores a computer program, and the processor is configured to determine voice coding information based on a preset large language model and according to an obtained user voice request, and determine similar application interface coding information based on a retrieval enhancement model and according to the voice coding information. The processor is configured to determine a target prompt word according to a preset prompt word template, the voice coding information, preconfigured general application interface coding and similar application interface coding information, and carry out voice interaction according to the target prompt word.

[0122] The embodiment of the application further provides a voice interaction device. The voice interaction method of the embodiment of the application can be implemented by the voice interaction device of the embodiment of the application. Specifically, the voice interaction device comprises a determination module and a voice interaction module. The determination module is configured to determine voice coding information based on a preset large language model and according to an obtained user voice request, and determine similar application interface coding information based on a retrieval enhancement model and according to the voice coding information. The determination module is further configured to determine a target prompt word according to a preset prompt word template, the voice coding information, preconfigured general application interface coding and similar application interface coding information. The voice interaction module is configured to carry out voice interaction according to the target prompt word.

[0123] Specifically, the user voice request refers to an instruction input by a user through a voice, such as 'turn on the air conditioner','set the kinetic energy recovery to the standard mode', etc., and is an input content of the in-vehicle voice interaction system.

[0124] The voice encoding information refers to a vector representation generated by a preset large language model (LLM) after encoding the user voice request.

[0125] The similar application interface encoding information refers to an encoding vector of a similar API retrieved from a preset application interface database by the RAG model according to the voice encoding information, and the feature dimension is consistent with the voice encoding information.

[0126] The preset prompt word template refers to a preset structured text framework for integrating the user voice request, vehicle knowledge information, and available application interfaces to form an input understandable by the large model. The preset prompt word template includes a system prompt, an output format, vehicle knowledge information, a list of available application interfaces, and a user voice request. In some embodiments, the preset prompt word template can be as follows:

[0127] "<|im_start|> System prompt

[0128] You are a car voice assistant, according to the user's instruction, combined with the given API and its parameters in the

API candidate list

vehicle knowledge information

[0129]

Output format

[0130] Thinking logic:

[0131] Whether there is a suitable API: yes or no

[0132] Selected API: {‘API’:‘api1’,‘ARGUMENTS’:{‘arg1’:‘value1’,‘arg2’:‘value2’,...,’argn’:‘valuen’}}

[0133]

Vehicle knowledge information

[0134]

Available application interface list

[0135]

Open_Or_Close

[0136] Name: Open_Or_Close

[0137] Description: Perform opening or closing of in-vehicle related devices or functions,

[0138] ARGUMENTS:

[0139] ‘action’ : [‘open’, ‘close’]

[0140] ‘device’: specifies the device in the vehicle

[0141] ‘function’: specifies the function related to the device in the vehicle ‘target_function’: specifies different modes or different functions related to the function of the device in the vehicle

[0142] ‘seat_position’: specifies the seat position in the vehicle, the value can be one of [‘main driver’, ‘co-driver’, …]

[0143] <|im_start|> User voice request

[0144] Kinetic energy recovery standard mode <|im_end|>

[0145] <|im_start|> Assistant.

[0146] The general application interface refers to the abstraction and integration of similar functions in the cockpit API, that is, integrating multiple specific APIs with similar functions and similar parameter structures in the cockpit into a standardized and unified interface to simplify the model processing process and improve retrieval efficiency. For example, the specific operation APIs such as “turn on the air conditioner” and “turn off the window” are combined into a general Open_Or_Close API, which distinguishes the specific operation (open / close) by adding parameters (such as action), and retains the core parameters (such as device to specify the device and function to specify the function type). In this way, when the preset large language model determines that the user voice request needs to use the general application interface, the general application interface is directly used, without the need to match all application interfaces in the traditional retrieval enhancement model based on all application interfaces in the application interface vector library to determine similar application interfaces, reducing the retrieval complexity.

[0147] It should be noted that the general application interface is generally directly presented on the preset prompt word template.

[0148] The target prompt word refers to the structured text generated by splicing the preset prompt word template, the user voice request, the vehicle knowledge information, and the general API, which is used as the input of the large model. For example, the user voice request is “kinetic energy recovery standard mode”, and the target prompt word is:

[0149] “<|im_start|> System prompt

[0150] You are a car voice assistant, according to the user's instructions, combined with the given API and its parameters in the

API candidate list

vehicle knowledge

[0151] [Output format]

[0152] Thinking logic:

[0153] Is there a suitable API: Yes or No

[0154] Selected API: {"API": "api1", 'ARGUMENTS': {'arg1': 'value1', 'arg2': 'value2',..., 'argn': 'valuen'}}

[0155] [Vehicle knowledge information]

[0156] X-pedal only needs to add parameters when opening and closing: 'function': ['energy recovery level', 'driving mode'], 'target_function': ['single pedal','stop mode'],

[0157] [List of available application interfaces]

[0158] [Open_Or_Close]

[0159] Name: Open_Or_Close

[0160] Description: Perform the opening or closing of in-vehicle related devices or functions,

[0161] ARGUMENTS:

[0162] 'action': ['open', 'close']

[0163] 'device': Specifies the device in the vehicle

[0164] 'function': Specifies the function related to the device in the vehicle 'target_function': Specifies different modes or different functions related to the function of the device in the vehicle

[0165] 'seat_position': Specifies the seat position in the vehicle, the value can be selected from one of ['main driver', 'co-driver',...]

[0166] [Energy_Recycle_Value_Set]

[0167] Name: Energy_Recycle_Value_Set

[0168] Description: Adjust the specific value of the energy recovery level (the user's voice request needs to clearly mention the specific value), the value is optional.

[0169] ARGUMENTS:

[0170] ‘set_type’:['to']

[0171] ‘set_value’:['large','medium'...]

[0172] <|im_start|>User voice request

[0173] Kinetic energy recovery standard mode<|im_end|>

[0174] <|im_start|>Assistant.

[0175] First, the pre-trained LLM is used to encode the user voice request to generate a fixed-dimensional (such as 1024-dimensional) semantic vector matrix, i.e. voice encoding information.

[0176] Next, the trained RAG model is used to search all API encodings in the pre-set application interface database, and the most similar API encoding to the voice encoding information is selected by calculating the semantic similarity.

[0177] Then, the server determines the target prompt word according to the pre-set prompt word template, voice encoding information, pre-configured general application interface encoding and similar application interface encoding information.

[0178] Finally, the target prompt word is input into the LLM, which infers the request category of the user voice request according to the semantic information in the prompt word. And according to the request category, the voice interaction is carried out. That is, in the case of a user voice request for a setting request, based on the retrieval enhancement model, the server determines the similar application interface encoding and similar voice request encoding associated with the user voice request according to the voice encoding information. Next, the server performs slot recognition processing according to the voice encoding information, similar voice request encoding and vehicle knowledge information to determine the slot recognition result. Then, the server determines the target application interface from the similar application interface encoding according to the voice encoding information, similar voice request encoding and vehicle knowledge information. Finally, the server performs application interface parameter filling based on the slot recognition result and the target application interface to obtain a natural language understanding result for voice interaction.

[0179] In the case of a user voice request for a general request, the server performs slot recognition processing according to the voice encoding information and vehicle knowledge information to determine the slot recognition result. Next, the server performs application interface parameter filling based on the slot recognition result and the general application interface encoding to obtain a natural language understanding result for voice interaction.

[0180] Please refer toFigure 8 , Figure 8 The complete flowchart of the voice interaction method provided by the embodiment of the application is shown. After the user voice request is input, the vehicle knowledge information is obtained through the AC automatic machine, and the user voice request is spliced based on the preset prompt word template, and then the request type of the user voice request is determined. Different voice interaction flows are performed based on different request types.

[0181] In this way, based on the preset large language model, the server determines the voice coding information according to the obtained user voice request. Then, based on the retrieval enhancement model, the server determines the similar application interface coding information according to the voice coding information, wherein the feature dimensions of the similar application interface coding information and the voice coding information are consistent. Then, the server determines the target prompt word according to the preset prompt word template, the voice coding information, the preconfigured general application interface code and the similar application interface coding information. Finally, the server performs voice interaction according to the target prompt word. In this way, by using the preset large language model and the retrieval enhancement model, the voice coding information and the similar application interface coding information with consistent dimensions are determined, and then the natural language processing result can be directly determined according to the voice coding information and the similar application interface coding information, thereby avoiding repeated coding of the similar application interface and eliminating the redundant work of twice coding of the large language model and the retrieval enhancement model for the similar application interface in the original flow.

[0182] Please refer to Figure 9 In some embodiments, the method further comprises:

[0183] 025: analyzing the user voice request, and determining a temporary generalization word from the preset generalization word library;

[0184] 026: performing deduplication processing on the temporary generalization word based on a preset rule to determine a target generalization word;

[0185] 027: determining a target entity according to the target generalization word;

[0186] 028: determining a similar public entity according to the target entity;

[0187] 029: determining associated vehicle knowledge information from the preset generalization word library according to the similar public entity.

[0188] In some embodiments, the determining module is further configured to analyze the user voice request, and determine a temporary generalization word from the preset generalization word library. Perform deduplication processing on the temporary generalization word based on a preset rule to determine a target generalization word. And determine a target entity according to the target generalization word. The determining module is further configured to determine a similar public entity according to the target entity. And determine associated vehicle knowledge information from the preset generalization word library according to the similar public entity.

[0189] In some embodiments, the processor is further configured to parse the user voice request, determine temporary generalization words from the preset generalization word library, and determine target generalization words by performing deduplication processing on the temporary generalization words based on preset rules, and determine the target entity based on the target generalization words. The processor is further configured to determine similar public entities based on the target entity, and determine associated vehicle knowledge information from the preset generalization word library based on the similar public entities.

[0190] Specifically, the parsing processing refers to scanning the user voice request using the AC automatic machine algorithm to extract keywords matching the preset generalization word library. That is, the AC automatic machine identifies all entity generalization words in the user voice request in a single scan by constructing a Trie tree and a failure pointer, such as extracting the words "kinetic energy", "kinetic energy recovery", and "standard mode" from "kinetic energy recovery standard mode".

[0191] The preset generalization word library refers to a database including generalization word-entity word correspondence and entity word-vehicle knowledge information correspondence.

[0192] The temporary generalization word refers to a set of entity generalization words initially matched by the AC automatic machine from the user voice request, which may include repeated or not completely matched words with the user voice request. For example, when the user voice request is "kinetic energy recovery standard mode", the temporary generalization word is ["kinetic energy", "kinetic energy recovery", "standard mode"].

[0193] The preset rule refers to the specific screening logic followed when processing the temporary generalization word, which can eliminate redundant generalization words, retain the target generalization word that best matches the user voice request, and ensure the accuracy of subsequent vehicle knowledge mapping. The preset rule includes longest common substring matching and similar public entity matching.

[0194] The longest common substring matching refers to when there is a containing relationship in the temporary generalization word (such as "kinetic energy" and "kinetic energy recovery"), the longer generalization word usually contains more specific semantic information, and has a higher matching degree with the user voice request, so the longest generalization word is preferentially retained to avoid entity mapping ambiguity caused by semantic ambiguity of short generalization words.

[0195] The similar public entity matching refers to mapping the generalized words after the longest common substring matching to the original entity words, if multiple generalized words correspond to the same original entity or there is a public entity subset, the generalized words corresponding to the public entity are retained, and the generalized words of irrelevant entities are excluded. For example, after the generalized words ["dynamic energy recovery", "standard mode"] obtained through the longest common substring matching are mapped to the original entity words, ["X-pedal driving mode, single pedal mode"] and ["driving mode", "standard mode", "energy recovery level", "X-pedal driving mode, single pedal mode"] are obtained, the common entity "X-pedal driving mode, single pedal mode" of the two is extracted, and finally the generalized words related to the entity are retained to ensure the uniqueness of the vehicle knowledge mapping.

[0196] The processing of the temporary generalized words refers to screening the temporary generalized words based on the preset rules, excluding redundant or irrelevant words, and retaining the generalized words most matched with the user voice request. In this way, by processing the temporary generalized words based on the preset rules, the redundancy of the generalized words can be reduced, invalid entity interference can be avoided, and the accuracy of the vehicle knowledge mapping can be improved.

[0197] The target generalized word refers to the generalized word most related to the user voice request obtained after processing based on the preset rules, which can be directly mapped to the unique vehicle entity and the corresponding knowledge.

[0198] The target entity refers to the core vehicle function entity directly related to the user voice request obtained by mapping the target generalized word, for example, the target generalized word "dynamic energy recovery, energy recovery and energy saving mode, etc." corresponds to the entity word "energy recovery"; the target generalized word "exterior light, wide light mode, position light and small light, etc." corresponds to the entity word "wide light", which is the target entity; the target generalized word "energy saving mode" corresponds to the entity word "driving mode".

[0199] The associated entity word refers to the associated entity word determined according to the target entity. In some embodiments, one target generalized word may correspond to multiple target entities, such as the target generalized word "energy saving mode" corresponding to "energy recovery" and "driving mode" described above, so the obtained target entities need to be processed to remove duplicates to determine the associated entity word.

[0200] The vehicle knowledge information refers to knowledge such as functional description, parameter requirement, and API calling rule related to the entity word, for example, "X-pedal driving mode, single-pedal mode includes multiple modes, and only opening and closing need to supplement parameters: function: {energy recovery level, driving mode} and target_function: {single-pedal, stop mode}". The acquisition method of the vehicle knowledge information is: through the mapping relationship of the preset generalization word library, the entity word is determined from the generalization word, and the vehicle knowledge information is derived from the entity word, for example, the generalization word 'kinetic energy recovery' is mapped to the vehicle knowledge information of 'X-pedal driving mode'.

[0201] In this way, the server analyzes and processes the user voice request, determines the temporary generalization word from the preset generalization word library. Then, based on the preset rule, the server performs deduplication processing on the temporary generalization word to determine the target generalization word. Then, the server determines the target entity according to the target generalization word. Subsequently, the server determines the similar public entity according to the target entity. Finally, the server determines the associated vehicle knowledge information from the preset generalization word library according to the similar public entity. In this way, through the above processing, the key semantics in the user voice request can be accurately extracted, and the entity retrieval efficiency and accuracy can be improved, thereby optimizing the input quality of the preset large language model reasoning.

[0202] Please refer to Figure 10 In some embodiments, step 023 (determining the target prompt word according to the preset prompt word template, the voice coding information, the pre-configured general application interface code, and the similar application interface coding information) comprises:

[0203] 0231: based on the preset label and the preset prompt word template, the voice coding information, the associated vehicle knowledge information, the pre-configured general application interface code, and the similar application interface coding information are spliced to determine the target prompt word.

[0204] In some embodiments, the determining module is further configured to splice the voice coding information, the associated vehicle knowledge information, the pre-configured general application interface code, and the similar application interface coding information based on the preset label and the preset prompt word template to determine the target prompt word.

[0205] In some embodiments, the processor is further configured to splice the voice coding information, the associated vehicle knowledge information, the pre-configured general application interface code, and the similar application interface coding information based on the preset label and the preset prompt word template to determine the target prompt word.

[0206] Specifically, the preset label refers to a special mark used to mark information of different sources when splicing multiple source coding information to generate a target prompt word, such as <CAR_Knows>, <\CAR_Knows>, <apis>, <\APIs>, <query>, <Query>, etc. The preset label can ensure that the format of the spliced feature is consistent with the format of the original prompt of the large language model (LLM), and will be deleted after splicing to eliminate coding redundancy and ensure that the LLM can correctly process the input information, thereby improving the efficiency and accuracy of voice interaction.

[0207] In this way, based on the preset label and the preset prompt word template, the server splices the voice coding information, the associated vehicle knowledge information, the pre-configured general application interface code and the similar application interface code information to determine the target prompt word. In this way, the voice coding information, the associated vehicle knowledge information, the pre-configured general application interface code and the similar application interface code information are fused by splicing to provide structured input for the preset large language model, and to guide the preset large language model to accurately generate the application interface and the parameters.

[0208] The present application also provides a computer readable storage medium having a computer program stored thereon. When the computer program processor is executed, the steps of the method described above are implemented.

[0209] It can be understood that the computer program includes computer program code. The computer program code can be in the form of source code, object code, executable files or some intermediate forms. The computer readable storage medium can include any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory) and software distribution medium, etc.

[0210] In the description of the present application, the description of the terms "specifically", "further", "particularly", "understandably" and the like means that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In the present application, the illustrative description of the above terms does not mean the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, the skilled in the art can combine and combine the different embodiments or examples described in the present application and the features of the different embodiments or examples without contradiction.

[0211] Any procedural or methodological descriptions in flow charts or otherwise described herein can be understood to represent modules, segments, or portions of executable request code including one or more steps for implementing the particular logic function or process, and the scope of preferred embodiments of the present application includes additional implementations in which the functions can be performed in an order other than that shown or discussed, including substantially simultaneously, or in reverse order, as will be understood by those having ordinary skill in the art to which embodiments of the present application pertain.

[0212] Although the embodiments of the present application have been shown and described above, it is understood that the above-described embodiments are exemplary and should not be construed as limiting the present application, and those ordinarily skilled in the art can make changes, modifications, substitutions, and variations to the above-described embodiments within the scope of the present application.< / query> < / apis>

Claims

1. A training method for a retrieval enhancement model, characterized in that: The method comprises: Based on a preset large language model, encoding the acquired training voice request to determine a voice request matrix, where the encoding length of the voice request matrix is ​​a preset length; Based on the current search enhancement model, encoding the application interface in the preset application interface database to determine the current application interface matrix, where the encoding length of the current application interface matrix is ​​the preset length; Determining a current similarity based on the voice request matrix and the current application interface matrix; The current retrieval enhancement model is trained according to the current similarity and a preset similarity threshold.

2. The method according to claim 1, characterized in that The training of the current retrieval enhancement model according to the current similarity and a preset similarity threshold includes: During the training of the current retrieval enhancement model, the parameters of the current retrieval enhancement model are updated according to the current similarity and the preset similarity threshold, and the current parameter settings of the preset large language model are maintained.

3. The method according to claim 2, characterized in that The updating of the parameters of the current retrieval enhancement model according to the current similarity and a preset similarity threshold includes: When the current similarity is less than the preset similarity threshold, determining a current loss value according to the current similarity; According to the current loss value, the parameters of the current retrieval enhancement model are updated.

4. The method according to claim 3, characterized in that The method further comprises: Based on the updated current retrieval enhancement model, encoding the application interface to determine the next application interface matrix of the preset length; determining a next similarity according to the voice request matrix and the next application interface matrix; The current retrieval enhancement model is trained according to the next similarity and the preset similarity threshold.

5. The method according to claim 2, characterized in that The updating of the parameters of the current retrieval enhancement model according to the current similarity and a preset similarity threshold includes: When the current similarity is greater than or equal to the preset similarity threshold, it is confirmed that the training of the current retrieval enhancement model is completed.

6. A voice interaction method, characterized in that: The voice interaction method is based on a retrieval enhancement model obtained by training using the training method according to any one of claims 1 to 5, and the method includes: Based on the preset large language model, determine the voice coding information according to the acquired user voice request; Determining similar application interface coding information based on the voice coding information based on the retrieval enhancement model, wherein the feature dimension of the similar application interface coding information is consistent with the feature dimension of the voice coding information; Determining a target prompt word according to a preset prompt word template, the voice coding information, a pre-configured universal application interface code, and the similar application interface code information; The voice interaction is performed according to the target prompt word.

7. The method according to claim 6, characterized in that The method further comprises: Parsing the user voice request and determining a temporary generalized word from the preset generalized word library; Based on preset rules, the temporary generalized words are deduplicated to determine the target generalized words; Determining a target entity according to the target generalized word; Determining similar public entities based on the target entity; According to the similar public entities, associated vehicle knowledge information is determined from the preset generalized vocabulary.

8. The method according to claim 7, characterized in that The step of determining a target prompt word according to a preset prompt word template, the voice coding information, the pre-configured universal application interface coding information, and the similar application interface coding information includes: Based on a preset tag and a preset prompt word template, the voice coding information, the associated vehicle knowledge information, the pre-configured universal application interface coding information and the similar application interface coding information are spliced ​​to determine the target prompt word.

9. A server, characterized in that: The server includes a processor and a memory, wherein a computer program is stored in the memory. When the computer program is executed by the processor, the method according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Citation Information

Patent Citations

  • Method and equipment for generating large model API (Application Program Interface) sequence based on retrieval enhancement

    CN118536607A

  • Retrieval enhancement generation method based on token-level word embedding

    CN119311814A

  • Retrieval enhancement method and device based on large model and storage medium

    CN119988602A

  • Model training method of large language model, server and storage medium

    CN120183389A

  • Response generation using a retrieval augmented ai model

    US20240346256A1