Voice interaction method, voice interaction device and computer readable storage medium

Abstract

The invention discloses a voice interaction method, a voice interaction device and a computer readable storage medium. The voice interaction method comprises the following steps: processing a current round of request by adopting a large language model in combination with a user memory database so as to determine a target application program interface from at least one alternative application program interface; wherein the alternative application program interfaces comprise the application program interfaces of which the similarity between the interface information of the application program interfaces and the current round request is higher than a preset similarity; the user memory database is at least used for storing user personalized information; and calling the target application program interface. Therefore, by introducing the user memory database as a decision basis, the decision of the large language model does not only depend on general semantics, but can fuse the long-term preference and historical habits of the user, so that the finally called target API can meet the individual requirements of the user.

Description

Technical Field

[0001] This invention relates to the field of voice interaction, and in particular to a voice interaction method, a voice interaction device, and a computer-readable storage medium. Background Technology

[0002] In-vehicle voice interaction systems can already use large language models to process user requests and call corresponding application programming interfaces (APIs) to execute specific functions (such as navigation, media playback, and vehicle control) based on the processing results. However, in related technologies, large language models generally only process user requests based on basic semantics, lacking consideration of users' long-term preferences and historical habits. For example, when a user issues the command "play music," the system can only randomly play music based on general rules or call a default music service API, and cannot intelligently select the most suitable music application or generate a precise playlist based on the user's historical preferences for a particular artist or style. This results in a monotonous interactive experience, failing to meet users' expectations for personalized services that "understand" them. Summary of the Invention

[0003] This application provides a voice interaction method, a voice interaction device, and a computer-readable storage medium.

[0004] This application provides a voice interaction method, the method comprising: A large language model combined with a user memory database is used to process the current round of requests in order to determine the target application interface from at least one candidate application interface; wherein, the candidate application interfaces include: application interfaces whose interface information has a similarity to the current round of requests higher than a preset similarity; the user memory database is used to store at least user personalized information; Invoke the target application interface.

[0005] Thus, by introducing the user memory database as a decision-making basis, the decision of the large language model no longer relies solely on general semantics, but can integrate the user's long-term preferences and historical habits, so that the target API ultimately called can meet the user's personalized needs.

[0006] In some implementations, the method further includes, prior to processing the current round of requests using a large language model in conjunction with a user memory database to determine the target application interface from at least one candidate application interfaces: The large language model is used to extract the user's personalized information from the current round of input; Based on the user's personalized information, add, replace, or delete target user memory data in the user memory database; The current round input includes at least one of the following: the current round request, and non-command input.

[0007] In other words, the large language model is responsible for the dynamic maintenance (creation, updating, or deletion) of the user memory database. This makes the user memory database not a static memory storage device, but an intelligent agent capable of autonomously learning from continuous interaction. The large language model can understand new, changed, or contradictory information in user input and automatically perform corresponding operations on the memory database. This ensures that the user memory database can reflect the user's latest and most authentic preferences in real time, providing a dynamic and reliable data foundation for accurate services. Furthermore, the maintenance of the user memory database is completely automated, requiring no manual settings by the user or frequent intervention from developers. Users complete the training of personalized models in everyday natural conversations, achieving "seamless learning," greatly simplifying the interaction process and improving the smoothness of the user experience.

[0008] In some implementations, extracting the user's personalized information from the current round input using the large language model includes: The current round input is filled into the memory retrieval prompt template to generate memory retrieval prompts; The large language model is used to process the memory retrieval prompts in order to extract the user's personalized information in the current round of input.

[0009] Thus, by using the memory retrieval prompt template to guide the large language model, key facts (such as "favorite singer: Jay Chou") can be accurately extracted from free text dialogues and formatted into structured "memory data." This results in stronger generalization ability and semantic understanding depth, enabling the handling of diverse expressions and significantly improving the coverage and accuracy of memory retrieval. Furthermore, the fixed memory retrieval prompt template ensures that the extracted memory data follows a unified format or schema. This standardization is a prerequisite for subsequent effective vectorization, storage, retrieval, and comparison operations (see below), laying a data foundation for the efficient operation of the user memory database.

[0010] In some implementations, adding, replacing, or deleting target user memory data in the user memory database based on the user's personalized information includes: The user's personalized information is entered into the memory maintenance prompt template to generate memory maintenance prompts; The large language model is used to process the memory maintenance prompts in order to extract the user's personalized information in the current round of input.

[0011] The addition, updating, and deletion of target user memory data are not simple additions, deletions, or modifications, but rather intelligent operations requiring semantic judgment (such as information merging and contradiction detection). By designing a specialized memory maintenance prompt template to encapsulate these complex logics, complex scenarios such as "information supplementation," "preference shifting," and "self-denial" can be reliably handled, ensuring the quality of memory bank maintenance decisions. Furthermore, embodying the maintenance logic in the memory maintenance prompt template makes the rules of memory management transparent, auditable, and optimizable. Developers can improve maintenance strategies by adjusting the prompts, which is more flexible and secure than modifying hard-coded algorithm rules.

[0012] In some implementations, adding, replacing, or deleting target user memory data in the user memory database based on the user's personalized information includes: Retrieve historical user memory data associated with the user's personalized information from the user memory database; If the historical user memory data is not found, the user's personalized information will be added to the user memory database. If the historical user memory data is retrieved and the amount of information in the user personalized information is greater than that in the historical user memory data, the historical user memory data is replaced with the user personalized information. If the historical user memory data is retrieved and the user's personalized information contradicts the historical user memory data, the historical user memory data is deleted.

[0013] In this way, the "maximization of information content" and "consistency" of the user memory database are ensured. Specifically, the rule of "adding if not found, and replacing if information content is greater" ensures that the memory content evolves in a richer and more specific direction (e.g., updating from "likes coffee" to "likes iced Americano"), making refined services possible. The "deleting if contradictory" rule maintains the logical consistency of the user memory database, promptly removing erroneous or outdated information and preventing the large language model from making incorrect inferences based on contradictory memories.

[0014] In some implementations, retrieving historical user memory data associated with the user's personalized information from the user memory database includes: The historical user memory data is retrieved from the user memory database using vector similarity retrieval technology.

[0015] Vector similarity retrieval technology can overcome the limitations of literal keyword matching and find the historical user memory data that is most semantically relevant to the current input. For example, if a user says "I want to listen to something refreshing," vector similarity retrieval technology can retrieve the associated memory phrase "favorite music genre: electronic music," even if there is no literal overlap. On the other hand, for massive amounts of user memory entries, vector similarity retrieval technology can complete the retrieval within milliseconds, meeting the stringent low-latency requirements of real-time interactive scenarios such as in-vehicle systems.

[0016] In some implementations, the step of using a large language model combined with a user memory database to process the current round of requests to determine the target application interface from at least one candidate application interfaces includes: In the presence of historical round requests and / or historical round dialogues, the large language model is used in conjunction with the historical round requests and / or historical round dialogues to process the current round request, so as to concatenate the current round request and the historical round requests and / or historical round dialogues to complete the current round request; The large language model is used in conjunction with the user memory database to process the completion request in the current round, so as to determine the target application interface from the at least one candidate application interface.

[0017] Thus, by using the large language model to concatenate the history and current request into a "complete current round request," it can automatically correct or complete unclear or incomplete user instructions. By introducing "historical round requests and / or dialogues," it possesses the ability to perceive dialogue context, understand referential expressions (such as "turn it up a bit"), omissions (such as saying "No, go home" directly after the user says "navigate to the company"), and other context-dependent expressions, greatly improving the naturalness and fluency of multi-turn dialogues.

[0018] In some implementations, processing the current round completion request using the large language model in conjunction with the user memory database includes: Vector similarity retrieval technology is used to search within the user memory database to obtain at least one alternative user memory data associated with the current round of completion request; Vector similarity retrieval technology is used to search within a pre-built application interface library to obtain at least one alternative application interface associated with the current round of completion request; The large language model is used to process the current round completion request, the at least one alternative user memory data, and the at least one alternative application interface to determine the target application interface.

[0019] First, vector similarity retrieval technology quickly filters the most relevant subset (candidate user memory data) from the massive user memory database, and then quickly filters potentially relevant subset (candidate APIs) from the massive API library. Then, the large language model performs deep reasoning on this narrowed, highly relevant candidate set. In this way, while ensuring decision accuracy, computational efficiency is greatly improved. By vectorizing the "API library" (API description documents), the large language model can dynamically discover and understand new service capabilities.

[0020] In some implementations, processing the current round completion request, the at least one alternative user memory data, and the at least one alternative API using the large language model includes: The current round request, the at least one alternative user memory data, and the at least one alternative API are filled into the instruction processing prompt word template to generate the instruction processing prompt word; The large language model is used to process the instruction processing prompt words to determine the target API.

[0021] In this way, a carefully designed "decision context" is constructed for the large language model. The instruction processing prompt word template can explicitly require the large language model to consider personalized preferences, compare the applicability of different APIs, and output formatted decision results. This greatly improves the reliability, consistency, and operability of the output results of the large language model, avoiding the arbitrariness or deviation that may arise from its free interpretation. On the other hand, by embedding the core interaction strategies and business logic into the instruction processing prompt word template, when it is necessary to adjust the decision rules or support new scenarios, only the prompt word template needs to be modified, without retraining or fine-tuning the large language model, making iteration and maintenance more agile and cost-effective.

[0022] Additional aspects and advantages of embodiments of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of embodiments of this application. Attached Figure Description

[0023] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, wherein: Figure 1-7 This is a flowchart illustrating a voice interaction method according to certain embodiments of this application. Detailed Implementation

[0024] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of this application, and should not be construed as limiting the embodiments of this application.

[0025] In-vehicle voice interaction systems are already able to use large language models to process user requests and call corresponding APIs to execute specific functions (such as navigation, media playback, and vehicle control) based on the processing results. However, in related technologies, large language models generally process user requests based only on basic semantics, lacking consideration of users' long-term preferences and historical habits.

[0026] For example, when a user issues a "play music" command, the system can only play music randomly based on general rules or call a default music service API. It cannot intelligently select the most suitable music application or generate a precise playlist based on the user's historical preferences for a particular artist or style. This results in a monotonous interactive experience that fails to meet users' expectations for personalized services that truly "understand" them.

[0027] Similar issues exist in vehicle control function adjustments. Over long-term vehicle use, users typically develop specific personalized habits, such as seat tilt angle, default air conditioning temperature settings (e.g., 24°C automatic fan speed), and default air vent direction selection. However, existing in-vehicle voice assistants cannot store these habits as personalized memories. Each time a user needs to make an adjustment, they must repeatedly state the specific parameters, such as "adjust the seat back 5 centimeters" or "turn the air conditioning to 24 degrees, blowing on your feet and face." This repetitive input not only lengthens the dialogue process and increases the user's interaction costs but also reduces operational efficiency. Especially while driving, cumbersome command input can even distract the user, posing a safety hazard.

[0028] Based on the above issues, please refer to Figure 1 This application provides a voice interaction method, which includes: 01: A large language model combined with a user memory database is used to process the current round of requests to determine the target application interface from at least one candidate application interface; wherein, the candidate application interfaces include: application interfaces whose interface information has a similarity to the current round of requests higher than a preset similarity; the user memory database is used to store at least user personalized information; and 02: Call the target API.

[0029] This application provides a voice interaction device, which includes a processing module and a calling module. The processing module is used to process the current round of requests using a large language model combined with a user memory database to determine a target application interface from at least one candidate application interfaces. The candidate application interfaces include those whose interface information has a similarity to the current round of requests that is higher than a preset similarity. The user memory database is used to store at least user-personalized information. The calling module is used to call the target API.

[0030] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor or at least one processor, the computer program is configured to process a current round of requests using a large language model combined with a user memory database to determine a target application interface from at least one candidate application interfaces. The candidate application interfaces include: application interfaces whose interface information has a similarity to the current round of requests higher than a preset similarity; the user memory database is used to store at least user personalized information; and the program invokes the target API.

[0031] In the voice interaction method, voice interaction device, and computer-readable storage medium of the embodiments of this application, by introducing a user memory database as a decision basis, the decision of the large language model no longer relies solely on general semantics, but can integrate the user's long-term preferences and historical habits, so that the target API called can meet the user's personalized needs.

[0032] Large language model (LLM): refers to a deep learning model that has been pre-trained on massive amounts of text data and has powerful semantic understanding, contextual reasoning and text generation capabilities. In this implementation, LLM acts as the core "inference engine" and is responsible for understanding user requests, associating memories, and making decisions.

[0033] User memory databases refer to structured data storage systems specifically designed for storing, indexing, and managing personalized user information, serving as a long-term, external knowledge source for LLM (Lifecycle Management) personalized reasoning. The content of a user memory database can include structured or semi-structured data extracted from historical conversations, such as user preferences, habits, and personal facts (e.g., {"Preference": "Music Genre", "Value": "Rock"}, {"Entity": "Frequently Visited Coffee Shop", "Value": "Starbucks in Building A"}). User memory databases are generally vector databases. That is, each piece of user memory data is not only stored as raw text or key-value pairs, but also transformed into a high-dimensional vector (i.e., "embedded vector") using text embedding models (such as BGE, Sentence-BERT) and indexed to enable efficient semantic similarity retrieval, rather than relying solely on keyword matching.

[0034] Current round request: refers to the latest instruction or query issued by the user to the system in a single interaction via voice, text, or other input methods. For example, "Navigate to my usual coffee shop" or "Play some relaxing music".

[0035] API: Refers to the functional interfaces provided by the in-vehicle system or connected external services that can be called by programs. Examples include the "Navigate to {location}" API, the "Play a song by {artist}" API, and the "Set the air conditioning temperature to {temperature}" API. These APIs encapsulate specific execution capabilities.

[0036] Alternative APIs: These refer to one or more potentially related APIs initially selected from all available APIs in the API library based on the semantics of the current round of requests. This is generally done in the "API description library" through vector similarity retrieval (see below).

[0037] Target API: The API that best matches the user's personalized and true intent is finally selected after comprehensive reasoning by LLM and user memory database. The target API will be actually called to execute the service.

[0038] Calls may involve the vehicle control domain: sending control commands to the body controller (such as setting the air conditioning temperature); the infotainment domain: launching a specific media application and executing a playback command; the navigation domain: planning a route and starting navigation; or external services: calling cloud services through the vehicle network (such as searching for restaurants or ordering coffee).

[0039] Please see Figure 1 In some implementations, before processing the current round of requests using a large language model in conjunction with a user memory database to determine the target application interface from at least one candidate application interfaces, the method further includes: 001: Use the large language model to extract the user's personalized information from the current round of input; 002: Based on the user's personalized information, add, replace, or delete target user memory data in the user memory database; The current round input includes at least one of the following: the current round request, and non-command input.

[0040] In other words, the large language model is responsible for the dynamic maintenance (creation, updating, or deletion) of the user memory database. This transforms the user memory database from a static memory storage device into an intelligent agent capable of autonomous learning from continuous interaction. The large language model can understand new, changed, or contradictory information in user input and automatically perform corresponding operations on the memory database. This ensures that the user memory database reflects the user's latest and most authentic preferences in real time, providing a dynamic and reliable data foundation for accurate services. Furthermore, the maintenance of the user memory database is completely automated, requiring no manual setup by the user or frequent intervention from developers. Users complete the training of personalized models in everyday natural conversations, achieving "seamless learning," greatly simplifying the interaction process and improving the smoothness of the user experience.

[0041] Current round input: This refers to the input source that triggers the memory maintenance process. Current round input can include current round requests. Specifically, current round requests typically refer to in-vehicle commands embedded in human-computer dialogue, that is, the target API that the large language model will decide to call after processing. Current round input only needs to be semantically clear user input, and it doesn't even have to be a human-computer dialogue. For example, casual conversations between users often contain a lot of personal information and preferences, which can be used as current round input, but not as current round requests. Of course, casual conversations between users and the in-vehicle voice system, which are non-command inputs, also fall under the category of current round input, but cannot be used as current round requests.

[0042] Target user memory data: refers to the specific user memory data that is determined to require operation (addition, update, deletion) in this maintenance decision. For "addition", the target is the newly extracted user memory data itself; for "update" or "delete", the target is a specific historical memory entry that already exists in the user memory database and is related to the new user memory data.

[0043] Create: Also known as "add", it refers to creating a new user memory data entry for entirely new user preferences or factual information that does not exist in the memory bank and storing it in the user memory database.

[0044] Update: This refers to modifying, supplementing, or replacing the content of an existing user memory record in the user memory database to make the information richer and more accurate, while the user attributes or entities it refers to remain unchanged. For example, updating from "Favorite singer: Jay Chou" to "Favorite singers: Jay Chou and JJ Lin".

[0045] Deletion: This refers to removing an existing record from the memory bank or marking it as invalid. It usually occurs when the system is certain that the memory information is no longer correct or contradicts the user's latest statement.

[0046] After processing the current round of input, the LLM output includes the specific operation (ADD, UPDATE, DELETE, NONE), the target memory ID (for updates and deletions), and a structured instruction for the new memory content.

[0047] Please see Figure 2 In some implementations, extracting the user's personalized information from the current round of input using the large language model includes: 0011: Fill the current round input into the memory retrieval prompt template to generate memory retrieval prompts; 0012: The large language model is used to process the memory retrieval prompt words in order to extract the user's personalized information in the current round of input.

[0048] Thus, by using memory retrieval cue templates to guide large language models, key facts (such as "favorite singer: Jay Chou") can be accurately extracted from free text dialogues and formatted into structured user memory data. This results in stronger generalization ability and semantic understanding depth, enabling the handling of diverse expressions and significantly improving the coverage and accuracy of memory retrieval. Furthermore, using fixed memory retrieval cue templates ensures that the extracted memory data follows a unified format or schema. This standardization is a prerequisite for subsequent effective vectorization, storage, retrieval, and comparison operations (see below), laying a data foundation for the efficient operation of the user memory database.

[0049] Memory retrieval prompt template: This refers to a predefined text framework containing specific task instructions and output format requirements. Its core function is to systematically guide the behavior of LLM (Local Language Management), focusing and constraining its powerful general text understanding capabilities to the specific task of "extracting user preferences and facts from dialogue." Memory retrieval prompt templates typically contain placeholders (such as {user input}) for dynamically filling in the actual "current round input."

[0050] In one example, the memory retrieval cue template is as follows: Please extract entity content related to cabin service, user preferences, and user personal information. Here are a few examples: Input: The weather is really nice today.

[0051] Output: {{"facts": []}} Input: I especially like Jay Chou.

[0052] Output: {{"facts": ["Favorite singer: Jay Chou"]}} Input: I love listening to rock music.

[0053] Output: {{"facts": ["Favorite music genre: rock"]}} Input: I've been craving a latte lately.

[0054] Output: {{"facts": ["Favorite coffee: Latte"]}} Input: I don't like listening to sports news.

[0055] Output: {{"facts": ["Disliked news type: sports news"]}} Input: My maternal grandmother's address is Jinyu Huafu.

[0056] Output: {{"facts": ["Grandma's house address: Jinyu Huafu"]}} Please return the relevant facts and user information in JSON format as described above. Personalized user information refers to the structured set of user information extracted from a single "current input" through the above process. It is usually represented as a list, where each item is an independent, uniformly formatted statement of a remembered fact, such as "favorite drink: latte". This is the original form of the "remembered data" before it enters the comparison and maintenance process.

[0057] Structured data refers to the process of converting information contained in free text into a standardized representation with clearly defined "attributes" (or "keys") and "values". Structured data facilitates precise storage, comparison, and retrieval by computers.

[0058] Please see Figure 2 In some implementations, adding, replacing, or deleting target user memory data in the user memory database based on the user's personalized information includes: 0021: Fill the user's personalized information into the memory maintenance prompt template to generate memory maintenance prompts; 0022: The memory maintenance prompt words are processed using the large language model to add, replace, or delete the target user memory data in the user memory database.

[0059] Adding, updating, and deleting target user memory data is not a simple matter of addition, deletion, or modification; rather, it requires intelligent operations based on semantic judgment (such as information merging and contradiction detection). By designing specialized memory maintenance prompt templates to encapsulate these complex logics, complex scenarios such as "information supplementation," "preference shift," and "self-denial" can be reliably handled, ensuring the quality of decisions in user memory database maintenance. Furthermore, embodying the maintenance logic in the memory maintenance prompt templates makes the rules for user memory database management transparent, auditable, and optimizable. Developers can improve maintenance strategies by adjusting the memory maintenance prompt templates, which is more flexible and secure than modifying hard-coded algorithm rules.

[0060] Memory Maintenance Prompt Template: A predefined, high-level instruction framework that encapsulates the business logic of memory maintenance. Its core task is to compare "newly retrieved memories" with "historically retrieved memories" and, based on a set of intelligent rules, decide whether to perform an "add," "update," "delete," or "no change" operation. This template serves as the carrier of the system's "memory management strategy."

[0061] In one example, the memory maintenance cue template is: You are a memory manager for a smart cockpit, responsible for controlling the cockpit system's memories. You can perform four operations: (1) add a memory, (2) update a memory, (3) delete a memory, and (4) leave it unchanged.

[0062] Based on the four operations described above, memories will change. The newly retrieved facts are compared with existing memories. For each new fact, a decision is made as to whether: Add (ADD): If the retrieved fact contains new information that is not in memory, it needs to be added and a new ID is generated in the id field.

[0063] Update: If the retrieved fact contains information already in memory, but the information is completely different, it needs to be updated. If the retrieved fact conveys the same content as an element in memory, the content with the most information should be retained. For example, if the memory contains "the user likes playing cricket," and the retrieved fact is "likes playing cricket with friends," the memory needs to be updated to the retrieved fact. If the memory contains "likes going to work," and the retrieved fact is "likes going to work," no update is needed because they convey the same information. If updating the memory is necessary, it must be done. During the update, the same ID must be retained. Note that the IDs returned in the output should only come from the IDs in the input; do not generate any new IDs.

[0064] DELETE: If the retrieved fact contains information that contradicts the information in your memory, it needs to be deleted. Alternatively, if you need to delete a memory, it must be deleted. Note that the IDs returned in the output should only come from the input IDs; do not generate any new IDs.

[0065] No change (NONE): If the retrieved facts contain information already in memory, no changes are required.

[0066] Here are specific examples: Add to: Old Memories: [ "id": "0", "text": "Favorite singer: Jay Chou" ] Extracted memory: ["Favorite song: As I Wish"] New Memories: [ "memory": [ { "id": "0", "text": "Favorite singer: Jay Chou", "event": "NONE" ] , [ "id": "1", "text": "Favorite song: As You Wish", "event": "ADD" ] ] ] renew Old Memories: [ ] [ ] "id" : "0", "text": "Favorite singer: Jay Chou" }, [ "id" : "1", "text": "Favorite song: As You Wish" ], ] Extracted memory: ["Favorite song: All the stars in the sky are you"] New Memories: [ "memory": [ [ "id" : "0", "text": "Favorite singer: Jay Chou", "event" : "NONE" ], [ "id" : "1", "text": Favorite songs: "As You Wish" and "All the Stars in My Eyes Are You". "event" : "UPDATE", "old_memory": Favorite song: As You Wish ], ] 3. Delete - Old Memories: - [ - [ - "id" : "θ", - "text": "Favorite singer: Jay Chou" -}, - [ - "id" : "1", - "text": Favorite songs: As You Wish; All the Stars in My Eyes Are You -} - ] - Extracted Memory: ['Singers I don't like to listen to: Jay Chou'] - New Memories: - { - "memory": [ - [ - "id" : "θ", - "text": "Favorite singer: Jay Chou", - "event" : "DELETE" -}, - [ - "id" : "1", - "text": "Favorite songs: As You Wish; All the Stars Are You". - "event" : "None" -} 4. No change - Old Memories: - [ - [ - "id" : "θ", - "text": "Favorite coffee: Latte" -}, - [ - "id" : "1", - "text": Favorite songs: As You Wish; All the Stars in My Eyes Are You -} - ] - Extracted memory: ['Favorite coffee: Latte'] - New Memories: - { - "memory": [ - [ - "id" : "θ", - "text": "Favorite coffee: Latte", - "event" : "NONE" -}, - [ - "id" : "1", - "text": "Favorite songs: As You Wish; All the Stars Are You". - "event" : "NONE" -}”} Please see Figure 3 In some implementations, the step of adding, replacing, or deleting target user memory data in the user memory database based on the user's personalized information employs a large language model to process memory maintenance prompts, including: 0021a: Retrieve historical user memory data associated with user personalized information from the user memory database; 0022a: If no historical user memory data is found, add the user's personalized information to the user memory database; 0023a: If historical user memory data is retrieved and the amount of information in the user's personalized information is greater than that in the historical user memory data, then the historical user memory data is replaced with the user's personalized information. 0024a: If historical user memory data is retrieved and the user's personalized information contradicts the historical user memory data, then the historical user memory data will be deleted.

[0067] In this way, the "maximization of information content" and "consistency" of the user's memory database are ensured. Specifically, the rule of "adding if not found" and "replacing if information content is greater" guarantees that the memory content will evolve in a richer and more specific direction (for example, updating from "likes coffee" to "likes iced Americano"), making refined services possible. "Deleting if contradictory" maintains the logical consistency of the user's memory database, and can promptly remove erroneous or outdated information to prevent large language models from making incorrect inferences based on contradictory memories.

[0068] Related historical user memory data: This refers to existing memory entries found in the user memory database through semantic retrieval that are identical or highly related to the user's personalized information in terms of theme or attributes. Associativity determination is a prerequisite for all subsequent decisions. For example, the current memory "favorite drink: iced Americano" may be associated with the historical memory "favorite drink: latte".

[0069] Information content: This is a comparative concept used to measure the richness, specificity, and accuracy of information carried by a piece of memory data. Judgment criteria typically include: whether the description is more detailed, whether it includes more limiting conditions, and whether it changes from a general term to a specific one. For example, "likes drinking coffee" (general) contains less information than "likes drinking iced Americano" (specific); "likes Jay Chou" contains less information than "likes Jay Chou's Chinese-style songs."

[0070] Please see Figure 4 In some embodiments, retrieving historical user memory data associated with the user's personalized information from the user memory database includes: 0021b: Use vector similarity retrieval technology to retrieve historical user memory data within the user memory database.

[0071] Vector similarity retrieval technology can overcome the limitations of literal keyword matching, finding historical user memory data that is semantically most relevant to the current input. For example, if a user says "I want to listen to something refreshing," vector similarity retrieval technology can retrieve information related to "favorite music genre: electronic music" from their memory, even if there is no literal overlap. On the other hand, for massive amounts of user memory entries, vector similarity retrieval technology can complete the retrieval within milliseconds, meeting the stringent low-latency requirements of real-time interactive scenarios such as in-vehicle systems.

[0072] Vector databases are database systems specifically optimized for storing, indexing, and retrieving high-dimensional vector data. Unlike traditional relational databases that store structured records, each record in a vector database contains a "vector" of floating-point numbers, along with optional raw text or metadata. It incorporates an efficient approximate nearest neighbor search algorithm, enabling it to quickly find the set of vectors most similar to the query vector.

[0073] Vector similarity retrieval technology refers to the technique of quickly finding several vectors that are closest to a given query vector from a massive number of vectors by calculating the distance or similarity between vectors (such as cosine similarity or Euclidean distance). Its core lies in transforming the semantic similarity problem into a distance calculation problem in space.

[0074] Embedding models: These are deep learning models that convert unstructured data such as text and images into numerical vectors (i.e., embeddings). In this implementation, it specifically refers to text embedding models (such as BGE and Sentence-BERT), which can map semantically similar text to similar positions in the vector space.

[0075] Please see Figure 5 In some implementations, a large language model combined with a user memory database is used to process the current round of requests, including: 011: In the case of historical round requests and / or historical round dialogues, the large language model is used to process the current round request in combination with historical round requests and / or historical round dialogues, so as to complete the current round request by splicing historical round requests and current round requests; 012: The large language model is used in conjunction with the user memory database to process the current round of completion requests in order to determine the target application interface from the at least one alternative application interface.

[0076] Thus, by using a large language model to concatenate historical requests with current requests to "complete the current round request," it can automatically correct or complete unclear or incomplete user instructions. By introducing "historical round requests and / or dialogues," it possesses the ability to perceive dialogue context, understand referential expressions (such as "turn it up a bit"), ellipsis (such as saying "No, go home" directly after the user says "navigate to the company"), and other context-dependent expressions, greatly improving the naturalness and fluency of multi-turn dialogues.

[0077] Historical round requests: refers to the most recent (e.g., the last 3-5) voice or text commands issued by the user before the current round of requests.

[0078] Historical round dialogue: refers to a complete record of interactions including historical round requests and corresponding system responses. It provides richer contextual information than a simple sequence of requests, such as previous system confirmations or execution results.

[0079] Complete the current request: This refers to the system using the understanding and generation capabilities of a large language model to rewrite or expand a context-dependent, incomplete "current request" into an independent, complete, and unambiguous semantic unit. This newly generated request itself contains all the contextual information needed to understand the current intent.

[0080] Please see Figure 6 In some implementations, processing the current round completion request using the large language model in conjunction with the user memory database includes: 0121: Use vector similarity retrieval technology to search within the user memory database to obtain at least one alternative user memory data associated with the current round request or to complete the current round request; 0122: Use vector similarity retrieval technology to search within a pre-built API library to obtain at least one alternative API associated with the current round request or to complete the current round request; 0123: Use a large language model to process or complete the current round of requests, at least one alternative user memory data, and at least one alternative API to determine the target API.

[0081] First, vector similarity retrieval technology can quickly filter the most relevant subset (candidate user memory data) from the massive user memory database, and then quickly filter the potentially relevant subset (candidate APIs) from the massive API library. Then, a large language model performs deep reasoning on this narrowed, highly relevant candidate set. In this way, while ensuring decision accuracy, computational efficiency is greatly improved. By vectorizing the "API library" (API description documents), the large language model can dynamically discover and understand new service capabilities.

[0082] Alternative user memory data: refers to a number of personalized memories returned from the user memory database through semantic retrieval that are most relevant to the current (or completed) user request in terms of intent, preference, or context. They are the "personalized context" for decision-making. For example, for a "play music" request, it might retrieve ["favorite singer: Jay Chou", "disliked music genre: jazz"].

[0083] API Library: A database that stores descriptions of all available service capabilities of the system. Its core innovation lies in the fact that the library stores not only API identifiers and parameter templates, but also vectorized representations of their natural language function description text. For example, the description of the "Navigate to {location}" API might be "Plan an optimal route to the specified location and begin navigation".

[0084] Please see Figure 7 In some implementations, a large language model is used to process or complete the current round of requests, at least one alternative user memory data, and at least one alternative API, including: 01231: Fill the instruction processing prompt template with the current round request or complete current round request, at least one alternative user memory data and at least one alternative API to generate instruction processing prompt; 01232: Use a large language model to process instruction prompts to determine the target API.

[0085] In this way, a carefully designed "decision context" is constructed for the large language model. The instruction processing prompt word template can explicitly require the large language model to consider personalized preferences, compare the applicability of different APIs, and output formatted decision results. This greatly improves the reliability, consistency, and operability of the large language model's output results, avoiding the arbitrariness or bias that may arise from its free interpretation. On the other hand, by embedding the core interaction strategies and business logic in the instruction processing prompt word template, when it is necessary to adjust the decision rules or support new scenarios, only the prompt word template needs to be modified, without the need to retrain or fine-tune the large language model, making iteration and maintenance more agile and cost-effective.

[0086] Instruction Processing Prompt Template: A predefined text framework for standardizing the LLM decision-making process. It is not a simple query, but a composite structure containing role settings, task descriptions, input data placeholders, thought process guidance, strict output format specifications, and possible examples. Its core function is to assemble unstructured, multi-source information (requests, memories, APIs) into an "assignment" that the LLM can precisely understand, and constrain its output to a specific "answer format."

[0087] In one example, the instruction processing prompt template is: You are a smart car's in-car voice assistant named "Xiao P". Your job is to attentively listen to all conversations between users inside the car. During these conversations, you need to combine the user's past conversations with the in-car voice assistant, as well as the user's personalized information, including but not limited to the user's daily travel habits, music preferences, frequent destinations, and personalized settings for vehicle functions. Through this information, you need to accurately infer the user's true intentions in the current conversation scenario, thereby providing the user with more considerate, accurate, and efficient services, ensuring a high-quality interactive experience for the user inside the car.

[0088] **Personalized user memory information (maximum 3 items)** ["User Memory 1", "User Memory 2", "User Memory 3"]**Multi-turn user dialogue (up to 3 rounds)** History Wheel: ["History Wheel 1", "History Wheel 2"] Current Round: "User's current round of requests" **Output Format** Xiao P's response: [Instructions deduced based on historical conversations and the user's personalized memories] Here are specific examples: **Personalized user memory information (maximum 3 items)** [Favorite singer: Jay Chou, Favorite music genre: Rock] **Multi-turn user dialogue (up to 3 rounds)** Current round: Recommend my favorite songs Little P responded: ["Play Jay Chou's rock song Huo Yuanjia", "Play Jay Chou's rock song Double Chopsticks"] **Personalized user memory information (maximum 3 items)** [Favorite coffee: Coconut Latte, Favorite coffee brand: Starbucks] **Multi-turn user dialogue (up to 3 rounds)** Current round: I'd like a cup of coffee. Little P responded: ["Give me a coconut latte from the Starbucks nearby"] Decision context: This refers to a temporary reasoning environment constructed for LLM using prompt templates, containing all the necessary information. This context explicitly tells LLM "who you are" (role), "what you are to do" (task), "what information you have available" (input), and "how you should think and output" (rules and format), thus confining open-domain LLM to a closed, well-defined decision problem.

[0089] Formatted decision results: This refers to requiring LLM outputs not free text, but data organized according to a predetermined structure (such as JSON, XML, or specific key-value pairs) that can be seamlessly parsed by machines. For example: {"action": "call_api", "api_name": "climate.set_temperature", "parameters": {"zone": "driver", "value_celsius": 22}} It is understood that a computer program includes computer program code. Computer program code can be in the form of source code, object code, executable files, or some intermediate form. Computer-readable storage media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), and software distribution media, etc.

[0090] In this specification, the terms "specifically," "furthermore," "particularly," "understandably," etc., refer to specific features, structures, materials, or characteristics described in connection with embodiments or examples that are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0091] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of executable request code comprising one or more steps for implementing a particular logical function or process, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order according to the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0092] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A voice interaction method, characterized in that, The method includes: A large language model combined with a user memory database is used to process the current round of requests in order to determine the target application interface from at least one candidate application interface; wherein, the candidate application interfaces include: application interfaces whose interface information has a similarity to the current round of requests higher than a preset similarity; the user memory database is used to store at least user personalized information; Invoke the target application interface.

2. The method according to claim 1, characterized in that, Before processing the current round of requests using a large language model combined with a user memory database to determine the target application interface from at least one candidate application interface, the method further includes: The large language model is used to extract the user's personalized information from the current round of input; Based on the user's personalized information, add, replace, or delete target user memory data in the user memory database; The current round input includes at least one of the following: the current round request, and non-command input.

3. The method according to claim 2, characterized in that, The step of extracting the user's personalized information from the current round of input using the large language model includes: The current round input is filled into the memory retrieval prompt template to generate memory retrieval prompts; The large language model is used to process the memory retrieval prompts in order to extract the user's personalized information in the current round of input.

4. The method according to claim 2, characterized in that, The step of adding, replacing, or deleting target user memory data in the user memory database based on the user's personalized information includes: The user's personalized information is entered into the memory maintenance prompt template to generate memory maintenance prompts; The memory maintenance prompt words are processed using the large language model to add, replace, or delete the target user's memory data in the user memory database.

5. The method according to claim 2, characterized in that, The step of adding, replacing, or deleting target user memory data in the user memory database based on the user's personalized information includes: Retrieve historical user memory data associated with the user's personalized information from the user memory database; If the historical user memory data is not found, the user's personalized information will be added to the user memory database. If the historical user memory data is retrieved and the amount of information in the user personalized information is greater than that in the historical user memory data, the historical user memory data is replaced with the user personalized information. If the historical user memory data is retrieved and the user's personalized information contradicts the historical user memory data, the historical user memory data is deleted.

6. The method according to claim 5, characterized in that, The step of retrieving historical user memory data associated with the user's personalized information from the user memory database includes: The historical user memory data is retrieved from the user memory database using vector similarity retrieval technology.

7. The method according to claim 1, characterized in that, The step of using a large language model combined with a user memory database to process the current round of requests to determine the target application interface from at least one candidate application interface includes: In the presence of historical round requests and / or historical round dialogues, the large language model is used in conjunction with the historical round requests and / or historical round dialogues to process the current round request, so as to concatenate the current round request and the historical round requests and / or historical round dialogues to complete the current round request; The large language model is used in conjunction with the user memory database to process the current round of completion requests in order to determine the target application interface from the at least one candidate application interface.

8. The method according to claim 7, characterized in that, The process of using the large language model in conjunction with the user memory database to process the completion request in the current round includes: Vector similarity retrieval technology is used to search within the user memory database to obtain at least one alternative user memory data associated with the current round of completion request; Vector similarity retrieval technology is used to search within a pre-built application interface library to obtain at least one alternative application interface associated with the current round of completion request; The large language model is used to process the current round completion request, the at least one alternative user memory data, and the at least one alternative application interface to determine the target application interface.

9. The method according to claim 8, characterized in that, The process of using the large language model to process the current round completion request, the at least one alternative user memory data, and the at least one alternative application interface to determine the target application interface includes: The current round request, the at least one alternative user memory data, and the at least one alternative application interface are filled into the instruction processing prompt word template to generate the instruction processing prompt word; The instruction processing prompts are processed using the large language model to determine the target application interface.

10. A voice interaction device, characterized in that, The device includes: The processing module uses a large language model combined with a user memory database to process the current round of requests, in order to determine the target application interface from at least one candidate application interface; wherein, the candidate application interfaces include: application interfaces whose interface information has a similarity to the current round of requests higher than a preset similarity; the user memory database is used to store at least user personalized information; and The calling module is used to call the target application interface.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by a processor, it implements the steps of the method as described in any one of claims 1-9.

Images