Vehicle interaction method based on multi-agent cooperation and vehicle

Abstract

This application discloses an in-vehicle interaction method and vehicle based on multi-agent collaboration. The method includes: standardizing the user's multimodal interactive input to generate a corresponding user task request; parsing the user task request based on a preset task allocation strategy to determine at least one target agent driven by a large language model that matches the user task request; driving the target agent to generate interactive response content corresponding to the user task request based on the dialogue context data corresponding to the user task request and the role feature data corresponding to the target agent; determining the target interaction result based on the interactive response content and outputting the target interaction result. The technical solution provided by this application can improve the processing efficiency of agent interaction tasks.

Description

Technical Field

[0001] This application belongs to the field of vehicle and intelligent cockpit technology, and particularly relates to an in-vehicle interaction method and vehicle based on multi-agent collaboration. Background Technology

[0002] With the rapid evolution of automotive intelligence and in-vehicle interaction technologies, vehicle systems have gradually shifted from single-function control to complex application scenarios such as natural language interaction, multi-task parallel processing, and personalized services. The widespread application of large language models enables intelligent agents driven by large language models to complete multiple tasks such as dialogue interaction, knowledge question answering, entertainment interaction, and decision support, providing users with a more convenient in-vehicle service experience.

[0003] However, in actual in-vehicle interaction, user needs are diverse, scenario-based, and complex. A single intelligent agent often struggles to simultaneously meet the requirements of task processing efficiency, natural interaction, personality consistency, and real-time response. When faced with multi-turn dialogues, multi-role interactions, complex reasoning, or entertainment game-like tasks, existing interaction methods generally suffer from problems such as inaccurate task parsing, unclear division of labor among intelligent agents, insufficient contextual coherence, poor consistency in role behavior, and difficulty in effectively integrating the outputs of multiple intelligent agents. This results in slow interaction response, logical breaks, and a stiff experience, failing to meet the needs of efficient, coherent, and stable collaborative interaction in in-vehicle scenarios.

[0004] Therefore, how to improve the processing efficiency of intelligent agent interaction tasks has become an urgent technical problem to be solved. Summary of the Invention

[0005] The embodiments of this application provide a vehicle interaction method, device, computer program product, computer-readable storage medium, and vehicle based on multi-agent collaboration, which can at least improve the processing efficiency of agent interaction tasks to a certain extent.

[0006] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0007] According to a first aspect of the embodiments of this application, a vehicle-mounted interaction method based on multi-agent collaboration is provided. The method includes: acquiring multimodal interaction input from a user and standardizing the multimodal interaction input to generate a corresponding user task request; parsing the user task request based on a preset task allocation strategy to determine at least one target agent matching the user task request, wherein each target agent is driven by a large language model; retrieving dialogue context data corresponding to the user task request and role feature data corresponding to the target agent, and driving the target agent to generate interactive response content corresponding to the user task request based on the dialogue context data and the role feature data; determining a target interaction result based on the interactive response content and outputting the target interaction result, wherein when the number of target agents is two or more, the interactive response content generated by all target agents is fused to obtain the target interaction result; when the number of target agents is one, the interactive response content generated by the target agent is determined as the target interaction result.

[0008] In some embodiments of this application, based on the foregoing scheme, the step of obtaining the user's multimodal interactive input and standardizing the multimodal interactive input to generate a corresponding user task request includes: obtaining interactive data submitted by the user through at least one of voice input, touch input, or gesture input; performing semantic recognition and intent extraction on the interactive data to generate a standardized user task request.

[0009] In some embodiments of this application, based on the foregoing scheme, the step of parsing the user task request based on a preset task allocation strategy to determine at least one target agent matching the user task request includes: classifying the user task request by domain and classifying the task difficulty to obtain task attribute labels; and using a task allocation model constructed based on the Q-learning algorithm to match at least one corresponding target agent according to the task attribute labels, wherein the allocation strategy of the task allocation model is optimized based on the feedback results of historical task execution.

[0010] In some embodiments of this application, based on the aforementioned scheme, the role feature data corresponding to the target intelligent agent is generated through the following steps: constructing corresponding personality feature vectors for different intelligent agents based on different personality models; fine-tuning the large language model corresponding to the intelligent agent through a proximal policy optimization algorithm to make the output behavior of the intelligent agent match the corresponding personality feature vector; and storing the matched personality feature vector and model fine-tuning parameters as the role feature data of the intelligent agent in a preset storage module.

[0011] In some embodiments of this application, based on the aforementioned scheme, the fine-tuning of the large language model corresponding to the agent using a proximal policy optimization algorithm includes: constructing a reward function, which is obtained by weighting the role behavior matching score and the historical speech consistency score; when the matching degree between the output content generated by the agent and the personality feature vector is higher than a preset matching threshold, and the consistency with the historical output content is higher than a preset consistency threshold, a positive reward is given, otherwise a negative reward is given; based on the output result of the reward function, the generation parameters of the large language model are iteratively optimized until the personality deviation rate of the agent is lower than a preset deviation threshold.

[0012] In some embodiments of this application, based on the foregoing scheme, retrieving the dialogue context data corresponding to the user task request includes: constructing and continuously updating a dialogue graph corresponding to the current user based on dialogue state tracking technology; extracting historical dialogue content related to the user task request from the dialogue graph through a hierarchical attention mechanism to generate dialogue context data, wherein the dialogue context data is used to maintain semantic coherence in multi-turn interaction processes.

[0013] In some embodiments of this application, based on the foregoing scheme, the method further includes: performing semantic embedding processing on the dialogue content of adjacent rounds to obtain corresponding semantic vectors; calculating the cosine similarity of the semantic vectors of adjacent rounds to obtain a context consistency score; and when the context consistency score is lower than a preset score threshold, retrieving the corresponding dialogue context data to optimize the interactive response content of the current round.

[0014] In some embodiments of this application, based on the foregoing scheme, before driving the target intelligent agent to generate interactive response content corresponding to the user task request, the method further includes: compressing the large language model through knowledge distillation technology to obtain a lightweight inference model; and deploying the lightweight inference model to an in-vehicle edge computing device.

[0015] In some embodiments of this application, based on the aforementioned scheme, the compression of the large language model using knowledge distillation technology includes: using a pre-trained large language model as a teacher model and a network model with fewer parameters than the teacher model as a student model; constructing a distillation loss function, which is obtained by weighting the relative entropy of the outputs of the teacher model and the student model and the cross-entropy loss of the student model; iteratively training the student model based on the distillation loss function until the inference accuracy of the student model reaches a preset accuracy threshold, thereby obtaining a lightweight inference model.

[0016] According to a second aspect of the embodiments of this application, an in-vehicle interaction device based on multi-agent collaboration is provided. The device includes: an acquisition unit, configured to acquire multimodal interaction input from a user and perform standardized processing on the multimodal interaction input to generate a corresponding user task request; a parsing unit, configured to parse the user task request based on a preset task allocation strategy and determine at least one target agent matching the user task request, wherein each target agent is driven by a large language model; a generation unit, configured to retrieve dialogue context data corresponding to the user task request and role feature data corresponding to the target agent, and drive the target agent to generate interactive response content corresponding to the user task request based on the dialogue context data and the role feature data; and an output unit, configured to determine a target interaction result based on the interactive response content and output the target interaction result, wherein when the number of target agents is two or more, the interactive response content generated by all target agents is fused to obtain the target interaction result; when the number of target agents is one, the interactive response content generated by the target agent is determined as the target interaction result.

[0017] According to a third aspect of the embodiments of this application, a computer program product is provided, the computer program product including computer instructions stored in a computer-readable storage medium and adapted to be read and executed by a processor to cause a computer device having the processor to perform an operation as described in any of the first aspects above.

[0018] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, the computer-readable storage medium storing at least one computer program instruction, the at least one computer program instruction being loaded and executed by a processor to perform the operation as described in any of the first aspects above.

[0019] According to a fifth aspect of the embodiments of this application, a vehicle is provided, the vehicle including one or more processors and one or more memories, the one or more memories storing at least one computer program instruction, the at least one computer program instruction being loaded and executed by the one or more processors to perform the operation as described in any of the first aspects above.

[0020] Based on the technical solution proposed in this application, by acquiring and standardizing the user's multimodal interactive input to generate user task requests, it can adapt to various commonly used interaction methods in in-vehicle scenarios, avoiding the inability of users to complete interactions due to operational limitations while driving. Simultaneously, it converts different forms of input into unified task requests, ensuring the system's accuracy in recognizing user needs and laying the foundation for subsequent task processing. By parsing user task requests based on a preset task allocation strategy and matching at least one corresponding target agent, it can flexibly match agents with corresponding capabilities according to the type and complexity of user needs, achieving precise task allocation and avoiding the problem of insufficient capabilities of a single agent in handling complex tasks, thus improving the adaptability and efficiency of task processing. By retrieving dialogue context data and the target agent's role feature data, it drives the target agent to generate interactive response content, ensuring that the agent's output always conforms to its own role positioning and aligns with the user's historical interaction habits, avoiding problems such as confused role behavior and broken context logic, and improving the naturalness and coherence of the interaction. By selecting either directly determining the target interaction result or fusing multiple response contents based on the number of target agents, the system can flexibly adapt to different interaction scenarios for single and multiple agents. This ensures that the content output by multiple agents is logically coherent and information is complete, avoiding the problems of messy and contradictory output content from multiple agents. It comprehensively improves the processing efficiency of agent interaction tasks and adapts to various interaction needs in in-vehicle scenarios, enhancing the user's in-vehicle interaction experience. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings: Figure 1 A flowchart of a vehicle interaction method based on multi-agent collaboration in an embodiment of this application is shown; Figure 2 The system architecture diagram for implementing the multi-agent collaborative in-vehicle interaction method in an embodiment of this application is shown. Figure 3 A block diagram of an in-vehicle interaction device based on multi-agent collaboration in an embodiment of this application is shown; Figure 4 A schematic diagram of the vehicle structure in an embodiment of this application is shown. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0024] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices. It should also be noted that, for the sake of simplicity, certain components in the drawings that do not affect the interpretation of the technical solution of this application have been appropriately omitted.

[0025] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined. Therefore, the actual execution order may change depending on the actual situation.

[0026] In the description of this application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "multiple" means two or more.

[0027] This application proposes a multi-agent collaborative in-vehicle interaction scheme. Its purpose is to solve the problems in existing in-vehicle interaction scenarios, such as the difficulty for a single agent to simultaneously meet the requirements of task processing efficiency, interaction naturalness, personality consistency and real-time response, as well as the problems of inaccurate task parsing, unclear division of labor, insufficient contextual coherence and difficulty in effectively integrating outputs during multi-agent interaction. The scheme aims to achieve efficient, coherent and stable collaborative interaction of multiple agents in in-vehicle scenarios, and comprehensively improve the processing efficiency of agent interaction tasks.

[0028] Next, this application will elaborate on the proposed in-vehicle interaction scheme based on multi-agent cooperation. (Refer to...) Figure 1 The diagram illustrates a flowchart of a multi-agent collaborative in-vehicle interaction method according to an embodiment of this application. This method can be executed by a device with computing processing capabilities, such as... Figure 1 As shown, the method includes at least steps 110 to 140, which are described in detail below: In step 110, the user's multimodal interaction input is obtained and the multimodal interaction input is standardized to generate a corresponding user task request.

[0029] In this application, the multimodal interactive input refers to input information submitted by the user through different interaction methods, covering all commonly used interaction forms in in-vehicle scenarios, thus avoiding user distraction caused by the need to consistently use a single interaction method while driving. The standardization processing refers to converting different forms and formats of interactive input into a standardized data format that the system can uniformly recognize and process, ensuring that subsequent task parsing and processing flows are not deviated due to different input formats. The user task request refers to the standardized task instruction that clarifies the user's core needs, serving as the core basis for all subsequent intelligent agent scheduling and processing flows.

[0030] In step 110 above, the step of obtaining the user's multimodal interaction input and standardizing the multimodal interaction input to generate the corresponding user task request can be performed according to the following steps 111 to 112: Step 111: Obtain interactive data submitted by the user through at least one of voice input, touch input, or gesture input.

[0031] Step 112: Perform semantic recognition and intent extraction on the interaction data to generate standardized user task requests.

[0032] In this application, voice input refers to interactive data submitted by the user through voice dialogue. Users can submit commands simply by speaking, without needing to operate the device, thus avoiding distraction while driving. Touch input refers to interactive data submitted by the user by touching the vehicle's screen, including clicks, swipes, and long presses, catering to the precise operation needs of users when resting at a parked location. Gesture input refers to interactive data submitted by the user through preset hand gestures. Users can submit commands simply by making corresponding gestures within the vehicle's recognition range, without touching the screen, adapting to scenarios where speaking or touching the screen is inconvenient while driving.

[0033] In this application, semantic recognition refers to the process of parsing user interaction data to identify the specific content expressed by the user, ensuring that the system can accurately understand the literal meaning of the user's input. Intent extraction, based on semantic recognition, further analyzes the core needs and usage intentions behind the user's input, distinguishing the user's true needs and avoiding biases that may occur when processing tasks solely based on literal content.

[0034] Based on the technical solutions in steps 111 to 112 above, by acquiring interaction data submitted by users through at least one of voice input, touch input, or gesture input, the interaction needs of users in different driving states and with different usage habits in in-vehicle scenarios can be comprehensively covered. Users can choose the appropriate interaction method according to their own driving state, without having to use a fixed operation method during driving, effectively reducing the distraction of driving attention during the interaction process and improving the safety and convenience of in-vehicle interaction. By performing semantic recognition and intent extraction on the interaction data, standardized user task requests can be generated. Based on an accurate understanding of the user's input content, the core usage intent of the user can be accurately captured, avoiding the deviation in demand recognition caused by different input forms or colloquial expressions, ensuring the accuracy of subsequent task allocation and processing, and further improving the processing efficiency of intelligent agent interaction tasks.

[0035] Continue to refer to Figure 1 In step 120, the user task request is parsed based on a preset task allocation strategy to determine at least one target agent that matches the user task request, wherein each target agent is driven by a large language model.

[0036] In this application, the task allocation strategy can refer to a rule system pre-set within the system, used to match intelligent agents with corresponding processing capabilities based on the attributes of user task requests, ensuring that tasks can be assigned to the most suitable intelligent agents and avoiding the problem of task-agent capability mismatch. The target intelligent agent can refer to an intelligent interactive unit driven by a Large Language Model (LLM), possessing specific role positioning, processing capabilities, and behavioral characteristics, capable of independently completing corresponding types of tasks. Different target intelligent agents can adapt to different user demand scenarios.

[0037] In step 120 above, the step of parsing the user task request based on a preset task allocation strategy and determining at least one target agent matching the user task request can be performed according to the following steps 121 to 122: Step 121: Classify the user task requests by domain and classify the task difficulty to obtain task attribute tags.

[0038] Step 122: The task allocation model constructed based on the Q-learning algorithm matches at least one target agent according to the task attribute label, wherein the allocation strategy of the task allocation model is optimized based on the feedback results of historical task execution.

[0039] In this application, the domain classification can refer to a categorization based on the industry, scenario, and functional type of the content corresponding to the user task request, used to distinguish the domain to which the task belongs and match the corresponding intelligent agent. The task difficulty grading can refer to a level grading based on the complexity of the user task request, the required inference depth, and the number of collaborative intelligent agents involved, used to distinguish the processing difficulty of the task and match intelligent agents with corresponding processing capabilities. The task attribute tag can refer to a tag generated for the user task request after domain classification and task difficulty grading, used to identify the core attributes of the task, and is the core basis for matching the target intelligent agent.

[0040] In this application, the Q-Learning algorithm is a value-based reinforcement learning algorithm used to construct a task allocation model. Through continuous trial and error and feedback, it optimizes the task allocation strategy, allowing the model to learn the optimal task allocation method. The feedback results of historical task execution can refer to feedback data generated after previous task allocations were completed, based on dimensions such as task completion rate, user satisfaction, and processing time, and are used to optimize the allocation strategy of the task allocation model.

[0041] Based on the technical solutions in steps 121 and 122 above, by classifying user task requests by domain and grading task difficulty, task attribute tags are obtained. This comprehensively and accurately identifies the core attributes of user task requests, clarifies the domain and processing difficulty of the task, and provides a clear basis for subsequent agent matching, avoiding problems such as domain mismatch and capability mismatch between tasks and agents. Through a task allocation model built based on the Q-learning algorithm, corresponding target agents are matched according to task attribute tags. The self-optimization capability of reinforcement learning allows the task allocation strategy to continuously adapt to changes in user needs, improving the accuracy and adaptability of task allocation. By optimizing the allocation strategy of the task allocation model based on feedback results from historical task execution, the model can continuously learn the optimal allocation method during continuous use, avoiding the lack of flexibility caused by fixed allocation strategies. This continuously improves the efficiency and accuracy of task allocation, further enhancing the overall efficiency of agent interaction task processing.

[0042] Continue to refer to Figure 1 In step 130, the dialogue context data corresponding to the user task request and the role feature data corresponding to the target agent are retrieved, and based on the dialogue context data and the role feature data, the target agent is driven to generate interactive response content corresponding to the user task request.

[0043] Dialogue context data refers to all historical interactions between the current user and the system prior to this interaction, as well as semantic association information related to the current task. This ensures that the system's responses do not suffer from contextual breaks or logical contradictions during multi-turn interactions. Role characteristic data refers to core data used to define the target agent's personality traits, behavioral logic, output style, and professional domain. This ensures that the target agent's output always aligns with its role positioning and avoids role behavior confusion.

[0044] In step 130 above, the role feature data corresponding to the target intelligent agent can be generated through steps 131 to 133 as follows: Step 131: Construct corresponding personality feature vectors for different agents based on different personality models.

[0045] Step 132: Fine-tune the large language model corresponding to the agent using the proximal policy optimization algorithm to match the agent's output behavior with the corresponding personality feature vector.

[0046] Step 133: The matched personality feature vector and model fine-tuning parameters are used as the role feature data of the intelligent agent and stored in a preset storage module.

[0047] In this application, the personality model can refer to a psychological model used to quantitatively describe the personality traits, behavioral tendencies, and output style of an intelligent agent, thereby constructing a unified and stable personality positioning for the intelligent agent and avoiding chaotic output behavior. The personality feature vector can refer to a multi-dimensional vector generated based on the personality model, used to quantitatively represent the personality traits of the intelligent agent. Each dimension of the vector corresponds to a personality trait in the personality model and is the core quantitative basis for the intelligent agent's role positioning.

[0048] In this application, the proximal policy optimization algorithm is a reinforcement learning algorithm used to fine-tune a large language model. Through a reward and punishment mechanism, it ensures that the output behavior of the large language model continuously conforms to a preset personality feature vector, thereby guaranteeing the consistency of the agent's behavior. The model fine-tuning parameters refer to the core parameters in the large language model used to control output behavior and conform to personality features after fine-tuning by the proximal policy optimization algorithm. These parameters, together with the personality feature vector, constitute the agent's role feature data, ensuring that the agent maintains consistent role behavior each time it starts.

[0049] For example, in a specific implementation, the system needs to construct a werewolf agent in a werewolf game. First, based on the five personality models, a corresponding personality trait vector can be constructed for this agent. A lower value is set for the agreeableness dimension, giving the agent a stronger tendency to disguise and antagonize; a higher value is set for the conscientiousness dimension, ensuring the agent always prioritizes the werewolf team's victory; and a moderate value is set for the extraversion dimension, ensuring the agent's speaking logic is sufficiently persuasive. Next, the system uses a proximal strategy optimization algorithm to fine-tune the large language model corresponding to the werewolf agent, ensuring the agent's output behavior matches the constructed personality trait vector. For instance, the agent might actively disguise its identity while cooperating with its werewolf teammates' speaking logic, avoiding any behavior that actively reveals its identity. After fine-tuning, the system stores the matched personality trait vector and model fine-tuning parameters as the werewolf agent's role feature data in a preset vector database. Subsequent calls to the werewolf agent can directly retrieve the corresponding role feature data, ensuring consistent role behavior. For example, in building a vehicle maintenance professional consultant intelligent agent, the system is based on the five personality models, setting high values ​​for the responsibility dimension and the emotional stability dimension, constructing corresponding personality feature vectors, and fine-tuning the model through a near-end strategy optimization algorithm to ensure that the intelligent agent's output always maintains a rigorous, professional, and concise style, adapting to the needs of the in-vehicle scenario, and then storing the corresponding role feature data in a preset storage module.

[0050] Based on the technical solutions in steps 131 to 133 above, by constructing corresponding personality feature vectors for different agents based on different personality models, a clear, stable, and quantifiable personality positioning can be set for each agent. This allows different agents to possess differentiated role characteristics, adapting to different task scenarios and user needs, and avoiding the problem of homogenized output styles among different agents. By fine-tuning the large language model corresponding to the agent through a proximal policy optimization algorithm, the agent's output behavior is matched with the corresponding personality feature vector. Through reinforcement learning, the output behavior of the large language model is ensured to always conform to the preset personality positioning, preventing the agent from deviating from its role behavior or experiencing personality confusion in multiple rounds of interaction, thus guaranteeing the consistency of the agent's role behavior. By storing the matched personality feature vectors and model fine-tuning parameters as the agent's role feature data in a preset storage module, the system can quickly retrieve complete role feature data each time it calls the corresponding agent. This ensures that the agent maintains consistent role behavior in different interaction scenarios and rounds, while improving the agent's calling efficiency and further enhancing the overall efficiency of interactive task processing.

[0051] In step 132 above, the fine-tuning of the large language model corresponding to the agent using the proximal policy optimization algorithm can be performed according to steps 1321 to 1323 as follows: Step 1321: Construct a reward function, which is obtained by weighting the role behavior matching score and the historical speech and behavior consistency score.

[0052] Step 1322: When the matching degree between the output content generated by the agent and the personality feature vector is higher than the preset matching threshold, and the consistency with the historical output content is higher than the preset consistency threshold, a positive reward is given; otherwise, a negative reward is given.

[0053] Step 1323: Based on the output of the reward function, iteratively optimize the generation parameters of the large language model until the personality deviation rate of the agent is lower than the preset deviation threshold.

[0054] In this application, the reward function refers to the function used in reinforcement learning to evaluate whether the agent's output behavior conforms to expectations. It is the core basis for driving the iterative optimization of the model, and the output of the function determines whether the agent receives a positive or negative reward. The role behavior matching score refers to the score used to evaluate the degree to which the agent's output content fits the preset personality feature vector. The higher the score, the more the agent's output behavior conforms to the preset role positioning. The historical speech and behavior consistency score refers to the score used to evaluate the logical consistency and behavioral coherence between the agent's current output content and its historical output content. The higher the score, the more consistent the agent's speech and behavior are, and the less likely there will be contradictions.

[0055] In this application, the preset matching threshold refers to a pre-set critical value used to determine whether the matching degree between the agent's output content and the personality feature vector meets the standard. Only when the matching degree is higher than this threshold will a positive reward be given. The preset consistency threshold refers to a pre-set critical value used to determine whether the consistency between the agent's current output and its historical output meets the standard. Only when the consistency is higher than this threshold can a positive reward be given.

[0056] In this application, the personality deviation rate refers to the proportion by which the output behavior of the agent deviates from the preset personality feature vector. The lower the value, the better the role consistency of the agent. The preset deviation threshold is a pre-set critical value used to determine whether the personality deviation rate of the agent meets the standard. Only when the personality deviation rate is lower than this threshold is the model fine-tuning considered complete.

[0057] For example, in a specific implementation, during the fine-tuning of the large language model of the werewolf agent, the system first constructs a reward function. This reward function is obtained by weighting the role behavior matching score and the historical speech consistency score, where the role behavior matching score has a weight of 0.4 and the historical speech consistency score has a weight of 0.6, prioritizing the consistency of the agent's speech. During the fine-tuning process, if the werewolf agent generates a speech disguised as a villager, the system first calculates the matching degree of this speech with the preset personality feature vector. Finding that the matching degree is higher than the preset matching threshold of 0.8, and simultaneously calculates the consistency of this speech with the agent's previous speech, finding that the consistency is higher than the preset consistency threshold of 0.85, the system gives the agent a positive reward, strengthening the probability of the model generating such content that aligns with the role's positioning. If the werewolf agent generates a speech that directly reveals its werewolf identity, the system calculates that the matching degree is lower than the preset matching threshold, and the consistency with historical speech is also lower than the preset consistency threshold, so the system gives a negative reward, reducing the probability of the model generating such content that does not align with the role's positioning. Based on the output of the reward function, the system continuously iterates and optimizes the generation parameters of the large language model until the agent's personality deviation rate is lower than the preset deviation threshold of 5%, thus completing the fine-tuning of the model and ensuring that the agent's output behavior always conforms to the positioning of the werewolf role.

[0058] Based on the technical solutions in steps 1321 to 1323 above, by constructing a reward function weighted by the role behavior matching score and the historical speech consistency score, the output behavior of the agent can be comprehensively evaluated from two dimensions: role positioning fit and historical speech consistency. This ensures that during model fine-tuning, the agent can fit the preset role positioning and maintain consistency in speech and behavior across multiple rounds of interaction, avoiding optimization direction deviation caused by a single evaluation dimension. By providing corresponding positive or negative rewards based on the matching degree and consistency of the agent's output content, the model can quickly learn the expected output behavior through the reward and punishment mechanism of reinforcement learning, accelerating the convergence speed of model fine-tuning and improving its efficiency. By iteratively optimizing the generation parameters of the large language model based on the output results of the reward function until the agent's personality deviation rate is lower than the preset deviation threshold, it can be ensured that the agent, after fine-tuning, maintains stable role behavior throughout the actual interaction process, preventing role deviation and personality collapse, improving the naturalness of interaction and user experience, and avoiding task processing failures due to agent behavior deviation, further improving the processing efficiency of interactive tasks.

[0059] In step 130 above, retrieving the dialogue context data corresponding to the user task request can be performed according to steps 134 to 135 as follows: Step 134: Based on dialogue state tracking technology, construct and continuously update the dialogue graph corresponding to the current user.

[0060] Step 135: Extract historical dialogue content related to the user's task request from the dialogue graph through a hierarchical attention mechanism to generate dialogue context data. The dialogue context data is used to maintain semantic coherence in the multi-turn interaction process.

[0061] In this application, the dialogue state tracking technology refers to a technology used to track, record, and update the dialogue state, core needs, and key information during multi-round interactions between the user and the system in real time. It is the core foundation for constructing a dialogue graph, ensuring that the system can continuously grasp changes in user needs. The dialogue graph refers to a structured graph constructed based on the user's historical interaction content, used to record the core semantics, entity relationships, and changes in needs during the dialogue process. It completely reconstructs the entire interaction process between the user and the system and is the core carrier for extracting dialogue context data.

[0062] In this application, the hierarchical attention mechanism refers to a mechanism for accurately extracting core information relevant to the current task from a large amount of historical dialogue content. Just as a person recalls a conversation, they focus on content related to the current topic and ignore irrelevant details, ensuring that the extracted contextual data accurately matches the current task requirements. Semantic coherence refers to the characteristic that, during multiple rounds of interaction, the system's response content maintains logical consistency and topic coherence with the user's historical dialogue content and changes in needs, avoiding issues such as contextual breaks, topic deviations, and contradictions.

[0063] For example, in a specific implementation, during multiple interactions with the system, the user first inquires about the vehicle's oil change interval, to which the system provides a corresponding response. The user then inquires about oil type selection, and subsequently requests a complete annual maintenance plan tailored to their vehicle. Throughout this process, the system, using dialogue state tracking technology, continuously records the core content of each round of dialogue and changes in the user's needs, constructing and continuously updating a dialogue graph corresponding to the current user. This dialogue graph records all relevant historical dialogue content, including the user's vehicle model information, previously inquired oil change intervals, and oil type selection. When the user requests the annual maintenance plan, the system, through a hierarchical attention mechanism, extracts key historical dialogue content related to the vehicle's annual maintenance from the constructed dialogue graph, including the user's vehicle model and previously inquired oil-related content, generating corresponding dialogue context data and passing it to the vehicle maintenance professional advisor agent. Based on this dialogue context data, the agent generates an annual maintenance plan that fits the user's vehicle model and meets their previously inquired maintenance needs, maintaining semantic coherence throughout the multi-round interaction process and preventing responses irrelevant to the user's previous requests.

[0064] For example, in a specific implementation, during the multi-round interaction of the Werewolf game, the system constructs a dialogue graph based on dialogue state tracking technology, records the speech content, voting records, and identity reasoning information of each role in each round. In each new speaking phase, the system extracts historical speech content related to the current game phase from the dialogue graph through a hierarchical attention mechanism, generates dialogue context data, and ensures that the speech content of each agent is consistent with the previous game process, so that there will be no contradictions in the speech.

[0065] Based on the technical solutions in steps 134 and 135 above, by constructing and continuously updating a dialogue graph corresponding to the current user using dialogue state tracking technology, the entire process of multi-turn interactions between the user and the system can be recorded completely and structurally. This allows for real-time tracking of changes in user needs and core dialogue information, preventing the loss of key information due to increased dialogue rounds and providing a complete carrier for contextual data extraction. Through a hierarchical attention mechanism, historical dialogue content related to the user's task request is extracted from the dialogue graph to generate dialogue context data. This allows for precise filtering of core information relevant to the current task from a large amount of historical dialogue content, ignoring irrelevant and redundant content. This ensures that the generated context data accurately matches the current task requirements and avoids irrelevant information interfering with the agent's output. Maintaining semantic coherence during multi-turn interactions through dialogue context data ensures that the agent's output always aligns with the user's historical interaction content and changes in needs, avoiding contextual breaks, logical contradictions, and topic deviations in multi-turn dialogues. This improves the naturalness and fluency of the interaction, while also preventing task processing errors caused by contextual misunderstandings, further enhancing the efficiency of interactive task processing.

[0066] In this application, the in-vehicle interaction method based on multi-agent cooperation may further perform the following steps 151 to 153: Step 151: Semantic embedding is performed on the dialogue content of adjacent rounds to obtain the corresponding semantic vectors.

[0067] Step 152: Calculate the cosine similarity of semantic vectors from adjacent rounds to obtain the context consistency score.

[0068] Step 153: When the context consistency score is lower than the preset score threshold, the corresponding dialogue context data is retrieved again to optimize the interactive response content of the current round.

[0069] In this application, semantic embedding processing refers to the process of converting the textual information of dialogue content into multidimensional semantic vectors that can be quantified and calculated. Through semantic embedding, different textual contents can be converted into vectors of the same dimension, thereby calculating the semantic similarity between different dialogue contents. The semantic vector refers to the multidimensional vector used to represent the core semantic information of the dialogue content after semantic embedding processing. The cosine similarity between vectors can reflect the degree of semantic association between corresponding dialogue contents.

[0070] In this application, the cosine similarity refers to a calculation method used to determine the similarity between two vectors by calculating the cosine value of the angle between them. The closer the cosine similarity value is to 1, the higher the semantic similarity of the dialogue content corresponding to the two vectors, and the better the contextual coherence. The contextual consistency score is a score calculated based on the cosine similarity of the semantic vectors of adjacent dialogue rounds, used to evaluate the semantic coherence of multi-turn dialogues. The higher the score, the better the contextual coherence of the multi-turn dialogues.

[0071] In this application, the preset scoring threshold can be a pre-set critical value used to determine whether the contextual consistency of a multi-turn dialogue meets the standard. When the score is lower than this threshold, it indicates that the current contextual coherence is insufficient and needs to be re-optimized.

[0072] For example, in a specific implementation, in a multi-turn dialogue, the user first asks about precautions for changing vehicle tires, and the system provides a corresponding response. In the second round, the user asks about tire brand recommendations, and the system generates a corresponding response. In this process, the system first performs semantic embedding processing on the dialogue content of the first and second rounds, obtaining corresponding 768-dimensional semantic vectors. Then, it calculates the cosine similarity between the two semantic vectors, obtaining a context consistency score of 0.88. This score is higher than the preset score threshold of 0.8, indicating that the current contextual coherence meets the standard, and the system directly outputs the generated response content. If the second-round response generated by the system is about vehicle oil maintenance, after semantic embedding processing of the two rounds of dialogue content, the calculated cosine similarity is 0.3, and the corresponding context consistency score is lower than the preset score threshold of 0.8. This indicates that the current response content lacks contextual coherence with the historical dialogue. The system will then retrieve the corresponding dialogue context data, i.e., the historical dialogue content related to tires previously asked by the user, and re-drive the agent to generate interactive response content that fits the tire brand recommendation needs, ensuring the contextual coherence of the multi-turn dialogue.

[0073] For example, in a specific implementation, during multiple rounds of speaking in the Werewolf game, the system performs semantic embedding processing on the adjacent speaking content in each round and calculates the context consistency score. If the consistency score between a certain agent's speaking content and the previous speaking content is lower than a preset threshold, the system will retrieve the game's dialogue context data again to optimize the agent's speaking content and avoid the problem of contradictory statements.

[0074] Based on the technical solutions in steps 151 to 153 above, semantic vectors are obtained by semantically embedding the dialogue content of adjacent rounds. This converts text-based dialogue content into quantifiable semantic vectors, providing a foundation for subsequent semantic similarity calculations and enabling quantitative evaluation of contextual coherence, thus avoiding subjective biases caused by manual evaluation. A contextual consistency score is obtained by calculating the cosine similarity of semantic vectors from adjacent rounds. This allows for precise numerical quantification of the semantic coherence of multi-round dialogues, clearly determining whether the current interaction content meets the requirements of contextual coherence, and providing a clear basis for subsequent content optimization. When the contextual consistency score is lower than a preset score threshold, the corresponding dialogue context data is retrieved again to optimize the interaction response content of the current round. This allows for timely correction of the agent's output when insufficient contextual coherence is detected, avoiding contextual breaks and logical contradictions, ensuring semantic coherence of multi-round interactions, improving the naturalness of interaction and user experience, and preventing task processing errors caused by contextual misunderstandings, further improving the processing efficiency of interaction tasks.

[0075] In step 130 above, before driving the target agent to generate the interactive response content corresponding to the user task request, the following steps 136 to 137 may also be performed: Step 136: The large language model is compressed using knowledge distillation technology to obtain a lightweight reasoning model.

[0076] Step 137: Deploy the lightweight inference model to the vehicle-mounted edge computing device.

[0077] In this application, the knowledge distillation technology is a model compression technique that transfers knowledge from a large language model with a large number of parameters and high inference accuracy to a smaller model with fewer parameters and faster inference speed. This allows the smaller model to retain its core inference capabilities while achieving faster inference speed and lower hardware requirements, thus adapting to the deployment needs of in-vehicle edge devices. The lightweight inference model refers to a large language model that, after knowledge distillation compression, has a significantly reduced number of parameters while retaining its core inference capabilities. This model can run quickly and stably on in-vehicle edge computing devices and serves as the core carrier driving intelligent agents to generate interactive response content.

[0078] In this application, the in-vehicle edge computing device refers to a hardware device installed inside the vehicle that has local computing capabilities. It can complete the inference calculation of the model without relying on a remote cloud server, avoiding the slow response problem caused by network latency and improving the real-time performance of in-vehicle interaction.

[0079] For example, in one specific embodiment, before deployment, the system can first acquire a pre-trained large language model with 7 billion parameters. This model has high inference accuracy, but its large number of parameters makes it unsuitable for real-time operation on in-vehicle edge computing devices. The system uses knowledge distillation technology to compress this large language model, obtaining a lightweight inference model. This lightweight inference model retains core natural language understanding and generation capabilities while significantly reducing the number of parameters, making it compatible with the hardware performance of in-vehicle edge computing devices. The system then deploys this lightweight inference model to the vehicle's in-vehicle edge computing device. During actual interaction, model inference is completed directly through the vehicle's local edge computing device, driving the target agent to generate interactive response content without relying on a remote cloud server, thus avoiding latency issues caused by network fluctuations.

[0080] Based on the technical solutions in steps 136 and 137 above, the large language model is compressed using knowledge distillation technology to obtain a lightweight inference model. This significantly reduces the number of model parameters and hardware performance requirements while retaining the core inference capabilities of the large language model, making the model adaptable to the hardware environment of in-vehicle scenarios and providing faster inference speeds. By deploying the lightweight inference model to in-vehicle edge computing devices, local inference can be achieved without relying on remote cloud servers. This avoids the problem of slow interactive response caused by network fluctuations and latency, greatly improving the real-time performance of in-vehicle interaction. User input can be responded to quickly, adapting to the high real-time requirements of in-vehicle scenarios. At the same time, it avoids the problem of interactive functions being unusable due to network interruptions, improving system stability and reliability, and further enhancing the processing efficiency of interactive tasks.

[0081] In step 136 above, the compression of the large language model using knowledge distillation technology can be performed according to steps 1361 to 1363 as follows: Step 1361: Use the pre-trained large language model as the teacher model and the network model with fewer parameters as the student model.

[0082] Step 1362: Construct a distillation loss function, which is obtained by weighting the relative entropy of the outputs of the teacher model and the student model and the cross-entropy loss of the student model.

[0083] Step 1363: Iteratively train the student model based on the distillation loss function until the inference accuracy of the student model reaches a preset accuracy threshold, thereby obtaining a lightweight inference model.

[0084] In this application, the teacher model refers to a pre-trained, large language model with a large number of parameters and high inference accuracy. It serves as the source of knowledge in the knowledge distillation process, providing learning goals and standards for the student model. The student model refers to a small network model with a much smaller number of parameters than the teacher model. It is the receiver of knowledge in the knowledge distillation process. By learning the inference logic and output distribution of the teacher model, it retains its core capabilities while possessing a smaller size and faster inference speed.

[0085] In this application, the distillation loss function refers to a function used to evaluate the difference between the output of the student model and the output of the teacher model during the knowledge distillation training process. It is the core basis for driving the iterative optimization of the student model; the smaller the value of the function, the smaller the ability gap between the student model and the teacher model. The relative entropy refers to an indicator used to measure the degree of difference between two probability distributions. In the distillation loss function, it is used to calculate the difference between the output distribution of the teacher model and the output distribution of the student model. The smaller the value, the closer the output distributions of the two models are. The cross-entropy loss refers to an indicator used to measure the difference between the output content of the student model and the true labeled content. It is used to ensure the inference accuracy of the student model itself and to prevent the student model from merely conforming to the output of the teacher model and losing its own inference accuracy.

[0086] In this application, the preset accuracy threshold refers to a pre-set critical value used to determine whether the reasoning accuracy of the student model meets the standard. Only when the reasoning accuracy of the student model reaches this threshold is the knowledge distillation process considered complete.

[0087] Based on the technical solutions in steps 1361 to 1363 above, by using a pre-trained large language model as the teacher model and a network model with fewer parameters as the student model, the learning objectives and recipients of knowledge distillation can be clearly defined. The mature reasoning capabilities and rich knowledge reserves of the teacher model provide precise guidance for the training of the student model, ensuring that the student model can learn the core capabilities of the teacher model. By constructing a distillation loss function weighted by the relative entropy of the outputs of the teacher and student models and the cross-entropy loss of the student model, the training process of knowledge distillation can be controlled simultaneously from two dimensions: the capability fit between the student and teacher models and the reasoning accuracy of the student model itself. This ensures that the student model can learn the core reasoning logic of the teacher model while also guaranteeing the accuracy of its own output, avoiding the problem of a significant drop in accuracy after model compression. By iteratively training the student model based on the distillation loss function until the inference accuracy of the student model reaches a preset accuracy threshold, a lightweight inference model is obtained. This ensures that the compressed lightweight model retains sufficient inference accuracy while having a smaller size and faster inference speed, meeting the needs of intelligent agent interaction in vehicle scenarios. At the same time, it adapts to the hardware performance of vehicle edge computing devices, further improving the real-time performance of interaction and task processing efficiency.

[0088] Continue to refer to Figure 1 In step 140, a target interaction result is determined based on the interaction response content, and the target interaction result is output. When there are two or more target agents, the interaction response content generated by all target agents is fused to obtain the target interaction result. When there is only one target agent, the interaction response content generated by that target agent is determined as the target interaction result.

[0089] In this application, the fusion processing refers to the process by which the system prioritizes, filters conflicts, and logically integrates multiple response contents when multiple target agents generate interactive response content simultaneously, ensuring that the final output content is logically coherent and information is complete, and avoiding the problem of contradictory or chaotic content from multiple agents.

[0090] Specifically, in this application, the process of fusing the interactive response content generated by all target agents to obtain the target interaction result can be performed according to the following steps 141 to 143: Step 141: Use the Borda counting method to prioritize the interactive response content generated by all target agents.

[0091] Step 142: Use the Delphi method to perform consistency checks and filter conflicting content on the sorted interactive response content.

[0092] Step 143: Integrate the validated and filtered interactive response content to generate a logically coherent target interactive result.

[0093] After step 140 above, that is, after outputting the target interaction result, the dialogue data and role operation data corresponding to this interaction can be updated to the preset storage module.

[0094] In this application, the preset storage module may include a cache database, a vector database, and a knowledge graph database. The cache database can be used to store real-time dialogue data and task execution data of the current interaction; the vector database can be used to store the agent's role feature data and semantic embedding data of historical dialogues; and the knowledge graph database can be used to store rule data, entity relationship data, and domain-specific knowledge data corresponding to the interaction scenario.

[0095] To enable those skilled in the art to better understand this application, the following will combine... Figure 2 The following is a specific embodiment illustrating a system architecture diagram for performing the methods described in this application.

[0096] Reference Figure 2 The diagram illustrates the system architecture of the in-vehicle interaction method based on multi-agent collaboration in an embodiment of this application.

[0097] like Figure 2As shown, in the system architecture, the top layer is the user interface layer, which is equipped with a core module supporting user input in three forms: voice, gesture, and touch. This module serves as the sole entry point for user interaction with the system, corresponding to the acquisition and standardization of multimodal interactive input in this application. After standardization, the user input enters the core agent collaboration layer, first reaching the scheduler module, which allocates tasks based on Q-Learning. This module is responsible for parsing and classifying the generated user task requests by domain and difficulty, matching at least one target agent with corresponding processing capabilities, corresponding to the task allocation and agent matching stage in this application. The scheduler simultaneously triggers the context manager module, which constructs a dialogue graph based on DST (Dialogue State Tracking) and a hierarchical attention mechanism. This module is responsible for retrieving dialogue context data matching the current user task request, as well as role feature data corresponding to the target agent, corresponding to the core technology stage of maintaining context consistency in this application. Subsequently, based on the retrieved context data and role feature data, the system drives the agent cluster module to generate corresponding interactive response content. This agent cluster is powered by LLM (Large Language Management). Driven by the Model (a large language model), the system uses BigFive (five personality models) vector embedding and PPO (proximal policy optimization algorithm) to complete the role modeling and personality consistency calibration of the agent, ensuring that the output content conforms to the role positioning and contextual logic. This corresponds to the core technology of agent role modeling and personality consistency maintenance in this application. When there are two or more target agents, the interactive response content generated by all agents will enter the decision fusion module. This module adopts a hybrid voting mechanism of Borda and Delphi to complete the priority sorting, conflict filtering and logical integration of the output content of multiple agents, generating the final target interaction result. This corresponds to the multi-agent output fusion processing stage in this application. The system's underlying layer... As the data storage layer, it provides bidirectional data support and high-speed read / write services for the entire interaction chain. This layer includes the ChromaDB vector database for storing agent role features and historical dialogue semantic embedding data, the Neo4j knowledge graph for storing entity relationships, scene rules, and domain knowledge, and the Redis cache for providing real-time context caching and ensuring stable high-concurrency communication. The three storage units work in real time with the core modules of the agent collaboration layer to complete the retrieval, updating, and persistent storage of various types of data during the interaction process. Finally, the target interaction result processed by the decision fusion unit will be output to the user terminal and presented to the user in three forms adapted to the in-vehicle scenario: voice, text, and graphics, completing the complete interactive loop from user input to result output.

[0098] Furthermore, to enable those skilled in the art to better understand this application, this application also provides several specific application scenarios to illustrate the proposed in-vehicle interaction method based on multi-agent collaboration.

[0099] Scenario 1: Multi-round chat interaction requests. This scenario can be applied to vehicle infotainment systems, specifically for multi-round vehicle maintenance consultation interactions during driving. The implementation process may include the following: First, the vehicle's infotainment system acquires the user's multimodal interactive input. While driving, if touch operation is inconvenient, the user can use voice commands to state, "My car has driven 10,000 kilometers, what maintenance is needed," while simultaneously making corresponding preset gestures to inquire about vehicle information. The system then collects both voice and gesture input data. The system performs semantic recognition and intent extraction on this data, identifying the user's vehicle model and mileage information. It extracts the user's core intent: to obtain vehicle maintenance recommendations based on their mileage. The system then standardizes the different input formats to generate corresponding user task requests.

[0100] Next, the system parses the generated user task requests based on a preset task allocation strategy. First, it categorizes the task requests by domain, determining that the task belongs to the vehicle professional maintenance domain. Simultaneously, it classifies the task difficulty, classifying it as a medium-difficulty reasoning task within a single domain, requiring targeted content generation based on the user's vehicle model and mileage, generating corresponding task attribute tags. The system's task allocation model, built on the Q-learning algorithm, matches a vehicle maintenance professional consultant agent as the target agent based on the generated task attribute tags. This target agent is driven by a lightweight, large-scale language model that has undergone knowledge distillation and compression, adapting to the deployment requirements of in-vehicle edge computing devices. After the task is processed, the system optimizes the task allocation strategy based on the processing time and user satisfaction feedback, improving the allocation accuracy of subsequent similar tasks.

[0101] Next, the system retrieves the dialogue context data corresponding to the current user task request, as well as the role feature data of the matched target agent. When retrieving the dialogue context data, the system constructs and continuously updates the dialogue graph corresponding to the current user based on dialogue state tracking technology. This dialogue graph records the user's previous interactions with the system, including previous inquiries about the same vehicle's oil change interval and tire maintenance. Through a hierarchical attention mechanism, the system extracts historical dialogue content related to the current 10,000-kilometer maintenance task from the dialogue graph, generating corresponding dialogue context data to maintain the semantic coherence of multi-turn interactions. Simultaneously, the system retrieves the role characteristic data corresponding to the vehicle maintenance professional consultant agent from a preset vector database. This role characteristic data is generated as follows: the system constructs a corresponding personality characteristic vector for the agent based on the five major personality models, setting high values ​​for the dimensions of responsibility and emotional stability to ensure that the agent's output is rigorous, professional, and concise, adapting to the needs of in-vehicle driving scenarios. Then, the system fine-tunes the large language model corresponding to the agent through a proximal strategy optimization algorithm. During the fine-tuning process, the system constructs a reward function weighted by the role behavior matching score and the historical speech consistency score. When the agent's output content conforms to the preset role positioning and is consistent with historical speech, a positive reward is given; otherwise, a negative reward is given. The model parameters are iteratively optimized based on the output of the reward function until the agent's personality deviation rate is lower than the preset deviation threshold of 5%. The final personality characteristic vector and model fine-tuning parameters are then used as the role characteristic data of the agent and stored in the preset vector database.

[0102] Based on the retrieved dialogue context data and role feature data, the system drives the vehicle maintenance professional consultant agent to generate interactive response content corresponding to the user's task request. This response content incorporates information previously asked by the user regarding engine oil and tires, and provides targeted maintenance suggestions based on the user's vehicle model and 10,000 kilometers of mileage. It also employs a concise and clear expression adapted to the driving scenario. During the response content generation process, the system performs semantic embedding processing on adjacent dialogue messages to obtain corresponding semantic vectors. It then calculates the cosine similarity of the semantic vectors from adjacent messages to obtain a context consistency score. In this embodiment, the score is 0.89, which is higher than the preset score threshold of 0.8, eliminating the need for content re-optimization.

[0103] Since there is only one target agent in this matching, the system directly determines the interactive response generated by that target agent as the target interaction result. This response is then simultaneously output to the user through the vehicle's voice broadcast and screen text display. At the same time, the system updates the dialogue data and role operation data corresponding to this interaction to the preset storage module, including updating the user's dialogue graph and storing the interaction content in the corresponding vector database to provide contextual support for subsequent interactions.

[0104] In this embodiment, the lightweight inference model used to drive the intelligent agent is obtained as follows: the system uses a pre-trained large language model with 7 billion parameters as the teacher model and a transformer network model with 3 billion parameters as the student model. A distillation loss function is constructed, which is obtained by weighting the relative entropy of the outputs of the teacher model and the student model and the cross-entropy loss of the student model. The weight of the relative entropy is 0.7 and the weight of the cross-entropy loss is 0.3. The system iteratively trains the student model based on the distillation loss function until the difference between the inference accuracy of the student model and the teacher model is less than 5%, reaching the preset accuracy threshold, thus obtaining a lightweight inference model. This model is then deployed to the vehicle's in-vehicle edge computing device to achieve local inference. In this embodiment, the average end-to-end response latency is 1.2 seconds, which fully meets the real-time requirements of in-vehicle driving scenarios.

[0105] Scenario 2: Werewolf Game Interaction Request. This scenario can be applied to the vehicle's infotainment system, specifically for in-vehicle Werewolf game interaction when a user stops for a rest at a service area. The implementation process may include the following: First, the vehicle's infotainment system acquires the user's multimodal interactive input. When the user is resting in the car, they say "I want to play a 6-player game of Werewolf" via voice and simultaneously tap the game's launch button on the infotainment screen. The system acquires both the user's voice input and touch input, performs semantic recognition and intent extraction on the interactive data, identifies the user's core need as starting a 6-player game of Werewolf, and after standardization processing, generates the corresponding user task request.

[0106] Next, based on a preset task allocation strategy, the system parses the user's task request, classifies it by domain, and determines that the task belongs to the in-vehicle entertainment and interactive, multi-role collaborative game domain. It then classifies the task difficulty, classifying it as a high-complexity multi-agent collaborative task requiring the cooperation of multiple agents with different roles, and generates corresponding task attribute tags. The system's task allocation model, built based on the Q-learning algorithm, matches six corresponding target agents according to the task attribute tags: two werewolf agents, two villager agents, one seer agent, and one witch agent. Each target agent is driven by a lightweight large language model deployed on the in-vehicle edge computing device.

[0107] Next, the system retrieves the dialogue context data corresponding to the Werewolf game task, including the game's initial rules, process settings, and role faction relationships. Simultaneously, it retrieves the role characteristic data for each target agent. Each agent's role characteristic data is constructed based on a five-dimensional personality model, creating a corresponding personality trait vector. For example, the werewolf agent has a lower value for agreeableness and a higher value for conscientiousness, ensuring the agent prioritizes the werewolf faction's victory and exhibits a tendency towards disguise and cooperation. The seer agent has higher values ​​for agreeableness and conscientiousness, ensuring its statements are guiding and logical. The system fine-tunes the large language model for each agent using a near-end strategy optimization algorithm, constructing a corresponding reward function to ensure the agent's output behavior matches the corresponding personality trait vector, and that consistency between words and actions is met. Finally, the system generates the role characteristic data for each agent and stores it in a pre-defined storage module. When retrieving dialogue context data, the system uses dialogue state tracking technology to construct a dialogue graph corresponding to the current game. As the game progresses, the graph is continuously updated, recording the content of each round of speech, voting records, and character action information. Through a hierarchical attention mechanism, the system extracts content related to the current game stage from the dialogue graph to generate dialogue context data, ensuring that each agent's speech is consistent with the game's progress.

[0108] Based on the retrieved dialogue context data and the role characteristic data of each target agent, the system drives the corresponding target agents to generate interactive response content in stages according to the game flow. For example, in the night action phase, it drives the werewolf agent to generate a decision to hunt targets, and the seer agent to generate a decision to check targets. In the daytime speaking phase, it drives each agent to generate corresponding speech content in sequence. During the content generation process in each stage, the system performs semantic embedding processing on the speech content of adjacent rounds and calculates the context consistency score. When the consistency score of a certain agent's speech content with previous speech is lower than a preset threshold, the system retrieves the dialogue context data again and optimizes the agent's speech content to ensure consistency.

[0109] Since there are 6 target agents in this matching, the system integrates the interactive response content generated by all target agents. Specifically, the Borda counting method is used to prioritize the response content of all agents according to the game flow, and the Delphi method is used to perform consistency verification and conflict filtering on the sorted content. Finally, according to the game progress, the system integrates and generates complete game interaction content as the target interaction result, which is output to the user through the vehicle's voice broadcast and screen display. At the same time, the game progress data and the interaction data of each agent are updated to the preset storage module to continuously update the dialogue graph corresponding to the game.

[0110] Scenario 3: Think Tank System Scenario. This scenario can be applied to the vehicle's infotainment system, specifically for providing consultation on a user's car purchase decision. The implementation process may include the following: First, the vehicle's infotainment system acquires the user's multimodal interactive input. The user says via voice, "I've recently taken a liking to a new energy vehicle. Could you analyze whether the purchase plan for this car is worthwhile?" At the same time, the user uploads the car's configuration information to the system via touch input on the vehicle's infotainment system. The system acquires the user's voice and touch input, performs semantic recognition and intent extraction, identifies that the user's core need is to conduct a multi-dimensional analysis of the purchase plan for the specified new energy vehicle, and generates a standardized user task request.

[0111] Next, based on a preset task allocation strategy, the system parses the user's task request, classifies it by domain, and determines that the task involves three domains: automotive technology analysis, economic cost analysis, and legal risk analysis for car purchase. It then classifies the task difficulty, classifying it as a complex decision-making support task requiring collaboration from multiple professional agents, and generates corresponding task attribute labels. The system's task allocation model, built based on the Q-learning algorithm, matches three corresponding target agents based on the task attribute labels: an automotive technology expert agent, an economic analyst agent, and a legal advisor agent. Each target agent is driven by a lightweight large language model deployed on an in-vehicle edge computing device.

[0112] Next, the system retrieves the dialogue context data corresponding to this decision-making consultation task, including historical interactions such as the user's previously mentioned car purchase budget, usage scenarios, and charging conditions. Simultaneously, it retrieves the role characteristic data corresponding to each target agent. Each agent has a unique personality feature vector constructed based on its corresponding professional domain. The model is fine-tuned using a proximal policy optimization algorithm to ensure that the output content meets the requirements of the corresponding professional domain. Based on dialogue state tracking technology, the system constructs a dialogue graph corresponding to the current user and extracts historical content related to this car purchase analysis through a hierarchical attention mechanism to generate dialogue context data, ensuring that the agent's analysis content aligns with the user's actual situation.

[0113] Based on the retrieved dialogue context data and the role characteristic data of each target agent, the system assigns corresponding sub-analysis tasks to each expert agent, driving each agent to generate corresponding professional analysis content. Among them, the automotive technology expert agent generates analysis content from the dimensions of vehicle technical configuration, performance, durability, etc., the economic analyst agent generates analysis content from the dimensions of vehicle purchase cost, vehicle use cost, residual value, etc., and the legal advisor agent generates analysis content from the dimensions of vehicle purchase contract terms, after-sales guarantee, relevant laws and regulations, etc.

[0114] Since there are three target agents in this matching process, the system integrates the professional analysis content generated by all agents, prioritizes the analysis content of different dimensions using the Borda counting method, verifies the consistency of the content using the Delphi method, and integrates to generate a logically coherent and dimensionally complete comprehensive car purchase plan analysis report. This report serves as the target interaction result and is output to the user through the vehicle's screen display and voice broadcast. At the same time, all data from this interaction is updated to the preset storage module to provide contextual support for the user's subsequent related inquiries.

[0115] Based on the aforementioned application scenarios, it can be seen that the multi-agent collaborative in-vehicle interaction method provided in this application comprehensively covers various user interaction needs in in-vehicle scenarios through multimodal input adaptation and standardization processing, improving the accuracy of need recognition and the security of interaction. Through a reinforcement learning-based task allocation strategy, it achieves precise matching between user tasks and agent capabilities, flexibly adapting to different interaction scenarios of single and multi-agent interactions. Through personality modeling and reinforcement learning fine-tuning, it ensures the consistency and stability of agent role behavior, avoiding issues such as personality confusion and behavioral deviation. Through the construction of a dialogue graph and the application of a hierarchical attention mechanism, it maintains the semantic coherence of multi-turn interactions, avoiding contextual breaks. To address issues of fission and logical contradictions, this approach utilizes knowledge distillation model compression and in-vehicle edge deployment to significantly improve the real-time performance and stability of interactions, avoiding sluggish responses caused by network latency. Furthermore, through the fusion processing of multi-agent outputs, it ensures logical coherence and complete information in the collaborative output of multiple agents. This comprehensively solves the problems of low agent task processing efficiency, insufficient naturalness of interaction, poor real-time performance, and weak multi-agent collaboration capabilities in existing in-vehicle interaction scenarios. It enables efficient, coherent, and stable collaborative interaction among multiple agents in in-vehicle scenarios, greatly enhancing the user's in-vehicle interaction experience. It also possesses strong scene adaptability, covering various interaction needs in in-vehicle scenarios such as knowledge consultation, entertainment interaction, and decision support.

[0116] The following describes an embodiment of the apparatus described in this application, which can be used to execute the in-vehicle interaction method based on multi-agent cooperation in the above embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the in-vehicle interaction method based on multi-agent cooperation described above in this application.

[0117] See Figure 3 The diagram shows a block diagram of an in-vehicle interaction device based on multi-agent collaboration in an embodiment of this application.

[0118] like Figure 3 As shown, the in-vehicle interaction device 300 based on multi-agent collaboration according to an embodiment of this application includes: an acquisition unit 301, a parsing unit 302, a generation unit 303, and an output unit 304.

[0119] The system includes: an acquisition unit 301, used to acquire the user's multimodal interaction input and standardize the input to generate a corresponding user task request; a parsing unit 302, used to parse the user task request based on a preset task allocation strategy to determine at least one target agent matching the request, wherein each target agent is driven by a large language model; a generation unit 303, used to retrieve the dialogue context data corresponding to the user task request and the role feature data of the target agent, and drive the target agent to generate interactive response content corresponding to the request based on the context data and the response data; and an output unit 304, used to determine the target interaction result based on the response content and output it. Specifically, when there are two or more target agents, the interaction response content generated by all target agents is fused to obtain the target interaction result; when there is only one target agent, the response content generated by that agent is determined as the target interaction result.

[0120] In some embodiments of this application, based on the foregoing scheme, the acquisition unit 301 is configured to: acquire interactive data submitted by the user through at least one of voice input, touch input, or gesture input; perform semantic recognition and intent extraction on the interactive data to generate a standardized user task request.

[0121] In some embodiments of this application, based on the foregoing scheme, the parsing unit 302 is configured to: classify the user task request into a domain and grade the task difficulty to obtain task attribute labels; and use a task allocation model constructed based on the Q-learning algorithm to match at least one target agent according to the task attribute labels, wherein the allocation strategy of the task allocation model is optimized based on the feedback results of historical task execution.

[0122] In some embodiments of this application, based on the foregoing scheme, the generation unit 303 is configured to: construct corresponding personality feature vectors for different agents based on different personality models; fine-tune the large language model corresponding to the agent through a proximal policy optimization algorithm so that the agent's output behavior matches the corresponding personality feature vector; and store the matched personality feature vector and model fine-tuning parameters as the agent's role feature data in a preset storage module.

[0123] In some embodiments of this application, based on the aforementioned scheme, the generation unit 303 is configured to: construct a reward function, which is obtained by weighting the role behavior matching score and the historical speech and behavior consistency score; when the matching degree between the output content generated by the agent and the personality feature vector is higher than a preset matching threshold, and the consistency with the historical output content is higher than a preset consistency threshold, a positive reward is given, otherwise a negative reward is given; based on the output result of the reward function, the generation parameters of the large language model are iteratively optimized until the personality deviation rate of the agent is lower than a preset deviation threshold.

[0124] In some embodiments of this application, based on the foregoing scheme, the generation unit 303 is configured to: construct and continuously update a dialogue graph corresponding to the current user based on dialogue state tracking technology; and extract historical dialogue content related to the user's task request from the dialogue graph through a hierarchical attention mechanism to generate dialogue context data, wherein the dialogue context data is used to maintain semantic coherence in multi-turn interaction processes.

[0125] In some embodiments of this application, based on the foregoing scheme, the device further includes: an optimization unit, configured to perform semantic embedding processing on the dialogue content of adjacent rounds respectively to obtain corresponding semantic vectors; calculate the cosine similarity of the semantic vectors of adjacent rounds to obtain a context consistency score; and when the context consistency score is lower than a preset score threshold, retrieve the corresponding dialogue context data again to optimize the interactive response content of the current round.

[0126] In some embodiments of this application, based on the foregoing scheme, the device further includes: a compression unit, used to compress the large language model using knowledge distillation technology to obtain a lightweight inference model before driving the target intelligent agent to generate interactive response content corresponding to the user task request; and to deploy the lightweight inference model to an in-vehicle edge computing device.

[0127] In some embodiments of this application, based on the aforementioned scheme, the compression unit is configured as follows: a pre-trained large language model is used as the teacher model, and a network model with fewer parameters than the teacher model is used as the student model; a distillation loss function is constructed, which is obtained by weighting the relative entropy of the outputs of the teacher model and the student model and the cross-entropy loss of the student model; the student model is iteratively trained based on the distillation loss function until the inference accuracy of the student model reaches a preset accuracy threshold, thereby obtaining a lightweight inference model.

[0128] Based on the same inventive concept, embodiments of this application provide a computer program product, the computer program product including computer instructions stored in a computer-readable storage medium and adapted to be read and executed by a processor, so as to cause a computer device having the processor to perform the operations performed by the in-vehicle interaction method based on multi-agent cooperation as described above.

[0129] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium storing at least one computer program instruction, which is loaded and executed by a processor to implement the operations performed by the multi-agent cooperative vehicle interaction method as described above.

[0130] Based on the same inventive concept, this application also provides a vehicle, see reference. Figure 4 The diagram shows a structural schematic of a vehicle according to an embodiment of this application. The vehicle includes one or more memories 404, one or more processors 402, and at least one computer program (computer program instruction) stored in the memory 404 and executable on the processor 402. When the processor 402 executes the computer program, it implements the vehicle interaction method based on multi-agent collaboration as described above.

[0131] Among them, Figure 4 In this document, a bus architecture (represented by bus 400) is used. Bus 400 may include any number of interconnected buses and bridges, linking various circuits including one or more processors represented by processor 402 and memory represented by memory 404. Bus 400 may also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. Bus interface 405 provides an interface between bus 400 and receiver 401 and transmitter 403. Receiver 401 and transmitter 403 may be the same element, i.e., a transceiver, providing a unit for communicating with various other devices over a transmission medium. Processor 402 is responsible for managing bus 400 and general processing, while memory 404 can be used to store data used by processor 402 during operation.

[0132] The functions described herein can be implemented in hardware, software executed by a processor, firmware, or any combination thereof. When implemented in software executed by a processor, the functions can be stored as one or more instructions or codes on or transmitted via a computer-readable medium. Other examples and embodiments are within the scope and spirit of this application and the appended claims. For example, due to the nature of software, the functions described above can be implemented using software executed by a processor, hardware, firmware, hardwired, or any combination thereof. Furthermore, the functional units can be integrated into a single processing unit, or each unit can exist physically separately, or two or more units can be integrated into a single unit.

[0133] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0134] The units described as separate components may or may not be physically separate. Similarly, the components of the control device may or may not be physical units; they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0135] When the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing computer program instructions, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0136] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A vehicle-mounted interaction method based on multi-agent collaboration, characterized in that, The method includes: Acquire the user's multimodal interaction input and standardize the multimodal interaction input to generate corresponding user task requests; Based on a preset task allocation strategy, the user task request is parsed to determine at least one target agent that matches the user task request, wherein each target agent is driven by a large language model. Retrieve the dialogue context data corresponding to the user task request and the role feature data corresponding to the target agent, and based on the dialogue context data and the role feature data, drive the target agent to generate interactive response content corresponding to the user task request; The target interaction result is determined based on the interaction response content and then output. When there are two or more target agents, the interaction response content generated by all target agents is fused to obtain the target interaction result. When there is only one target agent, the interaction response content generated by the target agent is determined as the target interaction result.

2. The method according to claim 1, characterized in that, The step of acquiring the user's multimodal interaction input and standardizing the multimodal interaction input to generate a corresponding user task request includes: Acquire interactive data submitted by the user through at least one of voice input, touch input, or gesture input; The interactive data is semantically recognized and intent extracted to generate standardized user task requests.

3. The method according to claim 1, characterized in that, The step of parsing the user task request based on a preset task allocation strategy and determining at least one target agent matching the user task request includes: The user task requests are classified by domain and task difficulty level to obtain task attribute tags; A task allocation model based on the Q-learning algorithm matches at least one target agent according to the task attribute labels. The allocation strategy of the task allocation model is optimized based on the feedback results of historical task execution.

4. The method according to claim 1, characterized in that, The role feature data corresponding to the target intelligent agent is generated through the following steps: Based on different personality models, corresponding personality feature vectors are constructed for different agents. The large language model corresponding to the agent is fine-tuned by a proximal policy optimization algorithm so that the agent's output behavior matches the corresponding personality feature vector. The matched personality feature vectors and model fine-tuning parameters are used as the role feature data of the intelligent agent and stored in a preset storage module.

5. The method according to claim 4, characterized in that, The fine-tuning of the large language model corresponding to the agent using a proximal policy optimization algorithm includes: Construct a reward function, which is obtained by weighting the role behavior matching score and the historical speech and behavior consistency score; When the matching degree between the output content generated by the agent and the personality feature vector is higher than the preset matching threshold, and the consistency with the historical output content is higher than the preset consistency threshold, a positive reward is given; otherwise, a negative reward is given. Based on the output of the reward function, the generation parameters of the large language model are iteratively optimized until the agent's personality deviation rate is lower than the preset deviation threshold.

6. The method according to claim 1, characterized in that, The retrieval of the dialogue context data corresponding to the user task request includes: Based on dialogue state tracking technology, a dialogue graph corresponding to the current user is constructed and continuously updated; By using a hierarchical attention mechanism, historical dialogue content related to the user's task request is extracted from the dialogue graph to generate dialogue context data, which is used to maintain semantic coherence in multi-turn interaction processes.

7. The method according to claim 6, characterized in that, The method further includes: Semantic embedding is performed on the dialogue content of adjacent rounds to obtain the corresponding semantic vectors; Calculate the cosine similarity of semantic vectors from adjacent rounds to obtain a context consistency score; When the context consistency score is lower than the preset score threshold, the corresponding dialogue context data is retrieved again to optimize the interactive response content of the current round.

8. The method according to claim 1, characterized in that, Before driving the target agent to generate interactive response content corresponding to the user task request, the method further includes: A lightweight reasoning model is obtained by compressing a large language model using knowledge distillation techniques. The lightweight inference model is deployed to an in-vehicle edge computing device.

9. The method according to claim 8, characterized in that, The compression process of the large language model using knowledge distillation technology includes: A pre-trained large language model is used as the teacher model, and a network model with fewer parameters than the teacher model is used as the student model. Construct a distillation loss function, which is obtained by weighting the relative entropy of the outputs of the teacher model and the student model and the cross-entropy loss of the student model; The student model is iteratively trained based on the distillation loss function until the inference accuracy of the student model reaches a preset accuracy threshold, thus obtaining a lightweight inference model.

10. A vehicle, characterized in that, The vehicle includes one or more processors and one or more memories, wherein at least one piece of program code is stored in the one or more memories, and the at least one piece of program code is loaded and executed by the one or more processors to implement the method as described in any one of claims 1 to 9.

Images