A multi-round dialogue instruction synthesis method, system and electronic device for an intelligent cockpit large model agent

Abstract

The application relates to a multi-round dialogue instruction synthesis method and system for an intelligent cockpit large model agent and electronic equipment, and belongs to the technical field of natural language processing. The method is realized through six steps: API data standardization processing, user instruction generation, multi-round dialogue flow planning, three-agent collaborative simulation dialogue, user instruction text enhancement and data quality evaluation. A large language model is used to automatically generate user instructions covering multiple intent types, and a three-agent architecture is used to simulate real vehicle voice interaction to synthesize high-quality multi-round dialogue data. In addition, a text enhancement mechanism and an automatic evaluation system are introduced to ensure the robustness and practicality of the data. The method significantly reduces the data construction cost and improves the efficiency, and is suitable for different vehicle models and language environments.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing technology and relates to a method, system and electronic device for synthesizing multi-turn dialogue instructions for intelligent cockpit large-scale model intelligent agents. Background Technology

[0002] In-vehicle intelligent cockpits refer to immersive and intelligent driving experience spaces centered on the user, created by integrating advanced infotainment systems, human-machine interfaces, intelligent voice assistants, driver status monitoring, multi-screen interaction, environmental perception, and personalized services. Large Language Models (LLMs), with their powerful natural language understanding and generation capabilities, are becoming a key driver for upgrading human-machine interaction in intelligent cockpits. In in-vehicle scenarios, LLMs can achieve more natural, context-aware, and multi-turn conversational voice interaction, supporting users to complete complex tasks such as navigation settings, media playback, vehicle control, and knowledge-based question answering through spoken commands.

[0003] However, the practical application of large language models for intelligent cockpits is severely hampered by a severe lack of high-quality training data. Limited by factors such as data privacy, security compliance, and the closed nature of automakers' data, the industry currently lacks publicly available, standardized, high-quality open-source datasets specifically designed for cockpit interaction tasks. Even for companies possessing internal data, the acquisition, anonymization, and sharing face strict legal and engineering restrictions, significantly hindering model training, fine-tuning, and continuous iteration in this field.

[0004] Traditional large-scale model training relies on massive amounts of high-quality manually labeled data, typically requiring a multi-stage process including rule selection, manual verification, intent classification, and slot labeling. This process is not only time-consuming and labor-intensive but also requires continuous maintenance by a professional team with domain knowledge, resulting in high data engineering costs. Especially in highly specialized, context-sensitive, and interactively complex scenarios like in-vehicle cockpits, the requirements for data quality are even more stringent—for example, accurately capturing the evolution of user intent across dialogue rounds and the impact of multimodal context (such as driving status and environmental information) on semantics.

[0005] To reduce data construction costs and improve efficiency, researchers have attempted to introduce data synthesis techniques, simulating real user interaction data through rule templates, language model self-generation, or Generative Adversarial Networks (GANs). However, current common data synthesis methods perform poorly in vertical fields such as in-vehicle cockpits: on the one hand, the generated content often lacks the pragmatic habits, terminology, and task logic of real cockpit scenarios; on the other hand, synthesized data struggles to meet practical deployment requirements in key capabilities involving multi-turn dialogue coherence, intent-action mapping consistency, and complex command recognition. Especially for complex interactive systems that require precise alignment of user intent, system response, and vehicle control commands, existing synthesis strategies cannot effectively reproduce the semantic depth and behavioral diversity of the real world.

[0006] Therefore, how to design an automated, high-quality dialogue data generation method specifically designed for the dialogue needs of in-vehicle intelligent cockpits is a technical problem that urgently needs to be solved in the field of natural language processing. Summary of the Invention

[0007] In view of this, the purpose of this invention is to provide a method, system and electronic device for synthesizing multi-turn dialogue instructions for intelligent cockpit large-scale model intelligent agents.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] On the one hand, a method for synthesizing multi-turn dialogue instructions for intelligent cockpit large-scale model agents is provided, which includes the following steps: S1. Obtain the raw data of the application programming interface (API) document for the intelligent cockpit, perform standardized cleaning and processing, and construct a structured API metadata database. S2. Establish a preset prompt word template, use one or more API samples from the API metadata database, construct structured input based on the prompt word template, and generate task-oriented user instructions through the first language model; S3. Based on the user instructions generated by the first language model, use the second language model to perform structured planning of the multi-turn dialogue process; S4. Construct a three-agent collaborative architecture including a user agent, an assistant agent, and a tool agent. The three-agent collaborative architecture simulates the real in-vehicle voice interaction process based on the structured planning results of the multi-turn dialogue process and synthesizes the multi-turn dialogue trajectory text. S5. Introduce a user instruction text enhancement mechanism based on a large language model to enhance and reconstruct the multi-turn dialogue trajectory text generated by the three-agent collaborative architecture. S6. Data format and basic language quality are filtered through predefined deterministic rules, and a third language model is used to perform fine-grained semantic scoring on each dialogue data. A comprehensive quality score is generated according to preset weights, and samples with a comprehensive quality score not lower than a preset threshold are retained in the final dataset.

[0010] Furthermore, in step S1, the API raw data sample includes at least the interface name, function description, parameter definition, and necessary parameter information; Standardized cleaning of raw API data includes completing missing parameter descriptions, standardizing parameter type definitions, converting unstructured fields, and merging or deduplication operations. Completing missing parameter descriptions involves adding descriptive information matching the functional attributes of parameter fields that lack natural language descriptions. Standardizing parameter type definitions involves clearly defining and uniformly constraining the types of input parameters, covering four basic types: strings, numbers, lists, and objects. Converting unstructured fields involves converting unstructured fields to structured JSON format according to OAS specifications. Merging or deduplication operations involve using a pre-trained semantic embedding model to vectorize the functional description text of each API, calculating the cosine similarity between any two API vectors, and merging or deduplicating texts with similarity exceeding a threshold.

[0011] Furthermore, in step S2, the preset prompt word template includes sampled API text, constraints, and output format specifications. The sampled API text includes API name, function description, and parameter information. The constraints refer to the specific requirements for the synthesis of single or multiple intentions. The output format specifications refer to the text that needs to be output in natural language or JSON format. Structured input is constructed based on prompt word templates and fed into the first major language model. The first major language model generates user instructions that comprehensively cover typical scenarios of multi-turn dialogue in intelligent cockpits. Among them, user instruction types are at least divided into single-intent type, parallel multi-intent type, and serial multi-intent type. Single-intent type means that the user expresses a single and clear goal, corresponding to one API call; parallel multi-intent type means that the user puts out multiple independent and undependent intents, corresponding to multiple API calls that can be executed concurrently; serial multi-intent type means that the user puts out multiple intents with logical dependencies or sequential constraints, requiring multiple APIs to be called in a specific order.

[0012] Furthermore, in step S3, the user commands output by the first language model are input into the second language model for structured planning of the multi-turn dialogue process. The structured planning information of the multi-turn dialogue process output by the second language model includes at least the total number of turns, the interaction content of each turn, and the interaction type of each turn. The interaction content refers to a simple description of the core interaction semantics of the turn. The interaction type includes two types: non-tool call type and tool call type. Non-tool call type refers to the interaction type in which the user's request does not need to call the API and can be responded to directly in natural language. Tool call type refers to the interaction type in which the user's request needs to call the API at least once.

[0013] Furthermore, in step S4, the user agent simulates user behavior through prompting engineering based on a general large language model; the user agent dynamically loads specific user profiles according to a pre-built multi-dimensional user profile template library, thereby constructing role prompts; The assistant agent simulates assistant behavior through prompting engineering based on a general large language model; The tool agent is based on a large language model that fine-tunes instructions on a specific tool call format. It generates structured API call instructions that conform to a predefined format based on user instructions and context.

[0014] Furthermore, the process of synthesizing multi-turn dialogue trajectory text based on the three-agent collaborative architecture is as follows: Input the API information for the dialogue flow sampling; The user agent initially loads the user profile based on the user profile template library; The user agent generates user command text based on the interaction content of the current round; Determine if the current round is a tool call type request; if so, it is handled by the tool agent; otherwise, it is handled by the assistant agent. The tool's intelligent agent parses user commands and generates structured API call text; The assistant agent understands user commands and generates natural language responses; Is the dialogue complete? If not, the user agent generates the instruction text again based on the content of the next round of interaction, and repeats the process until the dialogue is complete; if the dialogue is complete, the dialogue trajectory is output.

[0015] Furthermore, in step S5, the user instruction text enhancement mechanism includes three methods: context-aware generalization, spatial reference reconstruction, and intent multi-turn decomposition. Context-aware generalization refers to the multi-dimensional, controllable generalization of original user commands at the semantic or grammatical structure level, based on the current dialogue history, to generate ambiguous, ambiguous, or non-standard expressions. This simulates language variations caused by distraction, slips of the tongue, or habitual expressions in a real-world in-vehicle environment. Context-aware generalization includes three dimensions of transformation: grammatical expression transformation, entity substitution, and semantic association generalization. Specifically, it is represented as follows: Let the original user instruction be The historical dialogue context is Select the appropriate template from the pre-built three-dimensional generalized prompt word template library. , combined and Constructing structured prompts The prompt word is input into a large language model, which then outputs the generalized instruction. ,Right now ; Spatial reference reconstruction integrates the in-vehicle spatial topology with the user's current location context, guiding the large language model to reconstruct explicit location commands into implicit or relative location expressions that conform to human spoken language habits; let the original user command be... The user's current location is The predefined spatial topology inside the vehicle is Spatial relationship reasoning: First, based on and Clues for constructing relative spatial position reasoning within the vehicle Input into a large language model and output the correct relative position description. Instruction reconstruction: combining the inferred relative position and original user instructions Build command to reconstruct prompt words The prompt word is input into the large language model, and the reconstructed instruction is output. ; The intent decomposition method addresses the original single-turn user instruction by structurally decoupling it into multiple semantically coherent sub-intents, which are then distributed across consecutive rounds of user speech. This simulates the step-by-step expression behavior of real users due to factors such as delayed thinking and gradual information disclosure. Let the original user instruction to be decomposed be... Using large language models to Perform intent parsing to identify the included Individual Intention and their potential order or dependencies; for each sub-intent Based on the context, a large language model is used to generate specific sub-instructions. For adjacent sub-intents, preset or generated irrelevant content can be inserted. The final output is a multi-round instruction sequence. Complete the original instructions Multiple rounds of disassembly.

[0016] Furthermore, in step S6, the predefined deterministic rules for filtering data format and basic language quality include structure and protocol compliance checks and language standardization checks. Specifically, the structure and protocol compliance checks detect whether there are missing or disordered dialogue turn structures, and the language standardization checks use open-source grammar and style checking tools to detect and filter natural language text with obvious grammatical errors or semantic incoherence, ensuring that the language output meets basic readability standards. Based on the rule verification, a third large language model is called to perform fine-grained semantic scoring on each dialogue data. The scoring dimensions include context consistency, scene matching, reasonable task difficulty, and natural language fluency. Each dimension is weighted and summed according to preset weights to generate a comprehensive quality score. Only when the total score is greater than or equal to the threshold is the sample retained into the final dataset.

[0017] On the other hand, a system is also provided for executing the aforementioned method for synthesizing multi-turn dialogue instructions for intelligent cockpit large-scale model agents. This system includes an API data processing and metadata database construction module, a user instruction generation module, a multi-turn dialogue flow planning module, a multi-turn dialogue trajectory text synthesis module, a dialogue trajectory text enhancement and reconstruction module, and a data quality assessment and filtering module. The API data processing and metadata database construction module obtains the raw data of the application programming interface (API) document of the smart cockpit, performs standardized cleaning and processing, and constructs a structured API metadata database. The user instruction generation module establishes a preset prompt word template, uses one or more API samples from the API metadata database, constructs structured input based on the prompt word template, and generates task-oriented user instructions through the first language model. The multi-turn dialogue process planning module uses the second language model to perform structured planning of the multi-turn dialogue process based on the user instructions generated by the first language model. The multi-turn dialogue trajectory text synthesis module constructs a three-agent collaborative architecture including a user agent, an assistant agent, and a tool agent. The three-agent collaborative architecture simulates the real in-vehicle voice interaction process based on the structured planning results of the multi-turn dialogue process and synthesizes the multi-turn dialogue trajectory text. The dialogue trajectory text enhancement and reconstruction module introduces a user command text enhancement mechanism based on a large language model to enhance and reconstruct the multi-turn dialogue trajectory text generated by the three-agent collaborative architecture. The data quality assessment and screening module filters data format and basic language quality through predefined deterministic rules, and uses a third language model to perform fine-grained semantic scoring on each dialogue data. It generates a comprehensive quality score according to preset weights, and samples with a comprehensive quality score not lower than a preset threshold are retained in the final dataset.

[0018] An electronic device is also provided, which includes a memory storing a computer program for executing the aforementioned method for synthesizing multi-turn dialogue instructions for intelligent cockpit large-scale model agents.

[0019] The beneficial effects of this invention are as follows: (1) This invention does not rely on manual annotation. It can synthesize high-quality multi-turn dialogue data end-to-end using only API documentation as input, which significantly shortens the data production cycle and reduces manpower and time costs.

[0020] (2) This invention effectively restores the user's expression habits, system response logic and tool call closed loop in the vehicle environment through scene-driven dialogue planning and three-agent collaborative mechanism, so that the synthesized data is highly close to the real interaction in terms of semantics, structure and behavior.

[0021] (3) This invention is the first to create a user instruction text rewriting mechanism for intelligent cockpit scenarios, which fully covers the challenges of context dependence, spatial reference omission, and multi-turn intent continuation in in-vehicle voice dialogue, and generates training samples with more robustness and generalization ability.

[0022] (4) This invention introduces an automatic evaluation system driven by a large language model, establishes a quantifiable and reproducible automatic evaluation system, and ensures that the output data meets the model training requirements in terms of professionalism, consistency and practicality.

[0023] (5) This invention provides a modular and configurable framework. The overall framework supports flexible replacement of API documents, user profile libraries, large language models and evaluation strategies, and can be quickly adapted to different vehicle platforms, functional modules and multilingual environments, and has good engineering implementation value.

[0024] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0025] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1This is a schematic diagram of the overall process of the multi-turn dialogue instruction synthesis method for intelligent cockpit large model intelligent agents according to an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the process of synthesizing multi-turn dialogue trajectory text according to an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the enhancement process of the user command text enhancement mechanism according to an embodiment of the present invention. Detailed Implementation

[0026] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0027] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0028] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0029] Please see Figures 1-3 This invention relates to a method, system, and electronic device for synthesizing multi-turn dialogue instructions for intelligent agents in large-scale intelligent cockpit models.

[0030] Example 1 This embodiment first provides a method for synthesizing multi-turn dialogue instructions for intelligent agents in large-scale intelligent cockpit models, such as... Figure 1 As shown, it includes at least the following steps: S1. Obtain the raw data of the Application Programming Interface (API) document of the intelligent cockpit, perform standardized cleaning and processing, and build a structured API metadata database; S2. Establish a preset prompt word template, use one or more API samples from the API metadata database, construct structured input based on the prompt word template, and generate task-oriented user instructions through the first language model; S3. Based on the user instructions generated by the first language model, use the second language model to perform structured planning of the multi-turn dialogue process; S4. Construct a three-agent collaborative architecture including a user agent, an assistant agent, and a tool agent. The three-agent collaborative architecture simulates the real in-vehicle voice interaction process based on the structured planning results of the multi-turn dialogue process and synthesizes the multi-turn dialogue trajectory text. S5. Introduce a user instruction text enhancement mechanism based on a large language model to enhance and reconstruct the multi-turn dialogue trajectory text generated by the three-agent collaborative architecture. S6. Data format and basic language quality are filtered through predefined deterministic rules, and a third language model is used to perform fine-grained semantic scoring on each dialogue data. A comprehensive quality score is generated according to preset weights, and samples with a comprehensive quality score not lower than a preset threshold are retained in the final dataset.

[0031] In step S1 of this embodiment, the API raw data sample includes at least the interface name, function description, parameter definition, and necessary parameter information. Standardization cleaning of the API raw data includes completing missing parameter function descriptions, standardizing parameter type definitions, converting unstructured fields, and merging or deduplication operations. Completing missing parameter function descriptions refers to supplementing parameter fields without natural language descriptions with descriptive information matching their functional attributes. Standardizing parameter type definitions refers to clearly defining and uniformly constraining the types of input parameters, covering four basic types: string, number (including integer and float), array, and object. Converting unstructured fields refers to converting unstructured fields into structured JSON format according to the OAS (OpenAPI Specification). Merging or deduplication operations involve using the pre-trained Chinese semantic embedding model gte-small-zh to vectorize the function description text of each API and calculating the cosine similarity between any two API vectors. API pairs with a similarity exceeding a preset threshold of 0.8 are considered semantically redundant and are merged or deduplicated. Ultimately, a standardized API metadata database with a unified structure, no semantic redundancy, and conforming to the OAS specification was constructed as the core foundation for generating subsequent multi-round dialogue data.

[0032] In step S2 of this embodiment, the preset prompt template Prompt mainly includes three core elements: sampled API text, constraints, and output format specifications. The sampled API text includes the API name, function description, and parameter information. The constraints refer to the specific requirements for the synthesis of a single intent or multiple intents. The output format specifications refer to the text that needs to be output in natural language or JSON format.

[0033] Structured input is constructed based on prompt word templates and fed into the first major language model. The first major language model generates user commands that comprehensively cover typical scenarios of multi-turn dialogue in intelligent cockpits. Among them, user command types are at least divided into single-intent type, parallel multi-intent type, and serial multi-intent type. Single-intent type means that the user expresses a single and clear goal, corresponding to one API call; parallel multi-intent type means that the user puts out multiple independent and undependent intents, corresponding to multiple API calls that can be executed concurrently; serial multi-intent type means that the user puts out multiple intents with logical dependencies or sequential constraints, requiring multiple API calls in a specific order.

[0034] For example, based on the input air conditioning switch API (aircondition_switch), the following user commands can be synthesized: "Turn on the driver's side air conditioning", "Turn off the passenger side and rear air conditioning", "Turn off the rear air conditioning first, then turn on the driver's side air conditioning".

[0035] In step S3 of this embodiment, the user command output by the first language model is input into the second language model for structured planning of the multi-turn dialogue process. The structured planning information of the multi-turn dialogue process output by the second language model includes at least the total number of turns, the interaction content of each turn, and the interaction type of each turn. The interaction content simply describes the core semantics of the turn, such as "the user requests to turn on the air conditioner." The interaction type includes two types: non-tool call type and tool call type. Non-tool call type refers to the interaction type where the user's request does not require calling an API and can be responded to directly in natural language; tool call type refers to the interaction type where the user's request requires calling an API at least once. This step, by explicitly modeling the turn structure of the dialogue, ensures that the subsequently synthesized multi-turn dialogue possesses the task-oriented nature and interactive coherence of a real in-vehicle scenario.

[0036] In step S4 of this embodiment, in the constructed three-agent collaborative architecture, the user agent, based on a general large language model such as GPT-4 or Grok, simulates user behavior through prompt engineering. The user agent dynamically loads specific user profiles according to a pre-built multi-dimensional user profile template library, thereby constructing role prompts. The user profile template library contains information such as age, occupation, and language style. For example, the loaded user profile might be "30-year-old male programmer, preferring concise instructions." The constructed role prompts include role settings, language style, and expression habit constraints based on the user profile, guiding the model to generate user instruction text that conforms to the specific profile. This method constructs prompt words through diverse combinations of user profiles, eliminating the need for individual model fine-tuning for each user profile.

[0037] In the three-agent collaborative architecture, the assistant agent simulates assistant role behavior through prompts based on a general large language model. For example, inputting a role prompt such as "You are a professional, friendly, and efficient in-vehicle voice assistant 'Xiao Zhi,' capable of accurately understanding user intent and providing concise and clear answers" into the general large language model yields the assistant agent; it can then generate natural language responses that conform to the role settings.

[0038] The tool agent in the three-agent collaborative architecture is based on a large language model that fine-tunes instructions on a specific tool invocation format (such as the OpenAIFunction Calling format). Its core function is to accurately generate structured API invocation instructions that conform to a predefined format based on user instructions and context. To ensure that the generated invocation instructions strictly follow the API metadata specifications provided by S100, the model also needs to undergo targeted fine-tuning training to stably and accurately map user intents to legitimate API calls.

[0039] The dialogue trajectory synthesis process based on the above three intelligent agents is as follows: Figure 2As shown, the input includes a pre-planned dialogue flow and selected API information. Communication messages between agents consist of three parts: a complete dialogue history, the current dialogue flow node, and information about the selected API.

[0040] Specifically, when the dialogue process is not a tool call, the user and assistant agents will be called in sequence to generate the corresponding question and answer interaction text; when the dialogue process is a tool call request, the user, tool and assistant agents will be called in sequence to generate the corresponding tool API call trajectory, forming a complete "request-call-feedback" closed loop.

[0041] The specific process is as follows: Input the API information for the dialogue flow sampling; The user agent initially loads the user profile based on the user profile template library; The user agent generates user command text based on the interaction content of the current round; Determine if the current round is a tool call type request; if so, it is handled by the tool agent; otherwise, it is handled by the assistant agent. The tool's intelligent agent parses user commands and generates structured API call text; The assistant agent understands user commands and generates natural language responses; Is the dialogue complete? If not, the user agent generates the instruction text again based on the content of the next round of interaction, and repeats the process until the dialogue is complete; if the dialogue is complete, the dialogue trajectory is output.

[0042] In step S5 of this embodiment, the user instruction text enhancement mechanism includes three methods, such as... Figure 3 As shown, these are context-aware generalization, spatial reference reconstruction, and intent multi-round decomposition, respectively.

[0043] Specifically, context-aware generalization refers to the multi-dimensional and controllable generalization of original user commands at the semantic or grammatical structure level, based on the current dialogue history, to generate more challenging ambiguous, ambiguous, or non-standard expressions. This simulates language variations caused by distraction, slips of the tongue, or habitual expressions in a real-world in-vehicle environment. Context-aware generalization includes three dimensions of transformation: grammatical expression transformation, entity substitution, and semantic association generalization.

[0044] Sentence structure transformation includes inversion, verb omission, and negative transposition. For example, inversion is "Can the air conditioner be turned on?" → "Can the air conditioner be turned on?"; verb omission is "Turn up the volume" → "Turn it up a bit more"; negative transposition is "Open the sunroof" → "Don't close the sunroof".

[0045] Entity substitution includes synonym substitution, vague reference, and pronoun substitution. Synonym substitution is such as "car window" → "window" or "glass". Vague reference is such as replacing a specific location with "there". Pronoun substitution is such as "it" referring to the navigation location mentioned earlier.

[0046] Semantic association generalization refers to the equivalent reconstruction of intent based on common sense or cultural background. For example, "I want to listen to 'Blue and White Porcelain'" → "I want to listen to that Chinese-style song about traditional porcelain" or "Turn on the air conditioner to heat" → "Make the car warmer".

[0047] Let the original user instruction be The historical dialogue context is The system selects the appropriate template from a pre-built library of generalized prompt word templates across three dimensions. , combined and Constructing structured prompts The prompt word is input into a large language model, which then outputs the generalized instruction. ,Right now:

[0048] Spatial reference reconstruction integrates the in-vehicle spatial topology (such as "driver's seat," "passenger's seat," and "rear left side") with the user's current location context, guiding the large language model to reconstruct explicit location commands into implicit or relative location expressions that conform to human spoken language habits. For example, when the system knows that the current user is in the [driver's seat], the original command containing explicit location, "adjust the passenger's air conditioning temperature to 22 degrees," can be rewritten as "adjust the air conditioning temperature next to me to 22 degrees," where "next to me" relies on spatial knowledge to infer that it refers to the [passenger's seat] area.

[0049] Let the original user instruction be The user's current location is The predefined spatial topology inside the vehicle is (Including each functional area and their interrelationships).

[0050] Spatial Relationship Reasoning: First, based on and Clues for constructing relative spatial position reasoning within the vehicle Input into a large language model and output the correct relative position description. .

[0051] Instruction reconstruction: combining the inferred relative position and original user instructions Build command to reconstruct prompt words The prompt word is input into the large language model, and the reconstructed instruction is output. .

[0052] This mechanism not only significantly improves the naturalness of the language, but also enables the training of large language models to have spatial context understanding capabilities, which is closer to the typical characteristics of "position omission but semantic integrity" in real intelligent cockpit voice interaction.

[0053] The intent multi-round decomposition decouples the original single-round user instruction into multiple semantically coherent sub-intents, which are then distributed across multiple rounds of user speech to simulate the step-by-step expression behavior of real users due to factors such as delayed thinking and gradual disclosure of information.

[0054] Specifically, a large language model is used to perform intent parsing on the original command. Based on the intent parsing results, a single command is broken down into a multi-round command sequence. Optionally, irrelevant semantic fragments, such as casual conversations between users in the car, can be inserted between the decomposed multi-round command sequences to simulate voice interaction behavior in real-world in-car scenarios.

[0055] For example, the original command "increase driver's seat heating" can be broken down as follows: Round 1: "First, turn on the driver's seat heater." Round 2: "Mom, are you feeling better?" Round 3: "Turn the temperature up a bit more." The specific implementation method is as follows: Let the original user instruction to be disassembled be ; Intent parsing: Utilizing large language models to... Perform intent parsing to identify the included Individual Intention And their potential order or dependencies.

[0056] Instruction generation and insertion: For each sub-intention Based on the context, a large language model is used to generate specific sub-instructions. Between adjacent sub-intents, preset or generated irrelevant content can be inserted. .

[0057] Output result: The final output is a multi-round instruction sequence. Complete the original instructions Multiple rounds of disassembly.

[0058] Each decomposed sub-instruction maintains local semantic integrity while relying on dialogue history to achieve overall intent convergence. This mechanism effectively enhances the training value of generated data for multi-turn state tracking, intent accumulation, and context fusion capabilities, significantly improving the long-range interaction robustness of the dialogue system.

[0059] In step S6 of this embodiment, the predefined deterministic rules for filtering data format and underlying language quality include: The structure and protocol compliance check detects whether there are any missing or disordered dialogue rounds, such as missing user / assistant role identifiers; it also checks whether the tool call instructions conform to the preset protocol format, such as missing function names or incorrect parameter types in the OpenAI FunctionCalling format.

[0060] Language standardization verification uses open-source grammar and style checking tools (such as LanguageTool) to detect and filter natural language text with obvious grammatical errors or semantic inconsistencies, ensuring that the language output meets basic readability standards.

[0061] Based on rule validation, a third large language model is further invoked to perform fine-grained semantic scoring (0-5 points) on each dialogue data. The scoring dimensions include: Contextual consistency: Whether the multiple interactions between the user and the assistant are logically coherent, without factual contradictions or breaks in referential meaning; Scenario matching: Whether the user commands reasonably correspond to the functional scenarios of the in-vehicle cockpit; Task difficulty reasonableness: Rate the difficulty of the tasks in the dialogue to avoid tasks that are impossible or contradictory under the current conditions; Natural language fluency: Whether the sentences conform to real spoken language habits, without sounding mechanical or awkwardly connected.

[0062] Each dimension is weighted and summed according to preset weights to generate a comprehensive quality score. Only samples with a total score greater than or equal to a threshold are retained in the final dataset.

[0063] It is worth noting that in this embodiment, multiple steps involve assisted processing using a large language model. In this embodiment, the entire process is uniformly configured as a single large language model. Basic model parameter configurations, including the context window length and temperature coefficient during inference, must be completed before each step execution. Specific prompt word templates are designed for each step. Different steps trigger the model to execute the corresponding task and generate the corresponding output results by inputting a unique prompt word template (information such as the task objective, input data format, and output requirements for that step).

[0064] Example 2 This embodiment provides a system for executing the multi-turn dialogue instruction synthesis method for a large-scale intelligent cockpit model agent described in Embodiment 1 above. The system includes an API data processing and metadata database construction module, a user instruction generation module, a multi-turn dialogue flow planning module, a multi-turn dialogue trajectory text synthesis module, a dialogue trajectory text enhancement and reconstruction module, and a data quality assessment and filtering module. The API data processing and metadata database construction module acquires the raw data from the application programming interface (API) documentation of the smart cockpit. This raw data undergoes standardization and cleaning to remove noise, errors, and inconsistent data formats, ensuring accuracy and consistency. The processed data is then used to construct a structured API metadata database, enabling subsequent modules to easily query and utilize this API data.

[0065] The user command generation module internally stores preset prompt word templates, which provide a framework and guidance for generating user commands. It selects one or more API samples from the API metadata database, combines these sample data with the prompt word templates to construct structured input, and leverages the powerful text generation capabilities of a large language model to generate task-oriented user commands. These commands clearly define the tasks the user wants the intelligent cockpit to complete.

[0066] The multi-turn dialogue flow planning module receives user instructions generated by the first major language model. Based on these instructions, it uses the second major language model to perform structured planning of the multi-turn dialogue flow. The second major language model analyzes the user instructions, plans the multi-turn dialogue steps, sequence, and general content direction of each turn required to complete the task, and forms a structured multi-turn dialogue flow planning result, providing guidance for subsequent dialogue simulations.

[0067] The multi-turn dialogue trajectory text synthesis module employs a three-agent collaborative architecture: a user agent, an assistant agent, and a tool agent. The user agent simulates a real user posing needs and questions, the assistant agent simulates the intelligent assistant in a smart cockpit providing responses and operations, and the tool agent is responsible for calling relevant API tools to complete specific tasks. Based on the structured planning results of the multi-turn dialogue process, the three agents simulate a real in-vehicle voice interaction process, synthesizing multi-turn dialogue trajectory text through mutual information transmission and interaction, recording the content of each round of dialogue throughout the entire process.

[0068] The dialogue trajectory text enhancement and reconstruction module introduces a user command text enhancement mechanism based on a large language model to enhance and reconstruct the multi-turn dialogue trajectory text generated by the three-agent collaborative architecture. Leveraging the analysis and generation capabilities of the large language model, the module optimizes the dialogue trajectory text by increasing dialogue diversity, enriching dialogue details, and correcting grammatical errors, making the generated dialogue trajectory text more natural, accurate, and aligned with real-world scenario requirements.

[0069] The data quality assessment and filtering module uses predefined deterministic rules to filter the enhanced and reconstructed multi-turn dialogue trajectory text for data format and basic language quality, removing data that does not meet the requirements, such as data with format errors or serious language errors. Then, a third language model is used to perform fine-grained semantic scoring on each piece of dialogue data after the initial filtering, evaluating it from multiple dimensions such as semantic accuracy, coherence, and reasonableness. A comprehensive quality score is generated according to preset weights for each dimension. Samples with a comprehensive quality score not lower than a preset threshold are retained and included in the final dataset, ensuring the quality and usability of the final dataset.

[0070] Example 3 This embodiment provides specific examples of the methods and systems used in Embodiments 1 and 2.

[0071] In this embodiment, a sample of raw API data is given in Table 1 below: Table 2

[0072] This embodiment also provides a standardized API example in cleaned JSON format, as shown in Table 2 below: Table 2

[0073] Two APIs were sampled from the "On-board Equipment Control" API subset: ac_switch: Turns the vehicle's air conditioning on or off at the specified location; seat_massager_switch: Turns seat massage on / off at the specified location.

[0074] The constructed prompts are shown in Table 3 below: Table 3

[0075] Inputting the above prompt into the first selected large model (GPT-4o) will output the user command: "Turn on the driver's air conditioning and turn off the passenger's seat massage." Input the user question "Turn on the driver's air conditioning and turn off the passenger's seat massage" into the second large model (GPT-4o), and ask it to plan a 3-turn dialogue process. The output structured process is shown in Table 4 below: Table 4

[0076] User agent profile: 35-year-old male, Beijing office worker, concise language, with Northern Chinese accent; The role of the assistant intelligent agent: the voice assistant "Xiao Zhi".

[0077] API Calling Protocol for Tool Agents: OpenAI Function Calling The synthesized dialogue trajectory is shown in Table 5 below: Table 5

[0078] Enhance the user turn-by-turn instruction text, including: Context-aware generalization: Natural generalization is achieved through functional semantic substitution and component omission: "air conditioner" → function refers to "blowing air", and users often refer to the device by its effect; "seat massage" → omit the subject "seat" and retain the core functional word "massage", which conforms to spoken language habits.

[0079] Spatial reference reconstruction: Given that the current user is located in the [driver's seat], the relative position is reconstructed based on the driver's viewpoint: "Driver's seat" → "My side" From the driver's perspective, the driver's seat is the driver's current position, referred to in the first person; "Passenger's seat" → "Right side" From the driver's perspective, the passenger's seat is located on the right side.

[0080] Intended multi-round breakdown: The original single-round instruction "Turn on the driver's side air conditioning and turn off the passenger side seat massage" is broken down into multiple rounds: Round 1: "Let's turn on the air conditioning in the driver's seat first." Round 2: "Turn off the massage function on the passenger seat." Table 6 provides examples from three different dimensions: Table 6

[0081] The tool call format conforms to the OpenAI Function Calling specification, and LanguageTool is used to perform grammar and spell checks on all natural language text.

[0082] Multidimensional evaluation prompts were constructed using the large model GPT-4o to score the complete dialogue trajectory. The results are shown in Table 7 below: Table 7

[0083] If the overall score is greater than the set threshold of 4.5, the sample passes the quality screening and enters the final dataset.

[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for synthesizing multi-turn dialogue instructions for intelligent agents in large-scale intelligent cockpit models, characterized in that: The method includes the following steps: S1. Obtain the raw data of the application programming interface (API) document for the intelligent cockpit, perform standardized cleaning and processing, and construct a structured API metadata database. S2. Establish a preset prompt word template, use one or more API samples from the API metadata database, construct structured input based on the prompt word template, and generate task-oriented user instructions through the first language model; S3. Based on the user instructions generated by the first language model, use the second language model to perform structured planning of the multi-turn dialogue process; S4. Construct a three-agent collaborative architecture including a user agent, an assistant agent, and a tool agent. The three-agent collaborative architecture simulates the real in-vehicle voice interaction process based on the structured planning results of the multi-turn dialogue process and synthesizes the multi-turn dialogue trajectory text. S5. Introduce a user instruction text enhancement mechanism based on a large language model to enhance and reconstruct the multi-turn dialogue trajectory text generated by the three-agent collaborative architecture. S6. Data format and basic language quality are filtered through predefined deterministic rules, and a third language model is used to perform fine-grained semantic scoring on each dialogue data. A comprehensive quality score is generated according to preset weights, and samples with a comprehensive quality score not lower than a preset threshold are retained in the final dataset.

2. The method for synthesizing multi-turn dialogue instructions for intelligent cockpit large-scale model intelligent agents according to claim 1, characterized in that: In step S1, the API raw data sample includes at least the interface name, function description, parameter definition, and necessary parameter information; Standardized cleaning of raw API data includes completing missing parameter descriptions, standardizing parameter type definitions, converting unstructured fields, and merging or deduplication operations. Completing missing parameter descriptions involves adding descriptive information matching the functional attributes of parameter fields that lack natural language descriptions. Standardizing parameter type definitions involves clearly defining and uniformly constraining the types of input parameters, covering four basic types: strings, numbers, lists, and objects. Converting unstructured fields involves converting unstructured fields to structured JSON format according to OAS specifications. Merging or deduplication operations involve using a pre-trained semantic embedding model to vectorize the functional description text of each API, calculating the cosine similarity between any two API vectors, and merging or deduplicating texts with similarity exceeding a threshold.

3. The method for synthesizing multi-turn dialogue instructions for intelligent cockpit large-scale model intelligent agents according to claim 1, characterized in that: In step S2, the preset prompt word template includes sampled API text, constraints, and output format specifications. The sampled API text includes API name, function description, and parameter information. The constraints refer to the specific requirements for the synthesis of single or multiple intentions. The output format specifications refer to the text that needs to be output in natural language or JSON format. Structured input is constructed based on prompt word templates and fed into the first major language model. The first major language model generates user instructions that comprehensively cover typical scenarios of multi-turn dialogue in intelligent cockpits. Among them, user instruction types are at least divided into single-intent type, parallel multi-intent type, and serial multi-intent type. Single-intent type means that the user expresses a single and clear goal, corresponding to one API call; parallel multi-intent type means that the user puts out multiple independent and undependent intents, corresponding to multiple API calls that can be executed concurrently; serial multi-intent type means that the user puts out multiple intents with logical dependencies or sequential constraints, requiring multiple APIs to be called in a specific order.

4. The method for synthesizing multi-turn dialogue instructions for intelligent cockpit large-scale model intelligent agents according to claim 1, characterized in that: In step S3, the user commands output by the first language model are input into the second language model for structured planning of the multi-turn dialogue process. The structured planning information of the multi-turn dialogue process output by the second language model includes at least the total number of turns, the interaction content of each turn, and the interaction type of each turn. The interaction content refers to a simple description of the core interaction semantics of the turn. The interaction type includes two types: non-tool call type and tool call type. Non-tool call type refers to the interaction type in which the user's request does not need to call the API and can be responded to directly in natural language. Tool call type refers to the interaction type in which the user's request needs to call the API at least once.

5. The method for synthesizing multi-turn dialogue instructions for intelligent cockpit large-scale model intelligent agents according to claim 1, characterized in that: In step S4, the user agent simulates user behavior through prompting engineering based on a general large language model; The user agent dynamically loads specific user profiles based on a pre-built multi-dimensional user profile template library, thereby constructing role prompts; The assistant agent simulates assistant behavior through prompting engineering based on a general large language model; The tool agent is based on a large language model that fine-tunes instructions on a specific tool call format. It generates structured API call instructions that conform to a predefined format based on user instructions and context.

6. The method for synthesizing multi-turn dialogue instructions for intelligent cockpit large-scale model intelligent agents according to claim 5, characterized in that: The process of synthesizing multi-turn dialogue trajectory text based on a three-agent collaborative architecture is as follows: Input the API information for the dialogue flow sampling; The user agent initially loads the user profile based on the user profile template library; The user agent generates user command text based on the interaction content of the current round; Determine if the current round is a tool call type request; if so, it is handled by the tool agent; otherwise, it is handled by the assistant agent. The tool's intelligent agent parses user commands and generates structured API call text; The assistant agent understands user commands and generates natural language responses; Is the dialogue over? If not, the user agent generates the command text again based on the content of the next round of interaction, and repeats the process until the dialogue is over. Once the dialogue is complete, the dialogue sequence will be output.

7. The method for synthesizing multi-turn dialogue instructions for intelligent cockpit large-scale model intelligent agents according to claim 1, characterized in that: In step S5, the user command text enhancement mechanism includes three methods: context-aware generalization, spatial reference reconstruction, and intent multi-turn decomposition. Context-aware generalization refers to the multi-dimensional, controllable generalization of original user commands at the semantic or grammatical structure level, based on the current dialogue history, to generate ambiguous, ambiguous, or non-standard expressions. This simulates language variations caused by distraction, slips of the tongue, or habitual expressions in a real-world in-vehicle environment. Context-aware generalization includes three dimensions of transformation: grammatical expression transformation, entity substitution, and semantic association generalization. Specifically, it is represented as follows: Let the original user instruction be The historical dialogue context is Select the appropriate template from the pre-built three-dimensional generalized prompt word template library. , combined and Constructing structured prompts The prompt word is input into a large language model, which then outputs the generalized instruction. ,Right now ; Spatial reference reconstruction integrates the in-vehicle spatial topology with the user's current location context, guiding the large language model to reconstruct explicit location commands into implicit or relative location expressions that conform to human spoken language habits; let the original user command be... The user's current location is The predefined spatial topology inside the vehicle is Spatial relationship reasoning: First, based on and Clues for constructing relative spatial position reasoning within the vehicle Input to a large language model and output the correct relative position description. Instruction reconstruction: combining the inferred relative position and original user instructions Build command to reconstruct prompt words The prompt word is input into the large language model, and the reconstructed instruction is output. ; The intent decomposition method addresses the original single-turn user instruction by structurally decoupling it into multiple semantically coherent sub-intents, which are then distributed across consecutive rounds of user speech. This simulates the step-by-step expression behavior of real users due to factors such as delayed thinking and gradual information disclosure. Let the original user instruction to be decomposed be... Using large language models to Perform intent parsing to identify the included Individual Intention and their potential order or dependencies; for each sub-intent Based on the context, a large language model is used to generate specific sub-instructions. For adjacent sub-intents, preset or generated irrelevant content can be inserted. The final output is a multi-round instruction sequence. Complete the original instructions Multiple rounds of disassembly.

8. The method for synthesizing multi-turn dialogue instructions for intelligent cockpit large-scale model intelligent agents according to claim 1, characterized in that: In step S6, the predefined deterministic rules filter the data format and basic language quality, including structure and protocol compliance checks and language standardization checks. The structure and protocol compliance checks detect whether there are missing or disordered dialogue turn structures. The language standardization checks use open-source grammar and style checking tools to detect and filter natural language text with obvious grammatical errors or semantic incoherence, ensuring that the language output meets basic readability standards. Based on the rule verification, a third large language model is called to perform fine-grained semantic scoring on each dialogue data. The scoring dimensions include context consistency, scene matching, reasonable task difficulty, and natural language fluency. Each dimension is weighted and summed according to preset weights to generate a comprehensive quality score. Only when the total score is greater than or equal to the threshold is the sample retained into the final dataset.

9. A system for executing the multi-turn dialogue instruction synthesis method for intelligent cockpit large-scale model intelligent agents as described in any one of claims 1-8, characterized in that: The system includes an API data processing and metadata database construction module, a user command generation module, a multi-turn dialogue flow planning module, a multi-turn dialogue trajectory text synthesis module, a dialogue trajectory text enhancement and reconstruction module, and a data quality assessment and filtering module. The API data processing and metadata database construction module obtains the raw data of the application programming interface (API) document of the smart cockpit, performs standardized cleaning and processing, and constructs a structured API metadata database. The user instruction generation module establishes a preset prompt word template, uses one or more API samples from the API metadata database, constructs structured input based on the prompt word template, and generates task-oriented user instructions through the first language model. The multi-turn dialogue process planning module uses the second language model to perform structured planning of the multi-turn dialogue process based on the user instructions generated by the first language model. The multi-turn dialogue trajectory text synthesis module constructs a three-agent collaborative architecture including a user agent, an assistant agent, and a tool agent. The three-agent collaborative architecture simulates the real in-vehicle voice interaction process based on the structured planning results of the multi-turn dialogue process and synthesizes the multi-turn dialogue trajectory text. The dialogue trajectory text enhancement and reconstruction module introduces a user command text enhancement mechanism based on a large language model to enhance and reconstruct the multi-turn dialogue trajectory text generated by the three-agent collaborative architecture. The data quality assessment and screening module filters data format and basic language quality through predefined deterministic rules, and uses a third language model to perform fine-grained semantic scoring on each dialogue data. It generates a comprehensive quality score according to preset weights, and samples with a comprehensive quality score not lower than a preset threshold are retained in the final dataset.

10. An electronic device, characterized in that: It includes a memory storing a computer program for executing the multi-turn dialogue instruction synthesis method for intelligent cockpit large-scale model intelligent agents as described in any one of claims 1-8.

Images