A tool retrieval data generation method, system and device for a large model intelligent agent

Abstract

The application relates to a cockpit API retrieval training data generation method and system based on cluster iteration screening and hard negative mining, and belongs to the field of artificial intelligence and intelligent cockpits. A standardized natural language query instruction is generated according to an API description file of a cockpit system to serve as initial semantic Embedding, and a diversified generalization query instruction set is generated. Cluster analysis is performed on query instructions that fail to correctly match target APIs, representative Embedding of a preset number is obtained through iterative feature stripping screening, the coverage capability for long-tail semantic features is enhanced, candidate results highly similar to but not consistent with the target API are mined from model prediction results as hard negatives, training data samples containing positive examples and hard negatives are constructed, and a data set for tool retrieval model training is generated. The application reduces the risk of misidentification of similar function interfaces and is suitable for intelligent cockpit application scenarios with high requirements for safety and robustness.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and intelligent cockpit technology, and relates to a method, system and device for generating training data for tool retrieval of large model intelligent agents. It belongs to the training data generation technology for natural language command and function interface retrieval in high-safety scenarios such as intelligent cockpits, and particularly relates to a method, system and device for generating training data for cockpit API retrieval based on clustering iterative screening and hard negative example mining. Background Technology

[0002] With the increasing adoption of smart cockpit technology, users are increasingly demanding the ability to directly control in-vehicle devices (such as air conditioning, navigation, and windows) via natural language. This requires in-vehicle voice assistants to accurately map users' spoken commands to corresponding function interfaces or API calls in the vehicle's backend. To achieve this reliable mapping from natural language to function interfaces, high-quality and comprehensive training data has become one of the key factors affecting system performance.

[0003] To address the low accuracy of command recognition in early in-vehicle voice interaction, a joint model for intent recognition and slot filling based on slot-gated technology was proposed in 2018. This technique improves recognition accuracy within specific sentence structures and controlled semantic ranges by constructing predefined semantic slot templates and establishing explicit correspondences between user commands and API parameters. However, such template- and rule-based methods rely heavily on manually defined syntactic structures. When faced with diverse and non-standardized natural language commands in smart cockpit scenarios (e.g., indirectly expressing "open the window" through "it's too stuffy"), the lack of corresponding template coverage significantly limits its generalization ability, making it difficult to adapt to the complexity and diversity of real-world usage scenarios.

[0004] To address the insufficient generalization ability of template-based methods, related research has begun to explore the use of Large Language Models (LLMs) to generate synthetic training data. In 2022, the "Self-Instruct" framework was proposed. This method guides the LLM to automatically generate large-scale instruction data using a small number of seed instructions, reducing manual annotation costs and increasing data scale to some extent. However, subsequent practice has shown that this type of generation method, based on random sampling or simple prompts, easily leads to uneven distribution of generated data in the semantic space, resulting in the so-called "pattern collapse" phenomenon. This means that instruction content is overly concentrated in high-frequency semantic regions (such as repeatedly generating expressions like "turn on the air conditioner" or "turn on the cooling"), while insufficiently covering long-tail instructions that are used less frequently but are functionally important in smart cockpits (such as "turn down the passenger seat heating level"). This lack of long-tail semantics causes the trained model to degrade in performance when handling complex or obscure instructions.

[0005] To further improve the performance of retrieval models in semantic matching tasks, the industry has introduced contrastive learning and negative sampling techniques. In 2020, Dense Passage Retrieval (DPR) was proposed, which employs an in-batch negative sampling strategy during training, treating other samples in the same batch as negative examples to improve the model's discriminative ability. While this method has achieved good results in tasks such as open-domain question answering, it still has certain limitations in the specific application scenario of intelligent cockpit API retrieval. Because cockpit systems often contain many API interfaces with highly similar functional semantics but different execution consequences (e.g., "open sunroof" vs. "open window," "defrost" vs. "defog"), simple random negative sampling is insufficient to guide the model to learn and distinguish these subtle semantic differences, easily leading to functional confusion in practical applications, thus causing misoperation and affecting driving safety.

[0006] Based on the above analysis, existing technologies still generally suffer from uneven semantic coverage, insufficient long-tail instructions, and a lack of targeted and difficult-to-replicate examples during the training data generation process, making it difficult to meet the needs of application scenarios such as smart cockpits that have high requirements for safety and robustness. Summary of the Invention

[0007] To address the safety risks of functional misidentification that may arise from insufficient coverage of long-tail semantic distribution and lack of targeted difficult-to-bearing examples in the existing training data generation process, and considering the requirements of system robustness and recognition accuracy in intelligent cockpit application scenarios, this invention provides a cockpit API retrieval training data generation method based on clustering iterative screening and difficult-to-bearing example mining.

[0008] This method employs an iterative cluster center selection and data stripping strategy to filter the generated training instructions, thereby reducing semantic redundancy and enhancing coverage of long-tail semantic features. Simultaneously, it utilizes the high-confidence confusion results generated during model retrieval to automatically construct high-discrimination difficult negative example samples, thus generating a high-quality training dataset for tool retrieval model training, overcoming the aforementioned problems in existing technologies.

[0009] To achieve the above objectives, the present invention provides the following technical solution:

[0010] A method for generating training data for cockpit API retrieval based on clustering iterative screening and hard negative example mining, the method comprising the following steps: S1: Based on the API description file of the cockpit system, process the functional description of each tool interface, generate standardized natural language query instructions corresponding to each API, and use the standardized natural language query instructions as the initial semantic embedding. S2: Based on the API description file and the initial semantic embedding, the standardized natural language query instructions are semantically generalized and extended using a large language model to generate a set of generalized query instructions containing multiple forms of expression; S3: Extract features from the generalized query instruction set, and perform retrieval reasoning on the generalized query instructions based on the tool retrieval model. Filter query instructions that fail to correctly match the target API according to the retrieval results, perform cluster analysis on the incorrectly matched query instructions, select the cluster with the largest number of samples from the clustering results, and select the query instruction with the highest semantic similarity to the cluster center from the cluster as the dominant query instruction. Add the dominant query instruction to the Embedding set. S4: Repeat S3 until a preset number of Embeddings are obtained from the set of generalized query instructions; S5: Based on the Embedding obtained from the screening, perform retrieval reasoning on the generalized query command, obtain the top-ranked candidate APIs output by the model, mark the candidate APIs that are inconsistent with the target API but have high semantic similarity as difficult negative examples, and construct training data samples together with the corresponding target API positive examples to generate a training dataset for training the tool retrieval model.

[0011] Furthermore, the clustering analysis in step S3 employs a clustering algorithm based on vector semantic representation to cluster the vector sets that do not correctly match the query command, and measures the vector distance based on the cosine similarity between vectors.

[0012] Furthermore, the vector distance is expressed as follows:

[0013] in: For a set of vectors In the vector, n is the dimension of the vector. It is a vector The first in A number, For vectors The first in A number.

[0014] The goal of the K-Means algorithm is to minimize the sum of squared errors, and the total error of the objective is expressed by the following formula:

[0015] Where: J is the total error of the target, and k is the number of clusters. It refers to the j-th cluster. It is the mean vector of the j-th cluster, i.e., the centroid of the cluster.

[0016] Furthermore, in each round of clustering iteration, by stripping the cluster to which the selected dominant query instruction belongs, the subsequent iteration process is guided to focus on the long-tail semantic features of the remaining query instructions.

[0017] Furthermore, the preset number of Embeddings is k semantic Embeddings set according to the number of API functions or semantic complexity.

[0018] Furthermore, the top-ranked candidate APIs in step S5 are the Top-5 candidate APIs with the highest confidence in model prediction.

[0019] A cockpit API retrieval training data generation system based on clustering iterative screening and hard negative example mining includes: The query instruction generation module is used to generate standardized natural language query instructions based on the API description file and form the initial semantic embedding. A generalization extension module is used to generate a set of generalized query instructions based on the standardized natural language query instructions; The Embedding filtering module is used to perform cluster analysis on query commands that do not match correctly, and to filter out a preset number of Embeddings through an iterative feature stripping method. The difficult-to-bearing-example mining module is used to construct training data samples containing positive example APIs and difficult-to-bearing-example APIs based on the model retrieval results; The dataset generation module is used to output the training dataset for training the tool retrieval model.

[0020] This invention provides a training data generation method and system based on clustering iterative screening and hard negative example mining, and constructs a corresponding automated data generation process. Compared with existing training data generation techniques based on rule templates or random sampling of large models, this invention has the following technical advantages in terms of training data semantic coverage, discriminative ability, and generation efficiency: 1. Improve semantic coverage through clustering iterative screening mechanism The algorithm employs a "clustering-maximum cluster stripping-iterative filtering" approach. During data filtering, it guides the algorithm to focus on semantic features that are not yet fully covered, thereby reducing high-frequency semantic redundancy and enhancing coverage of long-tail semantics. The training data generated through this mechanism has a more uniform distribution in the semantic space, covering most cockpit API functions and their corresponding unconventional colloquial expressions, which helps alleviate the problem of functions not being correctly triggered due to differences in user expression.

[0021] 2. Improve semantic discriminative ability by constructing difficult negative examples based on retrieval confusion results. By constructing difficult negative example samples using high-confidence obfuscated results generated during model retrieval, the model is guided to focus on learning subtle semantic differences between similar functional interfaces during training. Practical results show that the training data generated based on the method of this invention can effectively reduce the model's misrecognition in high-risk obfuscated command scenarios, thereby improving the operational reliability and safety in intelligent cockpit applications.

[0022] 3. Reduce data building costs through automated data generation processes. A data generation process combining standardized query generation, generalization expansion, and automatic filtering reduces reliance on manual annotation. When a new functional interface is added to the cockpit system, only the corresponding API description information needs to be imported to automatically generate matching training data, thereby improving the efficiency of training data construction and updating. This approach is suitable for the application needs of continuous expansion and rapid iteration of intelligent cockpit functions.

[0023] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0024] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a schematic diagram of the overall process of the cockpit API retrieval training data generation method based on clustering iterative screening and hard negative example mining according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the API definition document for an embodiment of the present invention; Figure 3 This is a schematic diagram of complex API data in an embodiment of the present invention; Figure 4 This is a schematic diagram of the specific process of Step 3 in an embodiment of the present invention. Detailed Implementation

[0025] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0026] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0027] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0028] As intelligent cockpit functions become increasingly complex, natural language-based vehicle control is gradually becoming an important way to enhance user interaction. However, existing methods for generating large-scale model training data generally suffer from uneven distribution of training data, insufficient coverage of key long-tail instructions, and a lack of targeted negative samples, making it difficult to meet the requirements of accuracy and safety in function recognition in intelligent cockpit applications. Therefore, this invention focuses on how to achieve high coverage, high discriminative power, and low cost in generating API retrieval training data in a cockpit environment. Building upon existing large-scale model data synthesis techniques, an improved scheme combining data filtering mechanisms and difficult negative example construction is proposed.

[0029] Meanwhile, for the specific application scenario of in-vehicle cockpits, which has high requirements for safety and recognition accuracy, this invention designs a data generation process based on clustering iterative screening and mining of difficult negative examples (high-ranking confusing samples in the model retrieval results). This addresses the problems that traditional random sampling generation methods easily lead to, such as the lack of long-tail low-frequency commands (e.g., obscure vehicle settings) and the model's insufficient ability to distinguish similar functional APIs (e.g., misidentifying "open sunroof" as "open car window"). This process reduces semantic redundancy and enhances feature coverage through iterative cluster center selection and data stripping strategies. It also automatically constructs highly discriminative difficult negative example samples using the confusion results generated during model retrieval, thereby making the generated training data more evenly distributed and with clearer boundaries in the semantic space. This helps reduce the risk of false wake-ups and misoperations in cockpit interactions and improves the efficiency of model training and iteration.

[0030] Unlike traditional training data generation methods based on fixed rule templates or simply relying on random sampling from large models, this invention, for the cockpit API retrieval task, adopts a technical approach combining clustering iterative screening and hard negative example mining to improve the feature coverage and discriminative ability of the generated training data in the semantic space. This technical approach mainly includes the following two aspects: I. Embedding Filtering Mechanism Based on Iterative Feature Stripping Based on the structured description of the cockpit API, corresponding standard and regular query commands (Query) are generated. Multiple generalized query commands (Query) are then generated using a large model based on these standard and regular query commands, forming a set of generalized query commands. Instead of randomly sampling this set, an iterative filtering method involving "clustering—selecting the largest cluster center—stripping the data from that cluster" is used to process the generalized query commands. In each iteration, dominant semantic features are identified from query commands that fail to match correctly under the current model and embedding combination, and the corresponding query commands are selected as new embeddings, until a preset number k of the most representative embeddings are obtained. This method reduces high-frequency semantic redundancy and enhances the coverage of long-tail, low-frequency command semantics, resulting in a more even distribution of the embedding set in the cockpit functional semantic space. An embedding refers to a vectorized semantic representation obtained by mapping the semantic information of natural language query commands or tool interfaces through a feature extraction model, used to characterize the feature position of the query command or tool interface in the semantic space.

[0031] II. Difficult-to-Hard Example Mining Mechanism Based on Model-Based Retrieval of Confused Results Using the multiple embeddings obtained from the above screening as benchmarks, pre-retrieval inference is performed on the generalized query command to obtain the Top-5 retrieval results with the highest prediction confidence output by the model; interfaces in the Top-5 retrieval results that are inconsistent with the target tool interface but have high semantic similarity are marked as high-risk, difficult-to-bear examples, for example, the retrieval result of "open car window" corresponding to the "open sunroof" command is marked as a difficult-to-bear example.

[0032] Based on this, training data triples are constructed, containing input commands, the corresponding correct APIs, and the top-5 obfuscated API hard-to-handle examples. By introducing these hard-to-handle example samples during model training, the model is guided to learn subtle semantic differences between similar functional interfaces, thereby improving the model's anti-interference ability and tool retrieval accuracy in complex application scenarios of intelligent cockpits.

[0033] Please see Figure 1This is a schematic diagram of the overall process of the cockpit API retrieval training data generation method based on clustering iterative screening and hard negative example mining according to an embodiment of the present invention. The method mainly includes the following steps: 1. Generate a regular query command based on the tool interface description file (API). Based on the API description file of the cockpit system, the functional descriptions of each tool interface are processed to generate standardized natural language query instructions corresponding to each API, and the standardized query instructions are used as the initial semantic embedding.

[0034] 2. Generate a set of generalized query instructions based on the API description and initial embedding. By combining the API description information and the initial embedding, a large model is used to generate generalized query instructions in various forms, thereby constructing a set of generalized query instructions.

[0035] 3. Filter dominant query commands based on cluster analysis and update embedding. Feature extraction is performed on the generalized query instruction set, and retrieval reasoning is performed based on the original tool retrieval model. For query instructions that fail to match the target API correctly in the retrieval results, clustering is performed. Query instructions with high semantic similarity are divided into the same cluster, and the query instruction with the highest similarity to the center of the cluster is selected from the cluster with the largest number of samples as the dominant query instruction. The dominant query instruction is added to the Embedding set.

[0036] 4. Repeat step 3 until the preset number K of Embeddings are obtained. The clustering and filtering process in step 3 is iterated until a preset number of k Embeddings are obtained from the set of generalized query instructions.

[0037] 5. Construct a training dataset based on the selected embeddings. The original tool retrieval model is trained and inferred using the filtered query instructions and embeddings. Based on the retrieval results output by the model, interfaces that are inconsistent with the target API in the top-ranked (e.g., top-5) candidate results are marked as negative examples, and the corresponding target APIs are marked as positive examples, thereby generating a training dataset for training the tool retrieval model.

[0038] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments: Step 1: API standardization and seed query command generation First, the API description file of the cockpit system is obtained as the raw input. The API description file includes metadata information of multiple API interfaces. The metadata information includes at least the interface identifier, parameter list and functional semantic description.

[0039] To reduce noise in the original descriptions and standardize semantic expression, the functional descriptions of each API are processed according to natural language processing rules, transforming them into standardized and regular natural language query instructions. These query instructions serve as seed query instructions for the corresponding APIs. In this embodiment, the above standardization process is implemented by calling a large language model, but this invention does not limit the specific implementation method.

[0040] For example, for the API interface Air Conditioner Temperature, a standardized seed query instruction "set the air conditioner temperature to [number] degrees" is generated based on its parameter constraint information, and the semantic representation obtained by feature extraction of the seed query instruction is used as the embedding corresponding to the API. Through the above steps, the initial semantic reference point of each API in the semantic space is established.

[0041] Step 2: Building a Generalized Query Instruction Pool Based on the seed query instructions and corresponding API description files generated in step 1, the seed query instructions and API description information are input into the Large Language Model (LLM) to perform semantic generalization extension on the seed query instructions, thereby generating a set of generalized query instructions for simulating real cockpit interaction scenarios.

[0042] To enhance the coverage of generalized query commands with actual user expressions, the query commands are classified and expanded from multiple dimensions during the generalization process. For example, query commands with diverse expressions are generated through different sentence types (including imperative sentences, interrogative sentences, rhetorical questions, and declarative sentences) and different language styles (including inversion, ellipsis, or vague reference).

[0043] Specifically, for different API interfaces, a generalized query instruction candidate pool containing multiple expression methods is adaptively generated based on factors such as the number of parameters and the complexity of the functional description. Please see. Figure 2 This is a schematic diagram of the API definition document for an embodiment of the present invention: For air conditioning temperature adjustment APIs with fewer parameters, a relatively small number of generalized query commands can be generated, such as 200; please refer to... Figure 3The diagram illustrates complex API data in this embodiment of the invention: For navigation control APIs with many parameters and complex functional descriptions, a relatively large number of generalized query commands are generated, such as 800. While this method can obtain a large set of generalized query commands, the semantic distribution of this set may still be unbalanced, with high-frequency semantic expressions accounting for too high a proportion and long-tail semantic features easily being submerged.

[0044] Step 3: Embedding Filtering Based on Iterative Feature Stripping To address the issues of uneven semantic distribution and the tendency for long-tail semantic features to be overwhelmed by high-frequency representations in the generalized query instruction set, this embodiment employs an embedding filtering method based on iterative feature stripping to select from the generalized query instruction candidate pool. The selection process yields a predetermined number of k representative embeddings. Please refer to [link / reference]. Figure 4 This is a schematic diagram of the specific process of Step 3 in an embodiment of the present invention.

[0045] The specific process includes the following steps: (1) Feature extraction and preliminary matching judgment.

[0046] First, a text embedding model is used to extract features from generalized query commands, converting each generalized query command into a vectorized semantic representation. In this embodiment, the text embedding model can be a pre-trained text embedding model (e.g., BGE-M3), which... Convert the text in the document into a vector representation. .

[0047] For each generalized query instruction q The system performs retrieval inference within the current API Embedding set to obtain the Top-1 retrieval results with the highest model prediction confidence. If the API corresponding to the Top-1 retrieval result is consistent with the Embedding of the target API before generalization, the generalized query instruction is marked as a correct match instruction. If they are inconsistent, the generalized query instruction is marked as an incorrect match instruction and used as the processing object for subsequent clustering analysis.

[0048] (2) Cluster analysis For the vector set corresponding to the incorrectly matched instructions Clustering analysis is performed to identify instruction groups with high semantic similarity. In this embodiment, the K-Means clustering algorithm can be used to cluster the vector set. Clustering is performed, and the vector distance is measured based on the cosine similarity between the vectors. It can be expressed as the following formula:

[0049] in: For a set of vectors In the vector, n is the dimension of the vector. It is a vector The first in A number, For vectors The first in A number.

[0050] The goal of the K-Means algorithm is to minimize the sum of squared errors, and the total error of the objective is expressed by the following formula:

[0051] in: J It is the total error of the target. k It is the number of clusters in the clustering. It refers to the j-th cluster. is the mean vector of the j-th cluster, i.e., the centroid of the cluster. The goal of K-Means clustering is to minimize the distance error between the sample vectors within each cluster and the corresponding cluster center vector, so as to obtain multiple semantically relatively concentrated clusters. The above clustering method is only an example, and this invention does not limit the specific form of clustering algorithm.

[0052] (3) Maximum cluster stripping and dominant instruction selection After completing the cluster analysis, identify the cluster with the largest number of samples. This cluster represents the dominant semantic modality in the current data distribution. The centroid vector corresponding to this cluster is calculated, and the query instruction with the highest semantic similarity to the centroid vector from this cluster is selected as the dominant query instruction. The dominant query instruction is added to the Embedding set E.

[0053]

[0054] (4) Iterative update Repeat steps (1) to (3) above, and in each iteration, continue to filter new dominant query instructions from the remaining generalized query instructions and add them to the Embedding set until the number of Embeddings in the Embedding set reaches a preset value k. Through the above iterative feature stripping process, the filtering mechanism is guided to gradually focus on the semantic regions that have not yet been covered, thereby enhancing the coverage of long-tail semantic features.

[0055] Step 4: Adversarial Difficulty Example Mining Based on Top-5 Search Obfuscation Results The embedding set E obtained in step 3 is used to construct a retrieval index. For each generalized query instruction q, retrieval inference is performed in the retrieval index to obtain the Top-5 candidate retrieval results with the highest prediction confidence output by the model.

[0056] Based on the Top-5 candidate retrieval results, construct training data samples with high discriminative power: training triples :in: Mark the real target API corresponding to the generalized query instruction as a positive example. ; APIs other than the target API in the Top-5 candidate search results are marked as difficult-to-probate examples. .

[0057] For example, when the input query is "open sunroof", if the model's Top-5 candidate results include "open car window", then the API corresponding to "open car window" is marked as a difficult negative example. In this way, difficult negative example samples are constructed based on the high similarity confusion results generated by the model during the retrieval process, highlighting the semantic boundaries between similar functional interfaces.

[0058] Step 5: Formatting and Outputting the Training Dataset The training samples containing positive and hard negative examples generated in step 4 are formatted to generate a training dataset for training the tool's retrieval model. A single sample in the training dataset includes at least: User query command text; The positive example API tag corresponding to the query command; One or more negative API tags corresponding to the query command.

[0059] By outputting the structured training dataset mentioned above, data support is provided for the training and optimization of the subsequent tool retrieval model.

[0060] Table 1 shows the recall performance of different data generation methods: Table 1

[0061] In summary, this invention addresses the common problems of insufficient long-tail semantic feature coverage and semantic confusion arising from similar functional interfaces in the generation of training data for existing intelligent cockpit language models. It provides a training data generation method, system, and apparatus based on clustering iterative filtering and difficult-to-distribute example mining. This technical solution effectively enhances the coverage of training data in the functional semantic space by iteratively stripping and filtering the generated query commands. Simultaneously, by constructing difficult-to-distribute example samples based on the model's retrieval confusion results, it improves the training data's ability to distinguish subtle semantic differences between similar functional interfaces.

[0062] Practical results show that the training data generated using the method of this invention outperforms traditional generation methods based on random sampling or rule templates in terms of semantic distribution uniformity and hard-negative example discrimination, which helps improve the stability and reliability of tool retrieval models in complex cockpit application scenarios. Through the above methods, this invention can improve the efficiency of training data generation while reducing the cost of manual intervention, making it suitable for the application needs of continuous expansion of intelligent cockpit functions and rapid model iteration, and has good practical value and promising prospects for promotion.

[0063] This invention also proposes a cockpit API retrieval training data generation system based on clustering iterative screening and hard negative example mining, which mainly includes: The query instruction generation module is used to generate standardized natural language query instructions based on the API description file and form the initial semantic embedding. A generalization extension module is used to generate a set of generalized query instructions based on the standardized natural language query instructions; The Embedding filtering module is used to perform cluster analysis on query commands that do not match correctly, and to filter out a preset number of Embeddings through an iterative feature stripping method. The difficult-to-bearing-example mining module is used to construct training data samples containing positive example APIs and difficult-to-bearing-example APIs based on the model retrieval results; The dataset generation module is used to output the training dataset for training the tool retrieval model.

[0064] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for generating training data for cockpit API retrieval based on clustering iterative screening and hard negative example mining, characterized in that, The method includes the following steps: S1: Based on the API description file of the cockpit system, process the functional description of each tool interface, generate standardized natural language query instructions corresponding to each API, and use the standardized natural language query instructions as the initial semantic embedding. S2: Based on the API description file and the initial semantic embedding, the standardized natural language query instructions are semantically generalized and extended using a large language model to generate a set of generalized query instructions containing multiple forms of expression; S3: Extract features from the generalized query instruction set, and perform retrieval reasoning on the generalized query instructions based on the tool retrieval model. Filter query instructions that fail to correctly match the target API according to the retrieval results, perform cluster analysis on the incorrectly matched query instructions, select the cluster with the largest number of samples from the clustering results, and select the query instruction with the highest semantic similarity to the cluster center from the cluster as the dominant query instruction. Add the dominant query instruction to the Embedding set. S4: Repeat S3 until a preset number of Embeddings are obtained from the set of generalized query instructions; S5: Based on the Embedding obtained from the screening, perform retrieval reasoning on the generalized query command, obtain the top-ranked candidate APIs output by the model, mark the candidate APIs that are inconsistent with the target API but have high semantic similarity as difficult negative examples, and construct training data samples together with the corresponding target API positive examples to generate a training dataset for training the tool retrieval model.

2. The method for generating training data for cockpit API retrieval based on clustering iterative screening and hard negative example mining according to claim 1, characterized in that: The clustering analysis in step S3 uses a clustering algorithm based on vector semantic representation to cluster the vector sets that do not correctly match the query command, and measures the vector distance based on the cosine similarity between the vectors.

3. The method for generating training data for cockpit API retrieval based on clustering iterative screening and hard negative example mining according to claim 2, characterized in that: The vector distance is expressed by the following formula: in: For a set of vectors In the vector, n is the dimension of the vector. It is a vector The first in A number, For vectors The first in A number. The goal of the K-Means algorithm is to minimize the sum of squared errors, and the total error of the objective is expressed by the following formula: in: J It is the total error of the target. k It is the number of clusters in the clustering. It refers to the j-th cluster. It is the mean vector of the j-th cluster, i.e., the centroid of the cluster.

4. The method for generating training data for cockpit API retrieval based on clustering iterative screening and hard negative example mining according to claim 1, characterized in that: In each round of clustering iteration, by stripping the cluster to which the selected dominant query instruction belongs, the subsequent iteration process is guided to focus on the long-tail semantic features of the remaining query instructions.

5. The method for generating training data for cockpit API retrieval based on clustering iterative screening and hard negative example mining according to claim 1, characterized in that: The preset number of Embeddings is k semantic Embeddings set according to the number of API functions or semantic complexity.

6. The method for generating training data for cockpit API retrieval based on clustering iterative screening and hard negative example mining according to claim 1, characterized in that: The top-ranked candidate APIs in step S5 are the Top-5 candidate APIs with the highest confidence in model prediction.

7. A cockpit API retrieval training data generation system based on clustering iterative screening and hard negative example mining, characterized in that, include: The query instruction generation module is used to generate standardized natural language query instructions based on the API description file and form the initial semantic embedding. A generalization extension module is used to generate a set of generalized query instructions based on the standardized natural language query instructions; The Embedding filtering module is used to perform cluster analysis on query commands that do not match correctly, and to filter out a preset number of Embeddings through an iterative feature stripping method. The difficult-to-bearing-example mining module is used to construct training data samples containing positive example APIs and difficult-to-bearing-example APIs based on the model retrieval results; The dataset generation module is used to output the training dataset for training the tool retrieval model.

8. An electronic device comprising a processor and a memory, wherein the memory stores a computer program, characterized in that: When the computer program is executed on the processor, it causes the electronic device to perform the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When executed by a processor, the computer program is used to implement the method described in any one of claims 1 to 6.

Images