Agent training data set construction method and system combined with cross-modal learning

Abstract

The application discloses an agent training data set construction method and system combined with cross-modal learning, relates to the technical field of agent training, and comprises the following steps: acquiring multi-modal intelligence data from a multi-source database and extracting feature information, projecting the feature information to a preset semantic space to obtain cross-modal alignment features; constructing an agent function call syntax tree based on the cross-modal alignment features, analyzing the agent function call syntax tree into an instruction sequence, constructing an analysis and reasoning chain, generating sample labels and interaction trajectories; constructing a task dependency graph using the sample labels and the interaction trajectories, decomposing the task dependency graph into multiple parallel decision branches, dynamically adjusting a processing strategy to form a training data set; mapping the training data set to an agent target space, optimizing the analysis and reasoning chain through a training feedback channel, and outputting a standardized sample library; and finally inputting to-be-analyzed data into a trained agent, generating an intelligence analysis report according to the analysis and reasoning chain.

Description

Technical Field

[0001] This invention relates to the field of intelligent agent training technology, and more specifically, to a method and system for constructing intelligent agent training datasets that incorporate cross-modal learning. Background Technology

[0002] With the rapid development of artificial intelligence technology, intelligent agents are increasingly being used in fields such as intelligence analysis and knowledge discovery. However, intelligent agent training methods mainly rely on single-modal data for training, making it difficult to fully utilize cross-modal information in multi-source heterogeneous data. This limits the generalization ability and analytical performance of intelligent agents when dealing with complex scenarios.

[0003] The process of constructing training data for intelligent agents lacks semantic alignment and correlation analysis of multimodal data, making it impossible to effectively capture the inherent connections between different modalities. Existing methods often employ fixed processing strategies when constructing analytical inference chains, making it difficult to dynamically adjust the decision-making process according to task characteristics, thus limiting the agent's reasoning ability and analytical effectiveness.

[0004] Traditional training data optimization methods primarily focus on single-dimensional feature optimization, lacking a systematic utilization of feedback information during the training process, making it difficult to continuously improve the quality of training data. The lack of a standardized sample library construction mechanism leads to unstable training performance of the agent, making it difficult to guarantee the reliability of the analysis results. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for constructing an intelligent agent training dataset that combines cross-modal learning. This method provides an intelligent agent training dataset construction method that can make full use of multi-source heterogeneous data, achieve cross-modal feature alignment, and dynamically optimize the analysis inference chain, so as to improve the intelligent agent's analytical ability and generalization performance in complex scenarios.

[0006] The method for constructing an agent training dataset combining cross-modal learning provided in this embodiment of the invention includes the following steps:

[0007] Multimodal intelligence data is obtained from a multi-source database, feature information is extracted from the multimodal intelligence data, and the feature information is projected onto a preset semantic space to obtain cross-modal alignment features;

[0008] Based on cross-modal alignment features, an agent function call syntax tree is constructed, the syntax tree is parsed into an instruction sequence, and an analysis inference chain is constructed to generate sample labels and interaction trajectories;

[0009] A task dependency graph is constructed using sample labels and interaction trajectories. The task dependency graph is then decomposed into multiple parallel decision branches. The processing strategies of each branch are dynamically adjusted based on the analysis inference chain to form a training dataset.

[0010] The training dataset is mapped to the target space of the agent, and the analysis results are fed back to the analysis inference chain for optimization through the training feedback channel, and a standardized sample library is output.

[0011] The data to be analyzed is input into an agent trained on a standardized sample library, and an analysis report is generated based on the analysis inference chain.

[0012] Furthermore, multimodal intelligence data is obtained from multi-source databases, feature information is extracted from the multimodal intelligence data, and the feature information is projected onto a preset semantic space to obtain cross-modal alignment features, including:

[0013] Multimodal intelligence data is obtained from a multi-source database, and feature information is extracted from the multimodal intelligence data to generate multimodal features; the multi-source database includes text, image, voice and structured databases;

[0014] Calculate the mutual information values ​​between different dimensions of multimodal features, mark feature positions with mutual information correlation as positive sample pairs, and mark feature positions without correlation as negative sample pairs, and calculate feature fusion weights based on positive and negative sample pairs;

[0015] The multimodal features are weighted using feature fusion weights to obtain fused features, and intermodal feature constraints are constructed. The fused features are then optimized based on the intermodal feature constraints, and the optimized fused features are projected onto a preset semantic space.

[0016] Semantic constraint optimization is performed on the projected features in a predefined semantic space to generate cross-modal aligned features.

[0017] Furthermore, based on cross-modal alignment features, an agent function call syntax tree is constructed. This syntax tree is then parsed into instruction sequences, and an analysis inference chain is built to generate sample labels and interaction trajectories, including:

[0018] Action words and target words are extracted from cross-modal alignment features, and the action words and target words are combined to establish hierarchical dependency relationships and generate a semantic dependency tree;

[0019] The semantic features of each node in the semantic dependency tree are statistically analyzed, and the nodes with semantic features are clustered into function categories. Based on each function category, an agent function call template is constructed.

[0020] The nodes in the semantic dependency tree are matched with the agent function call templates for feature matching. Based on the matching results, the call relationships between functions are determined, and an agent function call syntax tree is constructed.

[0021] Based on the hierarchical structure of the agent function call syntax tree and the dependency relationships between nodes, an instruction sequence is generated, and the system state changes before and after the execution of each function in the instruction sequence are recorded as a state snapshot.

[0022] Based on the state snapshot, the causal relationship between instructions is extracted to construct an analysis and reasoning chain. The analysis and reasoning chain is transformed into an interaction trajectory, and the task objectives and constraints are extracted from the interaction trajectory to generate sample labels.

[0023] Furthermore, feature matching is performed between nodes in the semantic dependency tree and the agent function call template. Based on the matching results, the call relationships between functions are determined, and the agent function call syntax tree is constructed, including:

[0024] Extract the feature data of nodes in the semantic dependency tree and convert them into node feature vectors;

[0025] Extract template information from the agent function call template, and convert the template information into a template feature vector;

[0026] Calculate the similarity between the node feature vector and the template feature vector, and perform normalization to obtain the feature matching weights between the node and the template. Then, organize the feature matching weights into a feature matching matrix.

[0027] Obtain the node dependency relationships in the semantic dependency tree, optimize the feature matching matrix based on the node dependency relationships, and determine the calling relationships between functions;

[0028] The function call relationships are converted into directed edges, and the corresponding function nodes are connected based on the directed edges to generate an agent function call syntax tree.

[0029] Furthermore, a task dependency graph is constructed using sample labels and interaction trajectories. This graph is then decomposed into multiple parallel decision branches. The processing strategies of each branch are dynamically adjusted based on the analytical inference chain, resulting in a training dataset including:

[0030] Extract task attributes from sample labels and operation sequences from interaction trajectories;

[0031] Calculate the execution dependency between task attributes and the state transition degree between operation sequences, and combine the execution dependency and state transition degree into task dependency strength;

[0032] Cluster the task attributes, merge task attributes in the same group into a single node, and treat task attributes in different groups as independent nodes to form a node set.

[0033] In the node set, connections are established between nodes with dependencies, and the task dependency strength is assigned as the edge weight to generate a task dependency graph.

[0034] Calculate the sum of the weights of the incoming and outgoing edges of the nodes in the task dependency graph, take the node with the largest sum of weights as the split point, and divide the graph into multiple parallel decision branches along the split point.

[0035] The state transition characteristics of parallel decision branches in the analysis inference chain are analyzed, and the processing strategies of each parallel decision branch are dynamically adjusted based on the state transition characteristics to form a training dataset.

[0036] Furthermore, the training dataset is mapped to the agent's target space, and the analysis results are fed back to the analysis inference chain for optimization through the training feedback channel. The output standardized sample library includes:

[0037] State and action information are extracted from the training dataset and converted into a feature matrix. A feature attention matrix is ​​then generated based on the mutual information values ​​between feature pairs.

[0038] The feature attention matrix is ​​combined with the feature matrix to obtain the target space semantic vector. Based on the target space semantic vector, the training dataset is mapped to the agent's target space.

[0039] The state sequence of the training dataset is recorded at each node of the analysis and inference chain, and the node deviation value is calculated. The node deviation value is then fed back to the analysis and inference chain for optimization through the training feedback channel.

[0040] A quality score is generated based on the feature coverage and state consistency of samples in the optimized analysis inference chain. A standardized sample library is then output based on the quality score.

[0041] Furthermore, the data to be analyzed is input into an agent trained on a standardized sample library, and an analysis report is generated based on the analysis inference chain, including:

[0042] The feature score is obtained by calculating the similarity between the feature vector of the data to be analyzed and the standardized sample library, and the state score is obtained by calculating the transition probability of the state sequence of the data to be analyzed on the analysis inference chain.

[0043] A quality matrix of an agent is generated based on feature scores and state scores, and feature completion positions and state correction positions are determined based on the agent quality matrix.

[0044] The feature information of the sample with the highest feature score in the standardized sample library is transferred to the feature completion position to obtain the completed feature data;

[0045] The completed feature data is used for state sequence prediction. When the predicted state deviation exceeds a preset deviation threshold, state reconstruction is performed at the state correction position to obtain the reconstructed state sequence.

[0046] The reconstructed state sequence is input into the agent, and reasoning analysis is performed based on the analysis inference chain to extract the state information and inference path information of each node and generate an analysis report.

[0047] The present invention provides a system for constructing an agent training dataset that combines cross-modal learning, the system comprising:

[0048] The multimodal data processing module is used to obtain multimodal intelligence data from multi-source databases, extract feature information from the multimodal intelligence data, and project the feature information onto a preset semantic space to obtain cross-modal alignment features.

[0049] The syntax tree construction module is used to construct the agent function call syntax tree based on cross-modal alignment features, parse the syntax tree into instruction sequences and construct analysis inference chains, and generate sample labels and interaction trajectories.

[0050] The task decomposition module is used to construct a task dependency graph using sample labels and interaction trajectories, decompose the task dependency graph into multiple parallel decision branches, dynamically adjust the processing strategy of each branch based on the analysis inference chain, and form a training dataset.

[0051] The data optimization module is used to map the training dataset to the target space of the agent, feed the analysis results back to the analysis inference chain optimization through the training feedback channel, and output a standardized sample library.

[0052] The intelligent analysis module is used to input the data to be analyzed into an agent trained on a standardized sample library, and generate an analysis report based on the analysis inference chain.

[0053] One embodiment of the present invention also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the methods described above.

[0054] One technical solution provided in this embodiment of the invention is a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the steps in the method described in any of the preceding claims.

[0055] This invention effectively solves the problem of missing features in the data being analyzed by introducing an agent quality matrix to achieve precise location of feature completion and state correction. It improves the accuracy of feature completion by employing a feature transfer mechanism based on a standardized sample library. Through state sequence prediction and reconstruction, it achieves adaptive correction of abnormal states. Based on the extraction of node state information and the calculation of causal correlation strength in the analysis inference chain, the analysis report clearly presents the state changes and importance of key nodes, thereby improving the interpretability and credibility of the analysis results. The overall solution ensures both the accuracy and robustness of the analysis process and provides practically valuable analysis results. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 A flowchart illustrating a method for constructing an intelligent agent training dataset incorporating cross-modal learning, as provided in an embodiment of the present invention.

[0058] Figure 2 This embodiment of the invention provides a clustering heatmap of function categories.

[0059] Figure 3 This is a schematic diagram comparing the function call accuracy of different methods in various tasks according to embodiments of the present invention;

[0060] Figure 4 This is a schematic diagram of the structure of a system for constructing an agent training dataset that combines cross-modal learning, as described in an embodiment of the present invention. Detailed Implementation

[0061] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with one or more embodiments of this specification.

[0062] like Figure 1 As shown, Figure 1 This is a flowchart of a method for constructing a training dataset for an agent that combines cross-modal learning, provided in an embodiment of the present invention. The method includes the following steps:

[0063] Multimodal intelligence data is obtained from a multi-source database, feature information is extracted from the multimodal intelligence data, and the feature information is projected onto a preset semantic space to obtain cross-modal alignment features;

[0064] Based on cross-modal alignment features, an agent function call syntax tree is constructed, the syntax tree is parsed into an instruction sequence, and an analysis inference chain is constructed to generate sample labels and interaction trajectories;

[0065] A task dependency graph is constructed using sample labels and interaction trajectories. The task dependency graph is then decomposed into multiple parallel decision branches. The processing strategies of each branch are dynamically adjusted based on the analysis inference chain to form a training dataset.

[0066] The training dataset is mapped to the target space of the agent, and the analysis results are fed back to the analysis inference chain for optimization through the training feedback channel, and a standardized sample library is output.

[0067] The data to be analyzed is input into an agent trained on a standardized sample library, and an analysis report is generated based on the analysis inference chain.

[0068] As a preferred embodiment of the present invention, multimodal intelligence data is obtained from a multi-source database, feature information is extracted from the multimodal intelligence data, and the feature information is projected onto a preset semantic space to obtain cross-modal alignment features, including:

[0069] Multimodal intelligence data is obtained from a multi-source database, and feature information is extracted from the multimodal intelligence data to generate multimodal features; the multi-source database includes text, image, voice and structured databases;

[0070] Calculate the mutual information values ​​between different dimensions of multimodal features, mark feature positions with mutual information correlation as positive sample pairs, and mark feature positions without correlation as negative sample pairs, and calculate feature fusion weights based on positive and negative sample pairs;

[0071] The multimodal features are weighted using feature fusion weights to obtain fused features, and intermodal feature constraints are constructed. The fused features are then optimized based on the intermodal feature constraints, and the optimized fused features are projected onto a preset semantic space.

[0072] Semantic constraint optimization is performed on the projected features in a predefined semantic space to generate cross-modal aligned features.

[0073] First, multimodal intelligence data is acquired from multiple source databases, including text databases, image databases, voice databases, and structured databases. The acquired intelligence data can include text data such as research papers, technical reports, patent documents, meeting minutes, and technical blogs; image data such as technical diagrams, experimental charts, and product images; voice data such as technical explanations and expert interviews; and various structured technical indicators and experimental data.

[0074] For the acquired multimodal intelligence data, this method performs feature extraction processing. For text data, a pre-trained language model is used to extract semantic feature vectors; for image data, a deep convolutional neural network is used to extract visual features; for speech data, features are extracted after converting it into text using an acoustic model or by directly extracting acoustic features; for structured data, it is converted into feature vectors through data standardization and encoding. Optionally, textual features describing chip architecture are extracted from relevant literature, visual features are extracted from chip design diagrams, speech features are extracted from technical explanation videos, and structured features are extracted from performance parameter tables, collectively forming a multimodal feature set.

[0075] To effectively fuse feature information from different modalities, mutual information values ​​between different dimensions of multimodal features are calculated. Mutual information values ​​represent the degree of correlation between different feature dimensions and are measured by the logarithm of the ratio of the joint probability distribution to the marginal probability distribution of each feature dimension. For the calculated mutual information matrix, a threshold τ is set. Feature pairs with mutual information values ​​greater than τ are marked as positive sample pairs, indicating a semantic association between these features; feature pairs with mutual information values ​​less than or equal to τ are marked as negative sample pairs, indicating no significant association between these features.

[0076] Based on labeled positive and negative sample pairs, this method constructs a contrastive learning loss function. By minimizing the feature distance between positive sample pairs and maximizing the feature distance between negative sample pairs, a feature fusion weight matrix is ​​calculated. This weight matrix determines the importance of different modal features in the fusion process. Using the calculated feature fusion weights, a weighted summation operation is performed on the multimodal features to generate preliminary fused features.

[0077] To ensure that the fused features fully retain the key information of each modality, an inter-modal feature constraint mechanism is constructed. This constraint includes intra-modal consistency constraints and inter-modal complementarity constraints. Intra-modal consistency constraints ensure that key information of the same modality is preserved during the fusion process; inter-modal complementarity constraints encourage the capture of complementary information from different modalities by the fused features. The initial fused features are adjusted and optimized through iterative optimization of these two constraints.

[0078] The optimized fused features are projected onto a predefined semantic space through a nonlinear projection layer. This semantic space is a high-dimensional vector space where semantically similar concepts are close to each other. Within this predefined semantic space, this method introduces a semantic constraint optimization mechanism, including domain knowledge graph constraints and task-related semantic consistency constraints. Domain knowledge graph constraints utilize existing knowledge structures in the intelligence domain to ensure that the projected features conform to semantic relationships within the domain; task-related semantic consistency constraints ensure that the features support the needs of downstream tasks.

[0079] Through the semantic constraint optimization described above, cross-modal aligned features are generated, which are multimodal intelligence feature representations aligned in the semantic space. These features can capture the semantic relationships between different modalities, providing high-quality datasets for training large models in the intelligence field. The finally generated cross-modal aligned features can simultaneously express multi-dimensional information such as chip architecture, physical design, and performance parameters, while maintaining the semantic consistency and complementarity of this information.

[0080] This invention effectively addresses the challenges of strong heterogeneity and large semantic gaps between modalities in intelligence data through multimodal feature extraction and cross-modal alignment techniques, significantly improving the quality of the training dataset. By employing mutual information computation and contrastive learning, it captures semantic relationships between different modalities, avoiding information redundancy and noise interference issues common in traditional methods. An optimization strategy based on intermodal feature constraints and semantic space projection ensures the semantic consistency of fused features, enhancing the model's ability to understand complex intelligence.

[0081] In a preferred embodiment of the present invention, an agent function call syntax tree is constructed based on cross-modal alignment features. The syntax tree is then parsed into an instruction sequence, and an analysis inference chain is constructed to generate sample labels and interaction trajectories, including:

[0082] Action words and target words are extracted from cross-modal alignment features, and the action words and target words are combined to establish hierarchical dependency relationships and generate a semantic dependency tree;

[0083] The semantic features of each node in the semantic dependency tree are statistically analyzed, and the nodes with semantic features are clustered into function categories. Based on each function category, an agent function call template is constructed.

[0084] The nodes in the semantic dependency tree are matched with the agent function call templates for feature matching. Based on the matching results, the call relationships between functions are determined, and an agent function call syntax tree is constructed.

[0085] Based on the hierarchical structure of the agent function call syntax tree and the dependency relationships between nodes, an instruction sequence is generated, and the system state changes before and after the execution of each function in the instruction sequence are recorded as a state snapshot.

[0086] Based on the state snapshot, the causal relationship between instructions is extracted to construct an analysis and reasoning chain. The analysis and reasoning chain is transformed into an interaction trajectory, and the task objectives and constraints are extracted from the interaction trajectory to generate sample labels.

[0087] Action words and target words are extracted from the acquired cross-modal alignment features. Action words typically represent operations that the agent can perform, such as "analyze," "compare," and "predict." Target words represent the objects of these operations, such as "semiconductor materials," "integrated circuits," and "chip architecture." By using dependency parsing techniques, semantic relationships are extracted from the feature vectors to identify action words and target words.

[0088] The extracted action words and target words are combined to establish hierarchical dependency relationships. The combination method is based on semantic compatibility and domain knowledge constraints to ensure that the generated combinations have practical significance in the intelligence domain. Hierarchical dependency relationships represent the order and inclusion relationships between actions, determined through semantic relevance calculation. Dependency relationships include sequential execution, conditional execution, and loop execution types. Based on action words, target words, and their combination relationships, a semantic dependency tree is constructed. The root node of this tree is usually the main task objective, and the child nodes are subtasks or operational steps to achieve that objective. For example, a possible semantic dependency tree could have a root node of "analyzing chip technology trends," with lower-level nodes including "collecting chip parameter data," "comparing the performance of different architectures," and "predicting technology development directions," etc., with explicit dependency relationships existing between these nodes.

[0089] The semantic features of each node in the semantic dependency tree are analyzed, including the node's semantic category, contextual information, and association attributes. An unsupervised clustering algorithm is used to cluster nodes with similar semantic features into function categories. Common function categories include data acquisition, data processing, analysis and reasoning, and result presentation. These may include intelligence gathering functions, data cleaning functions, feature extraction functions, association analysis functions, and trend prediction functions. Based on the features and execution logic of each function category, a function call template for the intelligent agent is constructed. The function call template defines the function's input parameters, execution conditions, and output results, providing a standard for subsequent function matching and invocation.

[0090] The nodes in the semantic dependency tree are matched with the agent's function call templates, and the similarity between the semantic features of the nodes and the features of the function templates is calculated. The function template with the highest similarity is selected as the matching result. Based on the matching results, the call relationships between functions are determined, and an agent function call syntax tree is constructed. This syntax tree represents the sequence of function calls and its hierarchical structure required to complete a specific intelligence analysis task. The syntax tree may contain data collection function calls, parameter extraction function calls, performance comparison function calls, trend prediction function calls, etc., and these functions are organized into a tree structure according to their dependency relationships.

[0091] Based on the hierarchical structure of the agent's function call syntax tree and the dependencies between nodes, an instruction sequence is generated. The instruction sequence is a series of executable function call instructions arranged in a predetermined execution order. For conditional branches and loop structures, corresponding control flow instructions are generated. During the execution of the instruction sequence, the system state changes before and after each function execution are recorded as state snapshots. State snapshots contain information such as environmental state, data state, and the agent's internal state. State snapshots may also record database connection status, acquired chip parameter data, and intermediate analysis results.

[0092] Based on state snapshots, causal relationships between instructions are extracted to identify which instruction execution results affect the execution conditions or parameters of subsequent instructions. By comparing the differences between adjacent state snapshots, the correspondence between state changes and instruction execution is determined. Based on these causal relationships, an analytical reasoning chain is constructed to represent the reasoning process from the initial question to the final conclusion. The analytical reasoning chain is transformed into an interaction trajectory, which describes the interaction process between the agent and the environment or user, including steps such as input, processing, and output. Task objectives and constraints, such as time limits, resource limits, and accuracy requirements, are extracted from the interaction trajectory to generate sample labels. In the chip technology analysis example, the task objective might be "predicting the development direction of the next-generation chip architecture," and the constraints might include "based on the technology evolution data of the past five years" and "considering power consumption and performance balance factors," etc.

[0093] like Figure 2 The image shows a heatmap of function category clustering in this embodiment, which comprehensively demonstrates the effectiveness analysis of semantic node function category clustering and template construction. The similarity scores between different semantic nodes and each function category are intuitively displayed through color depth. For example, the similarity between the "Query Database" node and the "Intelligence Gathering" category is as high as 0.92, while the similarity with the "Trend Prediction" category is only 0.12; the similarity between the "Trend Modeling" node and the "Trend Prediction" category is 0.96, indicating an extremely high matching degree. In the heatmap, red areas (similarity > 0.85) indicate a high matching degree between nodes and function categories, orange areas (similarity 0.4-0.85) indicate a medium matching degree, and light yellow areas (similarity < 0.4) indicate a low matching degree.

[0094] This invention effectively addresses the problems of insufficient and low-quality training data for large-scale intelligence models by constructing an agent training dataset from cross-modal aligned features. It achieves automatic conversion from cross-modal features to agent function calls, constructing structured training samples and significantly improving the quality and diversity of training data. Through the construction of semantic dependency trees and function call syntax trees, it captures the deep semantic structure and execution logic of intelligence analysis tasks, enabling the trained model to better understand complex task instructions. The inference chain constructed based on state snapshots and causal association analysis enhances the model's logical reasoning ability and interpretability, contributing to the generation of more accurate and comprehensive intelligence analysis reports.

[0095] In a preferred embodiment of the present invention, feature matching is performed between nodes in the semantic dependency tree and the agent function call template, and the call relationship between functions is determined based on the matching result. Constructing the agent function call syntax tree includes:

[0096] Extract the feature data of nodes in the semantic dependency tree and convert them into node feature vectors;

[0097] Extract template information from the agent function call template, and convert the template information into a template feature vector;

[0098] Calculate the similarity between the node feature vector and the template feature vector, and perform normalization to obtain the feature matching weights between the node and the template. Then, organize the feature matching weights into a feature matching matrix.

[0099] Obtain the node dependency relationships in the semantic dependency tree, optimize the feature matching matrix based on the node dependency relationships, and determine the calling relationships between functions;

[0100] The function call relationships are converted into directed edges, and the corresponding function nodes are connected based on the directed edges to generate an agent function call syntax tree.

[0101] First, feature data of the nodes is extracted. A semantic dependency tree is a tree structure representing the dependency relationships between words in a natural language sentence, where each node represents a semantic unit, and edges represent the dependency relationships between nodes. For each node, feature data is extracted, including semantic type, associated entities, attribute qualifiers, and other multi-dimensional information. In the field of intelligence analysis, typical node features include operation type (e.g., retrieval, analysis, comparison), operation object (e.g., technical reports, market data), and limiting conditions (e.g., time range, technical field). The extracted feature data is then converted into node feature vectors using a feature encoder. The feature encoding process uses a pre-trained language model to map text features to a high-dimensional semantic space, resulting in fixed-dimensional feature vectors. This vectorized representation can capture the semantic information and contextual relationships of nodes.

[0102] Template information is extracted from the agent function call template library. Function call templates define the standard calling format and parameter specifications for various functions, forming the foundation for constructing a function call syntax tree. Template information includes function name, parameter types, execution conditions, and return value type. Typical function templates include information retrieval functions such as "document retrieval (keywords, time range, domain)", data processing functions such as "feature extraction (document set, feature type)", and analytical reasoning functions such as "trend analysis (time series data, prediction period)". When extracting template information, the focus is on the function's semantic function, input parameter requirements, and output characteristics, as these directly relate to the matching degree between the function and the node. The extracted template information is also converted into template feature vectors using a feature encoder to ensure that template features and node features are represented in the same semantic space, facilitating subsequent similarity calculations.

[0103] Calculate the similarity between the node feature vector and the template feature vector. For each node in the semantic dependency tree, calculate the cosine similarity between its feature vector and the feature vectors of all function templates. Cosine similarity measures the proximity of two vectors in a direction, with a value range of [-1, 1], where a larger value indicates greater similarity. The calculated raw similarity is normalized to convert it into feature matching weights in the interval 0 to 1. Normalization uses a soft maximization function to ensure that the sum of the matching weights of each node with all templates is 1, facilitating subsequent optimization and selection. For each node, its matching weights with all templates constitute a weight vector; the weight vectors of all nodes are combined to form a feature matching matrix. The rows of this matrix represent nodes in the semantic dependency tree, the columns represent function call templates, and the matrix element values ​​represent the matching weights of the corresponding node and template.

[0104] Obtain node dependency relations from the semantic dependency tree. Dependency relations describe the logical connections between semantic units, including dominance, coordination, and modification relations. Dependency relations can be extracted directly from the structure of the semantic dependency tree or obtained additionally using dependency parsing tools. Optimize the feature matching matrix based on node dependency relations. The optimization process considers dependency constraints between nodes to ensure that mutually dependent nodes match logically compatible function templates. A conditional random field model is used, with node dependency relations as constraints to adjust feature matching weights. During optimization, if two nodes with dependency relations cannot form a valid call relationship between their matched function templates, the corresponding matching weight is reduced; conversely, if a reasonable call relationship exists between the matched function templates, the matching weight is increased. Through multiple rounds of iterative optimization, the most suitable function template for each node is finally determined, and the call relationships between functions are determined accordingly.

[0105] The function call relationships are converted into directed edges. Each directed edge represents a function calling another function, with the direction of the edge pointing from the caller to the callee. Directed edges can include attribute information, such as parameter passing methods, calling conditions, and return value handling. For example, the directed edge from the "document retrieval" function to the "feature extraction" function indicates that feature extraction is called after document retrieval; the edge's attributes may include the method of passing the document set. Based on the constructed directed edges connecting the corresponding function nodes, an agent function call syntax tree is generated. In the syntax tree, each node represents a function call, and the node's attributes include the function name, parameter list, and return value type; the edges between nodes represent the call relationships. The root node of the syntax tree is usually the entry function for the entire task, and the leaf nodes are the basic operation functions.

[0106] like Figure 3The diagram illustrates a comparison of function call accuracy across various tasks using different methods in this embodiment. The data clearly shows that this technical solution significantly outperforms traditional methods across all task types. For simple tasks, this solution achieves an accuracy of 95.2%, 11.7 percentage points higher than the rule-matching method's 83.5%, 8 percentage points higher than the Word2Vec method's 87.2%, and 4.8 percentage points higher than the BERT method's 90.4%. As task complexity increases, the differences between methods become more pronounced. For highly complex tasks, this solution maintains a high accuracy of 88.4%, while the rule-matching method drops sharply to 58.7%, the Word2Vec method to 66.4%, and the BERT method to 75.6%. Particularly in domain-specific and cross-domain tasks, this solution demonstrates even greater advantages, achieving 86.5% and 82.9% accuracy respectively, while traditional rule-matching methods only achieve 54.3% and 48.9%. On average, the accuracy of this technical solution is 89.1%, significantly higher than the 63.3% of rule matching methods, 70.7% of Word2Vec, and 78.3% of BERT. Traditional rule matching methods (such as regular expressions and syntactic parsing) perform reasonably well on simple tasks, but their performance drops rapidly when faced with complex semantics. Word embedding methods like Word2Vec can capture some semantic information, but they struggle to handle contextual dependencies. Pre-trained models like BERT improve semantic understanding, but they are not optimized for function call relationships. This technical solution achieves best performance across various tasks by integrating vectorized representations and dependency optimization, and it has a significant advantage, especially in handling complex domain knowledge.

[0107] This invention achieves accurate conversion from semantic dependency trees to agent function call syntax trees through feature vectorization and similarity matching techniques, providing a high-quality training data generation method for large-scale intelligence models. It effectively solves problems such as inaccurate semantic understanding, low function matching efficiency, and unreasonable call structures in traditional methods. In particular, by introducing node dependency relationships to optimize matching weights, it ensures that the generated function call syntax tree has a reasonable structure and rigorous logic. The constructed training dataset can effectively guide large-scale models to learn the execution flow of complex intelligence analysis tasks, improving the model's capabilities in information retrieval, data processing, and trend analysis, providing strong technical support for intelligent analysis in the intelligence field.

[0108] In a preferred embodiment of the present invention, a task dependency graph is constructed using sample labels and interaction trajectories. This task dependency graph is then decomposed into multiple parallel decision branches. The processing strategies of each branch are dynamically adjusted based on the analysis and inference chain, resulting in a training dataset comprising:

[0109] Extract task attributes from sample labels and operation sequences from interaction trajectories;

[0110] Calculate the execution dependency between task attributes and the state transition degree between operation sequences, and combine the execution dependency and state transition degree into task dependency strength;

[0111] Cluster the task attributes, merge task attributes in the same group into a single node, and treat task attributes in different groups as independent nodes to form a node set.

[0112] In the node set, connections are established between nodes with dependencies, and the task dependency strength is assigned as the edge weight to generate a task dependency graph.

[0113] Calculate the sum of the weights of the incoming and outgoing edges of the nodes in the task dependency graph, take the node with the largest sum of weights as the split point, and divide the graph into multiple parallel decision branches along the split point.

[0114] The state transition characteristics of parallel decision branches in the analysis inference chain are analyzed, and the processing strategies of each parallel decision branch are dynamically adjusted based on the state transition characteristics to form a training dataset.

[0115] Extract task attributes and operation sequences from the generated sample labels. Task attributes include task type, task objective, execution conditions, resource constraints, and time constraints. The extraction process employs semantic parsing technology to convert text-based labels and trajectories into structured sets of attributes and operations.

[0116] This calculation computes the execution dependency between task attributes and the state transition degree between operation sequences. Execution dependency represents the degree to which one task attribute depends on another, determined by analyzing the semantic relationships and execution logic between attributes. The calculation of execution dependency is based on prior knowledge of the attributes and historical execution data, using a Bayesian network model for inference. State transition degree represents the degree to which a change in the system state after an operation affects the next operation, calculated by analyzing the differences in state snapshots between adjacent operations. State transition degree is calculated based on the cosine similarity of state difference vectors. The execution dependency and state transition degree are combined in a weighted average to form the task dependency strength, with the weights dynamically adjusted according to the task type.

[0117] Task attributes are clustered and grouped. A spectral clustering algorithm is used to construct an affinity matrix based on the semantic similarity and execution dependency between task attributes for attribute clustering. The clustering process considers the semantic associations and execution logic relationships between attributes, ensuring that closely related attributes are grouped together. Task attributes in the same group are merged into a single node, preserving key features of each attribute and constructing a combined feature representation during the merging process. Task attributes from different groups are treated as independent nodes, and all nodes form a node set, preparing for the subsequent construction of the task dependency graph.

[0118] Within a node set, connections are established based on dependencies between nodes. A dependency means that the execution of one node depends on the completion of another. Dependency is determined based on task dependency strength; when the dependency strength exceeds a preset threshold, a dependency is considered to exist between two nodes. When establishing a connection, the calculated task dependency strength is assigned as an edge weight, representing the strength of the dependency. In an intelligence analysis scenario, a connection is established between the "Information Acquisition" node and the "Data Processing" node, with an edge weight of 0.85, indicating a strong dependency; a connection is also established between the "Data Processing" node and the "Analysis and Reasoning" node, with an edge weight of 0.92, indicating a very strong dependency. By establishing these weighted connections, a task dependency graph is generated, which describes the dependencies and dependency strengths between various stages in the task execution process.

[0119] Calculate the sum of the incoming and outgoing edge weights of each node in the task dependency graph. The incoming edge weight represents the sum of the dependencies of other nodes on the current node, and the outgoing edge weight represents the sum of the dependencies of the current node on other nodes. The sum of weights reflects the criticality of a node in the entire task flow. The node with the largest sum of weights is used as the split point; this node is typically a critical link or bottleneck in the task flow. The task dependency graph is then divided into multiple parallel decision branches along this split point. The splitting process preserves the split point node in each branch to ensure task integrity. Using the "Data Processing" node as the split point, two parallel decision branches may be formed: "Information Acquisition - Data Processing" and "Data Processing - Analysis and Reasoning".

[0120] This study analyzes the state transition characteristics of parallel decision branches within the analysis and inference chain. State transition characteristics describe the pattern of system state changes during branch execution, including transition frequency, amplitude, and stability. By analyzing state snapshot sequences, state transition feature vectors are extracted for each branch. In the example, the "information acquisition-data processing" branch may exhibit high-frequency, low-amplitude state transitions, reflecting the gradual nature of data collection and processing; while the "data processing-analysis and inference" branch may exhibit low-frequency, high-amplitude state transitions, reflecting key breakthroughs in the analysis process. Based on the extracted state transition characteristics, the processing strategies for each parallel decision branch are dynamically adjusted. These strategies include resource allocation, execution priority, and monitoring granularity. For example, for branches with dramatic state transitions, monitoring granularity may be increased by setting more checkpoints; for branches with stable state transitions, monitoring frequency may be reduced to improve execution efficiency. These adjustments result in a more efficient and reasonable training dataset, providing high-quality training material for large-scale intelligence models.

[0121] This invention achieves structured representation and parallel processing of intelligence analysis tasks through the construction and segmentation of task dependency graphs, significantly improving the quality and diversity of the training dataset. Based on a task dependency strength calculation method using execution dependency and state transition degree, it accurately captures the complex relationships between task components, providing agents with training data that more closely resembles real-world intelligence analysis processes. The technique of dynamically adjusting strategies by analyzing state transition features enhances the adaptability and relevance of the training data, enabling large models to learn more efficient task execution strategies. The training dataset generated by this invention not only contains rich intelligence analysis knowledge but also encodes the decision-making logic and reasoning paths of professional analysts, greatly improving the analytical capabilities and report generation quality of large models in the intelligence field, providing a solid data foundation for the automation and intelligentization of intelligence analysis.

[0122] In a preferred embodiment of the present invention, the training dataset is mapped to the target space of the agent, and the analysis results are fed back to the analysis inference chain for optimization through the training feedback channel, outputting a standardized sample library including:

[0123] State and action information are extracted from the training dataset and converted into a feature matrix. A feature attention matrix is ​​then generated based on the mutual information values ​​between feature pairs.

[0124] The feature attention matrix is ​​combined with the feature matrix to obtain the target space semantic vector. Based on the target space semantic vector, the training dataset is mapped to the agent's target space.

[0125] The state sequence of the training dataset is recorded at each node of the analysis and inference chain, and the node deviation value is calculated. The node deviation value is then fed back to the analysis and inference chain for optimization through the training feedback channel.

[0126] A quality score is generated based on the feature coverage and state consistency of samples in the optimized analysis inference chain. A standardized sample library is then output based on the quality score.

[0127] State and action information are extracted from the constructed training dataset. State information describes the environmental, data, and internal states of the agent during task execution, including available resources, acquired information, and processing progress. Action information describes the specific operations performed by the agent, such as retrieval, filtering, analysis, and reasoning. The extracted state and action information are converted into a feature matrix using a feature encoder. Each row of the feature matrix represents a sample, and each column represents a feature dimension.

[0128] A feature attention matrix is ​​generated based on the mutual information values ​​between feature pairs. Mutual information measures the degree of interdependence between two features; a high mutual information value indicates a strong correlation between the two features. The mutual information values ​​between each pair of features in the feature matrix are calculated to form a mutual information matrix. This mutual information matrix is ​​then transformed into a feature attention matrix using an attention mechanism, which describes the importance of each feature in different contexts. The attention mechanism calculates the weight of each feature based on the distribution of mutual information values, giving higher weights to key features.

[0129] The target space semantic vector is obtained by combining the feature attention matrix with the feature matrix. This combination is achieved through weighted summation, where the features in the feature matrix are weighted using the weights from the feature attention matrix. This weighted combination considers the relationships between features, giving more influence to important features in the result. For intelligence analysis tasks, the target space semantic vector may emphasize key information such as technological innovation, application prospects, and market potential, while downplaying secondary information such as publication format and non-core authors. Based on the generated target space semantic vector, the training dataset is mapped to the agent's target space. The target space is a multi-dimensional vector space where samples with similar tasks and similar execution strategies are relatively close. The mapping process uses a non-linear mapping function to transform each training sample into a point in the target space. In the target space, intelligence analysis tasks with similar topics are mapped to nearby regions, forming topic clusters; tasks of similar difficulty also form a difficulty gradient distribution.

[0130] The analysis and inference chain records the state sequence of the training dataset at each node. The analysis and inference chain consists of multiple inference nodes, each representing a reasoning step. At each node, the system state at that moment is recorded, including acquired information, intermediate results, and decision-making basis. For intelligence analysis tasks, inference nodes may include "determining the stage of technological development," "identifying key technological bottlenecks," and "predicting development trends," with each node recording corresponding state information. The deviation value of each node is calculated based on a pre-defined standard state sequence. The standard state sequence is an ideal execution path defined by domain experts, reflecting the standard process of high-quality intelligence analysis. The node deviation value represents the degree of difference between the actual state and the standard state, calculated through the distance between state vectors. The node deviation values ​​are fed back to the analysis and inference chain through a training feedback channel to optimize the inference process. The feedback channel uses the gradient descent principle, adjusting the inference parameters according to the magnitude and direction of the deviation value to make the inference path closer to the standard path.

[0131] A quality score is generated based on the feature coverage and state consistency of samples in the optimized analysis inference chain. Feature coverage measures the degree to which a sample covers the target feature space, calculated as the similarity between the sample feature vector and the target feature vector. High coverage indicates that the sample contains most of the key features required for the task. State consistency measures the degree of matching between the sample state sequence and the standard state sequence, calculated as the dynamic time warp distance between state sequences. High consistency indicates that the execution path of the sample conforms to the standard analysis process. For example, high-quality samples should both cover key technical features and follow professional analysis procedures. The feature coverage and state consistency are combined into a quality score using a weighted average, with the weights dynamically adjusted according to the task type.

[0132] For deep technical analysis tasks, feature coverage may be more important; for trend prediction tasks, state consistency may be more important. Based on the calculated quality score, high-quality samples are selected and standardized to form a standardized sample library. Standardization includes steps such as format unification, feature normalization, and removal of redundant information to ensure the consistency and usability of the sample library. The standardized sample library serves as high-quality training data for large-scale intelligence models, used for model training and fine-tuning.

[0133] This invention achieves accurate representation and optimization of training datasets through feature attention matrices and target space mapping techniques, effectively solving the problems of low-quality and incomplete coverage of training data for large-scale intelligence models. Based on mutual information-based feature correlation analysis, it accurately captures key features and their inherent relationships in intelligence data, enabling the model to focus on truly important information. Through state sequence monitoring and bias feedback mechanisms in the analysis of the inference chain, dynamic optimization of training data is achieved, ensuring that the dataset conforms to professional analysis procedures and standards. The standardized sample library generated by this invention has high feature coverage and state consistency, providing a high-quality training foundation for large-scale intelligence models.

[0134] In a preferred embodiment of the present invention, the data to be analyzed is input into an agent trained on a standardized sample library, and an analysis report is generated based on the analysis inference chain, including:

[0135] The feature score is obtained by calculating the similarity between the feature vector of the data to be analyzed and the standardized sample library, and the state score is obtained by calculating the transition probability of the state sequence of the data to be analyzed on the analysis inference chain.

[0136] A quality matrix of an agent is generated based on feature scores and state scores, and feature completion positions and state correction positions are determined based on the agent quality matrix.

[0137] The feature information of the sample with the highest feature score in the standardized sample library is transferred to the feature completion position to obtain the completed feature data;

[0138] The completed feature data is used for state sequence prediction. When the predicted state deviation exceeds a preset deviation threshold, state reconstruction is performed at the state correction position to obtain the reconstructed state sequence.

[0139] The reconstructed state sequence is input into the agent, and reasoning analysis is performed based on the analysis inference chain to extract the state information and inference path information of each node and generate an analysis report.

[0140] Feature vectors are calculated for the intelligence data to be analyzed. A deep feature extraction network transforms multimodal intelligence data into feature vectors in a unified feature space. For example, the data to be analyzed may include multimodal information such as research paper text, chip design images, and performance test data; this information is converted into high-dimensional feature vectors. The cosine similarity between this feature vector and the feature vectors of each sample in a standardized sample library is calculated to obtain a feature score. The feature score reflects the closeness of the data to be analyzed to the standard samples in the sample library in the feature space, with a score ranging from 0 to 1; a higher value indicates greater feature similarity. A state sequence is extracted from the data to be analyzed, describing the trajectory of data changes during the analysis process. The transition probabilities of this state sequence on a pre-trained analysis inference chain are calculated to obtain a state score. The state score reflects the degree of conformity between the state transition pattern of the data to be analyzed and the standard analysis process, also with a value ranging from 0 to 1.

[0141] A quality matrix for the agent is generated based on the calculated feature scores and state scores. The quality matrix is ​​a two-dimensional matrix where rows represent feature dimensions, columns represent time steps in the state sequence, and matrix element values ​​represent the quality score of the corresponding feature at the corresponding time step. The quality score is calculated by a weighted combination of the feature scores and state scores. Chip architecture features may have high quality scores in the technology trend analysis phase but low scores in the market prospect prediction phase. Feature completion positions and state correction positions are determined based on the agent quality matrix. Feature completion positions are those where the feature score in the quality matrix is ​​lower than a preset feature threshold, indicating insufficient or low-quality feature information that needs to be supplemented. State correction positions are those where the state score in the quality matrix is ​​lower than a preset state threshold, indicating that these state transitions do not conform to the standard analysis process and need to be corrected. For semiconductor technology analysis, missing qubit entanglement characteristics might be found in the quantum computing potential assessment phase, and these positions would be marked as feature completion positions; similarly, state transitions that do not conform to the standard process might be found in the technology maturity assessment phase, and these positions would be marked as state correction positions.

[0142] Feature information from the highest-scoring sample in the standardized sample library is transferred to the feature completion location. The feature transfer process employs transfer learning techniques to transfer relevant feature knowledge from high-quality samples to the data to be analyzed. In the feature space, the feature distribution pattern within the neighborhood of the high-quality sample's feature vector is extracted, and then this distribution pattern is applied to the corresponding feature dimension of the data to be analyzed. For feature completion of qubit entanglement characteristics in semiconductor technology analysis, a high-quality quantum computing technology analysis sample may be selected from the sample library, and feature information regarding qubit entanglement characteristics may be extracted and transferred to the corresponding location in the data to be analyzed. Through feature transfer, completed feature data is obtained, which adds feature information transferred from the high-quality sample to the original features, resulting in a more complete feature dimension.

[0143] Completed feature data is used for state sequence prediction. State sequence prediction employs a recurrent neural network model, predicting the state at the next time step based on the current state and completed feature data. The deviation between the predicted state and the standard state is calculated. When the deviation exceeds a preset deviation threshold, state reconstruction is performed at the state correction point. The state reconstruction process is based on state transition patterns in a standardized sample library, adjusting the state sequence of the data to be analyzed to better conform to standard analysis procedures. In the state space, the state transition trajectories of standard samples are extracted, and the state of the data to be analyzed is then aligned with these trajectories. For the technology maturity assessment stage in technology analysis, it may be found that the predicted state deviates from the standard assessment process. In this case, state reconstruction is performed to adjust the assessment logic and indicators to conform to professional standards for intelligence analysis. Through state reconstruction, a reconstructed state sequence is obtained, which retains the core information of the original data while correcting state transitions that do not conform to standard analysis procedures.

[0144] The reconstructed state sequence is input into the agent, which then performs inference analysis based on the analysis inference chain. The agent employs an attention-based neural network architecture, capable of processing sequential data and capturing long-range dependencies. The analysis inference chain is a predefined standard process for intelligence analysis, containing multiple inference nodes, each corresponding to an analysis step. The agent performs forward inference along the analysis inference chain, processing the input state and generating an output state at each node. For technical analysis, the analysis inference chain may include nodes such as technical background analysis, core technology assessment, application scenario exploration, competitive landscape analysis, and development trend prediction. During the inference process, state information and inference path information for each node are extracted. State information includes the node's input state, processing procedure, and output state; inference path information includes the transition probabilities between nodes and the decision-making basis. This information collectively forms the skeleton of the analysis report.

[0145] Based on the extracted state information and reasoning path information, an intelligence analysis report is generated. The report generation combines template filling with natural language generation. The report framework, including chapter settings and content arrangement, is determined according to the structure of the analytical reasoning chain; the state information of each node is converted into natural language descriptions and filled into the corresponding chapters; logical transitions and conclusion derivations are generated based on the reasoning path information. The analysis report generated in this way possesses both professional depth and a clear logical structure, providing valuable reference for decision-making.

[0146] This invention achieves comprehensive quality assessment of intelligence data through a quality evaluation method combining feature scoring and state scoring, providing a precise basis for subsequent feature completion and state correction. The feature completion technology based on transfer learning effectively solves the problems of incomplete data and missing features in intelligence analysis, significantly improving the comprehensiveness and accuracy of intelligence analysis. The state sequence prediction and reconstruction mechanism ensures that the analysis process conforms to professional standards, avoiding analysis errors caused by process deviations. Based on the intelligent agent reasoning analysis of reconstructed state sequences, in-depth intelligence mining and systematic analysis are achieved, generating analysis reports that are highly professional, logically sound, and readable.

[0147] like Figure 4 As shown, Figure 4 This is a schematic diagram of the structure of a system for constructing an agent training dataset that incorporates cross-modal learning. The system includes:

[0148] The multimodal data processing module is used to obtain multimodal intelligence data from multi-source databases, extract feature information from the multimodal intelligence data, and project the feature information onto a preset semantic space to obtain cross-modal alignment features.

[0149] The syntax tree construction module is used to construct the agent function call syntax tree based on cross-modal alignment features, parse the syntax tree into instruction sequences and construct analysis inference chains, and generate sample labels and interaction trajectories.

[0150] The task decomposition module is used to construct a task dependency graph using sample labels and interaction trajectories, decompose the task dependency graph into multiple parallel decision branches, dynamically adjust the processing strategy of each branch based on the analysis inference chain, and form a training dataset.

[0151] The data optimization module is used to map the training dataset to the target space of the agent, feed the analysis results back to the analysis inference chain optimization through the training feedback channel, and output a standardized sample library.

[0152] The intelligent analysis module is used to input the data to be analyzed into an agent trained on a standardized sample library, and generate an analysis report based on the analysis inference chain.

[0153] One embodiment of the present invention also provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the methods described above.

[0154] One technical solution provided in this embodiment of the invention is a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the steps in the method described in any of the preceding claims.

[0155] The specific embodiments described above are preferred embodiments of the present invention and are not intended to limit the specific scope of the present invention. The scope of the present invention includes, but is not limited to, these specific embodiments. All equivalent changes made in accordance with the shape and structure of the present invention are within the protection scope of the present invention.

Claims

1. A method for constructing an agent training dataset combining cross-modal learning, characterized in that, Includes the following steps: Multimodal intelligence data is obtained from a multi-source database, feature information is extracted from the multimodal intelligence data, and the feature information is projected onto a preset semantic space to obtain cross-modal alignment features; the multi-source database includes text, image, speech and structured databases; Based on cross-modal alignment features, an agent function call syntax tree is constructed, the syntax tree is parsed into an instruction sequence, and an analysis inference chain is constructed to generate sample labels and interaction trajectories; A task dependency graph is constructed using sample labels and interaction trajectories. The task dependency graph is then decomposed into multiple parallel decision branches. The processing strategies of each branch are dynamically adjusted based on the analysis inference chain to form a training dataset. The training dataset is mapped to the target space of the agent, and the analysis results are fed back to the analysis inference chain for optimization through the training feedback channel, and a standardized sample library is output. The data to be analyzed is input into an agent trained on a standardized sample library, and an analysis report is generated based on the analysis inference chain. Based on cross-modal alignment features, an agent function call syntax tree is constructed. The syntax tree is parsed into an instruction sequence, and an analysis inference chain is built to generate sample labels and interaction trajectories, including: Action words and target words are extracted from cross-modal alignment features, and the action words and target words are combined to establish hierarchical dependency relationships and generate a semantic dependency tree; The semantic features of each node in the semantic dependency tree are statistically analyzed, and the nodes with semantic features are clustered into function categories. Based on each function category, an agent function call template is constructed. The nodes in the semantic dependency tree are matched with the agent function call templates for feature matching. Based on the matching results, the call relationships between functions are determined, and an agent function call syntax tree is constructed. Based on the hierarchical structure of the agent function call syntax tree and the dependency relationships between nodes, an instruction sequence is generated, and the system state changes before and after the execution of each function in the instruction sequence are recorded as a state snapshot. Based on the state snapshot, extract the causal relationship between instructions to construct the analysis and reasoning chain, transform the analysis and reasoning chain into an interaction trajectory, and extract the task objectives and constraints from the interaction trajectory to generate sample labels; A task dependency graph is constructed using sample labels and interaction trajectories. This graph is then decomposed into multiple parallel decision branches. The processing strategies of each branch are dynamically adjusted based on the analytical inference chain, resulting in a training dataset including: Extract task attributes from sample labels and operation sequences from interaction trajectories; Calculate the execution dependency between task attributes and the state transition degree between operation sequences, and combine the execution dependency and state transition degree into task dependency strength; Cluster the task attributes, merge task attributes in the same group into a single node, and treat task attributes in different groups as independent nodes to form a node set. In the node set, connections are established between nodes with dependencies, and the task dependency strength is assigned as the edge weight to generate a task dependency graph. Calculate the sum of the weights of the incoming and outgoing edges of the nodes in the task dependency graph, take the node with the largest sum of weights as the split point, and divide the graph into multiple parallel decision branches along the split point. The state transition characteristics of parallel decision branches in the analysis inference chain are analyzed, and the processing strategies of each parallel decision branch are dynamically adjusted based on the state transition characteristics to form a training dataset. The execution dependency degree represents the degree to which one task attribute depends on another task attribute, and is determined by analyzing the semantic relationships and execution logic between task attributes; the state transition degree represents the degree to which the change in system state after an operation is executed affects the next operation, and is calculated by analyzing the differences in state snapshots between adjacent operations.

2. The method as described in claim 1, characterized in that, Multimodal intelligence data is obtained from multiple source databases. Feature information is extracted from the multimodal intelligence data, and the feature information is projected onto a preset semantic space to obtain cross-modal alignment features, including: Multimodal intelligence data is obtained from a multi-source database, and feature information is extracted from the multimodal intelligence data to generate multimodal features; Calculate the mutual information values ​​between different dimensions of multimodal features, mark feature positions with mutual information correlation as positive sample pairs, and mark feature positions without correlation as negative sample pairs, and calculate feature fusion weights based on positive and negative sample pairs; The multimodal features are weighted using feature fusion weights to obtain fused features, and intermodal feature constraints are constructed. The fused features are then optimized based on the intermodal feature constraints, and the optimized fused features are projected onto a preset semantic space. Semantic constraint optimization is performed on the projected features in a predefined semantic space to generate cross-modal aligned features.

3. The method according to claim 1, characterized in that, The process involves matching the nodes in the semantic dependency tree with the agent's function call template, determining the call relationships between functions based on the matching results, and constructing the agent's function call syntax tree, including: Extract the feature data of nodes in the semantic dependency tree and convert them into node feature vectors; Extract template information from the agent function call template, and convert the template information into a template feature vector; Calculate the similarity between the node feature vector and the template feature vector, and perform normalization to obtain the feature matching weights between the node and the template. Then, organize the feature matching weights into a feature matching matrix. Obtain the node dependency relationships in the semantic dependency tree, optimize the feature matching matrix based on the node dependency relationships, and determine the calling relationships between functions; The function call relationships are converted into directed edges, and the corresponding function nodes are connected based on the directed edges to generate an agent function call syntax tree.

4. The method according to claim 1, characterized in that, The training dataset is mapped to the agent's target space, and the analysis results are fed back to the analysis inference chain for optimization through the training feedback channel. The output standardized sample library includes: State and action information are extracted from the training dataset and converted into a feature matrix. A feature attention matrix is ​​then generated based on the mutual information values ​​between feature pairs. The feature attention matrix is ​​combined with the feature matrix to obtain the target space semantic vector. Based on the target space semantic vector, the training dataset is mapped to the agent's target space. The state sequence of the training dataset is recorded at each node of the analysis and inference chain, and the node deviation value is calculated. The node deviation value is then fed back to the analysis and inference chain for optimization through the training feedback channel. A quality score is generated based on the feature coverage and state consistency of samples in the optimized analysis inference chain. A standardized sample library is then output based on the quality score.

5. The method according to claim 1, characterized in that, The data to be analyzed is input into an agent trained on a standardized sample library, and an analysis report is generated based on the analysis inference chain, including: The feature score is obtained by calculating the similarity between the feature vector of the data to be analyzed and the standardized sample library, and the state score is obtained by calculating the transition probability of the state sequence of the data to be analyzed on the analysis inference chain. A quality matrix of an agent is generated based on feature scores and state scores, and feature completion positions and state correction positions are determined based on the agent quality matrix. The feature information of the sample with the highest feature score in the standardized sample library is transferred to the feature completion position to obtain the completed feature data; The completed feature data is used for state sequence prediction. When the predicted state deviation exceeds a preset deviation threshold, state reconstruction is performed at the state correction position to obtain the reconstructed state sequence. The reconstructed state sequence is input into the agent, and reasoning analysis is performed based on the analysis inference chain to extract the state information and inference path information of each node and generate an analysis report.

6. A system for constructing an agent training dataset combining cross-modal learning, used to implement the method described in any one of claims 1-5, characterized in that, The system includes: The multimodal data processing module is used to obtain multimodal intelligence data from multi-source databases, extract feature information from the multimodal intelligence data, and project the feature information onto a preset semantic space to obtain cross-modal alignment features. The syntax tree construction module is used to construct the agent function call syntax tree based on cross-modal alignment features, parse the syntax tree into instruction sequences and construct analysis inference chains, and generate sample labels and interaction trajectories. The task decomposition module is used to construct a task dependency graph using sample labels and interaction trajectories, decompose the task dependency graph into multiple parallel decision branches, dynamically adjust the processing strategy of each branch based on the analysis inference chain, and form a training dataset. The data optimization module is used to map the training dataset to the target space of the agent, feed the analysis results back to the analysis inference chain optimization through the training feedback channel, and output a standardized sample library. The intelligent analysis module is used to input the data to be analyzed into an agent trained on a standardized sample library, and generate an analysis report based on the analysis inference chain.

7. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1 to 5.

Images