LLM-based cross-system heterogeneous metadata intelligent acquisition method and system

Abstract

The invention discloses a cross-system heterogeneous metadata intelligent acquisition method and system based on LLM, and the method comprises the following steps: S1, collecting and preprocessing heterogeneous system data, and constructing a heterogeneous graph; s2, analyzing nodes by the large language model, and splicing structural features to form initial representation of the nodes; s3, constructing a heterogeneous graph Transform, executing multi-layer propagation and low-rank decomposition, and outputting final node representation; s4, generating a field alignment path, an interface mapping path and a dependency path, and planning an acquisition path and an acquisition sequence; and S5, executing an acquisition task, converting a structured format, writing into a metadata center, and performing closed-loop optimization. According to the method, intelligent acquisition, semantic analysis and automatic integration of cross-system heterogeneous metadata are realized, and the efficiency and intelligent level of metadata management in a multi-source data environment are greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of data management, and particularly to an intelligent acquisition method and system for cross-system heterogeneous metadata based on LLM. Background Art

[0002] In the context of the increasing complexity of current information systems and the trend of data heterogeneity, enterprises and organizations usually need to integrate data systems from multiple sources, such as relational databases, non-relational databases, big data platforms, and distributed storage. These heterogeneous systems have significant differences in metadata structure, data type, semantic expression, and interface protocol, resulting in extremely heavy work in cross-system metadata acquisition and management. Traditional data integration methods usually rely on manual configuration of acquisition scripts or adaptation using fixed templates, and are unable to flexibly handle the frequent changes of source systems and complex semantic differences. Currently, some ETL-process-based tools lack the ability of fine-grained semantic understanding and structure recognition when dealing with complex semi-structured and unstructured data, and it is difficult to support the requirements of dynamic metadata management.

[0003] With the development of artificial intelligence, large language models have shown superior performance in semantic parsing, language understanding, etc. Graph neural networks, especially heterogeneous graph neural structures, have also been widely used in modeling complex multi-type relationships. However, the following defects still generally exist in the prior art: First, although large language models can support semantic parsing, when facing complex system metadata, they lack a deep fusion mechanism with system context and structural features; Second, when graph neural networks process high-dimensional features and multi-class edge nodes, they are easily interfered by redundant paths, resulting in semantic degradation and information redundancy, and it is difficult to output accurate node representations; Third, metadata acquisition paths and scheduling strategies are often set based on manual experience, lacking an interpretable semantic matching mechanism and structural stability evaluation, and unable to achieve automated planning; Fourth, there is a lack of a closed-loop optimization mechanism for feedback in the acquisition process, resulting in the inability of models and strategies to adaptively adjust.

[0004] Therefore, how to provide an intelligent acquisition method and system for cross-system heterogeneous metadata based on LLM is an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0005] An object of the present invention is to propose a method and system for intelligent acquisition of cross-system heterogeneous metadata based on LLM. The present invention fully integrates the semantic understanding ability of the large language model and the structural modeling advantages of the heterogeneous graph neural network, constructs a multi-layer information propagation network under the low-rank optimization mechanism, and combines field alignment, interface mapping and path planning strategies to achieve automatic identification, semantic parsing, path planning and dynamic acquisition of metadata for structured, semi-structured and unstructured data. This method has the advantages of high automation, strong semantic accuracy, strong cross-system adaptability and strong sustainable optimization ability, significantly improving the metadata integration efficiency and management intelligence level in a multi-source heterogeneous system environment.

[0006] The method for intelligent acquisition of cross-system heterogeneous metadata based on LLM according to an embodiment of the present invention includes the following steps:

[0007] S1. Collect structured, semi-structured and unstructured data in multiple heterogeneous information systems and perform preprocessing to construct a heterogeneous graph containing nodes and edges;

[0008] S2. Based on the constructed heterogeneous graph, extract the structural attribute information of each node to generate a node structure feature vector, call the large language model to perform semantic parsing on the original description information of each node to generate a semantic representation vector, and embed and splice it with the node structure feature vector to form an initial node representation;

[0009] S3. Construct a heterogeneous graph Transformer network, input the initial node representation, set the node type mapping and edge type attention mechanism, perform multi-layer information propagation, calculate the attention weight and message matrix in each layer, perform low-rank decomposition on the attention weight and message matrix, introduce a rank-aware mask mechanism in the multi-layer information propagation process, and combine the semantic representation vector with the edge connection strength in the heterogeneous graph to filter redundant paths and output the final node representation;

[0010] S4. Based on the final node representation, generate field alignment relationships, interface mapping tables and dependency paths, plan metadata acquisition paths, and formulate cross-system acquisition sequences and scopes according to confidence levels to determine metadata acquisition tasks;

[0011] S5. Execute the metadata acquisition task, uniformly convert the original metadata entries collected from each heterogeneous system into a structured format, and write them into the metadata center, dynamically update the prompt template of the large language model and the rank parameters of the heterogeneous graph Transformer, and construct a sustainable optimization closed-loop process for heterogeneous metadata acquisition.

[0012] Optionally, the structured, semi-structured and unstructured data specifically includes database table structures, log files, API documents, data directories and other source system metadata information.

[0013] Optionally, the preprocessing of the structured, semi-structured, and unstructured data specifically includes format parsing, field extraction, annotation cleaning, key-value pair extraction, structure mapping, and encoding normalization operations.

[0014] Optionally, the S2 specifically includes:

[0015] S21. For each node in the constructed heterogeneous graph, extract the associated original description information, where the original description information includes the field name, data type, name of the data table or interface where it is located, annotation content, and the corresponding source system identifier, to form an original metadata text set;

[0016] S22. Organize the original metadata text set into a natural language input sequence according to a preset structured input template, input it into a large language model based on the BERT structure and fine-tuned for domain tasks, perform semantic understanding and context encoding operations, and obtain the semantic representation vector of the node;

[0017] S23. For each node, extract the corresponding structural information features, where the structural information features include the encoded form of the field name, data type encoding, system type encoding, and the logical level or call depth of the node in the system, and construct a node structure feature vector;

[0018] S24. Perform a concatenation operation on the semantic representation vector and the node structure feature vector to construct a combined vector;

[0019] S25. Use the combined vector as the input representation of the node in the heterogeneous graph Transformer network to form the initial representation of the node.

[0020] Optionally, the S3 specifically includes:

[0021] S31. Construct a heterogeneous graph Transformer network, and set the initial representation of the node as the input;

[0022] S32. Define a node type mapping function to encode the types of all nodes in the heterogeneous graph into node type vectors, which are used as the information propagation conditions for the heterogeneous graph Transformer network. The node type mapping function is a type encoding function constructed based on the system source, data mode, and entity category to which the node belongs;

[0023] S33. Construct an edge type attention mechanism, set an independent attention weight matrix for each type of edge, and perform differential attention calculation based on the edge type attributes;

[0024] S34. Construct the heterogeneous graph Transformer network into a stacked network framework with multiple propagation structures. In each propagation layer, input the node representations output by the previous layer, combine the node type vectors and edge type attention weights, and based on the adjacency relationship in the heterogeneous graph, perform weighted aggregation on the neighbor node representations of each node to generate a message mapping matrix corresponding to the edge type, and execute the node representation update operation;

[0025] S35. In each propagation layer, perform low-rank approximation operations on the attention weight matrix and the message mapping matrix generated in the current layer respectively. Concatenate the attention weight matrix and the message mapping matrix into a joint representation matrix, and decompose it into the product form of two low-dimensional dense matrices by using the low-rank decomposition operation to generate a low-rank decomposition result matrix;

[0026] S36. On the basis of low-rank decomposition, introduce a rank-aware gating mechanism, construct an adjustment function that fuses rank contribution, attention saliency, and edge connection strength, and for each propagation path Construct a gating factor , set a gating threshold function based on the gating factor, perform a gating screening operation, and construct a structural path mask matrix: ;

[0027] Wherein, is the matrix trace function, is the path attention score, is the edge connection strength, is the rank upper limit corresponding to the node type, is the adjustment parameter, is the Sigmoid activation function, is the propagation path The left singular matrix representation corresponding to it in the low-rank decomposition;

[0028] S37. Based on the gating factor , perform weighted adjustment on the path attention scores of each propagation path in the heterogeneous graph Transformer network, and replace the path attention scores with the weighted results, only retaining the effective paths with significant gating factor values, and filtering the propagation information of low-correlation edges;

[0029] S38. According to the distribution of each node type in the heterogeneous graph Transformer network, fuse semantic entropy, intra-type structural centrality, and inter-type difference degree, and set the rank upper limit of the node type : ;

[0030] Wherein, is the node type The information entropy, is the node type The average degree centrality of is the node type The Jensen-Shannon divergence between the non-type node distribution is the adjustment factor, is the global base constant of the rank, and represents the probability distribution of the embedding vector, is the set of representation vectors of all nodes corresponding to the node type t;

[0031] S39. According to the rank upper limit corresponding to each node type, perform a rank truncation operation on the low-rank decomposition result matrix, and retain the rank-dimensional subspace corresponding to each type;

[0032] S310. Perform an element-wise multiplication operation on the low-rank decomposition result matrix after rank truncation and the structure path mask matrix to construct a node representation matrix with path selectivity and rank constraint characteristics, which is used as the output node representation of the current layer and input to the next propagation layer as the node initial input;

[0033] S311. After all propagation layers are propagated, output the representation of the nodes in the final layer as the final node representation of the heterogeneous graph Transformer network.

[0034] Optionally, the S4 specifically includes:

[0035] S41. Receive the final node representation output by the heterogeneous graph Transformer network, and set the feature matching function , where and respectively represent the final node representations of the field nodes in two heterogeneous systems: ;

[0036] Among them, is the weighting parameter, represents the semantic cosine similarity, represents the node and The structure dependency matrix in the heterogeneous graph Transformer network, represents the norm of the vector;

[0037] S42. Based on the similarity score calculated by the feature matching function, use the set threshold to screen the candidate field node pairs to generate a set of field alignment relationships ;

[0038] S43. Combine the field alignment relationship set with the node connection relationship in the heterogeneous graph Transformer network to construct a set of interface candidate pairs, and introduce an interface mapping function , map the field alignment relationship to the parameter docking relationship between system interfaces, and form an interface mapping table: ;

[0039] Among them, and are the interfaces of the source system and the target system respectively, and are weight coefficients, represents the node semantic similarity, represents the interface structure similarity, represents the pair when the function reaches the maximum value;

[0040] S44. Based on the interface mapping table and the heterogeneous graph Transformer network, execute the path extraction algorithm, generate the field collection path between field nodes, construct a path triple sequence including fields, interfaces, and call relationships, and form a dependency path graph;

[0041] S45. Comprehensively evaluate each field collection path according to the field matching degree, interface mapping consistency, and structural path stability corresponding to each path in the dependency path graph, and grade them according to the overall credibility, and output the initial score value of the field collection level;

[0042] S46. Set the confidence level threshold according to the initial score value of the field collection level, divide the field collection path into three levels: high, medium, and low, and formulate the corresponding cross-system collection priority and path coverage range, and generate the initial list of field collection tasks;

[0043] S47. Introduce a structural semantic consensus mechanism to perform a consensus aggregation operation on the redundant field nodes in the initial list of field collection tasks, fuse the path context features of the field nodes in the path triple sequence, the mapping deviation between the output of the interface mapping function and the interface mapping table, and reconstruct the field alignment relationship;

[0044] S48. Based on the reconstructed field alignment relationship, interface mapping table, and dependency path graph, finally determine the metadata collection task, and generate a multi-dimensional collection instruction set including field pairs, interface mapping pairs, path triples, and confidence levels.

[0045] Optionally, the specific content of S5 includes:

[0046] S51. Based on the determined metadata collection tasks, sequentially connect multiple heterogeneous information systems, execute the collection operations indicated by field pairs, interface mapping pairs, and path triples to obtain original metadata entries;

[0047] S52. According to the system types of the heterogeneous information systems, load the corresponding metadata collection plugins, perform structural parsing on the original metadata entries, and extract field content, interface parameters, and path context information;

[0048] S53. Uniformly convert the collected original metadata entries into a structured format according to the set field mapping rules, semantic tag templates, and system structure specifications, and construct a metadata structure unit with consistent system sources and complete semantics;

[0049] S54. Write the metadata structure unit converted into a structured format into the metadata center, and automatically trigger data consistency verification, redundancy elimination, and update marking;

[0050] S55. Dynamically update the prompt template of the large language model based on field semantic deviations, interface execution exceptions, and path coverage changes during the collection process;

[0051] S56. Combine the information propagation effects, attention distribution changes, and low-rank decomposition performance feedback of each layer of the heterogeneous graph Transformer during the collection process to update the rank parameters of the heterogeneous graph Transformer, and construct a sustainable optimization closed-loop process for heterogeneous metadata collection.

[0052] The intelligent cross-system heterogeneous metadata collection system based on LLM according to the embodiment of the present invention includes the following modules:

[0053] The data collection and preprocessing module is used to collect structured, semi-structured, and unstructured data, complete preprocessing, and construct a heterogeneous graph;

[0054] The semantic parsing module is used to call the large language model to parse the node description information, generate semantic representation vectors, and splice them with structural feature embeddings to form the initial node representation;

[0055] The graph modeling module is used to construct a heterogeneous graph Transformer network, input the initial node representation, execute multi-layer information propagation and low-rank decomposition, and output the final node representation;

[0056] The path planning module is used to generate field alignment relationships, interface mapping tables, and dependency path graphs based on the final node representation, plan the collection path, and determine the collection tasks;

[0057] The task execution module is used to execute the collection tasks, load the collection plugins to obtain the original metadata entries, convert them into structured metadata, and generate a standardized collection result to write into the metadata center;

[0058] An intelligent optimization module for dynamically updating the prompt template of the large language model according to the standardized acquisition results and optimizing the rank parameters of the heterogeneous graph Transformer network based on propagation feedback.

[0059] The beneficial effects of the present invention are as follows:

[0060] By constructing a metadata acquisition method that integrates a large language model and a heterogeneous graph Transformer network, the present invention significantly improves the intelligent level and automation ability of cross-system heterogeneous metadata management. Compared with existing methods, the present invention not only supports the unified modeling and parsing of structured, semi-structured, and unstructured data, but also realizes efficient information propagation and redundant path suppression by constructing a heterogeneous graph, generating initial node representations, introducing node type and edge type attention mechanisms, and combining low-rank approximation and rank-aware control mechanisms, enhancing the accuracy and efficiency of semantic modeling.

[0061] By constructing field alignment relationships, interface mapping tables, and dependency path graphs, the present invention effectively supports the dynamic planning and confidence evaluation of acquisition paths, and then improves the stability and accuracy of multi-source field mapping by combining a structural semantic consensus mechanism. Finally, during the execution of the acquisition task, with the help of a structure unified conversion module and a prompt parameter optimization mechanism, the present invention realizes the dynamic adjustment of the large language model prompt template and the Transformer rank parameters, constructs a metadata acquisition process with closed-loop optimization capabilities, and has strong robustness, adaptability, and scalability. This method can be widely applied to enterprise-level data governance and intelligent data asset management scenarios in multi-system, multi-format, and multi-structure data environments, and has significant practical value and promotion prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. They are used together with the embodiments of the present invention to explain the present invention, and do not constitute a limitation to the present invention. In the drawings:

[0063] Figure 1 is a flowchart of the cross-system heterogeneous metadata intelligent acquisition method based on LLM proposed by the present invention;

[0064] Figure 2 is a schematic structural diagram of the cross-system heterogeneous metadata intelligent acquisition system based on LLM proposed by the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0065] Now, the present invention will be further described in detail with reference to the accompanying drawings. These drawings are all simplified schematic diagrams, only showing the basic structure of the present invention in a schematic manner, so they only show the components related to the present invention.

[0066] Refer toFigure 1 , an LLM-based intelligent cross-system heterogeneous metadata acquisition method, comprising the following steps:

[0067] S1. Collect structured, semi-structured, and unstructured data from multiple heterogeneous information systems and perform preprocessing to construct a heterogeneous graph containing nodes and edges;

[0068] S2. Based on the constructed heterogeneous graph, extract the structural attribute information of each node to generate a node structure feature vector, call a large language model to perform semantic parsing on the original description information of each node to generate a semantic representation vector, and splice it with the node structure feature vector embedding to form an initial node representation;

[0069] S3. Construct a heterogeneous graph Transformer network, input the initial node representation, set the node type mapping and edge type attention mechanism, perform multi-layer information propagation, calculate the attention weight and message matrix in each layer, perform low-rank decomposition on the attention weight and message matrix, introduce a rank-aware mask mechanism during the multi-layer information propagation process, combine the semantic representation vector with the edge connection strength in the heterogeneous graph, filter redundant paths, and output the final node representation;

[0070] S4. Based on the final node representation, generate field alignment relationships, interface mapping tables, and dependency paths, plan the metadata acquisition path, and formulate the cross-system acquisition order and scope according to confidence levels to determine the metadata acquisition task;

[0071] S5. Execute the metadata acquisition task, uniformly convert the original metadata entries collected from each heterogeneous system into a structured format, and write them into the metadata center, dynamically update the prompt template of the large language model and the rank parameters of the heterogeneous graph Transformer, and construct a sustainable optimization closed-loop process for heterogeneous metadata acquisition.

[0072] The cross-system heterogeneous metadata intelligent acquisition method based on large language models and heterogeneous graph Transformer networks proposed by the present invention significantly improves the automated parsing and unified management capabilities of metadata in heterogeneous data environments. By constructing a heterogeneous graph containing nodes and edges, it effectively integrates complex information structures from structured, semi-structured, and unstructured data sources, enhancing the semantic and structural linkage between data. A large language model is introduced for node semantic parsing, and combined with structural embedding to form an initial node representation, realizing the fusion expression of semantic and structural information. Through multi-layer information propagation by the heterogeneous graph Transformer network, and the introduction of low-rank decomposition and rank-aware masking mechanisms, the information extraction efficiency and expression accuracy of the model are improved, and the interference of redundant paths is suppressed. Based on the final node representation, the present invention generates field alignment relationships, interface mapping tables, and dependency paths, forming a metadata acquisition task planning strategy with a confidence grading mechanism. Through the structured conversion of the acquisition results and the dynamic update of model parameters, the closed-loop optimization of the acquisition process is achieved. This method has the beneficial effects of strong structural unity, high semantic adaptation accuracy, excellent automatic adjustment ability, and good scalability, and is applicable to intelligent data governance and metadata integration scenarios in multi-source system environments.

[0073] In this embodiment, the structured, semi-structured, and unstructured data specifically includes database table structures, log files, API documents, data directories, and other source system metadata information.

[0074] In this embodiment, the preprocessing of the structured, semi-structured, and unstructured data specifically includes format parsing, field extraction, annotation cleaning, key-value pair extraction, structure mapping, and encoding normalization operations. Format parsing refers to identifying and parsing the content structure according to the data source type; field extraction refers to extracting field names, data values, and their attribute information from data records; annotation cleaning is used to remove irrelevant content, format noise, or redundant annotations in field descriptions to make the text description more standardized; key-value pair extraction mainly identifies and structurally expresses the "attribute-value" information hidden in unstructured text; structure mapping is to establish an association between the extracted fields and their corresponding business entities or interface structures to form nodes and edges in the graph structure; the encoding normalization operation is used to unify field naming rules, data type expressions, and encoding formats to ensure the semantic and structural consistency and alignability of data between different systems.

[0075] In this embodiment, S2 specifically includes:

[0076] S21. For each node in the constructed heterogeneous graph, extract the associated original description information, where the original description information includes field names, data types, the names of the data tables or interfaces where they are located, annotation contents, and the corresponding source system identifiers, to form an original metadata text set;

[0077] S22. Organize the original metadata text set into a natural language input sequence according to a preset structured input template, and input it into a large language model based on the BERT structure and fine-tuned by domain tasks to perform semantic understanding and context encoding operations to obtain the semantic representation vector of the node. The execution of semantic understanding and context encoding operations specifically refers to inputting the original metadata text sequence organized according to the structured input template into a large language model based on BERT and fine-tuned by specific domain metadata tasks. First, the model tokenizes and embeds the input text, encoding each field, data type, interface name, comment, etc. into high-dimensional vectors. Then, through multiple layers of self-attention mechanisms, the model deeply explores the semantic associations and context relationships between various parts of the text, capturing and aggregating the context dependencies of words, phrases, and even the entire record. Finally, the large language model outputs a semantic representation vector that combines the business meaning of the node itself and its context relationship;

[0078] S23. For each node, extract the corresponding structural information features, where the structural information features include the encoded form of the field name, data type encoding, system type encoding, and the logical level or call depth of the node in the system, and construct a node structure feature vector;

[0079] S24. Perform a concatenation operation on the semantic representation vector and the node structure feature vector to construct a combined vector;

[0080] S25. Use the combined vector as the input representation of the node in the heterogeneous graph Transformer network to form the initial representation of the node.

[0081] This paper proposes a method for generating initial node representations based on the fusion of a large language model and structural features for semantic understanding and structural feature modeling of nodes in heterogeneous information systems. This method significantly improves the semantic parsing accuracy and structural adaptability of metadata. By extracting multidimensional information such as field names, data types, interface names, and annotations to form a raw metadata text set, the method comprehensively captures the business context and contextual semantics of the node. A domain-tuned large language model based on the BERT architecture is introduced for natural language processing, enabling a deep understanding of domain terms and their contextual relationships, generating representation vectors with high semantic expressiveness. Structural information such as field encoding, system type, and logical hierarchy is integrated into a structural feature vector, which is then concatenated with the semantic representation vector to form a combined vector, achieving a deep fusion of node semantics and structural features. The resulting initial node representation not only possesses context-sensitive semantic recognition capabilities but also takes into account the differential expression of system hierarchical structure and logical position, providing a high-quality input foundation for multi-layer information propagation and precise modeling in heterogeneous graph Transformer networks. This method offers the advantages of accurate representation, strong structural awareness, and good transferability, providing key support for unified modeling and high-precision parsing of multi-source heterogeneous metadata.

[0082] In this embodiment, S3 specifically includes:

[0083] S31. Build a heterogeneous graph Transformer network and set the initial node representation as input;

[0084] S32. Define a node type mapping function to encode the types of all nodes in the heterogeneous graph into node type vectors as the information propagation condition of the heterogeneous graph Transformer network. The node type mapping function is a type encoding function constructed based on the system source, data mode, and entity category of the node.

[0085] S33. Construct an edge type attention mechanism, set an independent attention weight matrix for each type of edge, and perform differentiated attention calculation based on edge type attributes;

[0086] S34. Construct the heterogeneous graph Transformer network into a stacked network framework with multiple propagation structures. In each propagation layer, input the node representations output by the previous layer, combine the node type vectors and edge type attention weights, and according to the adjacency relationship in the heterogeneous graph, perform weighted aggregation on the neighbor node representations of each node to generate a message mapping matrix corresponding to the edge type, and perform node representation update operations. The weighted aggregation of the neighbor node representations of each node according to the adjacency relationship in the heterogeneous graph to generate a message mapping matrix corresponding to the edge type specifically means that in each propagation layer of the heterogeneous graph Transformer network, according to the adjacency relationship of the current heterogeneous graph, identify all the direct neighbor nodes of each target node. For edges of the same type or different types, apply the edge type attention weights set in advance or calculated dynamically to weight the representation vectors of these neighbor nodes. Each target node will combine the edge type attention weights according to the types, structures, and edge attributes of all its neighbor nodes, and weight and sum or aggregate the feature vectors of all neighbor nodes according to the weights, and finally form a message vector for this node and this edge type. All such weighted results are collected and arranged according to the edge type, which constitutes the message mapping matrix of this propagation layer;

[0087] S35. In each propagation layer, perform low-rank approximation operations on the attention weight matrix and the message mapping matrix generated in the current layer respectively. Concatenate the attention weight matrix and the message mapping matrix into a joint representation matrix, and decompose it into the product form of two low-dimensional dense matrices by using low-rank decomposition operations to generate a low-rank decomposition result matrix;

[0088] S36. On the basis of low-rank decomposition, introduce a rank-aware gating mechanism, construct a regulation function that fuses rank contribution, attention significance, and edge connection strength, and for each propagation path Construct a gating factor , set a gating threshold function based on the gating factor, perform gating screening operations, and construct a structural path mask matrix: ;

[0089] Among them, is the matrix trace function, is the path attention score, is the edge connection strength, is the rank upper limit corresponding to the node type, is the adjustment parameter, is the Sigmoid activation function, is the propagation path is the left singular matrix representation corresponding to the propagation path in the low-rank decomposition;

[0090] Gating factor Its core purpose is to dynamically adjust the retention and suppression of different paths during information dissemination to improve the accuracy of path selection and the efficiency of network expression. The gating factor combines three key factors: First, the rank contribution degree of the path in the low-rank decomposition result. By calculating the structural projection intensity of the path in the singular matrix, its impact on the low-rank feature space is measured; Second, the attention weight corresponding to this path, representing the degree of attention and significance of this edge in the heterogeneous graph structure; Third, the connection strength of the edge, which is used to reflect the structural compactness or interaction frequency between two nodes. The three factors are weighted and summed after being multiplied by the adjustment weights respectively, and the gating factor is generated through normalization by a non-linear activation function. The finally obtained gating factor can reflect the comprehensive role of the path in semantic contribution, structural stability, and sparse control, providing an accurate basis for path screening and information fusion, thereby suppressing redundant propagation paths while maintaining the transmission of effective information and improving the robustness and expression ability of the model under complex heterogeneous graph data.

[0091] S37. Based on the gating factor , the path attention scores of each propagation path in the heterogeneous graph Transformer network are weighted and adjusted, and the weighted result is used to replace the path attention scores. Only the effective paths with significant gating factor values are retained, and the propagation information of low-correlation edges is filtered. The weighted adjustment of the path attention scores of each propagation path in the heterogeneous graph Transformer network specifically means that in each propagation layer, the system calculates the corresponding gating factor for each path. The gating factor reflects the comprehensive importance of this path in multiple dimensions such as structural contribution, attention significance, and edge connection strength. The original attention score of each path is multiplicatively weighted with the gating factor of this path to obtain the new weighted attention score;

[0092] S38. According to the distribution of each node type in the heterogeneous graph Transformer network, fuse semantic entropy, intra-type structural centrality, and inter-type difference degree, and set the rank upper limit of the node type : ;

[0093] Among them, is the information entropy of the node type , is the average degree centrality of the node type , is the Jensen-Shannon divergence between the node type and the distribution of non-type nodes, is the adjustment factor, is the global basic constant of the rank, and Indicates the probability distribution of the embedding vector is the set of representation vectors of all nodes corresponding to node type t;

[0094] Node type rank upper bound Its core function is to assign different rank constraints to different node types during the low-rank approximation process, thereby enhancing the model's adaptability and expression accuracy for multi-type heterogeneous data. The rank upper bound comprehensively considers three aspects of structural and semantic features: First, the information entropy index of the node type, which measures the distribution uncertainty of nodes of this type in the semantic space. The higher the information entropy, the more complex the semantics carried by nodes of this type, and the stronger the required representation ability; Second, the average degree centrality of the node type in the structure diagram, which reflects the structural importance of this type of node in the system. The higher the centrality, the more extensive its connections and the greater its contribution to information dissemination; Third, the Jensen-Shannon divergence between the distribution of this node type and the distributions of other types of nodes, which is used to measure the semantic or structural distribution differences between different types of nodes. The greater the difference, the stronger the independence of this type, and the higher-dimensional expression space is required to maintain its unique features. The above three types of features are superimposed in a weighted manner on the basis of the global basic rank, and finally the rank upper bound of each node type is determined to guide the construction of each type of subspace in the low-rank approximation operation, realizing a more reasonable and accurate rank allocation mechanism.

[0095] S39. According to the rank upper bound corresponding to each node type, perform a rank truncation operation on the low-rank decomposition result matrix, and retain the rank dimension subspace corresponding to each type. The specific operation of performing a rank truncation operation on the low-rank decomposition result matrix refers to, according to the rank upper bound set in advance for each node type, in the result matrix obtained by low-rank decomposition, only retain the first several important principal components or feature dimensions of the eigen-subspace corresponding to each type of node, and the remaining high-dimensional or low-contribution features are discarded or set to zero;

[0096] S310. Perform an element-wise multiplication operation on the low-rank decomposition result matrix after rank truncation and the structural path mask matrix to construct a node representation matrix with path selectivity and rank constraint characteristics, which is used as the output node representation of the current layer and input to the next propagation layer as the node initial input. The construction of the node representation matrix with path selectivity and rank constraint characteristics specifically means that in each propagation layer, first perform a rank truncation operation on the low-rank decomposition result matrix obtained in the previous layer according to the node type, and only retain the effective feature subspace within the rank upper limit of each type of node; then multiply the rank-truncated matrix and the structural path mask matrix element by element, retain the path information marked as valid in the structural path mask, and set the path information that is invalid or gated and suppressed to zero. The finally obtained node representation matrix has two aspects of characteristics: on the one hand, it only contains the high-contribution low-dimensional features retained through dynamic rank allocation, realizing the adaptive control of the expression ability of different node types; on the other hand, it only retains the information on the effective propagation paths screened by the gating mechanism in the current propagation layer, filtering out irrelevant or low-value structural paths.

[0097] S311. After the propagation in all propagation layers is completed, output the representation of the nodes in the final layer as the final node representation of the heterogeneous graph Transformer network.

[0098] In the present invention, by constructing a heterogeneous graph Transformer network that integrates node type awareness and edge type attention mechanism, and combining a multi-layer propagation structure and a low-rank approximation optimization strategy, an efficient modeling of the complex semantic and structural relationships between nodes in a heterogeneous information system is achieved. In the construction stage, explicit modeling of heterogeneous features is realized through a node type mapping function and an edge type attention mechanism, enabling each type of node and edge to have different expression abilities during the propagation process. The introduction of the low-rank approximation decomposition operation effectively reduces the redundancy and resource overhead of propagation calculations while maintaining the main structural features of the representation matrix. Combining the rank-aware gating mechanism and the structural path mask matrix, guiding information to focus on propagation on high-contribution paths, significantly improving the utilization rate of effective edges and filtering out noise paths. Further, through a type-level dynamic rank control strategy, adaptively set the rank upper limit of each type of node based on semantic entropy and structural difference information, and construct a concise and discriminative node representation subspace. The finally output node representation integrates type awareness, structure selectivity, and rank constraint characteristics, providing a high-quality semantic structure representation for subsequent tasks such as field alignment and path planning, and having the beneficial effects of high propagation efficiency, high modeling accuracy, and strong cross-system generalization ability.

[0099] In this embodiment, the specific content of S4 includes:

[0100] S41. Receive the final node representation output by the heterogeneous graph Transformer network and set a feature matching function , where and respectively represent the final representations of the field nodes in two heterogeneous systems: ;

[0101] Among them, is the weighting parameter, represents the semantic cosine similarity, represents the node and in the heterogeneous graph Transformer network's structural dependency matrix, represents the norm of the vector;

[0102] The feature matching function is used to measure the matching degree between the field nodes in two heterogeneous systems. It comprehensively considers two aspects of factors: semantic similarity and structural dependency information, and constructs a field matching scoring function that takes into account both semantic and structural features. In the feature matching function, first, the semantic cosine similarity between the two node representations is calculated to reflect whether the two fields express the same meaning in the semantic space. The closer the semantics, the higher the cosine similarity, indicating that the field meanings are more consistent. Second, the product of the norms of the two node representation vectors and the inner product after their projection through the structural dependency matrix are considered. This part reflects the propagation paths, connection relationships, and upstream and downstream dependencies of the two fields in the heterogeneous graph structure. The overall function is controlled by a balance factor to adjust the weight ratio of the two parts, so that when the system structures are quite different or the semantics are ambiguous, more reliance can be placed on the structural information for matching, and when the structures are the same but the semantic differences are significant, more reliance can be placed on the semantic similarity. Through this function, it is possible to effectively identify field pairs with the same meaning but different names, or different structures but equivalent functions in heterogeneous systems.

[0103] S42. Based on the similarity scores calculated by the feature matching function, use a set threshold to screen the candidate field node pairs to generate a set of field alignment relationships , and the use of the set threshold to screen the candidate field node pairs specifically means that after calculating the feature matching similarity scores of all candidate field node pairs, a judgment threshold is preset in advance. All field node pairs with similarity scores higher than this threshold are retained, considering that they can achieve effective alignment semantically and structurally; while the node pairs with similarity scores lower than the threshold are excluded or not involved in the alignment. This screening operation ensures that only field node pairs with high semantic consistency and structural relevance are included in the set of field alignment relationships, effectively excluding invalid field pairs with inconsistent semantics, incompatible structures, or unclear mapping relationships, providing a high-quality and accurate field alignment basis for links such as interface mapping, path planning, and metadata collection;

[0104] S43. Combine the set of field alignment relationships with the node connection relationships in the heterogeneous graph Transformer network to construct a set of candidate interface pairs, and introduce an interface mapping function , map the field alignment relationships to the parameter docking relationships between system interfaces, and form an interface mapping table: ;

[0105] Among them, and are the interfaces of the source system and the target system to which they belong respectively, and are weight coefficients, represents the semantic similarity of nodes, represents the structural similarity of interfaces, represents the pair when the function obtains the maximum value;

[0106] The interface mapping function is used to construct the mapping relationship between system interfaces. Its core idea is to select a pair of interfaces with the best semantic and structural matching degree as the bridge for field docking among all possible combinations of source interfaces and target interfaces. By weighted fusion of two parts of information: one is the semantic similarity between field nodes, which is used to measure whether two fields are consistent in business meaning; the other is the structural similarity between interfaces, which reflects the consistency between two interfaces in terms of parameter form, call path or data organization method. Through the weighted sum of these two parts of similarity, it is possible to comprehensively evaluate whether a certain field pair is suitable for mapping to a specific set of interface pairs. Finally, the maximum value function is used to select the set with the highest matching score among all possible interface pairs as the interface docking relationship of the field between the two systems. It significantly improves the efficiency and accuracy of automatic interface matching between heterogeneous systems, especially suitable for scenarios such as inconsistent interface names, different parameter orders or complex interface nesting levels, and provides a highly confident and structurally consistent interface connection basis for generating interface mapping tables, planning dependency paths and performing metadata collection tasks.

[0107] S44. Based on the interface mapping table and the heterogeneous graph Transformer network, execute the path extraction algorithm to generate the field collection paths between field nodes, construct a path triple sequence containing fields, interfaces, and call relationships, and form a dependency path graph. The specific implementation of executing the path extraction algorithm based on the interface mapping table and the heterogeneous graph Transformer network means using the final node representations output by the heterogeneous graph Transformer network and the structured interface mapping table to embed the semantically aligned and structurally compatible field nodes, interface nodes, and their associated relationships in the same graph model. The system, according to the interface mapping relationship, combines the connection mode, edge weight, and attention distribution of nodes in the heterogeneous graph, and automatically performs heuristic search. Systematically starting from the source field node, along the interface mapping and the actual call or dependency relationships in the graph, it searches for data transmission paths that can achieve cross-system collection. The heuristic search not only focuses on the semantic and structural connectivity between field nodes but also dynamically considers factors such as interface call order, path dependency depth, and node type constraints;

[0108] S45. Comprehensively evaluate each field collection path according to the field matching degree, interface mapping consistency, and structural path stability corresponding to each path in the dependency path graph, and grade them according to the overall credibility level to output the initial score value of the field collection level. The specific implementation of comprehensively evaluating each field collection path means that the system examines multiple key indicators such as the semantic matching degree of field pairs, interface mapping consistency, and path structure stability on each field collection path in the dependency path graph. After quantitatively scoring each indicator, through weighted aggregation or fusion strategies, a comprehensive score for each collection path is formed. The field matching degree is mainly calculated based on the semantic similarity between node representations, the interface mapping consistency is evaluated according to the matching results in the interface mapping function and the interface mapping table, and the structural path stability is comprehensively determined by considering factors such as path length, path branch complexity, and historical collection success rate. After all scores are aggregated, the system divides each field collection path into different levels of high, medium, and low according to the preset credibility grading standard, and outputs it as the basis for the priority and scheduling order of the collection task;

[0109] S46. Set the confidence level threshold according to the initial score value of the field collection level , divide the field collection paths into three levels of high, medium, and low, and formulate the corresponding cross-system collection priorities and path coverage ranges to generate the initial list of field collection tasks;

[0110] S47. Introduce a structural semantic consensus mechanism to perform a consensus aggregation operation on redundant field nodes in the initial list of field collection tasks, fuse the path context features of field nodes in the sequence of path triples, the mapping deviation between the output of the interface mapping function and the interface mapping table, and reconstruct the field alignment relationship. The specific operation of performing a consensus aggregation operation on redundant field nodes in the initial list of field collection tasks refers to, in multiple collection paths or field alignment results, for field nodes with similar names, semantic similarities, or overlapping business functions, based on their context information in the sequence of path triples, the semantic and structural features output by the interface mapping function, and the mapping deviation in the interface mapping table, performing merging, clustering, or duplicate removal on these redundant or duplicate field nodes. During the consensus aggregation process, the system automatically calculates the path correlation degree, interface consistency, and structural position of similar field nodes, determines whether they actually have the same business meaning or collection target under different systems and different paths, unifies their field alignment relationships, merges them into a single collection target, and eliminates the risk of duplicate collection or conflicting mapping;

[0111] S48. Based on the reconstructed field alignment relationship, interface mapping table, and dependency path graph, finally determine the metadata collection task and generate a multi-dimensional collection instruction set including field pairs, interface mapping pairs, path triples, and confidence levels.

[0112] Through constructing a field alignment relationship, an interface mapping table, and a dependency path graph based on the final representation of nodes, the present invention realizes the precise planning of cross-system metadata collection tasks and the generation of task lists. During the feature alignment process, a feature matching function is set by combining semantic similarity and structural dependency information, so that the alignment relationship between field nodes not only has semantic consistency but also accurately transmits the original structural logic between systems. The design of the interface mapping function integrates double similarities of semantics and structure, realizes the effective expansion of the field alignment result to the interface-level mapping relationship, and constructs a stable interface mapping table. On this basis, a path extraction algorithm is introduced to generate path triples composed of fields, interfaces, and call relationships, forming a traceable and computable dependency path graph. By comprehensively evaluating path matching, interface consistency, and structural stability, different trust levels are assigned to collection paths to provide support for task hierarchical scheduling. Further combined with the structural semantic consensus mechanism, aggregation and correction are performed on the redundancy and ambiguity in field alignment and interface mapping, enhancing the accuracy and stability of the collection task. Finally, a multi-dimensional collection instruction set is formed to realize the efficient generation and organization scheduling of cross-system and multi-type field-level metadata collection tasks, having significant advantages of high recognition accuracy, reasonable task planning, and stable structural matching.

[0113] In this embodiment, the specific content of S5 includes:

[0114] S51. Based on the determined metadata collection tasks, sequentially connect multiple heterogeneous information systems, execute the collection operations indicated by field pairs, interface mapping pairs, and path triples to obtain original metadata entries;

[0115] S52. According to the system types of the heterogeneous information systems, load the corresponding metadata collection plugins to perform structure parsing on the original metadata entries, and extract field content, interface parameters, and path context information. The structure parsing of the original metadata entries specifically refers to, based on the system types of the heterogeneous information systems, using the corresponding collection plugins or adapters to parse and process the collected original metadata entries according to a preset data pattern, automatically extract and separate contents such as field names, field values, data types, table or interface names to which they belong, hierarchical relationships, interface parameters, field comments, primary and foreign key information, etc., and restore their structured hierarchy and context dependencies in the source system;

[0116] S53. Uniformly convert the collected original metadata entries into a structured format according to the set field mapping rules, semantic tag templates, and system structure specifications, and construct a metadata structure unit with consistent system sources and complete semantics;

[0117] S54. Write the metadata structure unit converted into a structured format into the metadata center, and automatically trigger data consistency verification, redundancy elimination, and update marking;

[0118] S55. Dynamically update the prompt template of the large language model based on field semantic deviations, interface execution exceptions, and path coverage changes during the collection process;

[0119] S56. Combine the information propagation effects, attention distribution changes, and low-rank decomposition performance feedback of each layer of the heterogeneous graph Transformer during the collection process to update the rank parameters of the heterogeneous graph Transformer, and construct a sustainable and optimized closed-loop process for heterogeneous metadata collection.

[0120] The present invention constructs an automated metadata acquisition and structured processing mechanism for multiple heterogeneous information systems by formulating a detailed metadata acquisition process. The method first connects heterogeneous systems one by one based on the planned field pairs, interface mapping pairs, and path triples, and performs precise acquisition operations, thereby improving the accuracy and execution efficiency of cross-system acquisition tasks. The system automatically loads corresponding acquisition plugins according to the type characteristics of various source systems, parses the structure of the obtained original metadata entries, and accurately extracts field content, interface parameters, and path context information. The original entries are converted into a structured format via unified field mapping rules, semantic tag templates, and system structure specifications, generating metadata units with unified semantics and consistent structures, and written into the metadata center. This process also synchronously triggers data consistency verification, redundancy elimination, and update marking, ensuring the accuracy and timeliness of the central data. During the acquisition execution process, the system also continuously adjusts and optimizes the prompt template of the large language model by dynamically monitoring field semantic deviations, interface exceptions, and path coverage changes, enhancing the semantic parsing ability. The system dynamically updates the rank parameters of the heterogeneous graph Transformer network based on the propagation effect, attention change, and low-rank decomposition performance feedback during the acquisition process, ensuring that the model structure can adapt to the dynamic characteristics of the actual data distribution. The overall process forms a sustainable optimization metadata acquisition closed-loop system, significantly improving the semantic understanding consistency and metadata integration efficiency between heterogeneous systems, and having good scalability and intelligent level.

[0121] Reference Figure 2 , an intelligent cross-system heterogeneous metadata acquisition system based on LLM, includes the following modules:

[0122] The data acquisition and preprocessing module is used to acquire structured, semi-structured, and unstructured data, complete preprocessing, and construct a heterogeneous graph;

[0123] The semantic parsing module is used to call the large language model to parse node description information, generate semantic representation vectors, and splice them with structural feature embeddings to form the initial node representation;

[0124] The graph modeling module is used to construct a heterogeneous graph Transformer network, input the initial node representation, perform multi-layer information propagation and low-rank decomposition, and output the final node representation;

[0125] The path planning module is used to generate field alignment relationships, interface mapping tables, and dependency path maps based on the final node representation, plan the acquisition path, and determine the acquisition task;

[0126] The task execution module is used to execute the acquisition task, load the acquisition plugin to obtain the original metadata entry, and convert it into structured metadata, generate a standardized acquisition result and write it into the metadata center;

[0127] An intelligent optimization module for dynamically updating the prompt template of the large language model according to the standardized acquisition results and optimizing the rank parameters of the heterogeneous graph Transformer network based on propagation feedback.

[0128] Example 1:

[0129] To verify the feasibility of the present invention in implementation, the present invention is applied to the information center of a certain city. The information center faces the task of metadata integration among a large number of government affairs subsystems, including multiple heterogeneous data sources such as the population management system, the social security business system, and the tax information system. These systems are respectively constructed by different manufacturers, with inconsistent data standards, large differences in field definitions, and poor interface compatibility, posing huge challenges to information integration and data governance. Especially when carrying out the construction of the city-wide unified public service data directory, problems such as difficult field alignment, complex interface dependency paths, low metadata acquisition efficiency, and unstable acquisition content are faced.

[0130] To solve the above problems, the platform deploys the cross-system heterogeneous metadata intelligent acquisition method and system based on LLM proposed by the present invention. The project is deployed in the data computer room of the information center in March 2025 and takes 2 weeks for testing and acquisition optimization.

[0131] In the system initialization stage, the platform first collects various types of data such as data structure information, interface definition documents, system logs, and database annotation content in 11 business systems. The collected data includes structured field definition tables, semi-structured Swagger interface documents, as well as unstructured description documents and API logs, with a total data volume reaching 2.8 GB. The system automatically constructs these information into a heterogeneous graph structure containing 11,743 nodes and 45,926 edges, and calls the BERT model fine-tuned with government affairs data to perform semantic parsing to form high-quality initial node representations.

[0132] By constructing an improved heterogeneous graph Transformer network and introducing a node type-specific multi-channel propagation mechanism and a low-rank decomposition structure compression mechanism, the platform efficiently models the field node representations in each business system, generates high-confidence field alignment relationships and interface mapping tables, and constructs a total of 1,125 path triples for the dependency path graph.

[0133] The system uses the field semantic cosine similarity not lower than 0.85 and the interface structure similarity not lower than 0.75 as the screening thresholds, improving the field alignment matching rate to 93.6%, enhancing the stability of the interface mapping table by 22.4%, and the overall field acquisition confidence reaching an average of 0.91. Finally, 326 groups of effective field pairs and 87 groups of interface mapping pairs are identified, the acquisition path length is controlled between 3 and 7, and the proportion of priority-level field acquisition reaches 62.8%.

[0134] After the platform executes the collection task, it uniformly writes the structured metadata entries into the government affairs metadata center and automatically completes data consistency verification and redundancy elimination. The system also dynamically adjusts the large language model prompt template and the Transformer network rank parameters based on the feedback during collection. After three consecutive optimizations, the accuracy of node semantic representation is improved by about 6.1%, and the stability of path selection is increased by 11.7%.

[0135] Table 1 Heterogeneous System Field Alignment and Collection Effect Evaluation Table

[0136] The above Table 1 shows the field mapping and collection evaluation results of ten groups from the three major government affairs subsystems of population, social security, and taxation to the data aggregation platform. Generally speaking, the semantic matching degree remains above 0.87, and the core primary keys such as ID number and date of birth and time attributes reach 0.93 - 0.95, indicating that after semantic parsing and structure alignment by the large language model, the key information has formed a highly consistent business meaning across systems. The confidence level of interface mapping is generally in the range of 0.81 - 0.92, showing that the interface structure differences have been effectively converged through the mapping function. Especially, the consistency of the population system fields at the interface layer is the highest, reflecting the best compatibility between the population system and the interface specifications of the aggregation platform. The path stability scores are mostly higher than 0.83, and the path stability of high-priority fields is significantly higher, proving that after the construction of the dependency path graph and low-rank gating screening, the data flow of the core fields is collected through stable paths, which can reduce the risk of task failure or redundant calls.

[0137] From the perspective of the priority level distribution, high-level fields account for 60%, concentrating on high-value information such as identity, household register, time, and amount; medium-level fields are mainly fields such as business status or type, still retaining relatively high matching and stability, but there are slight differences in interface definition or path depth, and dynamic updates need to be concerned about in subsequent collection tasks. Combining the three indicators, it can be seen that the semantic matching degree is positively correlated with the interface confidence level. High semantic similarity is often accompanied by higher interface compatibility; while the path stability is affected by the system topology and the complexity of the call chain, and together with the interface confidence level, it determines the collection priority. The field mapping results of this batch verify the effectiveness of the method of the present invention in achieving high-precision field alignment, interface mapping, and path planning in heterogeneous system scenarios.

[0138] As described above, it is only the preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, makes equivalent substitutions or changes, and should be covered by the protection scope of the present invention.

Claims

1. An LLM-based intelligent cross-system heterogeneous metadata acquisition method, characterized in that, It includes the following steps: S1. Collect structured, semi-structured, and unstructured data from multiple heterogeneous information systems and perform preprocessing to construct a heterogeneous graph containing nodes and edges; S2. Based on the constructed heterogeneous graph, extract the structural attribute information of each node, generate a node structure feature vector, call a large language model to perform semantic parsing on the original description information of each node, generate a semantic representation vector, and splice it with the node structure feature vector to form an initial node representation; S3. Construct a heterogeneous graph Transformer network, input the initial node representation, set the node type mapping and edge type attention mechanism, perform multi-layer information propagation, calculate the attention weight and message matrix in each layer, perform low-rank decomposition on the attention weight and message matrix, introduce a rank-aware masking mechanism during the multi-layer information propagation process, combine the semantic representation vector and the edge connection strength in the heterogeneous graph, filter redundant paths, and output the final node representation; S4. Based on the final node representation, generate field alignment relationships, interface mapping tables, and dependency paths, plan the metadata collection path, and formulate the cross-system collection order and scope according to the confidence level, and determine the metadata collection task; S5. Execute the metadata collection task, uniformly convert the original metadata entries collected from each heterogeneous system into a structured format, and write them into the metadata center, dynamically update the prompt template of the large language model and the rank parameter of the heterogeneous graph Transformer, and construct a sustainable and optimized closed-loop process for heterogeneous metadata collection.

2. The intelligent cross-system heterogeneous metadata acquisition method based on LLM according to claim 1, wherein, The structured, semi-structured, and unstructured data specifically includes database table structures, log files, API documents, data directories, and other source system metadata information.

3. The intelligent cross-system heterogeneous metadata acquisition method based on LLM according to claim 1, wherein, The preprocessing of the structured, semi-structured, and unstructured data specifically includes format parsing, field extraction, annotation cleaning, key-value pair extraction, structure mapping, and encoding normalization operations.

4. The intelligent cross-system heterogeneous metadata acquisition method based on LLM according to claim 1, wherein, The specific content of S2 includes: S21. For each node in the constructed heterogeneous graph, extract the associated original description information, where the original description information includes the field name, data type, the name of the data table or interface where it is located, annotation content, and the corresponding source system identifier, to form an original metadata text set; S22. Organize the original metadata text set into a natural language input sequence according to a preset structured input template, input it into a large language model based on the BERT structure and fine-tuned by domain tasks, and perform semantic understanding and context encoding operations to obtain the semantic representation vector of the node; S23. For each node, extract the corresponding structural information features, where the structural information features include the encoded form of the field name, data type encoding, system type encoding, and the logical level or call depth of the node in the system, and construct a node structure feature vector; S24. Perform a splicing operation on the semantic representation vector and the node structure feature vector to construct a combined vector; S25. Use the combined vector as the input representation of the node in the heterogeneous graph Transformer network to form an initial node representation.

5. The intelligent acquisition method of cross-system heterogeneous metadata based on LLM according to claim 1, characterized in that, The specific content of S3 includes: S31. Construct a heterogeneous graph Transformer network and set the initial node representation as the input; S32. Define a node type mapping function to encode the types of all nodes in the heterogeneous graph into node type vectors, which serve as the information propagation conditions for the heterogeneous graph Transformer network. The node type mapping function is a type encoding function constructed based on the system source, data mode, and entity category to which the nodes belong; S33. Construct an edge type attention mechanism, set an independent attention weight matrix for each type of edge, and perform differential attention calculations based on the edge type attributes; S34. Construct the heterogeneous graph Transformer network into a stacked network framework with multiple propagation structures. In each propagation layer, input the node representations output by the previous layer, combine the node type vectors and edge type attention weights, and based on the adjacency relationships in the heterogeneous graph, perform weighted aggregation on the neighbor node representations of each node to generate a message mapping matrix corresponding to the edge type, and perform node representation update operations; S35. In each propagation layer, perform low-rank approximation operations on the attention weight matrix and message mapping matrix generated in the current layer respectively. Concatenate the attention weight matrix and message mapping matrix into a joint representation matrix, and decompose it into the product form of two low-dimensional dense matrices using low-rank decomposition operations to generate a low-rank decomposition result matrix; S36. On the basis of low-rank decomposition, introduce a rank-aware gating mechanism, construct a regulation function that fuses rank contribution, attention saliency, and edge connection strength, and for each propagation path Construct a gating factor , set a gating threshold function based on the gating factor, perform a gating screening operation, and construct a structural path mask matrix; S37. Based on the gating factor , the path attention scores of each propagation path in the heterogeneous graph Transformer network are weighted and adjusted, and the weighted result is used to replace the path attention score. Only the effective paths with significant gating factor values are retained, and the propagation information of low-correlation edges is filtered; S38. According to the node type distribution in the heterogeneous graph Transformer network, fuse semantic entropy, intra-type structural centrality, and inter-type difference degree to set the upper bound of the rank of node types ; ; S39. According to the rank upper limit corresponding to each node type, perform rank truncation operations on the low-rank decomposition result matrix, and retain the rank dimension subspace corresponding to each type; S310. Perform an element-wise multiplication operation on the rank-truncated low-rank decomposition result matrix and the structure path mask matrix to construct a node representation matrix with path selectivity and rank constraint characteristics, which serves as the output node representation of the current layer and is input to the next propagation layer as the node initial input; S311. After the propagation in all propagation layers is completed, output the representations of the nodes in the final layer as the final node representations of the heterogeneous graph Transformer network.

6. The intelligent cross-system heterogeneous metadata acquisition method based on LLM according to claim 1, characterized in that The specific content of S4 includes: S41. Receive the final node representation output by the heterogeneous graph Transformer network and set a feature matching function , where and respectively represent the final node representations of field nodes in two heterogeneous systems; S42. Using a set threshold for the similarity score calculated based on the feature matching function to screen the candidate field node pairs and generate a set of field alignment relationships ; S43. Combine the field alignment relationship set with the node connection relationship in the heterogeneous graph Transformer network to construct a set of interface candidate pairs, and introduce an interface mapping function , map the field alignment relationship to the parameter docking relationship between system interfaces, and form an interface mapping table; S44. Based on the interface mapping table and the heterogeneous graph Transformer network, execute a path extraction algorithm to generate field collection paths between field nodes, construct a path triple sequence including fields, interfaces, and call relationships, and form a dependency path graph; S45. Comprehensively evaluate each field collection path according to the field matching degree, interface mapping consistency, and structural path stability corresponding to each path in the dependency path graph, and perform grading based on the overall credibility to output the initial score value of the field collection level; S46. Set the confidence level threshold according to the initial score value of the field collection level , divide the field collection paths into three levels: high, medium, and low, and formulate corresponding cross-system collection priorities and path coverage ranges to generate an initial list of field collection tasks; S47. Introduce a structural semantic consensus mechanism to perform consensus aggregation operations on redundant field nodes appearing in the initial list of field collection tasks, fuse the path context features of field nodes in the path triple sequence, the mapping deviation between the output of the interface mapping function and the interface mapping table, and reconstruct the field alignment relationship; S48. Based on the reconstructed field alignment relationship, interface mapping table, and dependency path graph, finally determine the metadata collection task and generate a multi-dimensional collection instruction set including field pairs, interface mapping pairs, path triples, and confidence levels.

7. The intelligent cross-system heterogeneous metadata acquisition method based on LLM according to claim 1, characterized in that The specific content of S5 includes: S51. Based on the determined metadata collection tasks, sequentially connect multiple heterogeneous information systems, execute the collection operations indicated by field pairs, interface mapping pairs, and path triples, and obtain the original metadata entries; S52. According to the system types of the heterogeneous information systems, load the corresponding metadata collection plugins, perform structure parsing on the original metadata entries, and extract field content, interface parameters, and path context information; S53. Uniformly convert the collected original metadata entries into a structured format according to the set field mapping rules, semantic tag templates, and system structure specifications, and construct a metadata structure unit with consistent system sources and complete semantics; S54. Write the metadata structure unit converted into a structured format into the metadata center, and automatically trigger data consistency verification, redundancy elimination, and update marking; S55. Dynamically update the prompt template of the large language model based on the field semantic deviation, interface execution exception, and path coverage change during the collection process; S56. Combine the information propagation effect, attention distribution change, and low-rank decomposition performance feedback of each layer of the heterogeneous graph Transformer during the collection process, update the rank parameters of the heterogeneous graph Transformer, and construct a sustainable optimization closed-loop process for heterogeneous metadata collection.

8. The LLM-based cross-system heterogeneous metadata intelligent acquisition system is applied to the LLM-based cross-system heterogeneous metadata intelligent acquisition method according to any one of claims 1 to 7, and is characterized in that, It includes the following modules: Data collection and preprocessing module, which is used to collect structured, semi-structured, and unstructured data, complete preprocessing, and construct a heterogeneous graph; Semantic parsing module, which is used to call the large language model to parse the node description information, generate semantic representation vectors, and splice them with the structural feature embeddings to form the initial node representation; Graph modeling module, which is used to construct a heterogeneous graph Transformer network, input the initial node representation, execute multi-layer information propagation and low-rank decomposition, and output the final node representation; Path planning module, which is used to generate field alignment relationships, interface mapping tables, and dependency path maps based on the final node representation, plan the collection path, and determine the collection tasks; Task execution module, which is used to execute the collection tasks, load the collection plugins to obtain the original metadata entries, convert them into structured metadata, and generate a standardized collection result to be written into the metadata center; Intelligent optimization module, which is used to dynamically update the large language model prompt template according to the standardized collection result, and optimize the rank parameters of the heterogeneous graph Transformer network based on the propagation feedback.

Images