Database logical topology construction method based on large language model

Abstract

The application discloses a database logical topology construction method based on a large language model, which comprises the following steps: obtaining target database Schema information, using a large language model to complete the semantic metadata of a missing service to generate enhanced metadata; based on the enhanced metadata, performing multi-stage reasoning arrangement to construct a global logical topology network, using an adaptive partition strategy and an early stop mechanism to solve the context limitation problem of the large language model; performing information entropy determination on key data fields to identify feature types, using an orthogonal boundary synthesis method to construct a virtual sample set covering global feature values to obtain an enhanced topology network; establishing a Schema change monitoring mechanism, triggering an incremental adsorption algorithm to integrate change elements, and maintaining the validity of the correlation relationship through a timeliness confidence decay model to obtain a dynamic global logical topology. The application realizes automatic correlation mining and topology dynamic maintenance of a foreign key-free database, and improves the SQL generation accuracy of an intelligent query scene.

Description

Technical Field

[0001] This invention relates to the fields of database management, artificial intelligence and natural language processing, and in particular to a method for constructing database logical topology based on a large language model. Background Technology

[0002] As enterprises deepen their digital transformation, the scale of data assets is growing exponentially. Traditional database entity relationship diagrams, which are based on manual maintenance, can no longer meet the needs of managing massive amounts of data tables. With the rapid popularization of intelligent data querying, i.e., natural language to SQL technology, the accuracy of SQL generated by large language models is highly dependent on the quality of database schema information, and there are three technical bottlenecks in actual enterprise applications.

[0003] First, the lack of physical foreign keys leads to a scarcity of relational information. Modern software engineering, in pursuit of performance optimization or adaptation to sharded database architectures, commonly uses logical foreign keys instead of physical foreign key constraints. Relationships are maintained at the code level rather than explicitly defined in database metadata. This makes it impossible for large language models to obtain inter-table join relationships from system tables, hindering the generation of multi-table join queries and severely limiting the multi-table reasoning capabilities of intelligent query models.

[0004] Second, the lack of semantic depth in metadata leads to ambiguity. Database table and field names heavily utilize English abbreviations, initials of pinyin, or internal codes, lacking standardized Chinese annotations. This makes it difficult for large language models to accurately understand the business meaning of fields. For example, the field `crtime` might be misunderstood as creation time, certificate time, or contract time, causing semantic discrepancies in SQL generation.

[0005] Third, unknown data distribution leads to predicate illusion. When there is only a schema structure but no data feature samples, large language models cannot determine whether a field is enumerated or range-based, making it difficult to know the range and distribution pattern of data values. As a result, the generated filtering conditions often contain invalid values ​​or exceed the actual range, leading to the predicate illusion problem.

[0006] Existing technical solutions have significant limitations. Rule-based foreign key inference methods rely on predefined patterns and cannot adapt to flexible and ever-changing business relationships; random sampling of data samples inevitably loses key feature values ​​under long-tail distributions; and full-scale inference methods are limited by the context capacity of large language models and struggle to handle large-scale databases with thousands of tables. Therefore, there is an urgent need for a fully automated method that can automatically infer missing metadata from large models, mine implicit inter-table relationships, and construct samples that reflect the true characteristics of the data, enabling the dynamic construction and continuous maintenance of high-quality, full-domain logical topology. Summary of the Invention

[0007] Therefore, this invention provides a database logical topology construction method based on a large language model to solve the aforementioned problems existing in the prior art.

[0008] To achieve the above objectives, this invention provides a database logical topology construction method based on a large language model, comprising:

[0009] Step S1: Obtain the schema information of the target database, and use the large language model to infer and complete the semantic metadata of the missing business to generate enhanced metadata;

[0010] Step S2: Perform multi-stage inference orchestration based on the enhanced metadata to construct a global logical topology network;

[0011] Step S3: Perform information entropy determination on the key data fields in the global logical topology network to identify feature types, extract the corresponding feature sets, and use the orthogonal boundary synthesis method to construct a virtual sample set covering the global feature values ​​to obtain the enhanced topology network.

[0012] Step S4: Establish a monitoring mechanism for changes in the schema information based on the enhanced topology network; when the schema information changes, trigger the incremental adsorption algorithm to integrate the changed elements into the enhanced topology network to form an updated topology network, and maintain the validity of each relationship in the updated topology network through a timeliness confidence decay model to obtain a dynamic global logical topology.

[0013] Furthermore, the process of step S2 includes:

[0014] Based on the semantic features in the enhanced metadata, the full data table is divided into several initial logical groups;

[0015] Calculate the semantic similarity between each initial logical group and the description of the preset target business domain, select one or more groups with the highest similarity as the core anchor set, and the remaining groups as the set to be absorbed;

[0016] Cross-association reasoning is performed between the core anchor set and the set to be absorbed; when the metadata scale involved in a single reasoning exceeds the context capacity constraint of the large language model, an adaptive partitioning strategy is triggered to split the set to be reasoned into subsets for matrix cross-association; an early stopping mechanism is implemented during the reasoning process, and the current reasoning path is terminated once a valid cross-set association is identified, and the associated set to be absorbed is aggregated to the corresponding core anchor set to form the global logical topology network.

[0017] Furthermore, the process of triggering the adaptive partitioning strategy to split the set to be inferred into subsets for matrix cross-inference includes:

[0018] Set a maximum metadata size threshold Tmax for a single inference iteration, and dynamically adjust the partition granularity based on the extent of metadata size expansion in the current core anchor set.

[0019] When the total metadata size of the core anchor point set and the target set to be adsorbed is... When Tmax is exceeded, the core anchor point set is divided into m subsets, the target set to be absorbed is divided into n subsets, and an m×n inference task matrix is ​​constructed.

[0020] First, fix a single subset Wl of the target set to be adsorbed, then sequentially traverse each subset Ak of ​​the core anchor set, and input the metadata of the paired subset (Ak, Wl) into the large language model to perform cross-association reasoning;

[0021] If the core anchor set aggregates and expands after this round of inference and the updated size exceeds Tmax, then dynamic repartitioning will be triggered before the next round of inference, and the number of subsets m and the partitioning method will be recalculated according to the updated size.

[0022] Furthermore, the process of dynamically adjusting the partition granularity based on the degree of metadata expansion of the current core anchor point set includes:

[0023] Set a base threshold Tbase and a hierarchical expansion coefficient k, and calculate the ratio r = |Aj| / Tbase of the metadata size |Aj| of the current core anchor set to the base threshold Tbase;

[0024] Calculate the cutting hierarchy based on the ratio r and the hierarchical expansion coefficient k. And determine the number of subsets m = 2^L;

[0025] When L=1, the core anchor point set is determined to be in the initial expansion stage. A bisection strategy is adopted to divide the core anchor point set into m=2 subsets.

[0026] When L=2, the core anchor set is determined to be in a moderate expansion stage. A four-part segmentation strategy is adopted to divide the core anchor set into m=4 subsets.

[0027] When L≥3, the core anchor point set is determined to be in a highly expanded stage, and an adaptive density segmentation strategy is adopted: spectral clustering is performed based on the semantic density of each data table within the core anchor point set, the number of subsets increases dynamically with the L value, and the size of each subset does not exceed Tmax / 2.

[0028] After each round of inference is completed, the updated r and L values ​​are recalculated, and the partition granularity of the next round of inference is dynamically adjusted according to the above rules.

[0029] Furthermore, step S2 also includes:

[0030] Establish a health index for the core anchor point set, which is comprehensively evaluated based on the metadata expansion rate, association stability, new set aggregation efficiency, and inference success rate of the set.

[0031] Monitor the health index of the core anchor point set. When the index is continuously lower than the health threshold for more than a preset number of rounds, determine that the core anchor point set is in a state of performance decay and obtain the anchor point set to be replaced.

[0032] For the set of anchor points to be replaced, initiate the freeze and re-initialization process: pause the expansion and aggregation operation of the set, release the computing resources it occupies, re-execute the anchoring phase based on the latest metadata, and generate a new core anchor point set to replace the original set.

[0033] Furthermore, the process of step S3 includes:

[0034] Calculate the cardinality ratio of the key data field, and determine whether the feature type is an enumeration feature or a range feature based on the cardinality ratio and a preset threshold;

[0035] Extract the corresponding feature set according to the feature type: extract the full set of values ​​for enumeration-type features, and extract the boundary value set for range-type features;

[0036] A round-robin selection strategy is used to construct virtual samples from the feature set, so that a single sample row carries the most differentiated feature combination to form the virtual sample set. The virtual sample set is then fused with the global logical topology network to obtain an enhanced topology network.

[0037] Furthermore, the process of constructing virtual samples from the feature set using a round-robin selection strategy includes:

[0038] For the i-th row and j-th column of the virtual sample, select values ​​from the feature set of field j in an indexed rotation manner. The index is calculated by taking the current row number modulo the number of elements in the feature set.

[0039] When field j is an enumerated feature, ensure that all of its values ​​appear at least once in the virtual sample; when field j is a range feature, ensure that its boundary values ​​appear at least once in the virtual sample.

[0040] By using differentiated combinations, adjacent sample rows can carry different feature values ​​on the same field to form the virtual sample set.

[0041] Furthermore, the process of step S4 includes:

[0042] Establish a schema change monitoring mechanism for the target database to capture data definition language change events;

[0043] When a change event is detected, the semantic affinity between the changed element and the existing core anchor set is calculated. Based on the comparison result of the semantic affinity and the aggregation threshold, the changed element is directly aggregated into the core anchor set, or classified into the set to be absorbed to perform cross-set inference, and the changed element is integrated into the enhanced topology network to form an updated topology network.

[0044] The validity of each association in the updated topology network is maintained by using a time-dependent confidence decay model to obtain the dynamic global logical topology.

[0045] Furthermore, the process of maintaining the validity of each association in the updated topological network through a time-dependent confidence decay model to obtain the dynamic global logical topology includes:

[0046] Assign an initial timeliness confidence level to each association in the enhanced topology network;

[0047] Calculate the current timeliness confidence level. When the timeliness confidence level of a certain relationship is lower than the verification threshold, trigger the re-inference verification of that relationship.

[0048] Update the timeliness confidence of the association based on the verification results;

[0049] The validity of each association is maintained by a time-dependent confidence decay model to obtain the dynamic global logical topology.

[0050] Furthermore, the timeliness confidence level decreases over time according to a preset attenuation coefficient, and increases with the number of verifications according to a preset gain coefficient.

[0051] Compared with existing technologies, the beneficial effects of this invention are as follows: Through the synergistic effect of semantic completion, multi-stage reasoning orchestration, orthogonal boundary synthesis, and dynamic maintenance, this invention achieves automatic mining of implicit associations and continuous topological evolution in databases without foreign keys. It utilizes the semantic understanding capabilities of a large language model to complete missing metadata, establishing a machine-interpretable business semantic layer. An adaptive partitioning strategy degrades ultra-large-scale reasoning tasks into subset-level atomic tasks, and combined with an early stopping mechanism, achieves full-domain coverage under limited contextual constraints, reducing token consumption through dynamic optimization of partition granularity. Based on information entropy, it distinguishes between enumeration-type and range-type features, employing a round-robin selection strategy to construct differentiated virtual samples, ensuring deterministic coverage of long-tail values ​​and boundary extremes, and eliminating probabilistic loss from random sampling. A schema change monitoring and incremental adsorption mechanism is established, enabling rapid attribution determination of changed elements through semantic affinity evaluation. By leveraging the decay of timeliness confidence and dynamic balance of gain, stable relationships receive high-frequency verification exemptions while potentially invalid relationships are promptly triggered for reconfirmation, thus achieving adaptive optimization between computational resource consumption, relationship recall rate, and topological timeliness. Attached Figure Description

[0052] Figure 1 A flowchart illustrating the database logical topology construction method based on a large language model provided in this embodiment of the invention;

[0053] Figure 2 This is a flowchart illustrating step S3 in the database logical topology construction method based on a large language model provided in an embodiment of the present invention. Detailed Implementation

[0054] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0055] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0056] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0057] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0058] Please see Figure 1 As shown, this invention provides a database logical topology construction method based on a large language model, comprising:

[0059] Step S1: Obtain the schema information of the target database, and use the large language model to infer and complete the semantic metadata of the missing business to generate enhanced metadata;

[0060] Specifically, the system connects to the target database via a program interface to obtain basic schema information, including table names, field names, field types, original comments, and data sampling. Metadata lacking business semantics in the basic schema information is filtered as metadata to be completed, including table names and field names missing Chinese comments. A semantic completion prompt template is constructed, and the metadata to be completed and contextual information are filled into the prompt template to form structured input. The contextual information includes existing comments of other fields in the same table, field data types, and field naming conventions. The structured input is submitted to a pre-trained large language model, which uses its reasoning capabilities to infer the business meaning of the metadata to be completed. The completion results returned by the large language model are received, and consistency checks are performed on the completion results, eliminating abnormal outputs that conflict with field data types or contradict existing semantics in the same table. The completed results that pass the check are written to the corresponding positions in the basic schema information to generate enhanced metadata containing a complete business semantic description.

[0061] Step S2: Perform multi-stage inference orchestration based on the enhanced metadata to construct a global logical topology network;

[0062] Specifically, step S2 includes the following process:

[0063] Based on the semantic features in the enhanced metadata, the full data table is divided into several initial logical groups;

[0064] Specifically, a hierarchical clustering algorithm based on cosine similarity is proposed to initially partition the entire data table. Clustering distance threshold. The setting is based on: when the semantic vectors of two tables have a cosine similarity of (cos(...)... When the precision is greater than 0.75, tables are considered to be strongly related within the same business domain and are preferentially grouped into the same initial logical group. This threshold was determined by optimizing an offline labeled dataset (containing 1000 manually labeled table relationships) using F1-score, achieving a balance between precision and recall. After clustering, the initial logical group set is obtained. Each group Gi contains a subset of semantically highly related data tables.

[0065] Calculate the semantic similarity between each initial logical group and the description of the preset target business domain, select one or more groups with the highest similarity as the core anchor set, and the remaining groups as the set to be absorbed;

[0066] Specifically, each initial logical group is calculated. Description of the preset target business domain The semantic similarity is calculated using a pre-defined target business domain description text, determined by business personnel based on the scope of the enterprise's core data assets, such as "user core transaction domain" or "supply chain master data domain." A weighted average of the semantic vectors of each table within a group is calculated, with weights equal to the proportion of fields in that table, resulting in the group center vector. Encode the target business domain description into a domain vector. ; Calculate semantic similarity Select the one with the highest similarity Each group serves as a core anchor point set. The remaining groups are used as the adsorption set. In this embodiment, The default value is 1. When the database size exceeds 1,000 tables or there is significant business domain isolation, it can be configured to 2-3 to form a multi-center parallel expansion pattern.

[0067] Specifically, during the expansion phase, the expansion efficiency metrics of each core anchor set are monitored, including the number of sets to be absorbed per round, average inference token consumption, and relationship recognition accuracy. When the expansion efficiency of a core anchor set falls below the competition threshold for multiple consecutive rounds, an anchor elimination mechanism is triggered, downgrading the set to a regular set to be absorbed and releasing its occupied computing resources. Simultaneously, when two core anchor sets identify strong semantic associations during expansion (i.e., data table pairs with high semantic similarity exceeding the association quantity threshold) and their connection density exceeds the merging threshold θmerge (typically 0.3-0.5), an anchor merging mechanism is triggered, merging the two sets into a new composite core anchor set and recalculating its semantic center and expansion strategy. When a semantic gap occurs within a single core anchor set (i.e., the semantic similarity of some data tables within the set falls below the gap threshold φthreshold, typically 0.4-0.6), an anchor splitting mechanism is triggered, using internal spectral clustering to decompose the set into two independent core anchor sets, each executing the subsequent expansion process separately.

[0068] By employing a multi-anchor competition and dynamic reorganization mechanism, the topology construction process is adaptively optimized, avoiding overall performance degradation caused by overload or performance decay of a single anchor point.

[0069] Cross-association reasoning is performed between the core anchor set and the set to be absorbed; when the metadata scale involved in a single reasoning exceeds the context capacity constraint of the large language model, an adaptive partitioning strategy is triggered to split the set to be reasoned into subsets for matrix cross-association; an early stopping mechanism is implemented during the reasoning process, and the current reasoning path is terminated once a valid cross-set association is identified, and the associated set to be absorbed is aggregated to the corresponding core anchor set to form the global logical topology network.

[0070] Specifically, the process of triggering the adaptive partitioning strategy to split the set to be inferred into subsets for matrix cross-inference includes:

[0071] Set a maximum metadata size threshold Tmax for a single inference iteration, and dynamically adjust the partition granularity based on the extent of metadata size expansion in the current core anchor set.

[0072] Specifically, large language models have an upper limit to their effective context window. When the input exceeds this limit, the model may truncate or experience attention degradation, leading to a significant increase in the reasoning illusion rate. In this implementation, the average token consumption described in the table schema is 150 tokens / table (including table name, field names, types, comments, and sampled data). Based on the stable inference range of 10K-16K tokens for mainstream commercial LLMs, and reserving 30% of the Prompt template and output space, the effective inference capacity is approximately 7K-11K tokens. Therefore: This value is a configurable parameter and can be increased proportionally when a model with a larger context window (such as the 32K / 128K version) or a simpler table structure is selected.

[0073] Specifically, the process of dynamically adjusting the partition granularity based on the degree of metadata expansion of the current core anchor set includes:

[0074] Set a base threshold Tbase and a hierarchical expansion coefficient k, and calculate the ratio r = |Aj| / Tbase of the metadata size |Aj| of the current core anchor set to the base threshold Tbase;

[0075] Specifically, the base threshold Tbase is set to 25 (table), and the hierarchical expansion coefficient k is set to 1.5.

[0076] Calculate the cutting hierarchy based on the ratio r and the hierarchical expansion coefficient k. And determine the number of subsets m = 2^L;

[0077] Specifically, the slicing hierarchy is calculated based on the ratio r and the hierarchical expansion coefficient k. ,in This represents the floor operation, i.e., the smallest integer not less than r / k.

[0078] When L=1, the core anchor point set is determined to be in the initial expansion stage. A bisection strategy is adopted to divide the core anchor point set into m=2 subsets.

[0079] When L=2, the core anchor set is determined to be in a moderate expansion stage. A four-part segmentation strategy is adopted to divide the core anchor set into m=4 subsets.

[0080] When L≥3, the core anchor point set is determined to be in a highly expanded stage, and an adaptive density segmentation strategy is adopted: spectral clustering is performed based on the semantic density of each data table within the core anchor point set, the number of subsets increases dynamically with the L value, and the size of each subset does not exceed Tmax / 2.

[0081] Specifically, this involves constructing a data table similarity matrix within the core anchor point set. ,in, Calculate the cosine similarity of the semantic vectors of tables u and v; then calculate the normalized Laplacian matrix: Where D is the degree matrix, for Perform eigenvalue decomposition, and take the eigenvectors corresponding to the first m smallest eigenvalues ​​to form the embedding matrix. K-means clustering is performed on the row vectors of U to obtain m subsets, ensuring that tables with high semantic density are grouped into the same subset.

[0082] After each round of inference is completed, the updated r and L values ​​are recalculated, and the partition granularity of the next round of inference is dynamically adjusted according to the above rules.

[0083] Specifically, during subset segmentation, a bridging table set is identified at the boundary of adjacent subsets. A bridging table is defined as a data table that has a high semantic similarity to both adjacent subsets, i.e., sim(T,Ak)>τbridge and sim(T,Ak+1)>τbridge, where τbridge is the bridging threshold (usually taken as 0.7-0.8).

[0084] By maintaining a copy of the bridging table in each of the two adjacent subsets, semantic continuity in cross-subset reasoning is ensured. In matrix-style cross-reasoning, when a pairing involving the bridging table identifies a valid association, cross-subset association propagation is triggered first, extending the association to adjacent subsets and reducing the overhead of repeated reasoning.

[0085] When the total metadata size of the core anchor point set and the target set to be adsorbed is... When Tmax is exceeded, the core anchor point set is divided into m subsets, the target set to be absorbed is divided into n subsets, and an m×n inference task matrix is ​​constructed.

[0086] Specifically, Divide into m subsets ;Will Divide into n subsets Construct an m×n reasoning task matrix, with matrix elements as follows: This indicates a paired subset.

[0087] First, fix a single subset Wl of the target set to be adsorbed, then sequentially traverse each subset Ak of ​​the core anchor set, and input the metadata of the paired subset (Ak, Wl) into the large language model to perform cross-association reasoning;

[0088] If the core anchor set aggregates and expands after this round of inference and the updated size exceeds Tmax, then dynamic repartitioning will be triggered before the next round of inference, and the number of subsets m and the partitioning method will be recalculated according to the updated size.

[0089] Specifically, once in any pairing In the reasoning process, when the large language model identifies a valid cross-set association (i.e., determines that a table in Wl has an implicit foreign key association with a table in Ak), it immediately terminates the remaining traversal operation of the current matrix; it aggregates the target set Wi to be absorbed into the core anchor set Aj, updates Aj←Aj∪Wi; and recalculates the updated set Wi. ,like If the previous iteration fails, dynamic repartitioning is triggered before the next round of inference, and the slicing level L and the number of subsets m are recalculated according to the updated scale. After each round of expansion, the system performs the following state updates: calculates the metadata scale |Aj| of the current core anchor set; recalculates the ratio r and slicing level L according to the updated |Aj|; if L changes, the subsets are re-partitioned using a new slicing strategy before the next cross-association inference; when the set to be absorbed is empty, or the maximum number of iterations (default 10 rounds) is reached, or no new associations are identified for 3 consecutive rounds, the expansion phase is terminated, and the global logical topology network is output.

[0090] Specifically, step S2 further includes:

[0091] Establish a health index for the core anchor point set, which is comprehensively evaluated based on the metadata expansion rate, association stability, new set aggregation efficiency, and inference success rate of the set.

[0092] Specifically, during long-term operation, the core anchor set may experience performance degradation due to excessive expansion. This implementation introduces a health monitoring mechanism to dynamically replace the anchor set. A health index for the core anchor set is established. A comprehensive evaluation of the operational performance of this set is conducted.

[0093]

[0094] Where w1, w2, w3, and w4 are weighting coefficients, and w1 + w2 + w3 + w4 = 1, the expansion rate is... Measure the growth rate of the scale in the most recent iteration, taking 1 for negative or zero growth, and 0 for a growth rate exceeding 50%; correlation stability. ,in, This represents the number of associations that remain valid after multiple verifications. The aggregation efficiency of the new set is the total number of associations. ,in, This represents the number of adsorbed aggregates that successfully polymerized. The number of sets to be attracted by the inference attempt, and the inference success rate. ,in The number of inference failures (such as timeouts, format errors, and hallucinations) in the large language model. This represents the total number of inference attempts. The weighting coefficients are set to w1=0.2, w2=0.3, w3=0.3, w4=0.2 by default, and can be adjusted according to business scenarios. Health metrics. ∈[0,1], the higher the value, the better the health status of the set.

[0095] Monitor the health index of the core anchor point set. When the index is continuously lower than the health threshold for more than a preset number of rounds, determine that the core anchor point set is in a state of performance decay and obtain the anchor point set to be replaced.

[0096] Specifically, setting health thresholds Monitoring rounds .

[0097] For the set of anchor points to be replaced, initiate the freeze and re-initialization process: pause the expansion and aggregation operation of the set, release the computing resources it occupies, re-execute the anchoring phase based on the latest metadata, and generate a new core anchor point set to replace the original set.

[0098] Specifically, when continuous Wheel below When the set of core anchor points is determined to be in a state of performance degradation, it is marked as the set of anchor points to be replaced. ,pause The expansion and aggregation operation releases the GPU / TPU computing resources and memory cache it occupies; China recently Unverified relationships are marked as pending confirmation, and their metadata summaries are extracted. Based on the latest full metadata, the semantic similarity between each logical group and the target business domain is recalculated, and a new set of core anchor points is selected. The residual relationships extracted in step 2 are incorporated into the data through incremental adsorption. Validated ones are kept, and those that fail to be validated are discarded. Replace the original .

[0099] Specifically, the health monitoring and dynamic replacement mechanism effectively solves the problem of inference performance decay caused by excessive expansion of the core anchor point set. By actively replacing the core anchor point set, load balancing is achieved, thereby improving the overall construction quality and efficiency of the global logical topology network.

[0100] Step S3: Perform information entropy determination on the key data fields in the global logical topology network to identify feature types, extract the corresponding feature sets, and use the orthogonal boundary synthesis method to construct a virtual sample set covering the global feature values ​​to obtain the enhanced topology network.

[0101] Specifically, such as Figure 2 As shown, the process of step S3 includes:

[0102] Step S31: Calculate the cardinality ratio of the key data field, and determine whether the feature type is an enumeration feature or a range feature based on the cardinality ratio and a preset threshold;

[0103] Specifically, key data fields include: related fields (foreign key fields) in identified relationships, frequently used query condition fields (based on historical query logs), and commonly used fields for business filtering conditions (such as status, type, time, etc.). For key data fields... Calculate its base ratio : ,in, This represents the number of unique values ​​to be removed from field j. The total number of records in the table to which this field belongs. Cardinality ratio. It measures the degree of dispersion of the values ​​in this field. The closer to 1, the more varied the field values ​​(e.g., user ID, order number). The closer to 0, the more concentrated the field values ​​are (such as status identifiers and type enumerations).

[0104] Specifically, setting an enumeration threshold Enumeration quantity threshold In typical business databases, enumeration fields (such as order status and payment type) usually have a limited number of possible values ​​(2-20) and a high rate of repetition; when and When the cardinality ratio is less than the enumeration threshold and the number of unique values ​​is less than or equal to the enumeration threshold, it is determined to be an enumeration feature. Typical examples include order status, such as pending payment, paid, and shipped, and review results, such as approved and rejected. Other cases are determined to be range features, typical examples of which include transaction amount, creation time, user age, etc., which are continuous or have a large range of discrete values.

[0105] Step S32: Extract the corresponding feature set according to the feature type: extract the full set of values ​​for enumeration type features, and extract the boundary value set for range type features;

[0106] Specifically, for enumeration-type features, the entire set of deduplicated values ​​is extracted, preserving the complete semantics of the original values ​​without aggregation or truncation. If the number of deduplicated values ​​exceeds the enumeration threshold, the top fifty high-frequency values ​​are selected in descending order of frequency and marked as a high-frequency enumeration subset. For range-type features, the set of boundary values ​​is extracted, including the minimum, median, and maximum values. The minimum and maximum values ​​represent the numerical boundaries of the field, and the median is the 50th percentile. For numerical fields, the 25th and 75th percentiles are additionally extracted as auxiliary boundaries to form a five-digit summary, better characterizing the distribution pattern.

[0107] Step S33: A round-robin selection strategy is used to construct virtual samples from the feature set, so that a single sample row carries the most differentiated feature combination to form the virtual sample set. The virtual sample set is then fused with the global logical topology network to obtain an enhanced topology network.

[0108] Specifically, the process of constructing virtual samples from the feature set using a round-robin selection strategy includes:

[0109] For the i-th row and j-th column of the virtual sample, select values ​​from the feature set of field j in an indexed rotation manner. The index is calculated by taking the current row number modulo the number of elements in the feature set.

[0110] Specifically, for the virtual sample in row i and column j, a value is selected from the feature set of field j using an index-based round-robin method. The index is calculated by taking the current row number modulo the number of elements in the feature set, i.e., the remainder when the row number is divided by the number of elements. The value at the corresponding position in the feature set is selected based on this index and used as the virtual sample value for that field.

[0111] When field j is an enumerated feature, ensure that all of its values ​​appear at least once in the virtual sample; when field j is a range feature, ensure that its boundary values ​​appear at least once in the virtual sample.

[0112] Specifically, for enumerated features, when the number of virtual sample rows is greater than or equal to the number of elements in the feature set, a round-robin mechanism ensures that all values ​​appear at least once. This implementation defaults to setting the number of virtual sample rows to the larger of ten and twice the number of elements in the feature set, ensuring coverage while controlling the sample size. For range-type features, the minimum and maximum values ​​are explicitly extracted and fixed at the beginning and end of the feature set to ensure that boundary values ​​appear at least once in the virtual samples.

[0113] By using differentiated combinations, adjacent sample rows can carry different feature values ​​on the same field to form the virtual sample set.

[0114] Specifically, adjacent sample rows carry different feature values ​​on the same field. When the number of elements in the feature set is greater than one, adjacent rows will inevitably take different values. A single sample row carries the maximum information entropy through differentiated combinations of different fields. The full range of values ​​for enumerated features is evenly distributed in the virtual samples, avoiding long-tail loss.

[0115] Step S4: Establish a monitoring mechanism for changes in the schema information based on the enhanced topology network; when the schema information changes, trigger the incremental adsorption algorithm to integrate the changed elements into the enhanced topology network to form an updated topology network, and maintain the validity of each relationship in the updated topology network through a timeliness confidence decay model to obtain a dynamic global logical topology.

[0116] Specifically, step S4 includes the following process:

[0117] Establish a schema change monitoring mechanism for the target database to capture data definition language change events;

[0118] Specifically, establish a data definition language change monitoring mechanism for the target database to capture a set of change events: ,in, For the event of adding a new data table, For events involving the addition of a field or a change in field type, For events involving deleting a relationship or deleting a table, Rename the table structure event.

[0119] When a change event is detected, the semantic affinity between the changed element and the existing core anchor set is calculated. Based on the comparison result of the semantic affinity and the aggregation threshold, the changed element is directly aggregated into the core anchor set, or classified into the set to be absorbed to perform cross-set inference, and the changed element is integrated into the enhanced topology network to form an updated topology network.

[0120] Specifically, when a change event is detected At that time, extract the metadata summary of the changed elements and generate semantic feature vectors. , where d is the feature dimension. Calculate the semantic affinity between the changed element and the existing core anchor set: ,in Let be the center vector of the core anchor point set. The element with the highest semantic affinity is taken as the overall affinity of the changed element. Set the aggregation threshold. .like In this case, the changed elements will be directly aggregated into the corresponding core anchor set: Update the metadata size of the collection. and semantic center vector .like When the change occurs, the changed element is assigned to the set W to be absorbed, triggering the adaptive partitioning strategy and early stopping mechanism, and cross-set inference is performed.

[0121] The validity of each association in the updated topology network is maintained by using a time-dependent confidence decay model to obtain the dynamic global logical topology.

[0122] Specifically, the process of maintaining the validity of each association in the updated topological network through a time-dependent confidence decay model to obtain the dynamic global logical topology includes:

[0123] Assign an initial timeliness confidence level to each association in the enhanced topology network;

[0124] Specifically, to enhance each connection in the topology network Assigning timeliness confidence The confidence level changes dynamically over time.

[0125] Calculate the current timeliness confidence level. When the timeliness confidence level of a certain relationship is lower than the verification threshold, trigger the re-inference verification of that relationship.

[0126] Specifically, the timeliness confidence level decreases over time according to a preset decay coefficient, and increases with the number of verifications according to a preset gain coefficient.

[0127] Specifically, the confidence level of timeliness decreases over time according to a decay factor:

[0128]

[0129] in, As the initial timeliness confidence level, Δt is the attenuation coefficient, and Δt is the time interval since the last verification. This is the gain coefficient; Number of successful validations. Confidence cap constraint: .

[0130] Specifically, setting a verification threshold When the timeliness confidence of a certain relationship is lower than the verification threshold, a re-inference verification is triggered. The re-inference uses the original metadata context when the relationship was established and inputs it into the large language model to determine the validity of the relationship.

[0131] Update the timeliness confidence of the association based on the verification results;

[0132] Specifically, if the verification passes, meaning the large language model confirms that the association remains valid, the timeliness confidence of the association is restored to its initial value, and the verification count is incremented. If the verification fails, meaning the large language model determines that the association is invalid or cannot be confirmed, the association is marked as invalid, removed from the topology network, and the reason for the invalidity is recorded for auditing.

[0133] The validity of each association is maintained by a time-dependent confidence decay model to obtain the dynamic global logical topology.

[0134] Specifically, this invention achieves automatic implicit association mining and continuous topology evolution in foreign key-free databases through the synergistic effects of semantic completion, multi-stage reasoning orchestration, orthogonal boundary synthesis, and dynamic maintenance: It utilizes the semantic understanding capabilities of a large language model to complete missing metadata, establishing a machine-interpretable business semantic layer; it degrades ultra-large-scale reasoning tasks into subset-level atomic tasks through an adaptive partitioning strategy, achieving full coverage under limited contextual constraints with an early stopping mechanism, thus reducing token consumption through dynamic optimization of partition granularity; it distinguishes between enumeration-type and range-type features based on information entropy, employing a round-robin selection strategy to construct differentiated virtual samples, ensuring deterministic coverage of long-tail values ​​and boundary extremes, and eliminating probabilistic loss from random sampling; it establishes a schema change monitoring and incremental adsorption mechanism, achieving rapid attribution determination of changed elements through semantic affinity assessment, and leveraging the decay and dynamic balance of timeliness confidence to grant high-frequency verification exemptions to stable relationships while triggering timely reconfirmation of potentially invalid relationships, thereby achieving adaptive optimization between computational resource consumption, relationship recall rate, and topology timeliness.

[0135] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

[0136] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for constructing a database logical topology based on a large language model, characterized in that, include: Step S1: Obtain the schema information of the target database, and use the large language model to infer and complete the semantic metadata of the missing business to generate enhanced metadata; Step S2: Perform multi-stage inference orchestration based on the enhanced metadata to construct a global logical topology network; Step S3: Perform information entropy determination on the key data fields in the global logical topology network to identify feature types, extract the corresponding feature sets, and use the orthogonal boundary synthesis method to construct a virtual sample set covering the global feature values ​​to obtain the enhanced topology network. Step S4: Establish a monitoring mechanism for changes in the schema information based on the enhanced topology network; when the schema information changes, trigger the incremental adsorption algorithm to integrate the changed elements into the enhanced topology network to form an updated topology network, and maintain the validity of each relationship in the updated topology network through a timeliness confidence decay model to obtain a dynamic global logical topology.

2. The database logical topology construction method based on a large language model according to claim 1, characterized in that, The process of step S2 includes: Based on the semantic features in the enhanced metadata, the full data table is divided into several initial logical groups; Calculate the semantic similarity between each initial logical group and the description of the preset target business domain, select one or more groups with the highest similarity as the core anchor set, and the remaining groups as the set to be absorbed; Cross-association reasoning is performed between the core anchor set and the set to be absorbed; when the metadata scale involved in a single reasoning exceeds the context capacity constraint of the large language model, an adaptive partitioning strategy is triggered to split the set to be reasoned into subsets for matrix cross-association; an early stopping mechanism is implemented during the reasoning process, and the current reasoning path is terminated once a valid cross-set association is identified, and the associated set to be absorbed is aggregated to the corresponding core anchor set to form the global logical topology network.

3. The database logical topology construction method based on a large language model according to claim 2, characterized in that, The process by which the adaptive partitioning strategy is triggered to split the set to be inferred into subsets for matrix cross-inference includes: Set a maximum metadata size threshold Tmax for a single inference iteration, and dynamically adjust the partition granularity based on the extent of metadata size expansion in the current core anchor set. When the total metadata size of the core anchor point set and the target set to be adsorbed is... When Tmax is exceeded, the core anchor point set is divided into m subsets, the target set to be absorbed is divided into n subsets, and an m×n inference task matrix is ​​constructed. First, fix a single subset Wl of the target set to be adsorbed, then sequentially traverse each subset Ak of ​​the core anchor set, and input the metadata of the paired subset (Ak, Wl) into the large language model to perform cross-association reasoning; If the core anchor set aggregates and expands after this round of inference and the updated size exceeds Tmax, then dynamic repartitioning will be triggered before the next round of inference, and the number of subsets m and the partitioning method will be recalculated according to the updated size.

4. The database logical topology construction method based on a large language model according to claim 3, characterized in that, The process of dynamically adjusting the partition granularity based on the current metadata size expansion of the core anchor point set includes: Set a base threshold Tbase and a hierarchical expansion coefficient k, and calculate the ratio r = |Aj| / Tbase of the metadata size |Aj| of the current core anchor set to the base threshold Tbase; Calculate the cutting hierarchy based on the ratio r and the hierarchical expansion coefficient k. And determine the number of subsets m = 2^L; When L=1, the core anchor point set is determined to be in the initial expansion stage. A bisection strategy is adopted to divide the core anchor point set into m=2 subsets. When L=2, the core anchor set is determined to be in a moderate expansion stage. A four-part segmentation strategy is adopted to divide the core anchor set into m=4 subsets. When L≥3, the core anchor point set is determined to be in a highly expanded stage, and an adaptive density segmentation strategy is adopted: spectral clustering is performed based on the semantic density of each data table within the core anchor point set, the number of subsets increases dynamically with the L value, and the size of each subset does not exceed Tmax / 2. After each round of inference is completed, the updated r and L values ​​are recalculated, and the partition granularity of the next round of inference is dynamically adjusted according to the above rules.

5. The database logical topology construction method based on a large language model according to claim 4, characterized in that, Step S2 further includes: Establish a health index for the core anchor point set, which is comprehensively evaluated based on the metadata expansion rate, association stability, new set aggregation efficiency, and inference success rate of the set. Monitor the health index of the core anchor point set. When the index is continuously lower than the health threshold for more than a preset number of rounds, determine that the core anchor point set is in a state of performance decay and obtain the anchor point set to be replaced. For the set of anchor points to be replaced, initiate the freeze and re-initialization process: pause the expansion and aggregation operation of the set, release the computing resources it occupies, re-execute the anchoring phase based on the latest metadata, and generate a new core anchor point set to replace the original set.

6. The database logical topology construction method based on a large language model according to claim 5, characterized in that, The process of step S3 includes: Calculate the cardinality ratio of the key data field, and determine whether the feature type is an enumeration feature or a range feature based on the cardinality ratio and a preset threshold; Extract the corresponding feature set according to the feature type: extract the full set of values ​​for enumeration-type features, and extract the boundary value set for range-type features; A round-robin selection strategy is used to construct virtual samples from the feature set, so that a single sample row carries the most differentiated feature combination to form the virtual sample set. The virtual sample set is then fused with the global logical topology network to obtain an enhanced topology network.

7. The database logical topology construction method based on a large language model according to claim 6, characterized in that, The process of constructing virtual samples from the feature set using a round-robin selection strategy includes: For the i-th row and j-th column of the virtual sample, select values ​​from the feature set of field j in an indexed rotation manner. The index is calculated by taking the current row number modulo the number of elements in the feature set. When field j is an enumerated feature, ensure that all of its values ​​appear at least once in the virtual sample; when field j is a range feature, ensure that its boundary values ​​appear at least once in the virtual sample. By using differentiated combinations, adjacent sample rows can carry different feature values ​​on the same field to form the virtual sample set.

8. The database logical topology construction method based on a large language model according to claim 7, characterized in that, The process of step S4 includes: Establish a schema change monitoring mechanism for the target database to capture data definition language change events; When a change event is detected, the semantic affinity between the changed element and the existing core anchor set is calculated. Based on the comparison result of the semantic affinity and the aggregation threshold, the changed element is directly aggregated into the core anchor set, or classified into the set to be absorbed to perform cross-set inference, and the changed element is integrated into the enhanced topology network to form an updated topology network. The validity of each association in the updated topology network is maintained by using a time-dependent confidence decay model to obtain the dynamic global logical topology.

9. The database logical topology construction method based on a large language model according to claim 8, characterized in that, The process of maintaining the validity of each association in the updated topological network through a time-dependent confidence decay model to obtain the dynamic global logical topology includes: Assign an initial timeliness confidence level to each association in the enhanced topology network; Calculate the current timeliness confidence level. When the timeliness confidence level of a certain relationship is lower than the verification threshold, trigger the re-inference verification of that relationship. Update the timeliness confidence of the association based on the verification results; The validity of each association is maintained by a time-dependent confidence decay model to obtain the dynamic global logical topology.

10. The database logical topology construction method based on a large language model according to claim 9, characterized in that, The timeliness confidence level decreases over time according to a preset decay coefficient, and increases with the number of verifications according to a preset gain coefficient.

Images