Relational database-oriented large language model-based self-supervised relation reasoning method and system

Abstract

This invention relates to the field of multi-table relational reasoning technology, and discloses a self-supervised relational reasoning method and system based on a large language model for relational databases. The method achieves robust relational reasoning on multi-table data by jointly modeling textual semantic information and structural topological information in the relational database. The method includes: constructing a relation-aware graph structure from the multi-table data and primary-foreign key relationships in the relational database, and generating a structural representation; embedding the structural representation as a soft cue into the large language model to align structural information with semantic representation; and gradually reducing the attribute context information in the input to guide the model from semantic reasoning to structure-driven cross-table reasoning. This invention achieves effective modeling of complex structural dependencies in relational databases without requiring manual data annotation, and improves the reasoning ability and application scalability of the large language model in multi-table data scenarios.

Description

Technical Field

[0001] This invention relates to the field of multi-table relational reasoning technology, specifically to a self-supervised relational reasoning method and system based on a large language model for relational databases. Background Technology

[0002] Relational databases (RDBs) serve as the data foundation for real-world mission-critical systems, widely supporting applications such as e-commerce, social media, and industrial platforms. RDBs organize multiple tables into a strongly structured network of relationships through primary and foreign key constraints, ensuring that the data contains not only the textual semantic information carried by table attributes but also the structural topological information depicted by cross-table joins. With the rapid development of Large Language Models (LLMs), research leveraging LLMs to empower relational databases has become a trend: early work focused on single-table scenarios (such as natural language query parsing and entity matching), while recent work has further expanded to contextual learning and reasoning modeling for tabular and multi-table data.

[0003] Regarding how large language models understand and reason about multi-table relational data, existing research has generally formed two representative technical approaches: (1) the text serialization paradigm, which serializes multi-table data and their relationships into a text sequence and directly inputs it into the large language model, for example, constructing a textual context by traversing relational links to complete the reasoning; (2) the structural encoding paradigm, which introduces structural encoders such as graph neural networks before inputting into the large language model, encodes topological structures such as primary and foreign key relationships into vector representations, and then combines them with semantic information before inputting into the large language model. The above methods have achieved good results on multiple tasks, but they have also exposed structural bottlenecks: such as Figure 1 As shown, relational databases inherently possess both semantic and structural attributes. For problems with rich contextual information that can be directly inferred from table attributes, semantic cues are often sufficient; however, for more complex cross-table reasoning problems (such as those requiring the integration of user and loan tables, and dynamic credit and income changes to determine the true extent of delinquency), structural topology is an indispensable key piece of evidence. However, text serialization methods are prone to encountering excessively long contexts and resulting in the loss of topological information; structural encoding methods typically rely on manually labeled signals for supervised fine-tuning, which is costly and difficult to scale.

[0004] To reduce reliance on annotations, some studies have begun to employ self-supervised objectives such as static mask prediction for fine-tuning, generating training signals through random masks. While this strategy is simple to implement and can effectively improve model adaptability, random masks struggle to control reasonable mask boundaries and systematically guide the model to adaptively select semantic or structural cues at different difficulty levels. The model may rely on "learned surface semantic shortcuts" during training, failing to stably develop genuine cross-table structural reasoning capabilities. Therefore, a critical issue remains: how to fully utilize the capabilities of large language models to achieve robust relational reasoning for relational databases without relying on task annotations, and how to achieve controllable and gradual capability transfer between semantic and structural inference. Summary of the Invention

[0005] To address the aforementioned technical issues, this invention provides a self-supervised relational reasoning method and system based on a large language model for relational databases. The aim is to guide a large language model from semantically driven shallow relevance to structure-driven cross-table reasoning through structure-guided alignment and progressive sparse learning, without relying on task-specific annotations. This avoids information redundancy and topological loss caused by excessively long contexts and reduces dependence on manual annotation-supervised fine-tuning, thereby improving the model's robustness and generalization ability in multi-table scenarios.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: In a first aspect, the present invention provides a self-supervised relational reasoning method based on a large language model for relational databases, comprising: A relational graph is constructed from multiple tables in a relational database and the primary key-foreign key relationships between the tables. A time-consistent subgraph is generated based on timestamp sampling, and the node attributes are encoded to obtain the initial node features. Each row record in the table is used as a node in the relational graph, and the primary key-foreign key relationship is used as an edge in the relational graph. Node attributes refer to the field information carried by the row record corresponding to each node in the relational graph. Based on the time-consistent subgraph, node features are updated using a heterogeneous graph neural network to obtain the node structure representation. The node structure representation is mapped to the embedding space of the large language model as a structure cue vector, and then concatenated with the semantic instruction embedding for self-supervised training of mask cell prediction to align the structural information with the semantic representation. By controlling the proportion of context retention, the number of attribute contexts in the input large language model is gradually reduced, and partial context instructions are constructed. The structural cue vector and sparse instruction embeddings constructed based on partial context instructions are concatenated and input into the large language model for self-supervised training of masked cell prediction, so as to guide the large language model from semantic reasoning to structure-driven cross-table reasoning.

[0007] In one embodiment, the generation of a time-consistent subgraph based on timestamp sampling specifically includes: At timestamp t, only times earlier than 1 are retained. Build a time-consistent subgraph using nodes and edges. ,in and Let represent the set of nodes and the set of edges of a time-consistent subgraph that satisfies the time constraint, respectively. This represents the time truncation threshold corresponding to the current prediction time, used to ensure that the time-consistent subgraph only contains node and edge information that can be observed before the prediction time.

[0008] In one embodiment, encoding the node attributes to obtain initial node features specifically includes: A multimodal column encoder is used to encode numerical, categorical, and textual attributes in a relational database to obtain initial node features.

[0009] In one embodiment, the step of updating node features using a heterogeneous graph neural network based on a time-consistent subgraph to obtain a node structure representation specifically includes: Based on time-consistent subgraph The node features are updated using a relation-aware heterogeneous graph neural network: ; ; in, Represents a node Based on primary key-foreign key relationship at level l The relationship-specific message representation obtained from aggregation, Representing relations Specific aggregate functions, This represents the node representation of neighbor node u at level l-1. Represents a node In relationship The set of neighboring nodes; This represents a set of relationship types, which includes at least primary key-foreign key relationships between tables. Represents a non-linear activation function. This represents the learnable parameter matrix of the l-th layer. This represents a combination function used to fuse the node's own representation with different relational message representations. This represents the node representation of node v after the update at layer l, which, after multiple propagations, yields the node structure representation. .

[0010] In one embodiment, the step of mapping the node structure representation to the embedding space of a large language model as a structure cue vector, and concatenating it with the semantic instruction embedding for self-supervised training of masked cell prediction to align the structure information with the semantic representation specifically includes: Using multilayer perceptron Representing the node structure Mapping to the embedding space of a large language model yields structural cue vectors. : ; structural cue vector Embedding with column-level semantic instructions By splicing the pieces together, we can obtain the splicing features. : ; Will Input a large language model and perform a masked cell prediction task. The training objective is as follows: ; Where Y is the set of all possible values ​​of the masked cell. This represents the training loss function for the masked cell prediction task. Indicates that the parameter is The conditional probability distribution output by the large language model. This represents the actual target value corresponding to the i-th masked cell.

[0011] In one embodiment, the step of gradually reducing the number of attribute contexts input to the large language model by controlling the context retention ratio and constructing partial context instructions specifically includes: Define the context retention ratio To control the number of visible column attributes during training: ; This represents the set of attributes for the i-th row record. The number of visible column attributes that are retained for the i-th row record in the current training step; Update the context retention ratio using a linear decay method. ,in S represents the current number of training steps, and S represents the total number of training steps. Indicates the minimum context retention ratio at the end of training; Based on reservation Each column attribute constructs a partial context instruction, and the target column value to be predicted is replaced with a mask symbol to generate a prediction task.

[0012] In one embodiment, the step of concatenating the structural cue vector with sparse instruction embeddings constructed based on partial context instructions and inputting them into a large language model for self-supervised training of masked cell prediction, in order to guide the large language model from semantic reasoning to structure-driven cross-table reasoning, specifically includes: Sparse instruction embeddings are obtained by encoding some contextual instructions through the embedding layer or text encoding module of a large language model. ; structural cue vector With sparse instruction embedding By splicing the features together, the second splicing feature is obtained. : ; Predicting target cell values ​​using large language models: ; in For the set of candidate values ​​for the target cell, This represents the training loss function for masked unit prediction based on partial context instructions. Indicates that the parameter is The conditional probability distribution output by the large language model. This represents the actual value corresponding to the i-th target cell.

[0013] In a second aspect, the present invention provides a computer system including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method of any embodiment of the first aspect.

[0014] Compared with the prior art, the beneficial technical effects of the present invention are: 1. This invention addresses the semantic and structural attributes of relational databases and proposes a self-supervised modeling approach centered on progressive transition, enabling robust cross-table relational reasoning using a large language model without relying on task annotation.

[0015] 2. This invention proposes a graph-guided prompt alignment mechanism, which models the primary and foreign key relationships of multiple tables as a relationship-aware graph and generates a structural representation from heterogeneous graph neural networks. The structural representation is then aligned to the semantic space of a large language model using soft prompts. This retains the original semantic modeling capabilities of the large language model while explicitly injecting structural constraints, thereby achieving a unified representation and collaborative reasoning of semantic information and structural topological information.

[0016] 3. This invention proposes a progressive sparsity context refinement strategy, which treats context sparsity as a controllable course variable, gradually reduces the proportion of visible attributes to construct information bottlenecks, and prompts the model to gradually shift from semantic-driven shallow correlation to structure-driven cross-table reasoning. This effectively overcomes the limitations of existing random mask self-supervised methods, which are difficult to control mask boundaries and guide the model to adaptively utilize semantic and structural cues, and maintains stable performance in different sparse scenarios. Attached Figure Description

[0017] Figure 1 This is an example diagram illustrating the motivation behind the present invention.

[0018] Figure 2 This is a schematic diagram of the method framework of the present invention. Detailed Implementation

[0019] A preferred embodiment of the present invention will now be described in detail with reference to the accompanying drawings.

[0020] To address the issue that existing large language models tend to over-rely on semantic cues within tables and struggle to reliably utilize primary and foreign key topologies for cross-table reasoning when processing multi-table data, this embodiment proposes a self-supervised relational reasoning method based on a large language model for relational databases. It designs a collaborative mechanism based on structural alignment and progressive sparsity learning: graph-guided prompt alignment injects the multi-table structural representation into the large language model with soft prompts to learn structural constraints and maintain semantic modeling capabilities; then, progressive sparsity context refinement gradually reduces the proportion of visible context, creating an information bottleneck. This prompts the model to gradually transition from semantically driven shallow correlations to structure-driven cross-table relational reasoning, thereby achieving robust reasoning for relational databases and stable generalization performance across different sparse scenarios without relying on task annotation.

[0021] The overall framework of the present invention is as follows Figure 2 As shown below, the invention will be described in detail using the user table, loan table, and repayment record table in a bank loan business as an example.

[0022] 1. Construction of time-constrained input representation.

[0023] Relational Database Graph Structure Construction: Multiple business tables in a relational database are represented as a relational graph structure. Specifically, each row in the User table, Loan table, and Repayment table is represented as a node; primary key-foreign key relationships are represented as edges, resulting in the relational graph. Here, V represents the set of nodes, and E represents the set of relationships. A primary key is a field or combination of fields in a table that uniquely identifies each row of data; a foreign key is a field in one table that points to the primary key of another table, thus establishing a link between the two tables. For example, "User ID=U001, Name=Zhang San, Position=Sales Manager, Monthly Income=18000, Credit Rating=A" can be represented as a user node. The primary key in the user table can be "User ID"; the primary key in the loan table can be "Loan ID," and its foreign key "User ID" is used to link to the user table. Therefore, the following relationship edge can be established: a "User applies for a loan" relationship edge is established between user node U001 and loan node L001.

[0024] Time-consistent subgraph sampling: To avoid leakage of time information, only times earlier than 1 are retained at timestamp t. Construct a time-consistent subgraph using the nodes and edges: ,in and Let represent the set of nodes and the set of edges that satisfy the time constraint, respectively. This represents the current prediction point in time. For example, if a prediction of user U001's loan risk or creditworthiness is needed on April 1, 2025, only data generated before that point in time will be retained. This constructs a... and It only includes information observable up to the prediction time, thus ensuring that the training and inference processes conform to real business processes.

[0025] Initial feature encoding for nodes: For fields in the user table, loan table, and repayment record table, a multimodal column encoder is used to encode the numerical, categorical, and textual attributes in the relational database to obtain initial feature vectors for nodes, which are then used as input to the graph neural network. Specifically, in this embodiment, for numerical attributes, such as "monthly income = 18,000 yuan" in the user table, "application amount = 200,000 yuan" in the loan table, and "overdue days = 0" in the repayment record table, normalization, bucketing, or numerical embedding methods can be used for encoding; for categorical attributes, such as "credit rating = A" and "marital status = married", category mapping and embedding lookup methods can be used for encoding; for textual attributes, such as "name = Zhang San" and "position = sales manager", text encoders or word vector methods can be used for encoding; then the encoding results of the above different types of attributes are concatenated or merged to form the initial node features of the node.

[0026] 2. The diagram guides the alignment.

[0027] Heterogeneous graph structure representation learning: Time-consistent subgraphs The node features are updated using a relation-aware heterogeneous graph neural network: ; in, Represents a node In relationship The following is a collection of neighbors. Represents a relation-specific aggregate function. Node representation updated as follows: ; The node structure representation is obtained after multiple propagations. For example, for user node U001, its neighbors may include: the loan node L1001 associated with it; and the repayment record node R9001, which is further associated with loan node L1001. Therefore, the structural representation of user node U001 not only includes its own local information such as "position, income, and credit rating," but also integrates cross-table relationship information such as "loan application behavior" and "repayment performance behavior." In this way, the node structural representation is no longer limited to single-table semantics, but integrates multi-table relationships, making the resulting structural representation closer to the relational reasoning needs in real financial risk control scenarios.

[0028] Furthermore, a multilayer perceptron is used to map the node structure representation to the embedding space of a large language model. ,in This represents a structure hint vector.

[0029] Structural semantic alignment training: aligning structural cue vectors Embedding with column-level semantic instructions spliced ​​together Then, the data is input into a large language model to perform a masked cell prediction task. The training objective is as follows: (3) In this invention, the "column-level semantic instruction embedding" is directly related to specific tables and fields. It is not a fixed vector set separately from the business table. Instead, it automatically constructs text instructions based on the table name, column name, column meaning, and the business scenario in which the column is located, and then encodes them by the embedding layer of the large language model.

[0030] For example, when the target column is the "Credit Rating" column in the user table, a semantic instruction can be constructed: "Based on the user's job title, monthly income, age, marital status, and other attributes in the user table, as well as their related information in the loan table and repayment record table, predict the user's credit rating value in the user table." This demonstrates the embedding of column-level semantic instructions. It has a direct correspondence with the "User Table", "Loan Table", "Repayment Record Table" and specific target fields. Its function is to tell the large language model: which column in which table is to be predicted, and what the semantic meaning of that column is in the business context.

[0031] In the aforementioned structural semantic alignment training, a masked cell prediction task is used to train the model. A target cell is selected from a specific table record, its original value is hidden, and the model is then required to recover the cell's true value by combining visible attributes and cross-table relationship information. Here, Y represents the set of candidate values, indicating the set of values ​​that the model can select or score for the current target column, and it is directly related to the specific table and the specific target column. For example, when the target column is the "Credit Rating" column in the user table, the candidate value set could be Y={A,B,C,D,E}.

[0032] 3. Progressive contextual sparse learning.

[0033] Context sparsity control: Define the context retention ratio To control the number of visible attributes during training: (4) in Let C represent the attribute set of the i-th row record. In this embodiment, if training is performed on user U001 in the user table, the attribute set of one row record can be represented as C. i ={Job Title, Monthly Income, Credit Rating, Age, Marital Status, City}. The specific attribute values ​​are {Sales Manager, 18,000 RMB, A, 35 years old, Married, Nanjing}. "Visible attributes" refer to the column attributes and their corresponding values ​​that are retained and provided to the model in the current training epoch. For example, if the user U001 row has 6 column attributes, and the context retention ratio for the current training step is 0.5, then... =3. That is, in this training step, only 3 visible attributes are retained.

[0034] Sparse scheduling strategy: Update the context retention ratio using a linear decay method. ,in Here, S represents the current training step count, and S represents the total training steps. For example, in the early stages of training... It can approach 1, at which point the model can see almost all the attributes in the user table or loan table, making it easier for it to learn the field semantics first; as training progresses, As the value gradually decreases, for example, from 1 to 0.5, 0.3 or even lower, the model can only see some fields, such as limited visible attributes like "job title, monthly income, and city". This forces the model to rely more on the relationship structure between "user-loan-repayment" for reasoning.

[0035] Sparse instruction construction: based on preservation The context instructions are constructed using individual column attributes, and the target column value is replaced with a mask symbol to generate a prediction task. If the current training step retains only three visible attributes: "Position = Sales Manager," "Monthly Income = 18,000 Yuan," and "City = Nanjing," and "Credit Rating" is selected as the target column, then the following partial context instructions can be constructed: "In the user table, a user's position is Sales Manager, their monthly income is 18,000 Yuan, their city is Nanjing, and their credit rating is [MASK]. Combining this user's association information in the loan table and repayment record table, predict the user's credit rating." In this example: "Target Column Value" is "A" in "Credit Rating = A" in the original record; "Mask Symbol" is "[MASK]"; "Prediction Task" is to recover the true value corresponding to "MASK" based on the visible attributes and association table information; "Target Cell Value" is the true value "A" of the masked cell in the original table.

[0036] 4. Structural guidance unit prediction.

[0037] Fusion of structural and semantic information: integrating structural cue vectors With sparse instruction embedding spliced ​​together .

[0038] in, The structure is mapped from the structural representation of the record in the relational graph. For example, the structural hint vector of user U001 not only contains the visible field information of the user table, but also the relational context obtained after connecting it with the loan table and the repayment record table. This corresponds to the text instruction embedding for the current sparse prediction task, such as "predict credit rating based on user income, occupation, and related loan information" or "predict loan type based on application amount and related user characteristics." The two are concatenated and input into a large language model, which can simultaneously utilize structural cues and some semantic cues to complete the prediction.

[0039] Masked cell prediction training: Predicting target cell values ​​using a large language model. ; in, This is the set of candidate values. For example, if the input is: "Job title = Sales Manager, Monthly income = 18,000 yuan, City = Nanjing, Credit rating = [MASK], and prediction is made based on the user's corresponding loan application and repayment information", and the user's original credit rating is "A", and the set of candidate values ​​is {A, B, C, D, E}, then the model should output "A", which is the target cell value.

[0040] Through the above steps, this invention eliminates the need for manual labeling of downstream tasks such as "default" or "risk level." Instead, it directly utilizes the original unit values ​​of the database to construct self-supervised training objectives, thereby improving the relational reasoning ability of large language models in multi-table relational databases.

[0041] To verify the effectiveness of this framework, this invention selects the RELBENCH benchmark to evaluate unlabeled relational reasoning capabilities on relational databases. RELBENCH consists of seven real-world datasets: rel-f1, rel-event, rel-avito, rel-hm, rel-stack, rel-trial, and rel-amazon, covering a total of 12 prediction tasks. Given that this invention does not rely on any explicit or implicit task labels during training, to ensure fairness in the comparative evaluation, this invention only selects self-supervised or unsupervised learning methods as the comparison baseline. Specifically: (1) In the direction of graph representation learning, DGI and InfoNCE are selected as representative self-supervised methods for learning structural representations from relational graphs; (2) In the direction of relational data learning (RDL), the advanced baseline RDL is selected and its training process is transformed into a fully self-supervised method so that it does not depend on any downstream task labels; (3) In the direction of language model (LM), BERT and RoBERTa are selected as baseline methods, and the serialized table rows are used as input for prediction to evaluate the ability of language models in the absence of explicit structural modeling; (4) In the direction of large language models (LLMs), ICL is selected as the table reasoning baseline; (5) In the direction of large language model and graph structure fusion, Rel-LLM is selected as a representative hybrid framework to achieve unlabeled reasoning ability representation by jointly utilizing relational structure and large language model. Since the method of this invention can be applied to actual business databases such as "user table - loan table - repayment record table" in specific implementation, the above experimental design can be understood as: repeatedly executing the unified process of "graph construction - time constraint sampling - structural hint alignment - progressive context sparsity - masked cell prediction" in different real relational databases to evaluate the model's comprehensive modeling ability for cross-table relationships and field semantics.

[0042] In terms of implementation, this invention uses LLaMA 3.2-1B as the backbone large language model and LoRA for efficient parameter fine-tuning; the structural encoder employs a two-layer heterogeneous GraphSAGE to capture multi-hop relational dependencies; to initialize node features with rich semantic information, a pre-trained MPNet is used to perform text encoding and vectorized embedding of the original attribute values. All models are implemented based on PyTorch and trained on a server equipped with two NVIDIA RTX 6000 GPUs; the optimizer is AdamW. The training hyperparameters are set as follows: batch size is 128, maximum learning rate is 0.001, and a cosine decay learning rate scheduling strategy is adopted; the model is trained for a maximum of 200 epochs.

[0043] Specifically, Table 1 shows the experimental results of the present invention on the above 12 tasks, and Table 2 shows the impact of the remaining modules on the framework of the present invention.

[0044] Table 1. Performance comparison (AUROC) of the present invention (ZeroRel) with different types of models under unlabeled settings.

[0045]

[0046] This invention achieves superior results on most datasets and tasks, demonstrating its strong unlabeled generalization ability.

[0047] Table 2. Verify the performance contribution of the two modules.

[0048]

[0049] Among them, PSCR is used to guide the model to gradually transition from surface semantic dependencies to structured reasoning and adaptively select context, while GrPA is used to align multi-table structural representations and inject them into the semantic space of a large language model to provide structural constraints.

[0050] Table 3. Performance comparison of the present invention under different data sparsity settings.

[0051]

[0052] Increasing sparsity typically leads to sustained improvement in model performance, suggesting that introducing higher sparsity later in training helps the model learn richer structural and topological information. Furthermore, the optimal sparsity setting varies across different relational database datasets, indicating that data with higher structural complexity and greater inference difficulty usually requires greater sparsity to enhance structure awareness.

[0053] As can be seen from the above analysis, the relational reasoning method proposed in this invention can achieve adaptive selection of context and robust cross-table reasoning, and is significantly better than the comparison methods.

[0054] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0055] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple steps or stages, which are not necessarily completed at the same time, but may be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but may be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0056] In one embodiment, a computer system is provided, which may be a server. The computer system includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data used in the methods described above. The network interface communicates with external terminals via a network connection. The computer program is executed by the processor to implement the methods described above.

[0057] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0058] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the scope of the claims.

[0059] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A self-supervised relational reasoning method based on a large language model for relational databases, characterized in that, include: A relational graph is constructed from multiple tables in a relational database and the primary key-foreign key relationships between the tables. A time-consistent subgraph is generated based on timestamp sampling, and the node attributes are encoded to obtain the initial node features. Each row record in the table is used as a node in the relational graph, and the primary key-foreign key relationship is used as an edge in the relational graph. Node attributes refer to the field information carried by the row record corresponding to each node in the relational graph. Based on the time-consistent subgraph, node features are updated using a heterogeneous graph neural network to obtain the node structure representation. The node structure representation is mapped to the embedding space of the large language model as a structure cue vector, and then concatenated with the semantic instruction embedding for self-supervised training of mask cell prediction to align the structural information with the semantic representation. By controlling the proportion of context retention, the number of attribute contexts in the input large language model is gradually reduced, and partial context instructions are constructed. The structural cue vector and sparse instruction embeddings constructed based on partial context instructions are concatenated and input into the large language model for self-supervised training of masked cell prediction, so as to guide the large language model from semantic reasoning to structure-driven cross-table reasoning.

2. The self-supervised relational reasoning method based on a large language model for relational databases according to claim 1, characterized in that, The generation of a time-consistent subgraph based on timestamp sampling specifically includes: At timestamp t, only times earlier than 1 are retained. Build a time-consistent subgraph using nodes and edges. ,in and Let represent the set of nodes and the set of edges of a time-consistent subgraph that satisfies the time constraint, respectively. This represents the time truncation threshold corresponding to the current prediction time, used to ensure that the time-consistent subgraph only contains node and edge information that can be observed before the prediction time.

3. The self-supervised relational reasoning method based on a large language model for relational databases according to claim 1, characterized in that, The encoding of node attributes to obtain initial node features specifically includes: A multimodal column encoder is used to encode numerical, categorical, and textual attributes in a relational database to obtain initial node features.

4. The self-supervised relational reasoning method based on a large language model for relational databases according to claim 1, characterized in that, The method of updating node features using a heterogeneous graph neural network based on a time-consistent subgraph to obtain a node structure representation specifically includes: Based on time-consistent subgraph The node features are updated using a relation-aware heterogeneous graph neural network: ； ； in, Represents a node Based on primary key-foreign key relationship at level l The relationship-specific message representation obtained from aggregation, Representing relations Specific aggregate functions, This represents the node representation of neighbor node u at level l-1. Represents a node In relationship The set of neighboring nodes; This represents a set of relationship types, which includes at least primary key-foreign key relationships between tables. Represents a non-linear activation function. This represents the learnable parameter matrix of the l-th layer. This represents a combination function used to fuse the node's own representation with different relational message representations. This represents the node representation of node v after the update at layer l, which, after multiple propagations, yields the node structure representation. .

5. A self-supervised relational reasoning method based on a large language model for relational databases according to claim 1, characterized in that, The process of mapping node structure representations to the embedding space of a large language model as structure cue vectors, and then concatenating them with semantic instruction embeddings for self-supervised training of masked cell prediction to align structural information with semantic representations, specifically includes: Using multilayer perceptron Representing the node structure Mapping to the embedding space of a large language model yields structural cue vectors. : ； structural cue vector Embedding with column-level semantic instructions By splicing the pieces together, we can obtain the splicing features. : ； Will Input a large language model and perform a masked cell prediction task. The training objective is as follows: ； Where Y is the set of all possible values ​​of the masked cell. This represents the training loss function for the masked cell prediction task. Indicates that the parameter is The conditional probability distribution output by the large language model. This represents the actual target value corresponding to the i-th masked cell.

6. The self-supervised relational reasoning method based on a large language model for relational databases according to claim 1, characterized in that, The method of gradually reducing the number of attribute contexts in the input large language model by controlling the context retention ratio and constructing partial context instructions specifically includes: Define the context retention ratio To control the number of visible column attributes during training: ； This represents the set of attributes for the i-th row record. The number of visible column attributes that are retained for the i-th row record in the current training step; Update the context retention ratio using a linear decay method. ,in S represents the current number of training steps, and S represents the total number of training steps. Indicates the minimum context retention ratio at the end of training; Based on reservation Each column attribute constructs a partial context instruction, and the target column value to be predicted is replaced with a mask symbol to generate a prediction task.

7. A self-supervised relational reasoning method based on a large language model for relational databases according to claim 1, characterized in that, The step of concatenating the structural cue vector with sparse instructions constructed based on partial context instructions and inputting the concatenation into a large language model for self-supervised training of masked cell prediction, in order to guide the large language model from semantic reasoning to structure-driven cross-table reasoning, specifically includes: Sparse instruction embeddings are obtained by encoding some contextual instructions through the embedding layer or text encoding module of a large language model. ; structural cue vector With sparse instruction embedding By splicing the parts together, we obtain the second splicing feature. : ； Predicting target cell values ​​using large language models: ； in For the set of candidate values ​​for the target cell, This represents the training loss function for masked unit prediction based on partial context instructions. Indicates that the parameter is The conditional probability distribution output by the large language model. This represents the actual value corresponding to the i-th target cell.

8. A computer system comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

Images