A database schema matching method based on relationship perception and incremental learning

Abstract

The application relates to a database mode matching method based on relationship perception and incremental learning, and belongs to the technical field of computer databases. First, the modal information of a target database and a source database is found. Then, feature extraction is performed on the sub-modal text information. After that, the feature extraction information is input into a Bayesian network for selection and screening. In this process, the quality of the features is also measured to ensure the accuracy and stability of the sub-modal classification. At the same time, neuron regularization and Bayesian incremental learning methods are introduced to fully exploit the continuous learning ability of the model. Finally, the obtained target database sub-modal and source database sub-modal are matched through a feature relationship perception mode to obtain a mode matching result. The application can effectively improve the query and mode matching processing efficiency and accuracy between different databases.

Description

Technical Field

[0001] The present invention relates to a database pattern matching method, in particular to a database pattern matching method based on relationship perception and incremental learning, and belongs to the technical field of computer databases. Background Art

[0002] Database schema matching, which is the task of querying semantic correspondence between elements (or attributes) of two given database schemas, is crucial for achieving data integration and system interoperability in fields such as e-commerce, geospatial, biology, and health.

[0003] The task of database schema matching is challenging for many reasons. On the one hand, schema elements (e.g., attributes representing the same concept) may have different names in different schemas. On the other hand, elements with similar names may refer to different concepts. In addition, equivalent elements in two schemas may have different structures. Finally, there may be cases where many elements in one schema represent concepts represented by a single element in another schema.

[0004] Domestic database systems are currently experiencing rapid development, with many excellent domestic database products emerging. Due to the large scale of information systems and the high complexity of business operations, data distribution is characterized by massive and heterogeneous data, with a large amount of data stored on overseas database platforms. In this context, enabling data query and migration between different databases has become a critical task.

[0005] Pattern matching is one of the key technologies for implementing data migration tasks. Figure 1 This shows an example of pattern matching. When migrating data from a source database to a target database, it is necessary to establish a matching relationship between the database sub-pattern and the field. Figure 1 As shown in the figure, database pattern matching helps users transform source data schemas into target data schemas by establishing mappings between database schemas. Traditional pattern matching techniques take two heterogeneous data sources as input and match the data source schemas based on their metadata or data instances. This data source matching process uses a method based on attribute similarity. After obtaining possible matches, users must manually specify which matches are truly desired. Summary of the Invention

[0006] The purpose of this invention is to address the shortcomings and improvement needs of existing manual matching, such as low efficiency and high human error rate, and to creatively propose a database pattern matching method based on relationship perception and incremental learning, which can effectively improve the efficiency and accuracy of query and pattern matching processing between different databases.

[0007] The present invention is implemented by adopting the following technical solutions.

[0008] A database pattern matching method based on relationship perception and incremental learning includes the following steps:

[0009] Step 1: Database sub-schema discovery.

[0010] The primary task of database pattern matching queries is database subpattern discovery. This is the process of automatically or semi-automatically identifying and extracting potentially useful subpatterns from a database. These subpatterns can be frequently occurring item sets, association rules, sequential patterns, and so on. This information helps discover useful patterns and associations in the data, thereby supporting data analysis, decision-making, and other tasks.

[0011] Sub-schema discovery primarily works on tabular data by searching for tables that can be joined, augmenting a given table with more data entries or additional attributes, and finding tables similar to a given table using different similarity measures.

[0012] Step 2: Sub-pattern feature representation.

[0013] Generally, a multi-dimensional feature vector is used to represent a sub-pattern data sample. The database sub-pattern information includes name, type, attribute description, etc. These description information are not easy to organize into an n-dimensional vector.

[0014] Therefore, when extracting sub-pattern features, we will refer to the pattern feature representation method based on entity classification, use the naive Bayes classifier or ResNet network to train an initial pattern matching sub-pattern classification model, combine different description information into short texts, and modify them to be suitable for the pattern matching scenario in this article.

[0015] Step 3: Sub-mode selection.

[0016] The present invention creates a quality feature to represent the reliability of low-quality pattern data. Sub-pattern samples are screened in the classification results, and the instances that are most conducive to improving classifier performance are selected and added to the training set, fully utilizing process data information and minimizing the impact of unreliable samples.

[0017] Specifically, extract reliable database data S, S∈R M×N , M and N represent the number of process variables and the number of data samples, respectively, and R represents the database format domain. Through the manual verification program, several reliable samples S are obtained under fixed working conditions. R , S R The mean vector of the trust center S T .

[0018] The quality characteristic calculation of process data X is designed to be the Mahalanobis distance from the measured mode to the trustworthy center. Compared with the typical Euclidean distance, the Mahalanobis distance method is a more reasonable method to measure modal similarity. It is scale-free and can eliminate the influence of variable correlation. Therefore, the sample mode S H The Mahalanobis distance d H As shown below:

[0019]

[0020] Among them, C is from S R The covariance matrix of the obtained process variables; Indicates transpose.

[0021] Step 4: Bayesian incremental learning based on neuron regularization and resource release.

[0022] The instance that meets the active selection and screening conditions and the pattern matching algorithm are assigned to the instance, and the classification model is trained again using the idea of ​​incremental learning: because data in the real world is constantly growing and changing, in order to avoid the need to retrain the entire model when new data is introduced, the model's adaptability and efficiency in processing complex data are enhanced.

[0023] Step 5: Introduce the relation-aware self-attention model. The relational connection between sub-patterns is realized through this model, realizing pattern matching based on relation-awareness.

[0024] The model embeds semi-structured input sequences in a way that jointly encodes the relational structure already present in the input and induces "soft" relations between sequence elements in the same embedding.

[0025] Preferably, during the sub-pattern selection process, the target database sub-pattern is trained by relying on a Bayesian network, and the source database sub-pattern is compared with a plurality of trusted centers and then output as a binary classification type.

[0026] Preferably, when performing incremental Bayesian network training, node-level regularization is used to keep the weights of target neurons of the same importance, and then a regularization of uniform strength is applied to the weights of target neurons according to the importance.

[0027] Preferably, the relation-aware attention model provides a method to communicate known relations to the attention mechanism module by adding representations of known relations to the attention mechanism.

[0028] Beneficial effects

[0029] Compared with the prior art, the present invention has the following advantages:

[0030] 1. In response to the shortcomings of traditional Bayesian learning methods when data is unreliable, this invention proposes an improved incremental learning method based on neuron regularization and combined with a resource release mechanism. To address the problem of excessive model computation, this method specifies the prior as a simpler Gaussian distribution and then performs analytical calculations on the regularization term. Secondly, this method does not use uncertainty as the basis for importance judgment, but instead groups the variance of weights by neuron and unifies them to the same value. This unified variance is then used as the regularization strength to collectively regularize all weights under the same neuron, achieving regularization of the entire neuron without introducing additional operations to calculate neuron importance.

[0031] 2. This invention improves upon typical Bayesian network learning methods and BIC-based structural learning methods by introducing weighted quality metrics to capture the most useful information from unreliable data sources. This significantly improves the probability model and network structure, making them more closely aligned with the true distribution and causal relationships, respectively. Furthermore, by using quality characteristics as weighting factors, a more reasonable control range for process monitoring is achieved.

[0032] 3. In order to more fully explore the model's continuous learning ability, the present invention introduces a resource release mechanism for the model, which facilitates the implementation of the Bayesian incremental learning algorithm. By selecting pattern matching instances during the test process and re-using them as training sets for pattern matching, it conforms to the trend of data growth and changes, thereby optimizing the matching effect.

[0033] 4. The present invention applies the relationship-aware self-attention framework to database sub-pattern matching, provides alignment modeling between the source database sub-pattern and the target database sub-pattern in a given query, defines the set of relationship types used in a way that directly addresses the challenges of pattern embedding and linking, and adds some auxiliary edges to help pattern encoding. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is an example of database pattern matching;

[0035] Figure 2 It is an overall flow chart of the method of the present invention;

[0036] Figure 3 It is a flowchart of the BIC-based structure learning method to improve the network structure through quality measurement;

[0037] Figure 4 This is a flowchart of regularization improvement and incremental learning based on Bayesian networks;

[0038] Figure 5 This is a flowchart of feature relationship perception based on the attention mechanism. DETAILED DESCRIPTION

[0039] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0040] like Figure 2 A database pattern matching method based on relationship perception and incremental learning is shown in Figure 2. By introducing the Bayesian network model and the self-attention-based relationship perception method in the machine learning model, it completes the pattern matching task between the target database and the source database, and can easily perform incremental learning after the initial training is completed.

[0041] First, we use SQL-based data dictionary and metadata query methods to find the modal information of the target and source databases. Alternatively, we can manually check the basic information of the databases to obtain the information.

[0042] Next, feature extraction is performed on the submodal text information. This extracted feature information is then fed into a Bayesian network for selection and screening. During this process, feature quality metrics are also performed to ensure the accuracy and stability of submodal classification. Furthermore, neuron regularization and Bayesian incremental learning methods are introduced to fully exploit the model's continuous learning capabilities.

[0043] Finally, the target database sub-modality and the source database sub-modality are compared through feature relationship perception mode to obtain the pattern matching result.

[0044] Example

[0045] This embodiment performs scheduling processing on a database written in SQL, and obtains metadata information of the database by executing a specific SQL query statement.

[0046] A database pattern matching method based on relationship perception and incremental learning includes the following steps:

[0047] Step 1: Database sub-schema discovery.

[0048] Step 2: Sub-pattern feature representation.

[0049] When extracting sub-pattern features, the present invention refers to the Bayesian network method based on entity classification, represents the sub-pattern with text, and then uses TF-IDF to generate the feature vector of the sub-pattern.

[0050] The frequency of occurrence of a single information in a sub-pattern SF is expressed as:

[0051]

[0052] Among them, SF i,j The information i representing the sub-pattern is in the sub-pattern S j The information frequency in N i,jThe information i representing the sub-pattern is in the sub-pattern S j The total amount of information in , k represents the amount of information contained in all sub-patterns.

[0053] Inverse mode frequency ISF of a single sub-mode message i Expressed as:

[0054]

[0055] Among them, |S| represents the total number of sub-patterns; |{j:N i ∈S j}| represents the number of sub-patterns containing information i. After TF-IDF processing, a sparse matrix S={s1,s2,…,s N}, as input for sub-pattern selection.

[0056] Step 3: Sub-mode selection.

[0057] By analyzing the formulas for learning the structure and parameters of Bayesian networks, the learning objective function is composed of the log-likelihood results of all data instances, which can be interpreted as the degree of fit of the model to the data. Because every data point is treated equally, traditional learning methods are obviously flawed when the data is unreliable, unable to correctly represent the true causal relationships and true distributions. Moreover, the performance of traditional process modeling based on Bayesian networks is affected by the quality of the data.

[0058] To solve this problem, the present invention uses quality features to fit the pattern and performs regularization and resource release operations on the Bayesian network.

[0059] Taking the fully connected Bayesian network as an example, since in the Bayesian network, the weight z exists in the form of probability distribution, θ = (μ, σ) represents the parameter set that defines the probability distribution of the weight, with a mean of μ and a standard deviation of σ.

[0060] Step 3.1: Pattern fitting.

[0061] In order to learn the Bayesian network structure from data, using the Bayesian Information Criterion (BIC) score function has been shown to be an effective trade-off between data fitting and model complexity. In this work, a BIC-based scoring function is used to learn the Bayesian network structure and is written as:

[0062]

[0063] Among them, G is the network structure, N is the number of samples of the sub-pattern in the data set, dim[G] is the dimension of the network, LL(θ G :S) is the log-likelihood function, θ GRepresents the most likely parameter set. Since the optimal solution of the Bayesian network structure is an NP problem, a heuristic algorithm is introduced to help search for the best fitting result.

[0064] After obtaining the Bayesian network structure, the corresponding model parameters θ are learned from the data set S through the maximum likelihood estimation method. G , the likelihood function is defined as:

[0065]

[0066] in, Represents a single node x m For its parent node π(x m ) According to the likelihood function of the variable relationship, the corresponding log-likelihood function is obtained as the objective function of the expectation maximization (EM) algorithm, and the optimal solution of the model parameters θ is iteratively searched. G .

[0067] Extract reliable database data S and express it as S∈R M×N , where M and N represent the number of process variables and the number of data samples, respectively, and R represents the database format domain. Through the manual verification procedure, several reliable samples are obtained under fixed working conditions and are recorded as S R , note S R The mean vector of the trust center S T .

[0068] In this method, the quality characteristic calculation of process data s is designed to be the Mahalanobis distance from the measured mode to the credible center. Compared with the typical Euclidean distance, the Mahalanobis distance method is a more reasonable method to measure modal similarity. It is scale-free and can eliminate the influence of variable correlation. Therefore, the sample mode S H The Mahalanobis distance can be written as follows, where C is the distance from S R Find the covariance matrix of the process variables.

[0069]

[0070] Then, the quality measurement result of each sample is obtained according to the exponential function of the defined distance result, which is expressed as M = {m1,m2,…,m N}, where m i Expressed as η d As the tuning factor, the optimized BIC-based scoring function is obtained, thereby updating the Bayesian network parameter θ in a negative feedback manner. G .like Figure 3 shown.

[0071] Step 3.2: Node regularization.

[0072] Taking the lth layer as an example, in the reparameterization stage, the standard deviation matrix σ l In , all weights of a single layer are divided into I groups, and the standard deviation of the weights of each group is limited to the same value, and then a new vector is formed with these I values:

[0073]

[0074] in, Indicates transpose.

[0075] Then, L(μ) is written as:

[0076]

[0077] Among them, L(μ) represents the mean vector of the lth layer, λ represents the update weight, μ l,(i) Represents the mean vector of the i-th group weights in the l-th layer, μ l,(i) and μ′ l,(i) Represent μ l,(i) The mean that needs to be optimized in the current task and the mean learned in the previous task; (σ′ l,(i) ) 2 Represents the variance of the i-th group of weights learned in the previous task, and its reciprocal will be used as the optimization μ l,(i) The weight of time.

[0078] In the previous model, each weight has a different update weight. After the above design, L(μ) will apply regularization to all elements in the mean vector according to a common update weight, that is, regularization is completed based on the node in a one-to-many form.

[0079] Step 4: Bayesian incremental learning based on neuron regularization and resource release.

[0080] In order to allocate resources more reasonably, a resource release incremental learning mechanism is introduced into the model.

[0081] First, sort the weights of layer l according to the size of the standard deviation.

[0082] Then, set a release ratio Based on this ratio, a threshold th is obtained. The i groups of weights whose standard deviation is greater than the threshold are released. The other Ii groups of weights whose standard deviation is less than the threshold are not released. The only exception is the first task. Since the prior for the first task is specified as a fixed Gaussian distribution, the prior for the weights in the lth layer is equal to the variance and cannot be compared. Therefore, they are randomly selected for release. The rest is as shown in the normal steps.

[0083] The specific release mechanism is: by adding the dynamic factor v into L(σ), the specific operation of the release mechanism is embedded in the loss function, so that L(σ) can dynamically switch between release and fixation according to the standard deviation of the weight. For weights with a standard deviation less than the threshold, v takes the value of 1. When the weight variance is higher than the threshold, v switches to a constant greater than 1.

[0084] The sub-pattern incremental correction model is a dynamic process of selecting instances from the sub-pattern incremental set S′ and adjusting the model parameters according to the newly added sub-patterns. If new pattern matching samples are added, P(θ|S,D) is updated, where P is the joint probability distribution of the Bayesian network and D is the prior knowledge of the Bayesian network. θ is the Bayesian network parameter. The overall Bayesian network operation process is as follows: Figure 4 shown.

[0085] Step 5: Relationship-aware pattern matching.

[0086] like Figure 5 As shown, the transformer structure is used to build an attention mechanism, which increases the controllability of sub-mode instances by active selection, and uses the attention mechanism to compare the sample instances of the incremental set with the sample instances of the target mode. j to match.

[0087] Consider a set of inputs S = {s i}, it is usually considered that a sub-pattern is an unordered set, and the relationship between the source sub-pattern and all sub-pattern categories is expressed in the form of probability.

[0088] The self-attention encoder introduces a stack of self-attention layers to convert each s i Convert to y i , which can be used as each source sub-pattern s i With target sub-pattern s j The confidence of is compared with the threshold ε. The specific calculation process is as follows:

[0089]

[0090]

[0091]

[0092]

[0093] y i ′=LayerNorm(s j +z i )

[0094] y i =LayerNorm(y i′+FC(ReLU(FC(y i ′))))

[0095] in, represents the attention vector of the h-th attention head, represents its weighting coefficient, represents the embedding vector of the h-th attention head; z i represents the total embedding vector, y i Represents the final output encoding vector, y i ' represents the variable to be output; d z represents the normalized value, H is the number of heads of the transformer; Q, K, V are query, key and value respectively, and W Q 、W K 、W V Represent the weight parameters of Q, K, and V respectively; FC is a fully connected layer, LayerNorm is layer normalization, and Concat represents the connection vector. Represents matrix transpose.

[0096] Considering the need to bias the self-attention model towards pre-existing relationships in the sub-modality while avoiding limiting the attention mechanism to encoders that maintain known existing relationships, which hinders the transformer's ability to learn new relationships, the present invention provides a method for communicating known relationships with the encoder by adding representations of known relationships to the attention mechanism, as follows:

[0097]

[0098]

[0099] in, For two input elements x i and x j The set of relationships between .

[0100] Although the present invention has been disclosed as above by way of embodiments, they are not intended to limit the present invention. Any appropriate modification or equivalent substitution of the technical solution of the present invention by a person skilled in the art should be included in the protection scope of the present invention. The protection scope of the present invention shall be based on that defined in the claims.

Claims

1. A database pattern matching method based on relationship perception and incremental learning, characterized in that: By introducing the Bayesian network model and the self-attention-based relationship perception method in the machine learning model, the pattern matching task between the target database and the source database is completed, including the following steps: Step 1: Database sub-schema discovery; Sub-schema discovery involves processing tabular data by searching for tables that can be joined, augmenting a given table with more data entries or additional attributes, and finding tables similar to a given table using different similarity measures; Step 2: Sub-pattern feature representation; When extracting sub-pattern features, we refer to the pattern feature representation method based on entity classification to train an initial pattern matching sub-pattern classification model, combine different description information into short texts, and modify them to be suitable for pattern matching scenarios; Step 3: Sub-mode selection; Quality features are used to represent the measurement results of the reliability of low-quality pattern data; sub-pattern samples are screened in the classification results, and the instances that are most conducive to improving classifier performance are selected to be added to the training set, making full use of process data information and minimizing the impact of unreliable samples; Extract reliable database data S, S∈R M×N , M and N represent the number of process variables and the number of data samples respectively, and R represents the database format domain; through the manual verification procedure, several reliable samples S are obtained under fixed working conditions R , S R The mean vector of the trust center S T ; The quality characteristic calculation of process data X is designed as the Mahalanobis distance from the measured mode to the credible center, and the sample mode S H The Mahalanobis distance d H As shown below: Among them, C is from S R The covariance matrix of the obtained process variables; T represents transpose; Step 4: Bayesian incremental learning based on neuron regularization and resource release; Assign the instance that meets the active selection and screening conditions and the pattern matching algorithm to the instance, and retrain the classification model using the idea of ​​incremental learning; Step 5: Introduce the relation-aware self-attention model, and realize the relational connection between sub-patterns through this model, thereby realizing pattern matching based on relation-awareness.

2. The database pattern matching method based on relationship perception and incremental learning according to claim 1, characterized in that: In step 2, when extracting sub-pattern features, the Bayesian network method based on entity classification is specifically referred to. After the sub-pattern is represented by text, TF-IDF is used to generate the feature vector of the sub-pattern. The frequency of occurrence of a single information in a sub-pattern SF is expressed as: Among them, SF i,j The information i representing the sub-pattern is in the sub-pattern S j The information frequency in N i,j The information i representing the sub-pattern is in the sub-pattern S j The total amount of information in , k represents the number of information contained in all sub-patterns; Inverse mode frequency ISF of a single sub-mode message i Expressed as: Among them, |S| represents the total number of sub-patterns; |{j:N i ∈S j | represents the number of sub-patterns containing information i. After TF-IDF processing, a sparse matrix S={s1,s2,…,s N }, as input for sub-pattern selection.

3. The database pattern matching method based on relationship perception and incremental learning according to claim 1, characterized in that: In step 3, the quality features are used to fit the pattern, and the Bayesian network is regularized and resource released; Step 3.1: Pattern fitting; The Bayesian network structure is learned using a scoring function based on the Bayesian Information Criterion, which can be written as: Among them, G is the network structure, N is the number of samples of the sub-pattern in the data set, dim[G] is the dimension of the network, LL(θ G :S) is the log-likelihood function, θ G represents the most likely parameter set; After obtaining the Bayesian network structure, the corresponding model parameters θ are learned from the data set S through the maximum likelihood estimation method. G , the likelihood function is defined as: in, Represents a single node x m For its parent node π(x m ) According to the likelihood function of the variable relationship, the corresponding log-likelihood function is obtained as the objective function of the expectation maximization algorithm, and the optimal solution of the model parameters θ is iteratively searched. G ; Extract reliable database data S and express it as S∈R M×N , M and N represent the number of process variables and the number of data samples respectively; through the manual verification procedure, a number of reliable samples are obtained under fixed working conditions and are recorded as S R , note S R The mean vector of the trust center S T ; The quality measurement result of each sample is obtained according to the exponential function of the defined distance result, which is expressed as M = {m1,m2,…,m N }, where m i Expressed as η d As the tuning factor, the optimized scoring function based on the Bayesian information criterion is obtained, thereby updating the Bayesian network parameter θ in a negative feedback manner. G ; Step 3.2: Node regularization; For the lth layer, in the reparameterization stage, the standard deviation matrix σ l In , all weights of a single layer are divided into I groups, and the standard deviation of the weights of each group is limited to the same value, and then a new vector is formed with these I values: in, represents transpose; Then, L(μ) is written as: Among them, L(μ) represents the mean vector of the lth layer, λ represents the update weight, μ l,(i) Represents the mean vector of the i-th group weights in the l-th layer, μ l,(i) and μ′ l,(i) Represent μ l,(i) The mean that needs to be optimized in the current task and the mean learned in the previous task; (σ′ l,(i) ) 2 Represents the variance of the i-th group of weights learned in the previous task, and its reciprocal will be used as the optimization μ l,(i) The weight of time; After the above design, L(μ) will apply regularization to all elements in the mean vector according to a common update weight, that is, regularization is completed based on the node in a one-to-many form.

4. The database pattern matching method based on relationship perception and incremental learning according to claim 1, characterized in that: In step 4, first sort the weights of layer l according to the size of the standard deviation; Then, set a release ratio A threshold th is obtained based on the ratio, and the i groups whose standard deviation of the weights is greater than the threshold are released; For the other Ii groups of weights that are less than the threshold, they are not released. The only exception is the first task. Since the prior of the first task is specified as a fixed Gaussian distribution, the weights in the lth layer are randomly selected for release. The release mechanism is: by adding the dynamic factor v to L(σ), the specific operation of the release mechanism is embedded in the loss function, so that L(σ) can dynamically switch between release and fixation according to the standard deviation of the weight. For weights with a standard deviation less than the threshold, v takes the value of 1. When the weight variance is higher than the threshold, v switches to a constant greater than 1. The sub-pattern incremental correction model is a dynamic process of selecting instances in the sub-pattern increment set S′ and adjusting the model parameters according to the newly added sub-patterns. If new pattern matching samples are added, P(θ|S,D) is updated, where P is the joint probability distribution of the Bayesian network and D is the prior knowledge of the Bayesian network.

5. The database pattern matching method based on relationship perception and incremental learning according to claim 1, characterized in that: In step 5, the attention mechanism is constructed using the transformer structure to increase the controllability of the sub-pattern instance through active selection. The sample instance of the incremental set is matched with the sample s1 of the target pattern through the attention mechanism. Consider a set of inputs S = {s i }, is considered as an unordered set, and the relationship between the source sub-pattern and all sub-pattern categories is expressed in the form of probability; The self-attention encoder introduces a stack of self-attention layers to convert each s i Convert to y i , as each source sub-pattern s i With target sub-pattern s j The confidence of is compared with the threshold ε, and the process is as follows: y i ′=LayerNorm(s j +z i ) y i =LaterNorm(y i ′+FC(ReLU(FC(y i ′)))) in, represents the attention vector of the h-th attention head, represents its weighting coefficient, represents the embedding vector of the h-th attention head; z i represents the total embedding vector, y i Represents the final output encoding vector, y i ' represents the variable to be output; d z represents the normalized value, H is the number of heads of the transformer; Q, K, V are query, key and value respectively, and W Q 、W K 、W V Represent the weight parameters of Q, K, and V respectively; FC is a fully connected layer, LayerNorm is layer normalization, and Concat represents the connection vector. Represents matrix transpose; We adopt the approach of communicating the known relationship to the encoder by adding the representation of the known relationship to the attention mechanism as follows: in, For two input elements x i and x j The set of relationships between .

Images