A method for detecting errors in a relational database with a learning-based attribute pattern

Abstract

The present invention belongs to the technical field of database data quality management, and specifically relates to a relational database error detection method with a learning-based attribute pattern. Steps: Data preprocessing, representing the input data sequence with input tokens constructed by interleaving data tokens and pattern tokens; Error recognition, learning the features of independence between attribute domains and sharing within attribute domains of corresponding attributes based on the input tokens, and storing the learned content in the learnable pattern module and the multi-head self-attention network weights for detection; Visualization and explanation, extracting the attention scores within the multi-head self-attention network, constructing the relationship between the input tokens and the output predictions based on this, and generating a visual attention map. The main improvement of the present invention is the attribute pattern-aware Transformer framework, which learns the features of the data values in the relational table and its attribute domain structure, is used to detect data errors in a given data set, improves the accuracy and efficiency of the detection task, and visualizes the process of error detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of database data quality management, and particularly relates to a relational database error detection method with a learning-based attribute pattern. Background Art

[0002] Error detection of data tables aims to discover various format violations and rule violations, which are constrained and restricted by relational databases. In the production, storage, and usage stages of data, error detection is a fundamental step in improving data quality. In addition, in subsequent data repair and data-driven tasks, accurate and efficient error detection can significantly reduce the burden on data analysts. Therefore, improving the accuracy and efficiency of error detection methods is crucial for actual data quality management.

[0003] Existing error detection methods are difficult to fully utilize the structural advantages of the relational table. Machine learning-based error detection methods need to handle multiple attribute domain problems of the relational table to learn the data distribution of multiple attributes. The existing methods mainly include three types: one is the unit-level data error detection method using multiple classifiers (Heidari A, et al. Holodetect: Few-shot learning for error detection[C] / / Proc. ACM SIGMOD Int. Conf. Manag. Data. 2019: 829-846.), which improves the accuracy by increasing the cost of model training and deployment. The second is the unit-level data error detection method using a single model (Miao Z, et al. Rotom: A meta-learned data augmentation framework for entity matching, data cleaning, text classification, and beyond[C] / / Proc. ACM SIGMOD Int. Conf. Manag. Data. 2021: 1303-1316.), which detects any data unit by obtaining the multi-column feature distribution of the data table. Due to the unique data profile of each attribute and the inclusion of different error types, this deviation between attributes inevitably affects the accuracy and generalization of the detection model. The third is the tuple-level data error detection method using large-scale models (Liu Z, et al. Picket: guarding against corrupted data in tabular data during learning and inference[J]. The VLDB Journal, 2022, 31(5): 927-955.), which needs to increase the model parameters and scale, so the training and inference costs are relatively high in actual use.

[0004] The main challenge in the data preprocessing stage is how to divide the data sequence to meet the requirements of the error detection model for the input structure. Currently, the data preprocessing methods are mainly divided into two types: Tokenizer and extracting predefined features. In the preprocessing stage of data error detection, the Tokenizer (Rico S, et al. Neural MachineTranslation of Rare Words with Subword Units [C]. ACL, 2016.) tokenizes the unit sequence into multiple input tokens, and its division follows the convention of natural language processing. However, the tokenization criteria of these Tokenizers do not consider factors such as error formats, error types, syntax, grammar, and inconsistent unit sequence lengths. Therefore, the vocabulary generated by these methods cannot be used as a suitable representation for the data quality detection management task and does not meet the limitations of token length and quantity. As another preprocessing method, the way of extracting predefined features (Neutatz F, et al. Ed2: A casefor active learning in error detection[C] / / Proc. ACM CIKM Int. Conf.Information & Knowledge Management. 2019: 2249-2252.) is not conducive to the model to understand the input sequence and improve the decision-making ability. Therefore, the deficiencies of the preprocessing methods lead to performance bottlenecks in the error detection model.

[0005] In summary, using deep learning to detect error data requires a trade-off between accuracy and efficiency. Achieving high efficiency requires the machine learning method to be simple, complete, and end-to-end to minimize time and space costs; while achieving high accuracy requires comprehensive task completion and high performance evaluation metrics.

[0006] In addition, currently few data error detection studies focus on the internal operation mechanism of the model. Some works in the fields of natural language processing (NLP) and computer vision (CV) conduct interpretability studies on unstructured data (paragraph texts and image fragments) using visualization and other methods. However, DL models used for relational table data detection may be difficult to provide sufficient internal attention parameters to establish the relationship between the data sequence and classification. As the infrastructure for data quality management, error detection methods are necessary to explore feasible visualization methods and explain the error detection mechanism. Summary of the Invention

[0007] To solve the problems existing in the prior art, the present invention provides a relational database error detection method with a learning-based attribute pattern, which is improved in three aspects: data preprocessing, the structure of the detection model, and the interpretability analysis of the model, so as to detect various types of errors in a data table with multiple attributes and perform visual analysis.

[0008] The technical solution adopted by the present invention to solve its technical problems is as follows: A relational database error detection method with a learning-based attribute pattern, the steps are as follows:

[0009] S1. Data preprocessing, representing the input data sequence with input tokens constructed by interleaving data tokens and pattern tokens;

[0010] S2. Error recognition, learning the features independent between attribute domains and shared within attribute domains of corresponding attributes based on the input tokens, and storing the learned content in the learnable pattern module and the weights of the multi-head self-attention network for detection;

[0011] S3. Visual interpretation, extracting the attention scores within the multi-head self-attention network, constructing the relationship between the input tokens and the output predictions based on this, and generating a visual attention map;

[0012] Adopt an attribute pattern-aware Transformer framework, which includes:

[0013] A data preprocessing module, configured with a program for executing a quasi-token arrangement algorithm and a hyperparameter setting algorithm;

[0014] A model structure module, configured with a program that takes the interleaved arrangement of data tokens and pattern tokens as input, learns the features independent between attribute domains and shared within attribute domains of corresponding attributes, stores the learning, and can output the detection results;

[0015] An interpretability module, configured with a program for extracting the parameters of the multi-head self-attention network, constructing the relationship between the input tokens and the output predictions based on this, and generating a visual attention map.

[0016] Preferably, the data preprocessing includes: dividing the input data sequence according to the length of the sequence and the type of words through a quasi-token arrangement algorithm to obtain data tokens; representing the attribute patterns of the data unit values in the input data sequence with pattern tokens according to the learnable patterns; and performing corresponding interleaved arrangements of the data tokens and the pattern tokens to obtain structured input tokens.

[0017] Preferably, the quasi-token arrangement algorithm depends on a tokenization hyperparameter setting algorithm for automatically configuring hyperparameters, and the tokenization hyperparameters include: the token dimension D and the number of tokens N.

[0018] Preferably, the steps of the tokenization hyperparameter setting algorithm are as follows:

[0019] S1. Obtain the histogram of token lengths and quantities from the error data table, and use the critical point search function to find a critical point to divide the elements of the input array into a high-frequency point group and a low-frequency point group;

[0020] S2. The token dimension is calculated by the critical point search function, with the input being the token length array and a preset threshold coefficient; in the default mode, the token quantity is calculated using the token length array as the input of the critical point search function, while in the compact mode, the token quantity is the length of the longest sequence in the dataset divided by the compact token dimension;

[0021] S3. The critical point search function determines the demarcation point between the high-frequency point group and the low-frequency point group through the product of the average value of the frequency histogram and the preset threshold coefficient; in the default mode, the preset threshold coefficient is β, and in the compact mode, it is .

[0022] Preferably, the quasi-token arrangement algorithm includes three stages: full word segmentation, rough word segmentation, and block word segmentation; in each stage, first apply the quasi-tokenizer to the input data sequence for division to obtain quasi-tokens, then set the tokenization parameters for the obtained quasi-tokens for arrangement, then perform the segmentation of tokens exceeding the dimension and the sliding window merging of short tokens, and finally map them to numerical representations according to the Unicode character dictionary to obtain data tokens.

[0023] Preferably, the quasi-token arrangement algorithm takes an input unit sequence with variable length and tokenization hyperparameters as input; first apply the quasi-tokenizer to the original input data sequence to obtain quasi-tokens for further processing ; subsequently, for each token in the quasi-tokenization that exceeds the dimension D limit, it is split into multiple tokens; for the quasi-tokenization that still does not meet the requirements of the token dimension D and the token quantity N, in the merging stage, adjacent tokens will be merged according to certain rules: when merging, words and punctuation marks are given priority, while numbers are considered last; tokens at the same merging level are merged first; as long as the tokens obtained after segmentation meet the requirements of the token dimension D and the token quantity N, the algorithm ends.

[0024] Preferably, the data preprocessing process is as follows: to imitate the structural characteristics of the unit value and its attribute domain, the data tokens and the pattern tokens are interleaved to form interleaved tokens, where the data tokens are obtained by tokenization of the input unit sequence in the preprocessing stage, the pattern tokens are obtained through training with the PAT model, N is the number of tokens, each data token and each pattern token All are embedding vectors of dimension D; then, a learnable CLS embedding token is appended to the head of the interleaved tokens of data and pattern ; The input tokens are obtained by combining the CLS token, data tokens, and pattern tokens: ; ; is the position information provided for the interleaved tokens of data and pattern;

[0025] Applying multiple structures, including multi-head self-attention, feed-forward layer, layer normalization, and residual connection, the overall computational structure of the attribute pattern-aware Transformer network is as follows:

[0026] ;

[0027] ;

[0028] ;

[0029] Among them, represents the number of stacked layers of the encoder; represents the output features of the self-attention network of the th layer; The output feature embedding of the th layer encoder is denoted as ; The state corresponding to the CLS token at the output of the model is used as the classification feature representation of the input unit sequence of the data table ;

[0030] Query, key, and value matrices are the main structures of the multi-head self-attention network, denoted as , , respectively, and they are computed in parallel: ;

[0031] Among them, is the internal weight matrix of the multi-head self-attention network; ; The dimension of the key is set to , the dimension of the query and the dimension of the value are the same as the dimension of the key;

[0032] Matrix output representation of the multi-head self-attention network: ;

[0033] Furthermore, to analyze the internal structure of the attention module, this method decomposes the steps therein. The present invention computes the attention score matrix to obtain , and this attention score matrix contains two types of sub-elements in the data tokens and pattern tokens. and can be described as and ;

[0034] ;

[0035] Attention score matrix fuses the features in the original data unit to be detected and the corresponding attribute pattern;

[0036] Value matrix can be described as ;

[0037] Matrix output of the multi-head self-attention network: .

[0038] Preferably, for visual interpretation, the first column vector is extracted from the attention score matrix to visualize the relationship between the input token and the classification prediction.

[0039] Preferably, the model structure module is based on the Transformer encoder architecture and consists of six identical encoder stacking layers. Each layer of the encoder is provided with multi-head self-attention and a feed-forward layer. The multi-head self-attention and the feed-forward layer are connected through an add-and-normalize operation. An MLP head for multi-layer perceptron classification is provided at the output end of the last layer of the encoder.

[0040] Compared with the prior art, the beneficial effects of the present invention are:

[0041] 1. The present invention proposes an attribute pattern-aware Transformer framework for error detection in data tables. The learnable pattern tokens can capture data features of different attributes during model training, thus achieving more effective detection.

[0042] 2. In the sequence preprocessing stage, the present invention introduces a quasi-token arrangement algorithm to separate the unit sequence into tokens according to appropriate lengths and token types. The tokens in QTA are suitable for detecting potential atomic error units. To maintain the low cost of the PAT model, the present invention provides a compact QTA, and it can meet the tokenization requirements of long-sequence data.

[0043] 3. The present invention visualizes the error detection process by extracting the attention scores in the MSA network and using them to present the relationship between the input data and the classification prediction.

[0044] In summary, the main improvement of the present invention is an attribute pattern-aware Transformer framework that learns the features of the data values in the relational table and its attribute domain structure, is used to find data errors in a given data set, and improves the accuracy and efficiency of the error detection task. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Flowchart of the relational database error detection method for the learning-based attribute pattern;

[0046] Figure 2 Classification of relational database data error types;

[0047] Figure 3 Structural diagram of the relational database error detection model for the learning-based attribute pattern;

[0048] Figure 4 Diagram of the operation process and attention score matrix of the multi-head self-attention structure;

[0049] Figure 5 Tree diagram of the quasi-token arrangement algorithm (QTA) in the data preprocessing stage;

[0050] Figure 6 Operation example of the quasi-token arrangement algorithm (QTA) in the data preprocessing stage;

[0051] Figure 7 D-accuracy result diagram of 4 data error detection methods on the Restaurants dataset;

[0052] Figure 8 D-accuracy result diagram of 4 data error detection methods on the Rayyan dataset;

[0053] Figure 9 D-accuracy result diagram of 4 data error detection methods on the Movies dataset;

[0054] Figure 10 D-accuracy result diagram of 4 data error detection methods on the Hospital dataset;

[0055] Figure 11 D-accuracy result diagram of 4 data error detection methods on the HOSP-100K dataset;

[0056] Figure 12 D-accuracy result diagram of 4 data error detection methods on the Adult dataset;

[0057] Figure 13 D-accuracy result diagram of 4 data error detection methods on the Beers dataset;

[0058] Figure 14 D-accuracy result diagram of 4 data error detection methods on the Food dataset;

[0059] Figure 15It is a graph of D-accuracy results of 4 data error detection methods on the Soccer dataset;

[0060] Figure 16 It is a graph of D-accuracy results of 4 data error detection methods on the Tax dataset;

[0061] Figure 17 It is a graph of N-accuracy results of 4 data error detection methods on the Restaurants dataset;

[0062] Figure 18 It is a graph of N-accuracy results of 4 data error detection methods on the Rayyan dataset;

[0063] Figure 19 It is a graph of N-accuracy results of 4 data error detection methods on the Beers dataset;

[0064] Figure 20 It is a graph of N-accuracy results of 4 data error detection methods on the Flights dataset;

[0065] Figure 21 It is a graph of the computational complexity-accuracy results of the method of the present invention;

[0066] Figure 22 It is a graph of the experimental results of the error detection attention visualization of the salary attribute value in the Adult data table;

[0067] Figure 23 It is a graph of the error detection attention visualization result of the act_dep_time attribute value in the Flights data table;

[0068] Figure 24 It is a graph of the error detection attention visualization result of the Language attribute value in the Movies data table;

[0069] Figure 25 It is a graph of the error detection attention visualization result of the Filming Locations in the Movies data table;

[0070] Figure 26 It is a graph of the error detection attention visualization result of the Website attribute value in the Restaurants data table;

[0071] Figure 27 It is a graph of the error detection attention visualization result of the author_list attribute value in the Rayyan data table;

[0072] Figure 28 It is a graph of the error detection attention visualization result of the MeasureName in the Hospital data table;

[0073] Figure 29 Visualization result graph of error detection attention for the extra - phones attribute value in the Hospital data table;

[0074] Figure 30 Visualization experimental result graph of error detection attention for the city attribute value in the Tax data table;

[0075] Figure 31 Visualization result graph of error detection attention for the measurename in the HOSP - 100K data table;

[0076] Figure 32 Visualization result graph of error detection attention for the birthplace in the Soccer data table;

[0077] Figure 33 Second visualization result graph of error detection attention for the MeasureName in the Hospital data table;

[0078] Figure 34 Visualization result graph of error detection attention for the City attribute value in the Beers data table;

[0079] Figure 35 Visualization result graph of error detection attention for the Ounces attribute value in the Beers data table. Detailed implementation manners

[0080] To facilitate the understanding of the present invention, the present invention will be described in more detail below with reference to the accompanying drawings and specific embodiments. However, the present invention can be implemented in many different forms and is not limited to the embodiments described in this specification. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosed content of the present invention more thorough and comprehensive.

[0081] For the error detection of relational data tables, the present invention proposes an attribute - wise PAttern - perceptive Transformer framework (PAT). The three modules of this framework are: data pre - processing module, model structure module, and interpretability module.

[0082] The specific process of the data pre - processing module: In the model structure, learnable patterns tokens are proposed. Under each attribute, this pattern token represents the underlying distribution jointly represented by the data unit values. The learnable pattern token is a vector (each attribute has A vector), the learnable pattern vector is accompanied by the original data token as input to facilitate data table error detection.

[0083] The specific process of the model structure module is as follows: In the data preprocessing module, to adapt to the input layout of the error detection model, the present invention designs a Quasi-Tokens Arrangement (QTA) algorithm. This algorithm divides the unit sequence according to the length of the unit sequence and the type of word, and provides compact tokenization parameters to achieve high-efficiency detection. The Quasi-Tokens Arrangement algorithm (QTA) can be abbreviated as QTA tokenization or QTA tokenizer.

[0084] The specific process of the interpretability module is as follows: In the visualization module, the present invention visualizes the attention scores to explain the internal detection mechanism of the PAT model.

[0085] The above-mentioned pattern-aware error detection method is the PAT method. The specific algorithm in the data preprocessing stage is as follows: In the data preprocessing stage, it mainly includes two parts: the Quasi-Tokens Arrangement algorithm QTA and the hyperparameter setting algorithm of the Quasi-Tokens Arrangement algorithm QTA.

[0086] The QTA tokenization algorithm takes unit data and tokenization hyperparameters as input. To process variable-length sequences and meet the restrictions on the token dimension D and the number N, this tokenization algorithm divides the input sequence into three processing stages according to the tokenization degree from fine to coarse: (1) Full Tokenization, (2) Rough Tokenization with numerics separated, and (3) Mass Tokenization without numerics separated. More specifically, all three procedures include a general process, namely quasi-tokenization and subsequent arrangement measures, which include the splitting and merging of tokens. However, the implementation steps of quasi-tokenization and arrangement measures in the three procedures are different, mainly depending on the declared delimiter and the division degree. Full Tokenization is suitable for processing short data sequences, and Rough Tokenization and Mass Tokenization are suitable for processing long sequences and extremely long sequences.

[0087] The QTA algorithm has a general processing step in each stage. The first step is to apply a Quasi-tokenizer to the original sequence to obtain quasi-tokens for further processing Subsequently, for each token in the quasi-tokenization that exceeds the dimension D limit, it needs to be split into multiple smaller tokens. For those quasi-tokenizations that still do not meet the requirements, adjacent tokens will be merged according to certain rules during the merging stage: when merging, word texts and punctuation marks have priority (higher merging level), while numbers are considered last (lower merging level); tokens with the same merging level are merged first. As long as the tokens obtained after splitting meet the requirements of token dimension D and quantity N, the algorithm ends.

[0088] Full tokenization stage. The quasi-tokenizer uses spaces, numbers, and punctuation marks as delimiters to split the unit content to obtain quasi-tokens . For tokens that exceed the dimension limit among them, they need to be further split first. If the finally obtained token T meets the limit of token quantity N, it is accepted as type 1; otherwise, if the number of punctuation tokens is greater than or equal to N, T is merged within the specified range, and the merged result is accepted as type 2. For scenarios that cannot meet the requirements (such as the unit data length is long), it will backtrack and execute the following two more coarse-grained tokenization methods (rough tokenization and block tokenization), as Figure 5 shown.

[0089] Rough tokenization stage. In the rough tokenization mode, the quasi-tokenizer uses numbers as delimiters (ignoring punctuation marks and spaces) to obtain rough quasi-tokens . In Figure 5 the type 3 branch, it is necessary to use the space tokenizer to split again to obtain , and after independently processing the numbers, perform splitting and merging to obtain the final quasi-tokens . If entering the type 4 branch, the tokens in that exceed dimension D will be split to obtain token T. If the corresponding acceptance conditions are met, these tokens are returned as .

[0090] Mass tokenization stage. For extremely long sequences, the third tokenization method no longer considers the priority of any token type. At this time, the quasi-tokenizer only uses spaces to split the unit content to obtain quasi-tokens . Compared with rough tokenization, mass tokenization uses fewer delimiters in the quasi-tokenizer. Subsequently, in Figure 5In the type 5 branch, the operation process is the same as that of the type 3 branch. Finally, in the type 6 branch, for the tokens T exceeding the quantity N, truncation is performed, and the remaining part is used as Accepted.

[0091] The last step is to obtain the Data Token Embeddings. After the above tokenization is completed, the QTA tokenizer will execute the data token embedding method designed specifically for the error detection task. First, the QTA algorithm constructs a symbol index dictionary by collecting all the characters that appear in the token vocabulary; subsequently, using the Unicode character index, is mapped to the data token embedding .

[0092] For the above-mentioned pattern-aware error detection method, the PAT method, the hyperparameter setting algorithm in the data preprocessing stage of the data preprocessing module is specifically as follows: PAT provides automatically configured hyperparameters to enable the QTA tokenization algorithm to adapt to different data tables. To achieve an orderly tokenization process, the present invention designs a tokenization hyperparameter setting algorithm and its sub-process Critical Point Find (CPF). Among them, the goal of the tokenization hyperparameter setting algorithm is to determine the token dimension D and the token quantity N. Moreover, the settings of D and N provide default and compact modes, that is, different hyperparameter configurations are provided for QTA tokenization to trade off between the accuracy and efficiency of error detection.

[0093] The first step is Critical Point Find (CPF), whose input is the array , coefficient , and . The sub-process of the tokenization hyperparameter setting algorithm is Critical Point Find (CPF), which generates a frequency histogram with the array and the number of bins to obtain the data point list and its corresponding frequency value . Subsequently, CPF will average to obtain the average frequency , and use it as the reference frequency threshold. Before selecting the critical point from the array , it is necessary to first use the percentile function with the proportional parameters and to calculate the 's and percentiles respectively, that is, and . Then use them as the array The two endpoints are used to narrow the region of interest (ROI) and find critical points within that range. For each data point located within the ROI, the CPF obtains its frequency value from and compares it with the reference threshold . If the frequency value is lower than the threshold, it is classified as a low-frequency bin; otherwise, it is classified as a high-frequency bin. In this way, all the frequency bins within the ROI form a type list that contains high and low types. The key step in the CPF process is to find and return a pivot point to divide the type list into two segments.

[0094] The second step is the token dimension and quantity setting algorithm. The input parameter is the original data table . Initialize the coefficients and their compact versions: , , , , , , . To meet the requirements of PAT in terms of accuracy and efficiency, the hyperparameters of the QTA tokenizer are provided in both default mode and compact mode. First, the tokenization hyperparameter setting algorithm. By applying the full quasi-tokenizer to the dirty tables, the vocabulary set , the list of data token sizes for the cells and the list of cell lengths can be obtained. Subsequently, the list of token lengths is calculated based on the vocabulary set . In the default mode, the token dimension D and the number of tokens N are calculated using the CriticalPointFind process (receiving the list of token lengths and the list of data token sizes S for the cells as parameters respectively). In the compact mode, the token dimension is calculated in the CriticalPointFind process, and its input parameters include the list of token lengths and the compact coefficient . Generally, the coefficient in the compact mode is higher than the coefficient β in the default mode (the specific value depends on different data tables), because a higher coefficient means obtaining a higher-frequency point, which is also a point with a shorter dimension. Since the PAT framework needs to reduce the computational amount in the compact mode, the number of tokens in the compact mode By dividing the length of the sequence in the dataset Divide by To obtain the appropriate number of word units. is a list of unit lengths of Percentile results, and The parameter is used to remove very few very long unit sequences. Finally, CPF will return four hyperparameters: D, N, and .

[0095] The above-mentioned pattern-aware error detection method PAT method, the model structure module said model module design is specifically as follows: the present invention constructs a Transformer-based method for error detection. The preprocessing stage of the method of the present invention converts the input unit of variable length into Sequential Tokens ,in is the number of tokens, each token All are of one dimension In addition, the learnable CLS embedding word ( ) is added to the front of the data and pattern tokens, The state corresponding to the output of the PAT model serves as the categorical feature representation of the data table cell input .

[0096] In order to explore the potential of combining Transformer with relational databases, since each column in the data table has a significantly different appearance and characteristics, the present invention embeds the learnable pattern (pattern token) As the corresponding attribute characteristics. Provide extended position information for the interleaved words of data and patterns. In the input structure of the PAT model, the data words of the unit sequence and the learnable pattern words of the corresponding attributes of the unit are arranged alternately as interleaved words.

[0097] ;

[0098] in, The interlaced words of data and patterns reflect the contrasting and contrasting relationship between data units and their attribute characteristics.

[0099] In the Attribute Pattern Aware Transformer (PAT) model, no trainable linear projection layer (i.e., Embedding layer) is added before the Transformer encoder. The PAT network with the Embedding layer removed can improve the accuracy of the error detection task. As shown in Figure 3As introduced, the PAT model includes = a 6-layer stacked encoder, and at the output end of the last-layer encoder an MLP head is added for multi-layer perception classification. According to the following overall calculation structure formula of the model, the encoder of the PAT model consists of multi-head self-attention (MSA) and a feed-forward layer (FFN or MLP). Among them, the output feature embedding of the th layer encoder is . At the same time, layer normalization (LayerNorm, LN) and residual connections are also used in the PAT model.

[0100] ;

[0101] ;

[0102] .

[0103] The self-attention mechanism module of the PAT model is described below. The attention module performs parallel computing. The self-attention function takes the interleaved token embeddings as input. In the first step, the input tokens are linearly projected into query, key, and value matrices, denoted as Q, K, and V.

[0104] ; among them, is the internal weight matrix of the multi-head attention structure, . The dimension of the key is set to , and the dimensions of the query and value ( , ) are the same as it.

[0105] .

[0106] Furthermore, to analyze the internal structure of the attention module, this method decomposes the steps therein. The present invention calculates the attention score matrix to obtain , and this attention score matrix contains two types of sub-elements in the data tokens and pattern tokens. and can be described as and . In the element vectors in , the example represents the query value corresponding to the CLS token. The same is true for the element vectors in K and V.

[0107] .

[0108] Attention score matrix Integrates the features in the original data unit to be detected and the corresponding attribute patterns. The latter helps to separate unique features from the tokens to be detected and retains or enhances the ability to identify incorrect features within the attribute domain.

[0109] Finally, the present invention calculates the matrix output of self-attention. Among them, the value matrix can be described as .

[0110] .

[0111] The above-mentioned pattern-aware error detection method, the PAT method, the specific visualization structure of the interpretability module is:

[0112] This visualization method aims to meet the potential need to examine and analyze the internal mechanism of the pattern-aware Transformer (PAT) to improve model transparency and reveal the reasons and ways for deep learning models to detect data errors.

[0113] In the pattern-aware Transformer (PAT), the present invention extracts attention scores from the multi-head self-attention (MSA) network as attention maps during the data detection inference process. To analyze the relationship between input tokens and predicted classifications, the present invention selects the last layer of the 6-layer encoder network for processing. In the encoder, since the multi-head self-attention network has h = 8 parallel attention heads, for each data sequence to be detected, a total of 8 attention score matrices are extracted. Specifically, each row vector of the attention score matrix corresponds to an input token, including the CLS token, data tokens, and pattern tokens; each column vector of the matrix represents an output token. Since the PAT model performs data error detection, only the CLS embedding vector in the first column is extracted for binary classification of error detection. Furthermore, in terms of model interpretability, the present invention extracts the first column vector from the attention score matrix of the PAT model as to visualize the relationship between input tokens and classification predictions (CLS outputs). In addition, the present invention removes the subsequent zero-padding part corresponding to the generated tokens in the QTA algorithm to highlight the true tokenized text.

[0114] When visualizing using the extracted attention vectors, the attention lines are too weak for most data sequences to represent the internal relationships of the PAT. To enhance the visualization effect, the present invention multiplies the attention scores by a coefficient η to highlight the visualization lines.

[0115] As Figure 1 shown, the execution process of the error detection method is divided into four stages: data reading, preprocessing, data input, and model execution. Among them, the execution process of the tokenizer in preprocessing is as Figure 5as shown; the model structure is as Figure 3 shown; the visualization graph generated during the model execution process is as Figures 22 to 35 shown. Additionally, the experimental results are shown in Tables 2 and 3, as well as Figures 7 to 21 shown.

[0116] (1) Preprocessing: Use the QTA tokenizer to partition the data sequence to obtain data tokens, and perform corresponding interleaved arrangements with the pattern tokens to obtain the input tokens of the model. Among them, the QTA tokenizer needs to set hyperparameters token dimension and the number of tokens .

[0117] Figure 5 Use a tree diagram to introduce the novel data table tokenizer designed in the present invention. The tokenizer uses the Quasi-Tokens Arrangement (QTA) algorithm, aiming to organize the basic tokenizer sequence segments through a quasi-tokenizer, and these segments ultimately form a set of tokens (the number is ) of the input sequence, so that the words in the sequence are arranged in a combination manner that is easy to detect error words. The present invention will have a data sequence of variable length After being processed by the tokenizer, it is decomposed into a set of tokens .

[0118] For the error detection task, when embedding the token sequence, the present invention makes the character and sentence format features of the original sequence retained in the embedding vector by creating a character index mapping. Specifically, the present invention uses the Unicode character mapping dictionary as the symbol index dictionary to create for the token fixed -dimensional embedded tokens .

[0119] Each original unit is actually a text sequence, which can be regarded as a combination of numbers, characters, and punctuation marks. The QTA algorithm tokenizes the entire text of the unit data based on the following principle: To process variable-length sequences, the input of the tokenizer algorithm has two hyperparameter configurations: default or compact. The execution process of the algorithm divides the input sequence into three steps according to the degree of fine or rough tokenization: 1) Full tokenization, 2) Rough tokenization with digit separation, 3) Block tokenization without digit separation. The program will end when the segmented tokens meet the range requirements of the hyperparameters and .

[0120] As a data preprocessing algorithm, the QTA tokenizer performs six different levels of tokenization on variable-length data sequences, and its evaluation factors consider two factors: the type priority of quasi-tokens and the actual length of the data sequence. After the original sequence is processed by the quasi-tokenizer, quasi-tokens are obtained. The categories of quasi-tokens are divided into three types: numerical, text, and punctuation. Among them, the independent priority of numerical tokens is the highest, the independent priority of punctuation tokens is the second, and the priority of text tokens is the lowest. In the QTA algorithm, tokens of the same type are preferentially merged.

[0121] (2)Model Structure

[0122] Such as Figure 3 , the model structure is specifically divided into multi-head self-attention, feed-forward layer, and its related operation structures. In the PAT model, the output corresponding to the CLS embedding is selected as the classification feature, and a linear layer is used to map it to binary classification, that is, two categories: error and correct.

[0123] The input form of the PAT model arranges two groups of tokens, data and pattern, in an interleaved order, and there are corresponding orders and categories in the network structure and output tokens. During the training and inference processes, the data and its corresponding attribute patterns serve as two network structures to jointly form the overall model. In the training stage, data with different attributes are randomly used as samples for training, and the corresponding pattern tokens of different attributes will be switched accordingly. Therefore, the pattern embedding tokens will serve as a bridge for the interaction of data with different attributes in the PAT model, realizing cross-attribute detection of the data error detection model. The PAT model can learn the features independent between attribute domains and shared within attribute domains of corresponding attributes, and store them in the learnable pattern module and MSA network weights to assist in detection. Figure 2 List various error types existing in the database and classify them according to the levels where the errors exist.

[0124] (3)Visualization Analysis

[0125] During the training and inference processes of the PAT model, in the front part of the multi-layer network, the parameters of the data and pattern structures learn multiple feature representations of data units. As the multi-layer network progresses backward, the two structures of data and pattern focus on misclassifications and play different roles in error detection. Specifically, during the training process of the model, data with the same attributes in the data table jointly learn N pattern embedding tokens. From the perspective of the same attribute domain, these pattern embeddings can learn the data and error features unique to the attribute, and these features are distinguishable. From the perspective between attribute domains, the pattern embedding features of different attributes are independent of each other, representing the feature differences of data and errors of different attributes. Therefore, the pattern embedding tokens are shared within the attribute domain and independent between attribute domains. On the other hand, the structural parameters of the data network of the PAT model learn the common data and error features of all the data in the data table, that is, the indistinguishable features of all attribute values. With the help of the two types of inputs and the internal network structure, the PAT model extracts the commonalities and characteristics of the structural features and content features of the data table, achieving high accuracy and high efficiency in database error detection.

[0126] (4)Method Performance Evaluation

[0127] To verify the performance advantages of the framework proposed in the present invention, the present invention conducted multiple groups of experiments on synthetic datasets and real datasets.

[0128] Experimental Environment: The framework of the present invention is implemented using the PyTorch deep learning framework. The experiment uses the Ubuntu 20.04 Linux system. The hardware is an Intel Xeon Gold 6226R CPU, 256GB of memory, and 4 NVIDIA GeForce RTX3090 GPUs.

[0129] Dataset: The information of the datasets used in the experiment is shown in Table 1. Among the 11 datasets, the real datasets are Adult, Beers, Flights, Food, Hospital, Movies, Rayyan, Restaurants, Soccer, and the synthetic datasets are HOSP-100K (Hospital 100K) and Tax (Employee Tax).

[0130] The experiment mainly conducts performance evaluation from the following aspects: In terms of accuracy, three metrics are adopted in this experiment. Precision (P) is defined as the proportion of true positives (TP) among all samples predicted as positive; Recall (R) is defined as the proportion of true positives (TP) among all misclassified samples (samples that are actually positive); the F1 score is a metric calculated by comprehensively considering P and R. For the efficiency metric of the model, this experiment uses the number of floating-point operations (FLOPs) as the measure of computational complexity.

[0131] Table 1 Dataset Information Table

[0132]

[0133] Table 2 Comparison Table of Accuracy Metrics of PAT, PATC Methods and 5 Advanced Methods on Real and Synthetic Datasets (The bold ones are the optimal measurement metrics of the detection methods among them)

[0134]

[0135] Table 3 Comparison Table of the Number of Parameters and Computational Complexity of 7 Methods on Real and Synthetic Datasets

[0136]

[0137] In this experiment, the PAT (PAttern-perceptive Transformer) and PATC (PAttern-perceptive Transformer Compact) methods are compared with the current state-of-the-art (SOTA) 5 error detection methods in terms of accuracy, including the P, R, and F1 metrics. The evaluation results are shown in Table 2, indicating that compared with multi-detectors (Raha, ED2, HOLODETECT) and single deep learning model methods (ETSB-RNN, ROTOM), PAT has significant advantages in terms of efficiency and detection accuracy. The PAT model belongs to the error detection method of a single model and surpasses the multi-detector method in terms of accuracy.

[0138] Figures 7 to 16 is to compare the accuracy of error detection using four Data Transformer methods (PAT, PATC, DT, DTC), where is the independent variable, is kept fixed and determined by the hyperparameter setting algorithm. The DT-based methods (DT, DTC) are Data Transformer methods that do not use learning-based attribute patterns in the model. Figures 17 to 20It is a comparison of the accuracy of error detection using four Data Transformer methods. Among them, is the independent variable, which is kept fixed and determined by the hyperparameter setting algorithm. In the Figures 7 to 20 accuracy comparison graph, each point is the corresponding F1 score value when the independent variable (D or N) is set (the error bar represents the standard deviation obtained by repeating each experiment 10 times).

[0139] In the Figures 7 to 20 accuracy comparison graph, the PAT-based methods (PAT and PATC) are significantly higher than the DT-based methods (DT and DTC), showing an obvious gap. In Figure 7 , Figure 8 , Figure 10 , Figure 11 , Figure 12 Figure 12 , the F1 score of the PAT method on the Restaurants, Rayyan, Hospital, HOSP-100K, and Adult datasets gradually increases with the increase of the token dimension D. This stable growth trend stems from the fact that this dataset contains a large number of long sequences (resulting in large chunks of words obtained by tokenization). When the token dimension D is larger, the model can accommodate larger chunks of words, and the F1 score of the PAT-based methods also increases accordingly. On the contrary, Figure 9 , Figure 14 Figure 14 , the F1 score of the PATC method on the Movies and Food datasets decreases instead as D increases because the two datasets lack medium-length sequences.

[0140] When the compact tokens can effectively accommodate long unit data sequences, the data table conversion method in the compact mode is better than the default QTA tokenization scheme. On the contrary, if the actual segmented sequence fragments are short, the token vectors generated by the default QTA algorithm will contain more zero-padding components. This makes it difficult for the default QTA in the default mode to effectively reorganize most quasi-tokens (fragments segmented by the quasi-tokenizer), and it will also cause a large amount of redundant occupation of vector components. Therefore, as Figure 8 , Figure 15 , Figure 16 , Figure 20 Figure 20 shows, the F1 scores of the compact modes of DTC and PATC on datasets such as Rayyan, Soccer, Tax, and Flights are higher than their default modes. It can be concluded that appropriate tokenization parameters (D and N) can adapt the model learning and thus affect the accuracy performance.

[0141] In terms of efficiency, this experiment uses two metrics, the number of parameters and FLOPs, to measure the computational efficiency of the model. Table 3 shows the computational efficiency results of seven methods on datasets containing short sequences, medium sequences, and long sequences, including three comparison methods and four Data Transformer methods. Figure 21 It is a trend chart of the computational volume (FLOPs) and accuracy (F1 Score) of five methods on the same dataset. It can be seen that DT, DTC, PAT, and PATC all show a trend of increasing accuracy with the increase in computational volume. In terms of the number of parameters and computational volume, DT, DTC, and HOLODETECT are at an approximate level with PAT and PATC; the PAT and PATC methods maintain high accuracy under different model computational volumes. In addition, at the same computational volume, the accuracy of PAT and PATC is higher than that of DT and DTC.

[0142] In terms of model interpretability, Figures 22 to 35 represents the model visualization process, and the attention map can effectively construct the relationship between the input token and the detection result. Among them Figure 29 is the DTC model, Figures 24 to 27 is the PATC model, and the other figures are PAT models. With the synergistic effect of the input data (CLS token, data token, and their corresponding pattern tokens), the multi-head attention network captures diverse features. In the MSA network, the 8-head attention mechanism undertakes different functions in the error detection task: some attention heads (such as Figure 23 the 1st and 3rd heads, Figure 25 the 3rd head, Figure 28 the 6th head, starting with 1) focus on the extraction of data token features, and some others (such as Figure 22 the 2nd head, Figure 30 the 1st head, Figure 35 the 2nd head) focus on the analysis of pattern tokens. This indicates that the PAT model can stably learn the features of data and patterns, and the arrangement of these features in the 8 parallel head attentions shows randomness.

[0143] The multi-head attention is mainly responsible for the error detection function of the unit sequence. There are some attention heads (even single ones) that focus on learning error-related features. All the multi-head attention features are calculated in parallel to make the final prediction together. The PAT method can accurately and efficiently apply the classification probe to the error tokens while weakening the attention to the non-error tokens. There are two typical phenomena in the attention feature distribution of error tokens: the error tokens may be in the data tokens ( Figure 25 the 3rd head, Figure 35 the 4th and 5th heads) or their PAT pattern tokens ( Figure 25 the 1st, 2nd, and 4th heads, Figure 32At the 3rd, 5th, and 8th heads), it shows dark color (high attention score), while the correct token shows light color (low attention score); it may also show light color or blank (low attention score) at the data tokens of the incorrect data ( Figure 28 the 6th head, Figure 34 the 2nd and 4th heads) or their PAT pattern tokens ( Figure 34 the 3rd and 6th heads, Figure 35 the 3rd head), while the normal tokens show dark color (high attention score). Generally, a high attention score means that the output pays more attention to the corresponding token, such as the former; for the latter, since the normal and incorrect tokens are two symmetric categories, there is a phenomenon of category feature swapping during multiple training processes, so there is an abnormal phenomenon of low attention at the incorrect tokens. At the same time, this indicates that the network parameters corresponding to the data tokens of some incorrect sequences can only learn local features relative to the multi-head attention network, and finally the MLP head module will integrate all attention parameters to complete the error detection task prediction.

[0144] It should be noted that, Figures 22 to 35 the corresponding MSA structure is located in the last layer encoder of the model. By observing the attention maps of the 6-layer encoder network, as the network depth increases, the attention mechanism can more clearly establish the association between incorrect words (input tokens containing incorrect information) and output classification features. This benefits from the reasonable arrangement of input tokens by the QTA algorithm, and incorrect words can be assigned to tokens with appropriate dimensions, thus accelerating the error detection learning process of the PAT model.

[0145] The technical features of the above-described embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combinations of these technical features do not conflict, they should all be considered as the scope described in this specification.

Claims

1. A relational database error detection method with a learning attribute model, characterized in that: Here are the steps: S1, data preprocessing, the input data sequence is represented by input word units constructed by interleaving data word units and pattern word units; S2, error recognition, learning the independent and shared features between attribute domains of the corresponding attributes based on the input word meta-words, and storing the learning content in the learnable pattern module and the multi-head self-attention network weights for detection; S3, visual explanation, extracts the attention scores in the multi-head self-attention network, builds the relationship between input word units and output predictions, and generates a visual attention map; Adopt the attribute pattern-aware Transformer framework, which includes: A data preprocessing module, configured with a program for executing a quasi-word arrangement algorithm and a hyperparameter setting algorithm; A model structure module is configured with a program that takes data words and pattern words as input, learns the independent features between attribute domains and the shared features within the attribute domains of the corresponding attributes, stores and learns them, and can output the detection results; the model structure module is based on the Transformer encoder architecture, and is composed of six identical encoder stacking layers, each encoder layer is provided with a multi-head self-attention and a feedforward layer, the multi-head self-attention and the feedforward layer are connected through a sum layer normalization operation, and the output end of the last encoder layer is provided with an MLP head for multi-layer perception classification; The interpretability module is configured with a program that extracts the parameters of a multi-head self-attention network to build a relationship between input tokens and output predictions and generate a visual attention map.

2. The relational database error detection method with a learning attribute model according to claim 1, characterized in that: Data preprocessing includes: dividing the input data sequence into data tokens according to the length of the sequence and the type of word through a quasi-term arrangement algorithm; representing the attribute pattern of the data unit value in the input data sequence with pattern tokens according to a learnable pattern; and arranging the data tokens and the pattern tokens in a corresponding staggered manner to obtain structured input tokens.

3. The relational database error detection method with a learning attribute model according to claim 2, characterized in that: The quasi-word arrangement algorithm relies on the word segmentation hyperparameter setting algorithm that automatically configures hyperparameters. The word segmentation hyperparameters include: word dimension D and word number N.

4. The relational database error detection method with a learning attribute model according to claim 3, characterized in that: The steps of word segmentation hyperparameter setting algorithm are as follows: S1. Obtain a histogram of word length and number from the error data table, find a critical point using a critical point search function, and divide the elements of the input array into a high-frequency point group and a low-frequency point group; S2. The word dimension is calculated by the critical point search function, and the input is the word length array and the preset threshold coefficient. In the default mode, the word length array is used as the input of the critical point search function to calculate the number of words, while in the compact mode, the number of words is the length of the longest sequence in the data set divided by the compact word dimension. S3, the critical point search function determines the dividing point between the high-frequency point group and the low-frequency point group by multiplying the average value of the frequency histogram by the preset threshold coefficient; in the default mode, the preset threshold coefficient is β, and in the compact mode is .

5. The relational database error detection method with a learning attribute model according to claim 4, characterized in that: The quasi-word arrangement algorithm includes three stages: full word segmentation, rough word segmentation, and block word segmentation; In each stage, the quasi-word segmenter is first applied to the input data sequence to divide it and obtain the quasi-word units, then the word segmentation parameters are set for the obtained quasi-word units to arrange them, and then the out-of-dimensional words are segmented and the short words are merged with sliding windows, and then they are mapped to numerical representations according to the Unicode character dictionary to finally obtain data words.

6. The relational database error detection method with a learning attribute model according to claim 5, characterized in that: The quasi-word arrangement algorithm uses variable-length unit sequences and word segmentation hyperparameters as input; first apply the quasi-word segmenter to the original input data sequence to obtain quasi-word elements for further processing ; Then, for the quasi-participle Each word that exceeds the dimension D limit is split into multiple word units; for quasi-segmented words that still do not meet the requirements of word unit dimension D and word unit number N, adjacent word units are merged according to certain rules in the merging stage: when merging, words and punctuation marks are given priority, and numbers are considered last; word units of the same merging level are merged first; as long as the word units obtained after segmentation meet the requirements of word unit dimension D and word unit number N, the algorithm ends.

7. The relational database error detection method with a learning attribute model according to claim 6, characterized in that: The data preprocessing process is as follows: To simulate the structural characteristics of unit values ​​and their attribute domains, the data word units are and pattern tokens Interleaved to form interleaved tokens, where a data token is a sequence of input units The preprocessing stage is obtained through word segmentation, and the pattern word is obtained through PAT model training. N is the number of word units. and each pattern token is an embedding vector of dimension D; then, a learnable CLS embedding token is appended to the head of the interleaved token of data and pattern ; The input word is obtained by combining the CLS word, the data word and the pattern word: ; ; It is the location information provided for the interleaved tokens of data and patterns; Applying a variety of structures, including multi-head self-attention, feed-forward layers, layer normalization, and residual connections, the overall computational structure of the attribute pattern-aware Transformer network is as follows; ； ； ； in, Indicates the number of stacked layers of the encoder; Indicates The output features of the self-attention module of the first layer; The output feature embedding of the layer encoder is recorded as ;CLS word The state corresponding to the output of the model is represented as the classification feature of the sequence of input cells in the data table ; The query, key, and value matrices are the main structures of the multi-head self-attention network, denoted as , , , which are computed in parallel: ; in, is the internal weight matrix of the multi-head self-attention network; ; The dimensions of the key are set to , the dimension of the query The dimension of the sum value Same dimension as the key; The matrix output representation of the multi-head self-attention network: ; Furthermore, in order to analyze the internal structure of the attention module, the method decomposes the steps therein; the present invention calculates the attention score matrix to obtain , this attention score matrix contains two types of sub-elements in data tokens and pattern tokens; and Can be described as and ; ; ； Attention score matrix It combines the features of the original data unit to be detected and the corresponding attribute pattern; Value Matrix Can be described as ; Matrix output of the multi-head self-attention network: .

8. The relational database error detection method with a learning attribute model according to claim 7, characterized in that: Visual explanation, from the attention score matrix Extract the first column vector from the to visualize the relationship between the input word and the classification prediction.

Images