Relational database error detection method with learning type attribute mode

Abstract

The invention belongs to the technical field of database data quality management, and particularly relates to a relational database error detection method with a learning type attribute mode. The method comprises the following steps of: preprocessing data, and representing an input data sequence by using an input lexical element which is constructed by staggering data lexical elements and mode lexical elements; error recognition: learning characteristics of independence between attribute domains and internal sharing of the attribute domains of corresponding attributes based on input lexical elements, and storing learning contents in a learnable mode module and a multi-head self-attention network weight for detection; and visual explanation: extracting attention scores in the multi-head self-attention network so as to construct a relationship between input lexical elements and output prediction, and generating a visual attention graph. The main improvement of the method is that a Transform framework perceived by an attribute mode learns data values of a relation table and characteristics of an attribute domain structure of the relation table and is used for detecting data errors of a given data set, the accuracy and efficiency of a detection task are improved, and the error detection process is visualized.

Description

Technical Field

[0001] The invention belongs to the technical field of database data quality management, and in particular relates to a relational database error detection method with a learning attribute model. Background Art

[0002] Error detection of data tables aims to find various format violations and rule violation errors, which are constrained and restricted by relational databases. Error detection is a basic step to improve data quality during the production, storage, and use of data. In addition, accurate and efficient error detection can significantly reduce the burden on data analysts in subsequent data repair and data-driven tasks. Therefore, improving the accuracy and efficiency of error detection methods is crucial to actual data quality management.

[0003] Existing error detection methods are difficult to fully utilize the structural advantages of relational tables. Error detection methods based on machine learning need to deal with multiple attribute domain problems of relational tables to learn the data distribution of multiple attributes. There are three main existing methods: one is the unit-level data error detection method using multiple classifiers (Heidari A, et al. Holodetect: Few-shot learning for error detection [C] / / Proc. ACM SIGMOD Int. Conf. Manag. Data. 2019: 829-846.), which improves accuracy by increasing the cost of model training and deployment. The second is the unit-level data error detection method using a single model (Miao Z, et al. Rotom: A meta-learned data augmentation framework for entity matching, data cleaning, text classification, and beyond [C] / / Proc. ACM SIGMOD Int. Conf. Manag. Data. 2021: 1303-1316.), which detects arbitrary data units by obtaining the multi-column feature distribution of the data table. Since each attribute has a unique data profile and contains different error types, the deviation between attributes inevitably affects the accuracy and generalization of the detection model. The third is the tuple-level data error detection method using a large-scale model (LiuZ, et al. Picket: guarding against corrupted data in tabular data during learning and inference[J]. The VLDB Journal, 2022, 31(5): 927-955.). This method requires increasing model parameters and size, so the training and inference costs are relatively high in actual use.

[0004] The main challenge in the data preprocessing stage is how to divide the data sequence to meet the requirements of the error detection model for the input structure. The current data preprocessing methods are mainly divided into two types: tokenizer and extraction of predefined features. In the preprocessing stage of data error detection, the tokenizer (Rico S, et al. Neural Machine Translation of Rare Words with Subword Units [C]. ACL, 2016.) tokenizes the unit sequence into multiple input tokens, and its division follows the convention of natural language processing. However, the tokenization criteria of these tokenizers do not consider the factors of error format, error type, syntax, grammar, and inconsistent length of unit sequences. Therefore, the vocabulary generated by these methods cannot be used as a suitable representation for data quality detection management tasks, and does not meet the restrictions on the length and number of tokens. As another preprocessing method, the method of extracting predefined features (Neutatz F, et al. Ed2: A case for active learning in error detection[C] / / Proc. ACM CIKM Int. Conf. Information & Knowledge Management. 2019: 2249-2252.) is not conducive to the model's understanding of the input sequence and improving its decision-making ability. Therefore, the inadequacy of the preprocessing method leads to a performance bottleneck in the error detection model.

[0005] In summary, using deep learning to detect erroneous data requires a trade-off between accuracy and efficiency. Achieving high efficiency requires machine learning methods to be simple, complete, and end-to-end to minimize time and space costs; while achieving high accuracy requires comprehensive task completion and high performance evaluation metrics.

[0006] In addition, there are currently few data error detection studies that focus on the internal operating mechanisms of the model. Some work in the fields of natural language processing (NLP) and computer vision (CV) uses visualization and other methods to conduct interpretability research on unstructured data (paragraph text and image fragments). However, DL models used for relational table data detection may have difficulty providing sufficient internal attention parameters to construct the relationship between data sequences and classifications. As the infrastructure of data quality management, it is necessary for error detection methods to explore feasible visualization methods and explain the error detection mechanism. Summary of the invention

[0007] In order to solve the problems existing in the prior art, the present invention provides a relational database error detection method with a learning attribute model, which makes improvements in three aspects: data preprocessing, the structure of the detection model and the interpretability analysis of the model, so as to detect various types of errors in a data table with multiple attributes and perform visual analysis.

[0008] The technical solution adopted by the present invention to solve the technical problem is as follows: a relational database error detection method with a learning attribute model, the steps are as follows: S1, data preprocessing, the input data sequence is represented by input word units constructed by interleaving data word units and pattern word units; S2, error recognition, learning the independent and shared features between attribute domains of the corresponding attributes based on the input word meta-words, and storing the learning content in the learnable pattern module and the multi-head self-attention network weights for detection; S3, visual explanation, extracts the attention scores in the multi-head self-attention network, builds the relationship between input word units and output predictions, and generates a visual attention map; Adopt the attribute pattern-aware Transformer framework, which includes: A data preprocessing module, configured with a program for executing a quasi-word arrangement algorithm and a hyperparameter setting algorithm; The model structure module is configured with a program that takes the data word unit and the pattern word unit in an interlaced arrangement as input, learns the independent features between the attribute domains and the shared features within the attribute domain of the corresponding attribute, stores and learns them, and can output the detection results; The interpretability module is configured with a program that extracts the parameters of a multi-head self-attention network to build a relationship between input tokens and output predictions and generate a visual attention map.

[0009] Preferably, data preprocessing includes: dividing the input data sequence into data words according to the length of the sequence and the type of word through a quasi-word arrangement algorithm; representing the attribute pattern of the data unit value in the input data sequence with pattern words according to a learnable pattern; and arranging the data words and pattern words in a corresponding staggered manner to obtain structured input words.

[0010] Preferably, the quasi-word-gram arrangement algorithm relies on a word segmentation hyperparameter setting algorithm that automatically configures hyperparameters, and the word segmentation hyperparameters include: word-gram dimension D and word-gram number N.

[0011] Preferably, the steps of word segmentation hyperparameter setting algorithm are as follows: S1. Obtain a histogram of word length and number from the error data table, find a critical point using a critical point search function, and divide the elements of the input array into a high-frequency point group and a low-frequency point group; S2. The word dimension is calculated by the critical point search function, and the input is the word length array and the preset threshold coefficient. In the default mode, the word length array is used as the input of the critical point search function to calculate the number of words, while in the compact mode, the number of words is the length of the longest sequence in the data set divided by the compact word dimension. S3, the critical point search function determines the dividing point between the high-frequency point group and the low-frequency point group by multiplying the average value of the frequency histogram by the preset threshold coefficient; in the default mode, the preset threshold coefficient is β, and in the compact mode is .

[0012] Preferably, the quasi-word-metagram arrangement algorithm includes three stages: full word segmentation, rough word segmentation and block word segmentation; in each stage, the quasi-word-metagram is first applied to the input data sequence for division to obtain the quasi-word-metagram, and then the word segmentation parameters are set for the obtained quasi-word-metagram to arrange them, and then the out-of-dimensional word-metagrams and the sliding window merging of the short word-metagrams are performed, and then they are mapped to numerical representations according to the Unicode character dictionary to finally obtain data word-metagrams.

[0013] Preferably, the quasi-word arrangement algorithm uses a variable-length input unit sequence and word segmentation hyperparameters as input; first apply the quasi-word segmenter to the original input data sequence to obtain quasi-word elements for further processing ; Then, for the quasi-participle Each word that exceeds the dimension D limit is split into multiple word units; for quasi-segmented words that still do not meet the requirements of word unit dimension D and word unit number N, adjacent word units are merged according to certain rules in the merging stage: when merging, words and punctuation marks are given priority, and numbers are considered last; word units of the same merging level are merged first; as long as the word units obtained after segmentation meet the requirements of word unit dimension D and word unit number N, the algorithm ends.

[0014] Preferably, the data preprocessing process is as follows: to imitate the structural characteristics of the unit value and its attribute domain, the data word unit and pattern tokens Interleaved to form interleaved tokens, where a data token is a sequence of input units The preprocessing stage is obtained through word segmentation, and the pattern word is obtained through PAT model training. N is the number of word units. and each pattern token is an embedding vector of dimension D; then, a learnable CLS embedding token is appended to the head of the interleaved token of data and pattern ; The input word is obtained by combining the CLS word, the data word and the pattern word: ; ; It is the location information provided for the interleaved tokens of data and patterns; Applying a variety of structures, including multi-head self-attention, feed-forward layers, layer normalization, and residual connections, the overall computational structure of the attribute pattern-aware Transformer network is as follows: ; ; ; in, Indicates the number of stacked layers of the encoder; Indicates The output features of the self-attention network of the first layer; The output feature embedding of the layer encoder is recorded as ;CLS word The state corresponding to the output of the model is represented as the classification feature of the sequence of input cells in the data table ; The query, key, and value matrices are the main structures of the multi-head self-attention network, denoted as , , , which are computed in parallel: ; in, is the internal weight matrix of the multi-head self-attention network; ; The dimensions of the key are set to , the dimension of the query The dimension of the sum value Same dimension as the key; The matrix output representation of the multi-head self-attention network: ; Furthermore, in order to analyze the internal structure of the attention module, the present method decomposes the steps. The present invention calculates the attention score matrix to obtain , this attention score matrix contains two types of sub-elements in data tokens and pattern tokens. and Can be described as and ; ; Attention score matrix It combines the features of the original data unit to be detected and the corresponding attribute pattern; Value Matrix Can be described as ; Matrix output of the multi-head self-attention network: .

[0015] Preferably, a visual explanation, from the attention score matrix Extract the first column vector from the to visualize the relationship between the input word and the classification prediction.

[0016] Preferably, the model structure module is based on the Transformer encoder architecture and is composed of six identical encoder stacking layers. A multi-head self-attention and a feedforward layer are set in each encoder layer. The multi-head self-attention and the feedforward layer are connected through a sum layer standardization operation. The output end of the last encoder layer is provided with an MLP head for multi-layer perception classification.

[0017] Compared with the prior art, the present invention has the following beneficial effects: 1. This paper proposes an attribute pattern-aware Transformer framework for error detection in data tables. The learnable pattern tokens can capture data features of different attributes during model training, thereby achieving more effective detection.

[0018] 2. In the sequence preprocessing stage, the present invention introduces a quasi-word arrangement algorithm to separate the unit sequence into words according to the appropriate length and word type. The words in QTA are suitable for detection as potential atomic error units. In order to keep the PAT model low cost, the present invention provides a compact QTA and can meet the word segmentation requirements of long sequence data.

[0019] 3. The present invention visualizes the process of error detection by extracting attention scores in the MSA network and using them to present the relationship between input data and classification predictions.

[0020] In summary, the main improvement of the present invention is the attribute pattern-aware Transformer framework, which learns the characteristics of relational table data values ​​and their attribute domain structures to find data errors in a given data set and improve the accuracy and efficiency of error detection tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A flow chart of a relational database error detection method for a learning attribute model; Figure 2 Categorize the types of data errors in relational databases; Figure 3 The structural diagram of the relational database error detection model for the learning attribute pattern; Figure 4 The operation process of the multi-head self-attention structure and the attention score matrix diagram; Figure 5 This is the tree diagram of the Quasi-Term Arrangement Algorithm (QTA) in the data preprocessing stage; Figure 6 An example of running the Quasi-Term Arrangement Algorithm (QTA) in the data preprocessing stage; Figure 7The D-accuracy results of four data error detection methods on the Restaurants dataset; Figure 8 The D-accuracy results of four data error detection methods on the Rayyan dataset are shown in the figure. Fig. 9 The D-accuracy results of four data error detection methods on the Movies dataset; Fig.10 The D-accuracy results of four data error detection methods on the Hospital dataset; Fig.11 The D-accuracy results of four data error detection methods on the HOSP-100K dataset are shown in the figure; Fig.12 The D-accuracy results of four data error detection methods on the Adult dataset; Fig.13 The D-accuracy results of four data error detection methods on the Beers dataset; Fig.14 The D-accuracy results of four data error detection methods on the Food dataset; Fig.15 The D-accuracy results of four data error detection methods on the Soccer dataset; Fig.16 The D-accuracy results of four data error detection methods on the Tax dataset; Fig.17 The N-accuracy results of four data error detection methods on the Restaurants dataset; Fig.18 This is the N-accuracy result diagram of four data error detection methods on the Rayyan dataset; Fig.19 The N-accuracy results of the four data error detection methods on the Beers dataset are shown in Figure 2. Fig. 20 The N-accuracy results of four data error detection methods on the Flights dataset; Fig.21 The figure is a result diagram of the calculation amount-accuracy of the method of the present invention; Fig. 22 This is the result of the error detection attention visualization experiment on the salary attribute value of the Adult data table; Fig.23 This is the error detection attention visualization result diagram of the act_dep_time attribute value of the Flights data table; Fig.24This is the error detection attention visualization result diagram of the Language attribute value of the Movies data table; Fig.25 This is the visualization result of the error detection attention of Filming Locations in the Movies data table; Fig.26 This is the visualization result of error detection attention for the Website attribute value of the Restaurants data table; Fig. 27 This is the error detection attention visualization result diagram of the author_list attribute value of the Rayyan data table; Fig.28 This is a visualization of the error detection attention results for the MeasureName table of the Hospital data table; Fig.29 This is the visualization result of error detection attention for the extra-phones attribute value of the Hospital data table; Fig.30 This is the result of the error detection attention visualization experiment on the city attribute value of the Tax data table; Fig.31 This is the visualization result of the error detection attention of the HOSP-100K data table measurename; Fig.32 Visualization result graph of error detection attention for birthplace of Soccer data table; Fig.33 Figure 2 is the visualization result of error detection attention for MeasureName in the Hospital data table; Fig.34 This is the error detection attention visualization result diagram of the City attribute value of the Beers data table; Fig.35 Visualization of error detection attention results for the Ounces attribute values ​​of the Beers data table. DETAILED DESCRIPTION

[0022] In order to facilitate the understanding of the present invention, the present invention is described in more detail below in conjunction with the accompanying drawings and specific embodiments. However, the present invention can be implemented in many different forms and is not limited to the embodiments described in this specification. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present invention more thorough and comprehensive.

[0023] For error detection of relational data tables, the present invention proposes an attribute-wise PAttern-perceptive Transformer framework (PAT), which consists of three modules: data preprocessing module, model structure module, and interpretability module.

[0024] The specific process of the data preprocessing module is as follows: In the model structure, a learnable pattern (Learnablepatterns) word is proposed. Under each attribute, this pattern word represents the underlying distribution represented by the data unit value. The learnable pattern word is vectors (each attribute has vectors), learnable pattern vectors accompany the original data tokens as input to facilitate data table error detection.

[0025] The specific process of the model structure module is as follows: In the data preprocessing module, in order to adapt to the input layout of the error detection model, the present invention designs a Quasi-Tokens Arrangement (QTA) algorithm, which divides the unit sequence according to the length of the unit sequence and the type of word, and provides compact tokenization parameters to achieve efficient detection. The Quasi-Token Arrangement Algorithm (QTA) can be referred to as QTA tokenization or QTA tokenizer.

[0026] The specific process of the explainability module is as follows: in the visualization module, the present invention visualizes the attention score to explain the internal detection mechanism of the PAT model.

[0027] The above-mentioned pattern-aware error detection method is a PAT method, and the algorithm of the data preprocessing stage is specifically as follows: in the data preprocessing stage, it mainly includes two parts: a quasi-word arrangement algorithm QTA and a hyperparameter setting of the quasi-word arrangement algorithm QTA.

[0028] QTA word segmentation algorithm is based on unit data and tokenization hyperparameters as input. In order to handle sequences of variable length and meet the restrictions on the dimension D and number N of tokens, the tokenization algorithm divides the input sequence into three processing stages according to the degree of tokenization from fine to coarse: (1) full tokenization, (2) rough tokenization with numerics separated, and (3) mass tokenization without numerics separated. More specifically, the three procedures all include a common process, namely quasi tokenization and subsequent arrangement measures, which include the segmentation and merging of tokens. However, the three procedures differ in the implementation steps of quasi tokenization and arrangement measures, which mainly depend on the declared delimiter and the division degree. Full tokenization is suitable for processing short data sequences, while rough tokenization and mass tokenization are suitable for processing long sequences and very long sequences.

[0029] The QTA algorithm has common processing steps at each stage. The first step is to apply a quasi-tokenizer to the original sequence to obtain quasi-tokens for further processing. Then, for the quasi-participle Each word that exceeds the dimension D limit needs to be split into multiple smaller words. For those quasi-segmented words that still do not meet the requirements, adjacent words will be merged according to certain rules during the merging stage: when merging, word text and punctuation marks are given priority (higher merging level), while numbers are considered last (lower merging level); words of the same merging level are merged first. As long as the word obtained after segmentation meets the requirements of word dimension D and number N, the algorithm ends.

[0030] Full tokenization stage. The quasi-tokenizer uses spaces, numbers, and punctuation marks as delimiters to tokenize the unit content. Split to get quasi-words . For the tokens that exceed the dimension limit, they need to be further segmented. If the final token T meets the limit of the number of tokens N, it is accepted as type 1; otherwise, if the number of punctuation tokens is greater than or equal to N, T is merged within the specified range, and the merged result is accepted as type 2. For scenarios that cannot meet the requirements (for example, the unit data length is long), it will backtrack and execute the following two more coarse word segmentation methods (coarse word segmentation and block word segmentation), such as Figure 5 shown.

[0031] Rough tokenization stage. In the rough tokenization mode, the quasi-tokenizer uses numbers as delimiters (ignoring punctuation and spaces) to obtain rough quasi-tokens. .exist Figure 5 In the type 3 branch, the space tokenizer needs to be used to split it again get , and then process the numbers independently and then split and merge them to obtain the final quasi-words If you enter the type 4 branch, you will The word units that exceed dimension D in are segmented to obtain word units T. If the corresponding acceptance conditions are met, these word units are used as return.

[0032] Mass tokenization stage. For very long sequences, the third tokenization method no longer considers the priority of any word type. At this time, the quasi-tokenizer only uses spaces to segment the unit content. Segment to get quasi-words Compared to coarse tokenization, chunk tokenization uses fewer delimiters in the quasi tokenizer. Figure 5 In the type 5 branch, the operation process is the same as that of the type 3 branch. Finally, in the type 6 branch, the word T that exceeds the number N will be truncated and the remaining part will be used as accept.

[0033] The last step is to obtain data token embeddings. After completing the above word segmentation, the QTA tokenizer will perform a data token embedding method designed specifically for error detection tasks. First, the QTA algorithm builds a symbol index dictionary by collecting all the characters that appear in the token vocabulary; then, it uses the Unicode character index to Mapping to data word embedding .

[0034] The above-mentioned pattern-aware error detection method PAT method, the data preprocessing module, the hyperparameter setting algorithm of the data preprocessing stage is specifically as follows: PAT provides automatically configured hyperparameters so that the QTA word segmentation algorithm can adapt to different data tables. In order to achieve an orderly word segmentation process, the present invention designs a word segmentation hyperparameter setting algorithm and its sub-process critical point finding (Critical Point Find, CPF). Among them, the goal of the word segmentation hyperparameter setting algorithm is to determine the word unit dimension D and the number of word units N. Moreover, the settings of D and N provide default (Default) and compact (Compact) modes, that is, different hyperparameter configurations are provided for QTA word segmentation, which is used to balance the accuracy and efficiency of error detection.

[0035] The first step is critical point finding (CPF), whose input is the array ,coefficient , and The sub-process of the word segmentation hyperparameter setting algorithm is the critical point find (CPF), which is an array And the number of boxes Generate a frequency histogram to get a list of data points and its corresponding frequency value Subsequently, CPF will Find the average frequency , and use it as the reference frequency threshold. Before selecting the critical point in and The percentile function of the array is calculated separately of and Percentile, that is and . Then put them as an array The two endpoints of are used to narrow the range of interest (ROI) and find the critical point within the range. For each data point within the ROI, CPF calculates Get its frequency value and compare it with the reference threshold If the frequency value is lower than the threshold, it is classified as a low-frequency column; otherwise, it is classified as a high-frequency column. In this way, all frequency columns within the ROI range will form a type list , which contains high and low types. The key step in the CPF process is to find and return a pivot point to convert the type list Split into two segments.

[0036] The second step is the word dimension and quantity Set the algorithm, the input parameter is the original data table . Initialize the coefficients and their compact versions: , , , , , , To meet the accuracy and efficiency requirements of PAT, the hyperparameters of the QTA tokenizer are provided in default mode and compact mode. First, the tokenization hyperparameters set the algorithm. The vocabulary is obtained by applying the full quasi-tokenizer to the dirty tables. , data word size list of units and a list of unit lengths Then, according to the vocabulary Calculate the word length list In default mode, the CriticalPointFind process is used (receives a list of word lengths and the unit data token size list S as parameters) to calculate the token dimension D and the number of tokens N. In compact mode, the token dimension It is calculated in the CriticalPointFind process, whose input parameters include the word length list and compactness factor . Typically, the coefficients in compact mode It will be higher than the coefficient β in the default mode (its specific value depends on different data tables), because a higher coefficient means that a higher frequency point can be obtained, and this point is also a point with a shorter dimension. Since the PAT framework needs to reduce the amount of calculation in compact mode, the number of tokens in compact mode By dividing the length of the sequence in the dataset Divide by To obtain the appropriate number of word units. is a list of unit lengths of Percentile results, and The parameter is used to remove very few very long unit sequences. Finally, CPF will return four hyperparameters: D, N, and .

[0037] The above-mentioned pattern-aware error detection method PAT method, the model structure module said model module design is specifically as follows: the present invention constructs a Transformer-based method for error detection. The preprocessing stage of the method of the present invention converts the input unit of variable length into Sequential Tokens ,in is the number of tokens, each token All are of one dimension In addition, the learnable CLS embedding word ( ) is added to the front of the data and pattern tokens, The state corresponding to the output of the PAT model serves as the categorical feature representation of the data table cell input .

[0038] In order to explore the potential of combining Transformer with relational databases, since each column in the data table has a significantly different appearance and characteristics, the present invention embeds the learnable pattern (pattern token) As the corresponding attribute characteristics. Provide extended position information for the interleaved words of data and patterns. In the input structure of the PAT model, the data words of the unit sequence and the learnable pattern words of the corresponding attributes of the unit are arranged alternately as interleaved words.

[0039] ; in, The interlaced words of data and patterns reflect the contrasting and contrasting relationship between data units and their attribute characteristics.

[0040] In the Attribute Pattern Aware Transformer (PAT) model, no trainable linear projection layer (i.e., Embedding layer) is added before the Transformer encoder. The PAT network with the Embedding layer removed can improve the accuracy of the error detection task. As shown in Figure 3 As introduced in the PAT model, = 6 layers of stacked encoders, and at the output of the last layer of encoder The MLP head is added to perform multi-layer perception classification. According to the following model overall calculation structure formula, the encoder of the PAT model consists of multi-head self-attention (MSA) and feed-forward layer (FFN or MLP). The output feature embedding of the layer encoder is At the same time, layer normalization (LayerNorm, LN) and residual connections are also used in the PAT model.

[0041] ; ; .

[0042] The following focuses on the self-attention mechanism module of the PAT model. The attention module is calculated in parallel. The self-attention function embeds the staggered word units into As input, in the first step, the input tokens are linearly projected into query, key, and value matrices, denoted as Q, K, and V.

[0043] ;in, is the internal weight matrix of the multi-head attention structure, The dimensions of the key are set to , the dimensions of the query and value ( , ) is the same as it.

[0044] .

[0045] Furthermore, in order to analyze the internal structure of the attention module, the present method decomposes the steps. The present invention calculates the attention score matrix to obtain , this attention score matrix contains two types of sub-elements in data tokens and pattern tokens. and Can be described as and .exist In the element vector in, example represents the query value corresponding to the CLS word. The same is true for the element vectors in K and V.

[0046] .

[0047] Attention score matrix It combines the features of the original data unit to be detected and the corresponding attribute pattern. The latter helps to separate the unique features from the word unit to be detected and retain or enhance the recognition ability of erroneous features in the attribute domain.

[0048] Finally, the present invention calculates the matrix output of self-attention. Among them, the value matrix Can be described as .

[0049] .

[0050] In the above-mentioned pattern-aware error detection method PAT method, the visualization structure of the interpretability module is specifically as follows: This visualization method aims to meet the potential need to inspect and analyze the internal mechanisms of the Pattern-Aware Transformer (PAT) to improve model transparency and reveal why and how deep learning models detect data errors.

[0051] In the Pattern-Aware Transformer (PAT), the present invention extracts attention scores from the multi-head self-attention (MSA) network as the attention map in the data detection reasoning process. In order to analyze the relationship between the input word and the predicted classification, the present invention selects the last layer in the 6-layer encoder network for processing. In the encoder, since the multi-head self-attention network has h=8 parallel attention heads, a total of 8 attention score matrices are extracted for each data sequence to be detected. Specifically, each row vector of the attention score matrix corresponds to an input word, including CLS words, data words, and pattern words; each column vector of the matrix represents an output word. Since the PAT model detects data errors, only the CLS embedding vector of the first column is extracted for error detection binary classification. Furthermore, in terms of model interpretability, the present invention extracts the CLS embedding vector from the attention score matrix of the PAT model. Extract the first column vector as , to visualize the relationship between the input word unit and the classification prediction (CLS output). In addition, the present invention removes the subsequent zero-filling part corresponding to the generated word unit in the QTA algorithm to highlight the real segmented text.

[0052] When using the extracted attention vectors for visualization, the attention lines are too weak for most data sequences to represent the internal relationships of PAT. To enhance the visualization effect, the present invention multiplies the attention scores by a coefficient η to highlight the visualization lines.

[0053] like Figure 1 As shown in Figure 1, the execution process of the error detection method is divided into four stages: data reading, preprocessing, data input, and model execution. Figure 5 As shown; the model structure is as Figure 3 As shown in the figure; the visualization diagram generated during the model execution is as follows Figures 22 to 35 In addition, the experimental results are shown in Tables 2 and 3, as well as Figures 7 to 21 shown.

[0054] (1) Preprocessing: Use the QTA tokenizer to divide the data sequence, obtain data tokens, and interleave them with the pattern tokens to obtain the input tokens of the model. The QTA tokenizer needs to set the hyperparameter token dimension and word number .

[0055] Figure 5 The present invention uses a tree diagram to introduce a novel data table tokenizer designed by the present invention. The tokenizer uses a Quasi-Tokens Arrangement (QTA) algorithm, which aims to organize basic tokenizer sequence fragments through the quasi-tokenizer, and these fragments eventually form a set of tokens (number of tokens) of the input sequence. ), so that the words in the sequence are arranged in a combination in which the wrong words are easy to detect. The decomposition obtained by the word segmenter is divided into a set of word units .

[0056] For the error detection task, when embedding a word sequence, the present invention creates a character index mapping so that the character and word format features of the original sequence can be retained in the embedding vector. Specifically, the present invention uses the Unicode character mapping dictionary as the symbol index dictionary for the word create Fixed Dimensional embedding tokens .

[0057] Each primitive unit is actually a text sequence that can be considered as a combination of numbers, characters, and punctuation marks. The QTA algorithm tokenizes the entire text of the unit data based on the following principles: In order to handle variable-length sequences, the input of the tokenizer algorithm has two hyperparameter configurations, default or compact. The execution process of the algorithm divides the input sequence into three steps according to the degree of fine or coarse tokenization: 1) full tokenization, 2) coarse tokenization with number separation, and 3) block tokenization without separating numbers. The program will tokenize the word units that meet the hyperparameters in the segmentation. and The range ends when requested.

[0058] As a data preprocessing algorithm, the QTA tokenizer divides variable-length data sequences into six different degrees of tokens. Its evaluation factors take into account two factors: the type priority of the quasi-token and the actual length of the data sequence. The original sequence is processed by the quasi-tokenizer to obtain quasi-tokens. The categories of quasi-tokens are divided into three categories: numerical, text, and punctuation. Among them, the independent priority of numerical tokens is the highest, the independent priority of punctuation tokens is the second, and the priority of text tokens is the lowest. In the QTA algorithm, tokens of the same type are merged first.

[0059] (2) Model structure like Figure 3 ,The model structure is specifically divided into multi-head self-attention, feed-forward layers and their related ,operation structures. In the PAT model, the output corresponding to CLS embedding is selected as ,classification feature, and a linear layer is used to map it into two ,classes, namely, error and correct.

[0060] The input form of the PAT model arranges the two groups of words, data and pattern, in an interleaved order, and there are corresponding orders and categories in the network structure and output words. During the training and inference process, the data and its corresponding attribute patterns are used as two network structures to form the overall model. In the training phase, data of different attributes are randomly used as samples for training, and the corresponding pattern words of different attributes will switch accordingly. Therefore, the pattern embedding word will serve as a bridge for data interaction of different attributes in the PAT model to realize cross-attribute detection of the data error detection model. The PAT model can learn the independent features between attribute domains and the shared features within the attribute domains of the corresponding attributes, and store them in the learnable pattern module and MSA network weights to assist in detection. Figure 2 List the various error types that exist in the database and divide them according to the level at which the errors exist.

[0061] (3) Visual analysis During the training and reasoning process of the PAT model, in the front part of the multi-layer network, the parameters of the data and pattern structures will learn multiple feature representations of the data unit. As the multi-layer network progresses, the two structures of data and pattern will focus on error classification and play different roles in error detection. Specifically, during the training process of the model, the data with the same attributes in the data table will jointly learn N pattern embedding tokens. From the perspective of the same attribute domain, these pattern embeddings can learn the data and error features unique to the attribute, and these features are identifiable. From the perspective of the attribute domain, the pattern embedding features of different attributes are independent of each other, indicating the feature differences of data and errors of different attributes. Therefore, the pattern embedding tokens are shared within the attribute domain and independent between attribute domains. On the other hand, the structural parameters of the data network of the PAT model will learn the common data and error features of all data in the data table, that is, the indistinguishable features of all attribute values. With the help of two inputs and the internal structure of the network, the PAT model extracts the commonality and characteristics of the structural characteristics and content characteristics of the data table, achieving high accuracy and efficiency in database error detection.

[0062] (4) Method performance evaluation

[0063] In order to verify the performance advantages of the framework proposed in the present invention, the present invention conducted multiple groups of experiments on synthetic datasets and real datasets.

[0064] Experimental environment: The framework of the present invention is implemented using the PyTorch deep learning framework. The experiment uses the Ubuntu 20.04 Linux system. The hardware is Intel Xeon Gold 6226R CPU, 256GB memory and 4 NVIDIA GeForce RTX3090 GPUs.

[0065] Datasets: The dataset information used in the experiment is shown in Table 1. Among the 11 datasets, the real datasets are Adult, Beers, Flights, Food, Hospital, Movies, Rayyan, Restaurants, Soccer, and the synthetic datasets are HOSP-100K and Tax.

[0066] The experiment mainly evaluates performance from the following aspects: In terms of accuracy, this experiment uses three indicators. Precision (P) is defined as the proportion of true positive samples (TP) among all samples predicted to be positive; Recall (R) is defined as the proportion of true positive samples (TP) among all wrong samples (samples that are actually positive); F1 score is a metric calculated by comprehensively considering P and R. For the efficiency metric of the model, this experiment uses floating-point operations (FLOPs) as a measure of the amount of computation.

[0067] Table 1 Dataset information table

[0068]

[0069] Table 2 Comparison of the accuracy of PAT, PATC and five advanced methods on real and synthetic datasets (the ones in bold are the ones where the detection methods have achieved the best measurement indicators)

[0070]

[0071] Table 3 Comparison of parameters and computational complexity of 7 methods on real and synthetic datasets

[0072]

[0073] This experiment compares the accuracy of PAT (PAttern-perceptive Transformer) and PATC (PAttern-perceptive Transformer Compact) methods with the five most advanced (SOTA) error detection methods, including P, R, and F1 metrics. The evaluation results are shown in Table 2, which show that PAT has significant advantages in efficiency and detection accuracy compared with multi-detector (Raha, ED2, HOLODETECT) and single deep learning model methods (ETSB-RNN, ROTOM). The PAT model is a single-model error detection method that surpasses the multi-detector method in terms of accuracy.

[0074] Figures 7 to 16 This is a comparison of the accuracy of error detection using four Data Transformer methods (PAT, PATC, DT, DTC), among which, is the independent variable, Remain fixed and determined by the hyperparameter setting algorithm. DT-based methods (DT, DTC) are Data Transformer methods that do not use learned attribute patterns in the model. Figures 17 to 20 This is a comparison of the accuracy of error detection using four Data Transformer methods, among which, is the independent variable, remains fixed and is determined by the hyperparameter setting algorithm. Figures 7 to 20 In the accuracy comparison chart, each point is the corresponding F1 score value under the set independent variable (D or N) (the error bar indicates the standard deviation obtained by repeating the experiment 10 times).

[0075] exist Figures 7 to 20 In the accuracy comparison chart, PAT methods (PAT and PATC) are significantly higher than DT methods (DT and DTC), with a clear gap. Figure 7 , Figure 8 , Fig.10 , Fig.11 , Fig.12 In the data, the F1 scores of the PAT method on the Restaurants, Rayyan, Hospital, HOSP-100K, and Adult datasets gradually increase with the increase of the word unit dimension D. This stable growth trend is due to the fact that the dataset contains a large number of long sequences (large blocks of words and sentences obtained by word segmentation). When the word unit dimension D is larger, the model can accommodate larger blocks of words and sentences, and the F1 score of the PAT method also increases accordingly. On the contrary, Fig. 9 , Fig.14 The F1 scores of the PATC method for the Movies and Food datasets decrease as D increases because the two datasets lack sequences of medium length.

[0076] When compact tokens can effectively accommodate long unit data sequences, the data table conversion method in compact mode is better than the default QTA tokenization scheme. On the contrary, if the actual segmented sequence fragments are shorter, the token vector generated by the default QTA algorithm will contain more zero-filled components. This makes it difficult for QTA in default mode to effectively reorganize most quasi-tokens (fragments segmented by the quasi-tokenizer) and will cause redundant occupancy of a large number of vector components. Therefore, Figure 8 , Fig.15 , Fig.16 , Fig. 20As shown in the figure, the F1 scores of the compact modes of DTC and PATC on Rayyan, Soccer, Tax, Flights and other datasets are higher than those of the default modes. From this, we can conclude that appropriate lemma parameters (D and N) can adapt to model learning and thus affect accuracy performance.

[0077] In terms of efficiency, this experiment uses two indicators, parameter count and FLOPs, to measure the computational efficiency of the model. Table 3 shows the computational efficiency results of 7 methods on datasets containing short sequences, medium sequences, and long sequences, including 3 comparison methods and 4 Data Transformer methods. Fig.21 This is a trend chart of the computational load (FLOPs) and accuracy (F1 Score) of the five methods on the same dataset. It can be seen that DT, DTC, PAT, and PATC all show a trend of increasing accuracy as the computational load increases. In terms of parameter volume and computational load, DT, DTC, and HOLODETECT are at a similar level to PAT and PATC; PAT and PATC methods maintain high accuracy under different model computational loads. In addition, under the same computational load, the accuracy of PAT and PATC is higher than that of DT and DTC.

[0078] In terms of model interpretability, Figures 22 to 35 Represents the model visualization process, and the attention map can effectively construct the relationship between the input word and the detection result. Fig.29 It is a DTC model. Figure 24 to Figure 27 The figure is the PATC model, and the other figures are the PAT model. With the help of the synergy of input data (CLS tokens, data tokens and their corresponding pattern tokens), the multi-head attention network captures diverse features. In the MSA network, the 8-head attention mechanism plays different roles in the error detection task: some attention heads (such as Fig.23 1st and 3rd heads, Fig.25 No. 3 Fig.28 The sixth head, starting with 1, focuses on extracting word features from data, while others (such as Fig. 22 Head 2, Fig.30 No. 1, Fig.35 The second head focuses on pattern meta-analysis. This shows that the PAT model can stably learn the characteristics of data and patterns, and these characteristics are randomly arranged in 8 parallel heads.

[0079] Multi-head attention is mainly responsible for the error detection function of unit sequences. Some attention heads (even a single one) focus on learning error-related features. All multi-head attention features are calculated in parallel to make the final prediction. The PAT method can accurately and efficiently apply classification probes to error words, while weakening the attention to error-free words. There are two typical phenomena in the distribution of attention features of error words: the error word may be in the data word ( Fig.25 No. 3 Fig.35 4th and 5th heads) or its PAT pattern word ( Fig.25 Heads 1, 2, and 4, Fig.32 The 3rd, 5th, and 8th heads are dark (high attention scores), and the correct word is light (low attention scores); it may also appear in the data word of the wrong data ( Fig.28 No. 6 Fig.34 2nd and 4th heads) or their PAT pattern words ( Fig.34 The 3rd and 6th heads, Fig.35 The third head) position shows light or blank (low attention score), while the normal word is dark (high attention score). Generally speaking, a high attention score means that the output pays more attention to the corresponding word, such as the former; for the latter, since normal and incorrect words are two symmetrical categories, there is a phenomenon of swapping category features during multiple training processes, so there is an abnormal phenomenon of low attention at the incorrect word. At the same time, this shows that the network parameters corresponding to the data words of some error sequences can only learn local features relative to the multi-head attention network. Finally, the MLP head module will integrate all attention parameters to complete the error detection task prediction.

[0080] It is worth noting that Figures 22 to 35 The corresponding MSA structure is located in the last encoder layer of the model. By observing the attention graph of the 6-layer encoder network, as the number of network layers increases, the attention mechanism can more clearly establish the association between the wrong words (input words containing wrong information) and the output classification features. This is due to the reasonable arrangement of the input words by the QTA algorithm. The wrong words can be assigned to words with appropriate dimensions, thereby accelerating the error detection learning process of the PAT model.

[0081] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A relational database error detection method with a learning attribute model, characterized in that: Here are the steps: S1, data preprocessing, the input data sequence is represented by input word units constructed by interleaving data word units and pattern word units; S2, error recognition, learning the independent and shared features between attribute domains of the corresponding attributes based on the input word meta-words, and storing the learning content in the learnable pattern module and the multi-head self-attention network weights for detection; S3, visual explanation, extracts the attention scores in the multi-head self-attention network, builds the relationship between input word units and output predictions, and generates a visual attention map; Adopt the attribute pattern-aware Transformer framework, which includes: A data preprocessing module, configured with a program for executing a quasi-word arrangement algorithm and a hyperparameter setting algorithm; The model structure module is configured with a program that takes the data word unit and the pattern word unit in an interlaced arrangement as input, learns the independent features between the attribute domains and the shared features within the attribute domain of the corresponding attribute, stores and learns them, and can output the detection results; The interpretability module is configured with a program that extracts the parameters of a multi-head self-attention network to build a relationship between input tokens and output predictions and generate a visual attention map.

2. The relational database error detection method with a learning attribute model according to claim 1, characterized in that: Data preprocessing includes: dividing the input data sequence into data tokens according to the length of the sequence and the type of word through a quasi-term arrangement algorithm; representing the attribute pattern of the data unit value in the input data sequence with pattern tokens according to a learnable pattern; and arranging the data tokens and the pattern tokens in a corresponding staggered manner to obtain structured input tokens.

3. The relational database error detection method with a learning attribute model according to claim 2, characterized in that: The quasi-word arrangement algorithm relies on the word segmentation hyperparameter setting algorithm that automatically configures hyperparameters. The word segmentation hyperparameters include: word dimension D and word number N.

4. The relational database error detection method with a learning attribute model according to claim 3, characterized in that: The steps of word segmentation hyperparameter setting algorithm are as follows: S1. Obtain a histogram of word length and number from the error data table, find a critical point using a critical point search function, and divide the elements of the input array into a high-frequency point group and a low-frequency point group; S2. The word dimension is calculated by the critical point search function, and the input is the word length array and the preset threshold coefficient. In the default mode, the word length array is used as the input of the critical point search function to calculate the number of words, while in the compact mode, the number of words is the length of the longest sequence in the data set divided by the compact word dimension. S3, the critical point search function determines the dividing point between the high-frequency point group and the low-frequency point group by multiplying the average value of the frequency histogram by the preset threshold coefficient; in the default mode, the preset threshold coefficient is β, and in the compact mode is .

5. The relational database error detection method with a learning attribute model according to claim 4, characterized in that: The quasi-word arrangement algorithm includes three stages: full word segmentation, rough word segmentation, and block word segmentation; In each stage, the quasi-word segmenter is first applied to the input data sequence to divide it and obtain the quasi-word units, then the word segmentation parameters are set for the obtained quasi-word units to arrange them, and then the out-of-dimensional words are segmented and the short words are merged with sliding windows, and then they are mapped to numerical representations according to the Unicode character dictionary to finally obtain data words.

6. The relational database error detection method with a learning attribute model according to claim 5, characterized in that: The quasi-word arrangement algorithm uses variable-length unit sequences and word segmentation hyperparameters as input; first apply the quasi-word segmenter to the original input data sequence to obtain quasi-word elements for further processing ; Then, for the quasi-participle Each word that exceeds the dimension D limit is split into multiple word units; for quasi-segmented words that still do not meet the requirements of word unit dimension D and word unit number N, adjacent word units are merged according to certain rules in the merging stage: when merging, words and punctuation marks are given priority, and numbers are considered last; word units of the same merging level are merged first; as long as the word units obtained after segmentation meet the requirements of word unit dimension D and word unit number N, the algorithm ends.

7. The relational database error detection method with a learning attribute model according to claim 6, characterized in that: The data preprocessing process is as follows: To simulate the structural characteristics of unit values ​​and their attribute domains, the data word units are and pattern tokens Interleaved to form interleaved tokens, where a data token is a sequence of input units The preprocessing stage is obtained through word segmentation, and the pattern word is obtained through PAT model training. N is the number of word units. and each pattern token is an embedding vector of dimension D; then, a learnable CLS embedding token is appended to the head of the interleaved token of data and pattern ; The input word is obtained by combining the CLS word, the data word and the pattern word: ; ; It is the location information provided for the interleaved tokens of data and patterns; Applying a variety of structures, including multi-head self-attention, feed-forward layers, layer normalization, and residual connections, the overall computational structure of the attribute pattern-aware Transformer network is as follows; ； ； ； in, Indicates the number of stacked layers of the encoder; Indicates The output features of the self-attention module of the first layer; The output feature embedding of the layer encoder is recorded as ;CLS word The state corresponding to the output of the model is represented as the classification feature of the sequence of input cells in the data table ; The query, key, and value matrices are the main structures of the multi-head self-attention network, denoted as , , , which are computed in parallel: ; in, is the internal weight matrix of the multi-head self-attention network; ; The dimensions of the key are set to , the dimension of the query The dimension of the sum value Same dimension as the key; The matrix output representation of the multi-head self-attention network: ; Furthermore, in order to analyze the internal structure of the attention module, the method decomposes the steps therein; the present invention calculates the attention score matrix to obtain , this attention score matrix contains two types of sub-elements in data tokens and pattern tokens; and Can be described as and ; ; ； Attention score matrix It combines the features of the original data unit to be detected and the corresponding attribute pattern; Value Matrix Can be described as ; Matrix output of the multi-head self-attention network: .

8. The relational database error detection method with a learning attribute model according to claim 7, characterized in that: Visual explanation, from the attention score matrix Extract the first column vector from the to visualize the relationship between the input word and the classification prediction.

9. The relational database error detection method with a learning attribute model according to claim 1, characterized in that: The model structure module is based on the Transformer encoder architecture and consists of six identical encoder stacking layers. A multi-head self-attention and a feedforward layer are set in each encoder layer. The multi-head self-attention and the feedforward layer are connected through a sum layer standardization operation. The output end of the last encoder layer is provided with an MLP head for multi-layer perception classification.

Images