An efficient entity segmentation and matching method
By combining a block-based approach integrating term inverted indexes and vector similarity with unsupervised contrastive learning, along with a pre-trained BERT model and a matching neural network that combines memory and generalization, the problems of polysemy and scarce labeled data in entity matching are solved, achieving efficient and accurate entity matching and data cleaning.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIHANG UNIV
- Filing Date
- 2022-11-11
- Publication Date
- 2026-05-26
AI Technical Summary
Existing entity matching methods cannot effectively solve the problems of polysemy and complex and varied wording in entity records during the segmentation stage. Furthermore, the matching stage relies heavily on expert knowledge, has insufficient generalization effect, and lacks hierarchical modeling from token to attribute to record, resulting in poor efficiency and performance.
We adopt a block-based approach that integrates term inverted index and vector similarity, and combine unsupervised contrastive learning to obtain vector representations of entity records. We perform semantic embedding through a pre-trained BERT model, combine a memory and generalization matching neural network, and use the TF-IDF-improved BM25 algorithm and convolutional neural network to extract matching signals and optimize the matching process.
It improves the recall rate in the segmentation stage and the accuracy in the matching stage, reduces manual maintenance costs, enhances the matching effect of multiple data sources, solves the problems of scarce labeled data and heterogeneous information representation, and achieves efficient and accurate entity matching.
Smart Images
Figure CN116226368B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of web search technology, and in particular to an efficient entity segmentation and matching method. Background Technology
[0002] In the information age, the problems of data redundancy and conflict are becoming increasingly prominent, and people are placing higher demands on data quality and information sharing. At the same time, the growth in data scale has also brought significant challenges to the execution efficiency and effectiveness of algorithm design.
[0003] To address these real-world challenges, entity matching aims to identify whether data records point to the same entity. Various methods have been developed in the past, yet they still face numerous difficulties and significant room for improvement. These difficulties stem from factors such as poor data quality; real-world data is rarely completely clean, structured, and homogeneous. Manually inserted data may contain spelling errors, substitutions, or patterns that do not conform to the pattern (e.g., mixing first and last names), as well as data loss. Furthermore, we cannot always expect different data sources to follow the same patterns, formats, and syntax.
[0004] Currently, mainstream methods for solving entity matching divide the process into two stages: segmenting records and matching record pairs. The former aims to reduce the computational cost of entity matching by limiting the matching scope to a set of entities that are more likely to match, while the latter represents the matching result based on a predetermined similarity threshold. Among the segmentation methods, rule-based and traditional machine learning methods are simple and efficient, but they cannot solve the problems of polysemy and complex word variations in entity records, and are poor in representing entity semantic information. Currently, some works use pre-trained language models such as BERT to supplement semantic information, but they often rely on a large amount of labeled data. In the matching stage, rule-based methods require expert participation and strong domain knowledge, which are difficult to maintain; traditional machine learning methods rely on meticulous feature engineering; existing deep learning methods have insufficient generalization performance and lack hierarchical modeling from token to attribute to record during modeling. Losing information in any dimension will result in a loss of recognition capability in the final matching.
[0005] Blocking is an essential stage in entity matching, reducing the number of matches. The main idea is to divide entities into different blocks based on characteristics such as key values. Blocking can be considered a search problem for Approximate Nearest Neighbor (ANN) networks in a similarity space.
[0006] Methods for selecting and partitioning keys can be categorized into rule-based and machine learning (ML) / deep learning (DL)-based approaches. The former assigns entities to blocks based on rules derived from expert knowledge or purely heuristics, while the latter two require a training set to learn how to map entities to corresponding blocks. Among rule-based methods, Standard Blocking (SB) is a typical example, proposing an adjacency ranking method to address issues like misspellings. To avoid problems caused by noise and missing values, subsequent methods based on suffix arrays and filtering based on attribute-based text features (N-Gram, edit distance, etc.) were proposed. Their advantage is that similar text forms are grouped into the same block, but their disadvantage is poor generalization ability due to relying solely on character and numeric features to select candidate entity records. In machine learning methods, the earliest approach viewed record partitioning as a clustering task. Canopy clustering-based partitioning methods group similar records into the same cluster. TF-IDF-based vector representation methods have the advantage of considering word importance but the disadvantage of not considering the specific meaning of words. Locality Sensitive Hashing (LSH) is a popular probabilistic search technique where the more similar the input vectors, the higher the probability of placing them in the same block. Subsequent papers have attempted this method and its variations, but its drawback lies in its failure to identify the true semantics of records. In deep learning methods, self-supervised distributed word vector representations, such as Word2vec and FastText, are used. Their advantage is that they consider the context of words and have fewer vector dimensions; their disadvantage is that they cannot handle polysemy. When using sentence similarity as a block segmentation method, sentence vectors are generated by word vector mean-pooling, max-pooling, or other unsupervised methods such as SIF (Smooth Inverse Frequency) and TF-IDF, which can compromise sentence semantics. Block segmentation based on Sentence-BERT semantic similarity has the advantage of encoding sentence semantics based on a pre-trained model, resulting in better performance; its disadvantage is that it is supervised, and the quality of the negative sampling method greatly affects the experimental results. The block segmentation methods mentioned in Ditto include a brief introduction to the classic TF-IDF method and Sentence-BERT.
[0007] Research on the matching phase has evolved from rule-based methods to machine learning and crowdsourcing, and now to the mainstream deep learning methods.
[0008] Rule-based entity matching algorithms generally determine whether two entities match based on manually defined rules. These algorithms are relatively simple in concept. Rules can be divided into exact matching rules and approximate matching rules. For example, "same last name" is an exact matching rule, while "similar first name" is an approximate matching rule. A simple example is: if two people have the same last name, the same gender, the same address, or similar first names, they can be considered the same person. This type of method is relatively intuitive and has low code complexity. However, the formulation of entity matching rules relies on prior knowledge from experts, requiring significant manpower and time. Furthermore, the rationality and effectiveness of the rules directly determine the algorithm's performance.
[0009] Most machine learning methods are variations of the classic Fellegi-Sunter model, primarily including Support Vector Machines (SVM), active learning, and clustering. These methods learn a classifier from a set of samples and then use the classifier to determine whether record pairs match. Magellan is a typical machine learning framework for solving this problem, integrating the aforementioned methods.
[0010] Crowdsourcing, precisely defined, is the distributed allocation of tasks that machines cannot handle to a group of people; its essence is collective wisdom, pooling the collective intelligence of the masses to help solve a problem. Platforms like Amazon MechanicalTurk (AMT) or CrowdFlower allow users to post tasks on websites. These tasks are typically simple, such as image annotation or sentiment analysis, and anyone (the worker) can perform them and earn a commission. Crowdsourcing methods for entity matching also require expert involvement in determining the similarity function and threshold settings.
[0011] Recently, entity matching has achieved promising results using deep learning. DeepER is based on an LSTM neural network architecture and uses word embeddings such as GloVe to train the EM model. DeepMatcher designs a deep learning-based matching framework. Ditto replaces the attribute embedding and similarity representation components in its architecture with a pre-trained language model (LM) (such as BERT), thus providing a simpler overall architecture. DeepER, DeepMatcher, and Ditto all treat matching as a binary classification problem.
[0012] The main drawbacks are as follows:
[0013] 1. Currently, methods for solving entity matching divide the process into two stages (blocking and matching), and there is no complete and unified technical solution in the existing technology.
[0014] 2. Existing block-based methods, such as rule-based and traditional machine learning methods, while simple and efficient, cannot solve problems related to polysemy and complex variations in entity record wording, resulting in poor representation of entity semantic information. Currently, some works utilize pre-trained language models such as BERT to compensate for semantic information deficiencies, but these often rely on large amounts of labeled data.
[0015] 3. In the matching stage, existing technologies rely on rule-based methods that require expert participation and strong domain knowledge, making them difficult to maintain; traditional machine learning methods rely on meticulous feature engineering; and existing deep learning methods have insufficient generalization effects and lack hierarchical modeling from tokens to attributes to records, resulting in matching performance that needs improvement on some datasets. Summary of the Invention
[0016] To this end, the present invention first proposes an efficient entity segmentation and matching method. The entity records input into e-commerce or academic databases are efficiently matched by three modules: a segmentation module, a preprocessing module, and a matching module.
[0017] Specifically, the entity records are first input into the segmentation module. The segmentation module first constructs a semantic index block through an offline module. The semantic index block is formed through model training, entity record embedding inference, and semantic index builder. The queried entity records are semantically embedded. At the same time, the entity records are input into the inverted index block after being cleaned by term segmentation. The results of the above two are sent to the semantic index block and the inverted index block respectively for segmentation and filtering of entity records.
[0018] The results from the semantic index block and the inverted index block are then merged through the preprocessing module, followed by data processing and transformation to obtain the information representation of the entity.
[0019] The data is then sent to the matching module, where redundant retrieval and category classification are performed on the input entity record pairs to obtain redundant and similar data entity records from the database. This allows for the aggregation of e-commerce or academic data, avoiding the waste of physical space caused by redundant storage of duplicate product or monograph data. The final output is a summarized and redundancy-free e-commerce or academic database structure. As described in claim 1, an efficient entity block segmentation and matching method is characterized by: the block method employing a block method based on term inverted index and vector similarity integration. To address the problem of scarce labeled data, unsupervised contrastive learning is introduced to obtain vector representations of entity records. The term inverted index method obtains candidates by building a term inverted index on entity records and returns their entity records according to a certain ranking score. In the process of measuring the relevance of entity records (the similarity between two entity records), a TF-IDF-improved BM25 algorithm is used to rank the relevance of entity records.
[0020]
[0021]
[0022] Where Q and D represent two entity records, q i Let be a Term i for entity record Q. In TF-IDF, the importance of a Term increases proportionally to its frequency in the entity record and decreases inversely proportionally to its frequency in the entire table. IDF stands for Inverse Document Frequency, where N represents the total number of entity records to be queried, and n(q i ) represents the entire database containing entity q. i The number of entity records. In the BM25 formula, k1 and b are adjustment factors, which are usually k1 = 2 and b = 0.75 based on experience. f represents the frequency of q in entity D, |D| represents the length of entity record D, and avgdl represents the average length of all entity records.
[0023] The unsupervised contrastive learning specifically involves: using a pre-trained BERT model as the backbone network for encoding entity records; performing dropout on the same entity record, treating it as a positive sample, and treating other entity records within the same batch as negative samples; and employing a contrastive loss function. Indicates the current sample h i The loss, of which h is the positive sample corresponding to this sample. jFor each sample in a batch, the number is N, and τ is the temperature coefficient. After obtaining the embedded representation of the entity record, the approximate nearest neighbor algorithm is used to quickly find the top-k candidate entity records and merge and filter them with the candidate records of the inverted index for the entire entity record set. Then, the result is processed and passed to the matching module.
[0024] The information representation is achieved using a heterogeneous entity information representation method based on pre-trained models and feature engineering. Specifically, firstly, important relevant features are selected based on feature engineering and shallowly memorized by the model. Then, entities, proper nouns, etc. in the records are identified and injected into the pre-trained BERT model. Vector representations of entity records are obtained through the pre-trained model. Finally, the vector representations are optimized through a matching neural network that combines memorization and generalization.
[0025] The memory and generalization matching neural network structure is divided into a wide side and a deep side. The input features on the wide side are type words that reinforce the signal, which are learned and memorized shallowly through a linear perceptron layer. The input on the deep side consists of two entity records. A pre-trained BERT entity vector representation layer obtains the vector representations of the entity records, which are then input to the matching layer. The matching layer primarily extracts the matching relationships between tokens, N-grams, and entity granularities. In the fusion layer, information at the token level, N-gram level, and entity record semantic level is fused. Finally, the deep side result is obtained through a linear perceptron layer. The wide side result and the deep side result are then processed using a formula... And through the sigmoid activation function Obtain the final matching probability
[0026] The deep-side model captures information from the term level to the N-Gram level and then to the entity level for two entity records. The matching layer is designed to calculate the matching matrix for term exact match, generalized match, and relevance match for two records. Then, the convolutional neural network is used to extract matrix features. At the same time, semantic information is extracted at the entity level to enhance the overall matching signal.
[0027] The term precise match is calculated by using a matching matrix that shows a complete match between two entities. The formula is as follows: The value is 1 if all terms are completely equal, and 0 otherwise. The generalized hit rate is a matching matrix obtained by calculating the similarity of each token-level vector output by BERT, using the following formula:
[0028] The relevance matching matrix adopts a neural tensor network (NTN) structure, which considers the relationship between any two tokens between entity records as a feature of the matching signal. A scoring model function is used to score it, and the specific scoring function is as follows: Where f = tanh is a nonlinear activation function. It is a tensor, bilinear tensor product The result is a vector h∈R k Each entry is computed from a tensor slice i = 1, ..., k: V R ∈R k×2d and U∈R k ,b R ∈R k K represents the number of relationships. After the exact match, generalized match, and relevance match mentioned above, a relationship matrix between tokens of entities is obtained. The number of matrices is equivalent to the number of input channels of the image, and the cell values in the matrix are equivalent to the pixels of the image. A convolutional neural network is used to extract the matching signal.
[0029] For matching two entity records e1 and e2, the convolutional neural network is specifically designed with a 4-layer convolutional kernel structure: the first layer has 12 convolutional kernels of size 1×3, the second layer has 12 convolutional kernels of size 3×3, the third layer has 24 convolutional kernels of size 3×5, and the fourth layer has 24 convolutional kernels of size 5×5. The convolution operation between the input matrix and the kernel matrix is called the cross-correlation operation, specifically O(i,j)=(K*I)(i,j)=∑ m ∑ n The expression I(i+m,j+n)K(m,n) is used, where the input matrix is I, the kernel matrix is K, and the kernel matrix K has an m×n shape. This yields the element at position (i,j) in the output matrix O. An odd-numbered convolutional kernel is used, and matrix padding is applied to both ends to ensure the output matrix has the same dimension as the input matrix. Finally, pooling is performed on the output channels along the e² dimension to obtain the matching features of the two entities. In the fusion layer, the entity-level semantic matching information, obtained from the pre-trained model's CLS vector and the token sequence vector difference, is incorporated into the final features. The specific calculation method is: deep_match = [cls_embedding, term_matching, seq_diff]. The fusion result is passed through a linear perceptron layer to obtain a one-dimensional vector representation. Furthermore, for the optimization objective, focal loss is used as the loss function, specifically: Where γ is called the focusing parameter, γ>=0, (1-p i )γ This is called the modulation coefficient.
[0030] The technical effects to be achieved by this invention are as follows:
[0031] 1. This invention meets the requirements of speed, efficiency and high recall in the segmentation stage, and also effectively solves the problem of insufficient labeled data in entity matching scenarios.
[0032] 2. This invention solves the problem of difficult learning of heterogeneous information representations in entity matching.
[0033] 3. This invention addresses the challenges of modeling and learning using neural networks in entity records, such as misaligned attributes, abbreviations in words, and a large number of numerical features, thereby improving the performance of existing technical solutions.
[0034] 4. The block-based method of this invention improves the recall rate of Top-10 by an average of 1.21% compared to the baseline, and the designed matching method improves the F1 score across multiple data sources. Meanwhile, the overall time efficiency of each module in this invention remains consistent. This method can be applied to database cleaning, deduplication, and retrieval matching, significantly reducing manual maintenance costs. Attached Figure Description
[0035] Figure 1 Framework design diagram;
[0036] Figure 2 Overall design of the inverted index block system;
[0037] Figure 3 Entity record embedding vector model;
[0038] Figure 4 Top-k candidate records;
[0039] Figure 5 Entity record vector representation;
[0040] Figure 6 Matching network structure;
[0041] Figure 7 Matching structure on the deep side;
[0042] Figure 8 Interrelationship diagram; Detailed Implementation
[0043] The following are preferred embodiments of the present invention, which are described in conjunction with the accompanying drawings. However, the present invention is not limited to these embodiments.
[0044] This invention proposes an efficient entity segmentation and matching method. It can be applied in scenarios such as an e-commerce platform wanting to query the inventory of a certain type of product (same brand, different stores, diverse descriptions, and varying prices). Instead of a one-to-one comparison of all products in the database, this invention can quickly and efficiently use a model method to find products with high matching scores, select a threshold score, or sort them according to the scores to calculate the quantity. Another example is in scenarios such as population statistics and information registration, where an individual has registered duplicate information in multiple locations such as their hometown, workplace, and family's hometown. When data is being uniformly organized nationwide, this invention can be used to identify redundant registrations in cases where certain ID information is missing, thereby achieving data cleaning and purification.
[0045] The overall design of the solution proposed in this invention is as follows: Figure 1 The system is divided into three modules: Blocker, Handler, and Matcher. The Blocker module primarily consists of offline components including model training, entity record embedding inference, and a semantic index builder. In the online service, it performs semantic embedding and term segmentation and cleaning on the queried entity records, then sends the results to the semantic index block and inverted index block respectively for entity record filtering. The Handler module combines the results from the two blocks, performs data processing and transformation, and then sends it to the Matcher module. Finally, the Matcher module performs matching and scoring on the input entity record pairs.
[0046] The inverted index block employs a BLOCK method based on the integration of term inverted index and vector similarity, as designed in this invention. The term-based method guarantees a certain recall rate, while the vector similarity method enhances semantic understanding capabilities, addressing to some extent the issue of term-based methods failing to recall matching entity records due to rewriting or miswriting. Furthermore, this invention introduces unsupervised contrastive learning to obtain vector representations of entity records to address the problem of scarce labeled data, such as... Figure 2 As shown. The term-based method obtains candidates by building a term inverted index on entity records and returns their entity records according to a certain sorting score. In the process of calculating the similarity of entity records, the BM25 algorithm improved by TF-IDF is used to sort the relevance of entity records, and its formula is shown in equation (2.1-2.2).
[0047]
[0048]
[0049] Where IDF represents inverse document frequency, N represents the total number of entity records to be queried, and n(q i )
[0050] This indicates that it contains entity q. i The number of entity records is BM25, which, compared to TF-IDF, improves upon the unfairness that longer entity records are more likely to contain a certain word, while shorter entity records are more likely to contain it. Here, k1 and b are adjustment factors, which are usually k1=2 and b=0.75 based on experience. f represents the frequency of q in the entity, |D| represents the length of the entity record, and avgdl represents the average length of all entity records.
[0051] The block-based method based on entity record embedding vectors employs an unsupervised contrastive learning approach. Specifically, this invention utilizes a pre-trained BERT model as the backbone network encoding entity records. Dropout is applied to the same entity record, treating it as a positive sample, while other entity records within the same batch are treated as negative samples. The model structure is as follows: Figure 3 As shown. This invention also explores data augmentation techniques such as synonym rewriting and back-translation of entity records as positive examples.
[0052] The entity record embedding contrastive learning method uses the contrastive loss function (InfoNCEloss), as shown in formula (2.3), which represents the current sample h. i The loss, of which h is the positive sample corresponding to this sample. j Let N be the number of samples in a batch. The contrastive loss function is a loss function with the property of self-discovery of difficult negative samples. This property is crucial for learning high-quality self-supervised representations; loss functions without this property will significantly degrade the performance of self-supervised learning. The role of focusing on difficult samples is: for those samples that have already moved away, there is no need to continue to push them away; instead, the focus is on how to push those samples that have not moved away away, thereby making the resulting representation space more uniform. τ in the formula is a temperature coefficient, which adjusts the degree of attention given to difficult samples. A smaller temperature coefficient means more attention is paid to separating the current sample from difficult negative samples. This invention sets it to 0.07, referencing a certain paper. The essence of this contrastive loss function is to make the vector representations of entities that are closer in semantic space more similar, and to make the distance between entities that are less similar greater.
[0053]
[0054] After obtaining the embedded representation of the entity record, the Approximate Nearest Neighbor (ANN) algorithm is used to quickly find the top-k candidate entity records and merge them with the candidate records of the inverted index for filtering. The result is then processed and passed into the matching stage for the next stage of calculation.
[0055] The pre-training module is based on the heterogeneous entity information representation of the pre-trained model and feature engineering, that is, representing information on entity records. Information representation, as we usually understand it, refers to how the human brain stores, processes, and expresses information. Similarly, the model needs to represent the information of entity records before it can be further transformed.
[0056] First, important features are selected based on feature engineering, and shallow features are allowed to be memorized by the model. Table 1 shows the relevant feature descriptions.
[0057] Table 1. Description of Relevant Features
[0058]
[0059]
[0060] For the representation of entity records, this invention identifies entities, proper nouns, etc., in the records and injects them into a pre-trained BERT model. (See model details...) Figure 5 Among them, token injection based on part-of-speech and entity recognition, such as: ...token <adj> basic< / adj> token…token… <org> Microsoft< / org>
[0061] Table 2 lists some of the main types of enhanced signals.
[0062] Table 2 Types of Enhanced Signals
[0063] type meaning <adj> < / adj> adjective <person> < / person> people <precent> < / precent> percentage <gpe> < / gpe> Place <WORK_OF_ART> Book titles, songs, etc. <date> < / date> date <org> < / org> mechanism <money> < / money> monetary value <product> < / product> thing <cardinal> < / cardinal> number
[0064] For example, entity records a and b, as shown in Table 3, can be represented using a pre-trained model as follows: Figure 5 As shown:
[0065] Table 3 Entity Record Format
[0066]
[0067]
[0068] 2.2.3 Matching Neural Networks with Memory and Generalization and Target Optimization
[0069] After representing the information of entity records, this information needs to be used to enter the matching network to capture the matching relationship between the features of entity records.
[0070] This invention combines the characteristics of heterogeneous information and misaligned entities in entity records with the advantages of machine learning and deep learning, and designs an overall model as follows: Figure 6 The model learns by using machine learning feature engineering to memorize features and by using deep learning to improve its generalization ability. The network structure consists of a wide side and a deep side. The features input to the wide side are those listed in Table 3, which are learned and memorized shallowly through a linear perceptron layer. The input to the deep side consists of two entity records, in the form of... Figure 5 The vector representation of entity records is obtained through a pre-trained BERT entity vector representation layer and input into the matching layer. The matching layer mainly extracts the matching relationship between token, N-gram, and entity granularity. In the fusion layer, information at the token level, N-gram level, and entity record semantic level is fused. Finally, the deep side result is obtained through the linear perception layer. The wide side result and the deep side result are processed by formula (2.4) and the sigmoid activation function, as shown in formula (2.5), to obtain the final matching probability.
[0071]
[0072]
[0073] The model structure on the Deep side is as follows: Figure 7 As shown, information is captured from the term level to the N-gram level and then to the entity level for two entity records. The matching layer is designed to calculate the matching matrix for term exact match, generalized match, and relevance match for two records, and then uses a convolutional neural network to extract matrix features. At the same time, semantic information is extracted at the entity level to enhance the overall matching signal.
[0074] Term exact match is calculated by performing a matching matrix on complete word matches between two entities, as shown in formula (2.6). A value of 1 indicates that terms are completely equal, while a value of 0 indicates otherwise. Generalized match, on the other hand, is a matching matrix calculated by performing similarity calculations on the vectors at each token level output by BERT, as shown in formula (2.7).
[0075]
[0076]
[0077] For the relevance matching matrix, this invention employs a Neural Tensor Network (NTN) model. The task of this model is to determine the relationship between two entity pairs. For example, consider entity pairs...<cat,dog> and<supervised learning,machinelearning> What are the relationships between them? Based on this idea, this invention designs a feature that considers the relationship between any two tokens between entity records as a feature of the matching signal, and uses a scoring model function to score it, the specific scoring function being as shown in (2.8-2.9).
[0078]
[0079] related_match (i，j) =g(a i b j (2.9)
[0080] Here, f = tanh is a nonlinear activation function. It is a tensor, bilinear tensor product The result is a vector h∈R k Each entry is computed from a tensor slice i = 1, ..., k: V R ∈R k×2d and U∈R k ,b R ∈R k Here, K relationships are designed. This structural design allows the relationships between tokens to be mapped to different spaces to learn features, thereby improving the model performance. After the aforementioned exact hit, generalized hit, and relevance hit, the relationship matrix between tokens between entities is obtained. The number of matrices here is equivalent to the number of input channels of the image, and the cell values in the matrix are equivalent to the pixels of the image. For matching two entity records e1 and e2, this invention uses a convolutional neural network (CNN) to extract the matching signal. Specifically, the convolution kernel is designed as shown in Table 4. The convolution operation between the input matrix and the kernel matrix is called the cross-correlation operation, as shown in equation (2.10). Converting this cross-correlation matrix multiplication into a formula, such as if the input matrix is I, the kernel matrix is K, and the shape of the kernel matrix K is m×n, then obtaining the element at position (i,j) in the output matrix O is actually multiplying each number in the matrix and finally summing them together, such as Figure 8 For detailed calculations, please refer to (2.11). In convolutional neural networks, generally speaking, it is assumed that the input shape is n. h *n w The shape of the convolution kernel is k h *k wThen the output shape will be (n h -k h +1)*(n w -k w +1). This invention assumes that the output Y[i,j] is obtained by cross-correlation calculation between the input window centered at X[i,j] and the convolution kernel, that is, the dimension of the output matrix is the same as the dimension of the input matrix. Generally speaking, if p is filled on both sides of the height... h Line, fill p on both sides of the width. w If the column is n, then the output shape will be (n h -k h +p h +1)*(n w -k w +p w +1). That is to say, the output height and width will increase by p respectively. h and p w If you want the output dimension to have the same height and width as the input dimension, set p. h =k h -1 and p w =k w -1, assuming k here h If it's an odd number, we'll fill the gaps on both sides of the height with p. h / 2 lines. If k h If it's even, one possibility is to pad with p on one side of the top of the input. h / 2+1 rows, and fill with p on one side of the bottom edge. h / 2 rows, with similar padding on both sides. This invention uses an odd-numbered convolutional kernel, so the number of padding elements at both ends is equal. This design is to extract information from N-Gram level entity records. Finally, pooling is performed on the output channels in the e2 dimension to obtain matching features for the two entities. In the fusion layer, semantic understanding of the entities is also emphasized. The semantic matching information at the entity level is integrated into the final features from the CLS vector obtained by the pre-trained model and the difference between the token sequence vectors, as shown in formula (2.12). The fusion result is passed through a linear perceptron layer to obtain a one-dimensional vector representation.
[0081]
[0082]
[0083] deep_match=[cls_embedding,term_matching,seq_diff] (2.12)
[0084] Table 4. Design of Convolution Kernels
[0085] kernel size Number of convolution kernels 1*3 12 3*3 12 3*5 24 5*5 24
[0086] In the real world and in test samples, positive samples are sparse, resulting in a serious imbalance problem, which leads to unfair model learning and biased results. To enable the model to learn more fully, this invention compares the impact of different losses on the model results, targeting the optimization objective. Equation (2.13) is the cross-entropy function, and Equation (2.14) is the weighted cross-entropy loss function, mainly used to balance the imbalance between positive and negative samples. This loss function is a modification of the standard cross-entropy loss, which can reduce the weight of easily classified samples, allowing the model to focus more on difficult-to-classify samples during training. Although the weighted cross-entropy can control the weight of positive and negative samples, it cannot control the weight of easily classified and difficult-to-classify samples. Therefore, focal loss is used, as shown in Equation (2.15).
[0087] Here, γ is called the focusing parameter, and γ>=0. This allows for both adjustment of the weights of positive and negative samples and control of the weights of samples classified as easy or difficult. (1-p) i ) γ This is called the modulation factor.
[0088]
[0089]
[0090]
Claims
1. An efficient entity segmentation and matching method, characterized in that: The entity records input into e-commerce or academic databases are efficiently matched using three modules: a segmentation module, a preprocessing module, and a matching module. Specifically, the entity records are first input into the segmentation module. The segmentation module first constructs a semantic index block through an offline module. The semantic index block is formed through model training, entity record embedding inference, and semantic index builder. The queried entity records are semantically embedded. At the same time, the entity records are input into the inverted index block after being cleaned by term segmentation. The results of the above two are sent to the semantic index block and the inverted index block respectively for segmentation and filtering of entity records. The results from the semantic index block and the inverted index block are then merged through the preprocessing module, followed by data processing and transformation to obtain the information representation of the entity. Then, it is sent to the matching module to perform redundancy retrieval and category classification on the input entity record pairs, obtaining redundant and similar data entity records in the database. This is used to summarize e-commerce or academic data, avoiding the waste of physical space caused by redundant storage of duplicate product or monograph data, and finally forming the output summary and redundancy-removed e-commerce or academic database structure. The block method employs a fusion of term inverted indexing and vector similarity. Addressing the scarcity of labeled data, it introduces unsupervised contrastive learning to obtain vector representations of entity records. The term inverted indexing method acquires candidates by building a term inverted index on entity records and returns the entity records according to a certain ranking score. Specifically, in measuring the relevance of entity records—that is, the similarity between two entity records—a TF-IDF-modified BM25 algorithm is used to rank the relevance of entity records. Where Q and D represent two entity records, For a specific Term i in entity record Q, in TF-IDF, the importance of a Term increases proportionally to its frequency of occurrence in the entity record, and decreases inversely proportionally to its frequency of occurrence in the entire data table. IDF represents Inverse Document Frequency, and N represents the total number of entity records to be queried. This indicates that the entire database contains entities. The number of entity records in the BM25 formula and It is a modulating factor, where f represents the frequency of q in entity D. The length of entity record D is represented by avgdl, and the average length of all entity records is represented by avgdl. The unsupervised contrastive learning specifically involves: using a pre-trained BERT model as the backbone network for encoding entity records; performing dropout on the same entity record, treating it as a positive sample, and treating other entity records within the same batch as negative samples; and employing a contrastive loss function. Indicates the current sample h i The loss, where This is the positive sample corresponding to this sample. Let N be the number of samples in a batch. As a temperature coefficient, after obtaining the embedded representation of the entity record, the approximate nearest neighbor algorithm is used to quickly find the top-k candidate entity records and merge them with the candidate records of the inverted index for the entire entity record set. The result is then processed and passed to the matching module.
2. The efficient entity segmentation and matching method as described in claim 1, characterized in that: The information representation is achieved using a heterogeneous entity information representation method based on pre-trained models and feature engineering. Specifically, firstly, important relevant features are selected based on feature engineering and shallowly memorized by the model. Then, entities and proper nouns in the records are identified and injected into the pre-trained BERT model. Vector representations of entity records are obtained through the pre-trained model. Finally, the vector representations are optimized through a matching neural network that combines memorization and generalization.
3. The efficient entity segmentation and matching method as described in claim 2, characterized in that: The memory and generalization matching neural network structure is divided into a wide side and a deep side. The input features on the wide side are type words that reinforce the signal, which are learned and memorized shallowly through a linear perceptron layer. The input on the deep side consists of two entity records. A pre-trained BERT entity vector representation layer obtains the vector representations of the entity records, which are then input to the matching layer. The matching layer primarily extracts the matching relationships between tokens, N-grams, and entity granularities. In the fusion layer, information at the token level, N-gram level, and entity record semantic level is fused. Finally, the deep side result is obtained through a linear perceptron layer. The wide side result and the deep side result are then processed using a formula... And through the sigmoid activation function The final matching probability is obtained.
4. The efficient entity segmentation and matching method as described in claim 3, characterized in that: The deep-side model captures information from the term level to the N-Gram level and then to the entity level for two entity records. The matching layer is designed to calculate the matching matrix for term exact match, generalized match, and relevance match for two records. Then, the convolutional neural network is used to extract matrix features. At the same time, semantic information is extracted at the entity level to enhance the overall matching signal. The term precise match is calculated by plotting a matching matrix where words in two entities completely match. The formula is as follows: If the terms are completely equal, the value is 1; otherwise, it is 0. The generalized hit is a matching matrix obtained by calculating the similarity of each token-level vector output by BERT, using the following formula: ; The relevance matching matrix adopts a neural tensor network (NTN) structure, which considers the relationship between any two tokens between entity records as a feature of the matching signal. A scoring model function is used to score it, and the specific scoring function is as follows: ,in It is a non-linear activation function. It is a tensor, bilinear tensor product The result is a vector Each entry consists of a slice of a tensor. calculate: , , The number of relationships is represented by the exact match, generalized match, and relevant match results, which yield a relationship matrix between tokens of entities. The number of matrices corresponds to the number of input channels of the image, and the cell values in the matrix correspond to the pixels of the image. A convolutional neural network is then used to extract the matching signal.
5. The efficient entity segmentation and matching method as described in claim 4, for two entity records... and The matching process is characterized by: The convolutional neural network is specifically designed with a four-layer convolutional kernel structure: the first layer has 12 kernels of size 1×3, the second layer has 12 kernels of size 3×3, the third layer has 24 kernels of size 3×5, and the fourth layer has 24 kernels of size 5×5. The convolution operation between the input matrix and the kernel matrix is called the cross-correlation operation, specifically... The input matrix is I, and the kernel matrix is K, with K having an m×n shape. The element at position (i, j) in the output matrix O is obtained. An odd-numbered convolution kernel is used, and matrix padding is applied to both ends to ensure the output matrix has the same dimension as the input matrix. Finally, the output channels are... Pooling operations are performed on the dimension to obtain matching features between the two entities; In the fusion layer, entity-level semantic matching information, obtained from the pre-trained model's CLS vector and the token sequence vector difference, is incorporated into the final feature. The specific calculation method is: deep_match = [cls_embedding, term_matching, seq_diff]. The fusion result is then passed through a linear perceptron layer to obtain a one-dimensional vector representation. For the optimization objective, focal loss is used as the loss function, specifically: γ is called the focusing parameter, and γ>=0. This is called the modulation coefficient.