A cross-domain data matching method based on a large language model

Abstract

The application discloses a cross-domain data matching method based on a large language model, comprising the following steps: inputting source data and target data, wherein the source data and the target data are respectively from patient information and drug data; extracting entity pairs from the source data and the target data respectively, wherein the entity pairs in the source data have labels, i.e., are marked as matched or unmatched, and the entity pairs in the target data do not have labels; active candidate source data generation; context demonstration selection; injecting domain information of each entity pair into a predefined format, sending a prompt to a large language model for processing, and the large language model returning a result of a specific entity pair according to the received prompt. The application explores the context learning ability of the large language model in performing cross-domain entity resolution. The application improves the performance of the large language model in the cross-domain entity resolution task, and can select more valuable demonstrations and construct higher-quality prompts.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of medical information technology, and particularly relates to a cross-domain data matching method based on a large language model. BACKGROUND

[0002] Entity resolution (ER), also known as entity matching or data matching, is the process of identifying records that represent the same real-world entity, despite differences in how entities are represented or described in different data sources. This process is crucial in data integration and is widely applied in various fields. In the field of medical and health information management, in the scenarios of electronic medical records, clinical trials, and drug research and development, entity matching helps to integrate patient information and drug data, and promotes the rational allocation of medical resources and research collaboration.

[0003] Existing entity resolution methods use deep learning-based methods and pre-trained language models (PLMs) such as BERT to automatically train effective classifiers, which requires a large amount of labeled training data. However, the test data (i.e., target data) and the training data (i.e., source data) are usually from the same domain, i.e., the content of the entity description is similar, and the structure of the entity attribute is the same.

[0004] However, in the field of medical and health information management, source data and target data are usually from different domains. For example, there are significant differences between entity information in the fields of patient information and drug data, not only in the content of entity description, but also in the structure of entity attributes. This is also known as the cross-domain entity resolution task, which has received increasing attention in recent years.

[0005] Cross-domain entity resolution presents many new challenges, the most notable of which is the difference in data distribution between source and target data, i.e., domain shift. To address this issue, in addition to the general entity resolution solution, a feature extractor and matcher are usually used, current cross-domain entity resolution methods also design a feature aligner, which aims to align the distribution of source and target entity pairs, and then update the feature extractor and matcher accordingly, thereby mitigating the impact of domain shift. However, these methods usually require additional computational resources and fine-tuning efforts to achieve effective matching.

[0006] The advent of large language models (LLMs) provides a potential solution that can alleviate the heavy training cost required by the above cross-domain entity resolution methods using neural networks, where users no longer need to focus on the design and training of domain-adaptive models, but instead for large language models (design effective prompts to handle, which is also known as context learning. In fact, large language models (have been used to solve general entity resolution problems. These methods explore various methods to measure the similarity between entity pairs in training and testing data, aiming to select records with the highest similarity to each test record from the training data, which are called demonstrations, will be used to build prompts and help large language models reason.

[0007] However, the above general demonstration selection methods do not work well in cross-domain entity resolution. This is because (1) they only consider the semantic or structural similarity between entity pairs in the source and target domains, while the heterogeneity of cross-domain entity attribute structures makes current structural similarity calculation methods unsuitable. For example, the entity pairs in the source domain contain five attributes, while the entity pairs in the target domain have only four attributes. Current methods will generate five-dimensional structure vectors for entity pairs in source data, while generating four-dimensional structure vectors for entity pairs in target data, making it impossible to calculate structural similarity by multiplication; (2) in the cross-domain setting, not all records in the source data are beneficial for learning and transfer to the target domain, which requires the use of features contained in the data to preliminarily filter the source data. However, existing methods ignore this.

[0008] Entity resolution is a fundamental task in data integration and has been extensively studied for decades. Most of the state-of-the-art methods currently utilize pre-trained language models (PLMs) to extract features from entity pairs, framing entity resolution as a binary classification problem, and then fine-tune the model to improve its performance on entity resolution. Ditto is one of the pioneering models that uses pre-trained models such as BERT, DistilBERT, and RoBERTa in the entity resolution task, achieving significant performance improvements. Since these methods often require a significant amount of time for fine-tuning, Li et al. proposed a method to improve the efficiency of entity resolution based on pre-trained models. In addition, Yao et al. introduced HierGAT, a model based on hierarchical graph attention transformation networks. HierGAT can effectively model and utilize the interdependence between different entity resolution decisions, providing improved interpretability and performance.

[0009] However, all these methods require a large amount of labeled data for training, and they are designed to handle data from the same domain, so it is challenging to apply them directly to cross-domain entity resolution tasks.

[0010] With the development of large language models (LLMs), in-context learning (ICL) has become an emerging learning paradigm and has achieved good performance in many tasks, including entity resolution. Narayan et al. first tried entity resolution for entity resolution and achieved good results through manually selected demonstrations, but this method requires a lot of physical labor and consumes a lot of time. As the feasibility of entity resolution for entity resolution is constantly being proven, Peeters et al. proposed a method to calculate the Jaccard similarity between serialized entity pairs, and select a few most relevant records for demonstration. Fan et al. further designed a structure vector that calculates the correlation between the values of each attribute dimension of the two entities within an entity pair. These methods aim to select more suitable demonstrations through which large language models can grasp the rules of entity resolution and apply them to subsequent unlabeled entity pair judgments. However, these methods cannot be directly applied to cross-domain entity resolution because they ignore the heterogeneity of cross-domain entity attribute structures and the fact that not all instances in the source domain are beneficial to reasoning in the target domain

[0011] The focus of cross-domain entity resolution is to leverage existing labeled data to improve the performance of models in another domain's entity resolution task. Due to the high cost of data labeling and the increasing number of publicly available datasets, the demand for cross-domain entity resolution is becoming more and more common. Current methods use domain adaptation techniques to align the distributions of source and target data and achieve good performance. Among them, Tu et al. first systematically explained how to apply domain adaptation techniques to pre-trained model-based cross-domain entity resolution methods and proposed the DADER framework, which is the latest method to perform cross-domain entity resolution tasks.

[0012] Although these methods can cope with the challenges of cross-domain entity resolution, they often require a large amount of computational resource consumption and manual fine-tuning. Recently, with the excellent performance of entity resolution in general entity resolution tasks and other cross-domain tasks such as knowledge base question answering and information retrieval, the present application attempts to explore the application of entity resolution in cross-domain entity resolution tasks. SUMMARY

[0013] Therefore, the application first explores the context learning ability of large language models in performing cross-domain entity resolution and proposes a new framework, CiDER, which effectively selects useful demonstrations to improve the performance of large language models in handling cross-domain entity resolution. The most significant contribution of CiDER is the first step, i.e., active candidate source data generation, which aims to select data with higher value in the source domain for the large language model to learn using the idea of active learning, forming candidate source data. Then, in the subsequent context-in-demonstration selection module, the application designs a domain-invariant method to calculate the structural similarity between entity pairs in the candidate source data and the target data, and combines it with semantic similarity to select useful demonstrations for each target instance. Finally, the application uses the selected source entity pairs and their label information, the target instance, and the cross-domain information to form the prompt of the large language model, and then obtains the prediction result. A large number of experiments show that CiDER can obtain better performance than existing methods in cross-domain entity resolution.

[0014] To achieve the above-mentioned purpose, the application discloses a cross-domain data matching method based on a large language model, comprising the following steps:

[0015] S1: obtaining source data and target data, the source data and the target data are respectively from patient information and drug data;

[0016] S2: extracting entity pairs from the source data and the target data respectively, the entity pairs in the source data have labels, marked as matched or unmatched, and the entity pairs in the target data do not have labels;

[0017] S3: using an active learning method, selecting a subset of the source data as candidate source data to preliminarily filter the source data for large language model learning; specifically including: selecting records from the source data as candidates according to the uncertainty of each record in the source data and its relevance to the target data;

[0018] S4: changing the range of demonstration selection from the source data to the candidate source data; combining structural and semantic similarity to select more valuable demonstrations;

[0019] S5: injecting domain information of each entity pair into a predefined format, sending the prompt to the large language model for processing, and the large language model returns the result of the specific entity pair according to the received prompt;

[0020] S6: obtaining the label information of the target data entity pairs and outputting.

[0021] Preferably, the labeled source data is represented as , wherein is the set of all entity pairs in the source data, is the source data entity pair, is the label set of all entity pairs; the target data without label is represented as wherein is the set of all entity pairs in the source data, is the entity pair in the target data, is the label information to be judged.

[0022] Preferably, step S3 comprises: using uncertainty sampling to select candidate source data, and the strategy of the uncertainty sampling is entropy, and the calculation formula of the entropy is as follows:

[0023] ;

[0024] wherein is the semantic representation vector of the entity pair , is the probability of the match of the two entities in the record, is the probability of the mismatch of the two entities in the record, and ;

[0025] The matching probability of each entity pair is calculated , so as to calculate the uncertainty of the entity pair;

[0026] Convert the entity pair into a semantic vector representation using a pre-trained language model, and use SBERT to generate entity pair embedding;

[0027] Calculate the relevance to the target data, that is, calculate the distance between the representation vector of the entity pair in the source data and the average representation vector of all entity pairs in the target data, to obtain the value of each record The value of the large language model in performing the cross-domain entity resolution task on the target data;

[0028] According to the calculated value, all records in the source data set are sorted, and a plurality of records with the highest value are selected as candidate source data.

[0029] Preferably, the matching probability of each entity pair is calculated , so as to calculate the uncertainty of the entity pair, comprising:

[0030] A probability classifier is constructed to calculate the matching probability of the entity pair: the source data is evenly divided into folds; for the records in the th fold , the classifier is constructed using the entity pairs and their label pair information in the other folds; let judge the matching probability of the entity pair in the th fold.The compromise entity pair, so as to obtain the matching probability; thus, repeat iteration K rounds, obtain the matching probability of all entity pairs in all source data;

[0031] When calculating the matching probability of the entity pair using the Naive Bayes classifier, the calculation is based on the Bayes theorem and the feature condition independent assumption. Specifically, according to the feature values of each dimension in the vector representation of the entity pair, the conditional probability of these feature values under the given matching label is calculated. Then, the conditional probability and the Bayes theorem are used to calculate the probability of the entity pair matching. The specific calculation method is as follows:

[0032] ;

[0033] Wherein, is 0 or 1, 0 represents not matching, and 1 represents matching, is the semantic vector representation of the entity pair in the lth dimension in the source data, represents dimension, is the prior probability, which is calculated according to the training data.

[0034] Preferably, for a given entity pair , first, the attribute value pair of each entity is serialized into a token sequence, represented as:

[0035] ;

[0036] respectively represent the kth attribute and attribute value;

[0037] The serialized token sequence of the two entities is connected with a special mark [SEP] to form a sequence of the entity pair. The purpose of using [SEP] is to help the model distinguish different entity information; and the entity pair (a, b) is converted into the following sequence:

[0038] ;

[0039] Finally, the SBERT encodes the sequence to convert the entity pair into a semantic vector representation:

[0040] ;

[0041] represents the encoder.

[0042] Preferably, given the source data and the entity pair in the target data , for , define each record The value of the large language model in performing the cross-domain entity resolution task on the target data is:

[0043] ;

[0044] wherein is the representation vector of the entity pair , is the average semantic representation vector of all entity pairs in the target data, is the distance function, is a hyperparameter, and when the value is equivalent to the entropy value.

[0045] Preferably, the S4 step aims to select a plurality of records from the candidate source data as demonstrations for each data record in , and then forward them to the prompt generation module, with the goal of finding the most useful examples from the candidate source data compared to randomly sampled demonstrations from the same pool to improve the performance of the large language model on the target data;

[0046] Generate domain-invariant structure vectors in cross-domain environments. Specifically, for a given entity pair , calculate the similarity under each attribute respectively, and take the average as the similarity value:

[0047] ;

[0048] wherein n is the number of attributes contained in the entity pair , denotes the indicators used to calculate the structural similarity, including three metrics;

[0049] According to , a three-dimensional structure feature vector is constructed for each entity pair, wherein the three dimensions correspond to each of the three metrics:

[0050] ;

[0051] In this way, even if different entity pairs contain different numbers of attributes, the constructed structure vectors have the same dimension; the structural similarity between two entity pairs is the cosine similarity between two structure feature vectors; the final similarity between two entity pairs is calculated by weighting the sum of semantic similarity and structural similarity, i.e.

[0052] ;

[0053] wherein is a hyperparameter; in this way, for a certain entity pair with a judgment in , the calculation is The similarity of each record with it, and the record with the highest similarity is selected The record.

[0054] Preferably, Φ includes the following three metrics:

[0055] Jaccard similarity, defined as the ratio of the number of elements in the intersection of two sets to the number of elements in the concatenation of the two sets;

[0056] Levenstein distance, the minimum number of operations required to convert one string to another by insertion, deletion and replacement operations;

[0057] Jaro-Winkler distance, which calculates the number and order of matching characters between two strings, and then improves the score by adding a prefix ratio factor to the prefix.

[0058] Preferably, after obtaining each target entity pair demonstration, it is organized according to a predefined format, and a piece of text, i.e. prompt, is generated and provided to the large language model to generate the result; specifically, for m demonstration records, the domain information of each entity pair is injected into the task description, and the task description is provided to describe the overall requirements and background of the task, so that the large language model can clearly understand the task goal; then, the demonstration part provides a series of specific entity pairs and their label information, and through learning the potential patterns in these data, the large language model deepens the understanding of the task; finally, the task input part contains the specific information of the entity pairs to be judged, and the large language model judges them according to the knowledge learned from the given information.

[0059] The beneficial effects of the present application are as follows:

[0060] Using the context learning strategy and the domain-invariant similarity calculation method, the demonstration selection of cross-domain entity resolution is improved. BRIEF DESCRIPTION OF DRAWINGS

[0061] Figure 1 The framework diagram of the present application. DETAILED DESCRIPTION

[0062] The present application will be further described below in conjunction with the drawings, but in no way limits the present application, any transformation or replacement based on the teaching of the present application belongs to the protection scope of the present application.

[0063] Artificial intelligence is the theory, method, technology and application system of using digital computer or digital computer controlled machine to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0064] Natural language processing (NLP) is an important direction in the field of computer science and artificial intelligence. It studies various theories and methods that can realize effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science and mathematics. Therefore, the research in this field will involve natural language, i.e. the language used in daily life, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, robot question and answer, knowledge graph and other technologies.

[0065] The technical scheme provided by the embodiments of the present application relates to machine learning and natural language processing of artificial intelligence, and is specifically introduced and explained by the following embodiments.

[0066] The input of the cross-domain entity resolution problem includes source data and target data, and both come from different fields. The entity pairs in the source data are labeled, that is, marked as matching or not matching. The labeled source data is represented as , wherein is the set of all entity pairs in the source data set. is the set of labels of all entity pairs. The unlabeled target data is represented as , wherein is the set of all entity pairs in the source data set. is the label information to be judged.

[0067] Cross-domain entity resolution is a process of obtaining the label information of the target data entity pair using the given source data .

[0068] With the development of large language models, in-context learning (ICL) has become an emerging learning paradigm. Entity resolution aims to better release the potential of large language models by constructing high-quality prompts without adjusting the internal structure of large language models. The key idea of entity resolution is to learn from analogy, which connects the question to be queried and the selected demonstration (i.e., example) and provides it to the large language model to obtain the inference result.

[0069] Prior art finds that using entity resolution can help large language models perform entity resolution tasks. These methods select demonstrations by computing semantic similarity or structural similarity between entity pairs. However, due to different data distributions and different attribute structures, these methods do not show good results in cross-domain scenarios. In addition, these methods ignore the information contained in the data itself when selecting demonstrations. Therefore, the present application aims to design a method to mitigate the impact of the above problems and enable large language models to better perform cross-domain entity resolution tasks.

[0070] Prior art finds that using entity resolution can help large language models perform entity resolution tasks. These methods select demonstrations by computing semantic similarity or structural similarity between entity pairs. However, due to different data distributions and different attribute structures, these methods do not show good results in cross-domain scenarios. In addition, these methods ignore the information contained in the data itself when selecting demonstrations. Therefore, the present application aims to design a method to mitigate the impact of the above problems and enable large language models to better perform cross-domain entity resolution tasks. Figure 1 The present application provides an overview of the framework CiDER proposed by the present application. CiDER consists of three key modules: active candidate source data generation, context demonstration selection, and prompt generation.

[0071] In the active candidate source data generation module, the present application uses the concept of active learning to select a subset of labeled source data as candidate source data, which will be used in the subsequent context demonstration selection module. Candidate source data generation can be regarded as a preliminary filtering of source data, aiming to select records with higher values for large language models to learn. To achieve this goal, the present application considers the inherent uncertainty of each record in the source data and its association with the target data, thereby selecting useful records from the source data as candidates.

[0072] In the context demonstration selection module, the present application changes the range of demonstration selection from source data to candidate source data. The present application redefines the structural similarity between entity pairs to address the challenges posed by the heterogeneity of data structures in different domains. The present application also fully considers the similarity of structure and semantics and combines them to select more valuable demonstrations. For a certain entity pair to be judged in the target data, the present application uses this method to select some records in the candidate source data as demonstrations.

[0073] In the prompt generation module, the domain information of each entity pair is injected into the existing predefined format to help the large language model better understand the task background. The prompt is sent to the large language model for processing, and the large language model returns the results of the specific entity pair according to the received prompt.

[0074] CiDER consists of three key modules: active candidate source data generation, context demonstration selection, and prompt generation. In this section, we will introduce each of the three modules in more detail. Active candidate source data generation.

[0075] In cross-domain learning tasks, it is generally believed that not all source data can help improve the model's performance in the target domain, for example, some records may be noisy or even contain harmful knowledge. Therefore, the goal of this stage is to refine the source data by utilizing the general information of the source data and the target data, and to retain the data that is beneficial for predicting the target data, i.e., the candidate source data. The candidate source data is then forwarded to the context demonstration selection module to help select more beneficial samples. Compared with direct demonstration selection using all source data, the refined candidate source data ensures the quality of the selected demonstrations, thereby improving the overall performance.

[0076] The generation of candidate source data draws on the idea of active learning (AL), which aims to select the most helpful data for training and labeling by a labeler, thereby improving performance under a fixed budget. This application formulates the selection of candidate demonstrations in the context as a single-iteration active learning problem. Specifically, we use uncertainty sampling to select candidate source data, which is a classic query strategy in active learning that aims to improve performance by identifying the most uncertain samples in the model.

[0077] Common strategies for uncertainty sampling are uncertainty degree sampling, margin sampling, and entropy. We use entropy to measure the uncertainty of a record (a, b) in the source data. The greater the entropy, the greater the uncertainty of the record. The formula for calculating entropy is as follows

[0078] ;

[0079] where is the semantic representation vector of the entity pair , which will be described in detail in the following. is the probability of matching of the two entities in the record, is the probability of not matching of the two entities in the record, and .

[0080] Next, we need to calculate the matching probability of each entity pair This allows for the calculation of the uncertainty of entity pairs. CiDER attempts to construct a probabilistic classifier. This is used to calculate the matching probability of entity pairs. First, CiDER divides the source data into equal parts. Fold. Then, for the first one... fold In the records, CiDER uses other A compromise between entity pairs and their label pairs is used to construct a classifier. Next, CiDER let judge By finding suitable entity pairs, CiDER obtains the matching probability. This process is repeated K times until CiDER acquires the matching probability for all entity pairs across all source data. It's important to note that... The parameters are initialized at the beginning of each iteration. Classifier It is necessary to output the probability of entity pairs matching or not matching. Common choices include Naive Bayes classifier, logistic regression, support vector machine, etc.

[0081] CiDER uses a Naive Bayes classifier. When CiDER uses it to calculate the matching probability of entity pairs, it bases the calculation on Bayes' theorem and the feature conditional independence assumption. Specifically, CiDER calculates the conditional probabilities of the feature values ​​in each dimension of the entity pair's vector representation, given a matching label. Then, CiDER uses these conditional probabilities and Bayes' theorem to calculate the probability of an entity pair matching. The specific calculation method is as follows:

[0082] ;

[0083] Where 0 indicates no match and 1 indicates a match. Entity pairs in the source data semantic vector representation, express Dimensions. and These are prior probabilities, all calculated based on training data.

[0084] Next, this application will discuss how to convert entity pairs. The approach involves using d-dimensional vector-based representations, i.e., semantic features, from pre-trained language models such as BERT and SBERT. In this work, we use SBERT to generate entity pair embeddings.

[0085] For a given pair of entities First, serialize the attribute value pairs of each entity into a token sequence, represented as:

[0086] ;

[0087] Next, the serialized token sequence of two entities is connected with a special token [SEP] to form the sequence of entity pairs. The purpose of using [SEP] is to help the model distinguish different entity information. And the entity pair (a, b) is converted into the following sequence

[0088] ;

[0089] Finally, SBERT encodes the sequence to convert the entity pair into a semantic vector representation:

[0090] ;

[0091] Generally, the sample with the highest uncertainty is considered to have higher value. However, in the cross-domain setting, the present application also considers the correlation between the source data and the target data, because the information instances in the source domain are not necessarily relevant to the target domain.

[0092] Specifically, in addition to the uncertainty, the present application also considers the distance between the representation vector of the entity pair in the source data and the average representation vector of all entity pairs in the target data, where a larger distance results in a lower value. Formally, given the source data and the entity pair in the target data , the present application defines the value of each record in the source data to the cross-domain entity resolution task performed by the large language model on the target data as:

[0093] ;

[0094] where is the representation vector of the entity pair , is the average semantic representation vector of all entity pairs in the target data, is a distance function, such as Euclidean distance, cosine distance, etc. is a hyperparameter, and when , the value is equivalent to the entropy value.

[0095] Finally, CiDER sorts all records in the source data set according to the calculated value, and selects a number of records with the highest value as candidate source data. After determining the size of the candidate source data, CiDER can generate candidate source data using the present method and use it in the following modules.

[0096] The context demonstration selection module aims to select multiple records from the candidate source data as For each data record, a demonstration is generated, and then forwarded to the prompt generation module. The goal is to find the most useful examples from the candidate source data that are expected to improve the performance of the large language model on the target data compared to randomly sampled demonstrations from the same pool.

[0097] It should be noted that since the source data and the target data come from different domains, the selection of the demonstration is crucial. By selecting samples similar to those in the target domain, while being able to reveal the rules and patterns of the ER task, the large language model can better understand the task and thus transfer this knowledge to the target domain.

[0098] Existing methods mainly use semantic similarity or structural similarity between two entity pairs to represent their similarity. Semantic similarity can be easily obtained by calculating the cosine similarity between the two semantic vectors of the entity pair. Structural similarity mainly reflects the structural difference between the two entity pairs, where the structural feature vector of the entity pair is constructed by considering the difference in the value of each attribute dimension between the two entities in the entity pair. However, this representation of structural information cannot be directly transferred to the cross-domain scenario, as the number of attributes in the source domain and the target domain may not be the same, and the structural vectors may have different dimensions and cannot be directly compared.

[0099] As an improvement, the present application designs a new method to generate domain-invariant structural vectors in a cross-domain environment. Specifically, for a given entity pair , the similarity under each attribute is calculated respectively, and the average value is taken as the similarity value:

[0100] ;

[0101] where n is the number of attributes contained in the entity pair , and represents the index used to calculate the structural similarity.

[0102] The present application uses the following indexes to realize Φ:

[0103] Jaccard similarity, defined as the ratio of the number of elements in the intersection of two sets to the number of elements in the concatenation of the two sets, is a statistical measure used to measure the degree of similarity between two sets.

[0104] Levenstein distance, a measure used to measure the degree of difference between two strings. It represents the minimum number of operations required to convert one string to another through insertion, deletion, and replacement operations.

[0105] Jaro-Winkler distance, which calculates the number and order of matching characters between two strings, and then improves the score by adding a prefix ratio factor for common prefixes.

[0106] Based on these three metrics, this application constructs a three-dimensional structural feature vector for each entity pair. The three dimensions correspond to each of the three metrics mentioned above:

[0107] ;

[0108] Thus, even if different entity pairs contain different numbers of attributes, the constructed structure vectors will all have the same dimension. The structural similarity between two entity pairs is the cosine similarity between the two structural feature vectors. The final similarity between two entity pairs is calculated by weighting the sum of semantic similarity and structural similarity, i.e.

[0109] ;

[0110] in It's a hyperparameter.

[0111] Thus, for A certain entity pair with a condition can be calculated. The similarity score between each record and the given record is calculated, and the record with the highest similarity score is selected. This record, as a demonstration, It can be adjusted according to specific needs.

[0112] After obtaining the demonstrations for each target entity pair, we organize them according to the predefined format and generate a piece of text, i.e., the prompt, which is then provided to the large language model to generate the result. Since the prompt design is not the focus of this paper, we directly follow the previous entity resolution solution based on large language models (Ralph Peeters and Christian Bizer. 2023. Entity Matching using Large Language Models. CoRR abs / 2310.11244 (2023). https: / / doi.org / 10.48550 / ARXIV.2310.11244arXiv:2310.11244). Specifically, for m demonstration records, we first provide a task description to describe the overall requirements and background of the task, enabling the large language model to clearly understand the task goal. It is worth noting that we inject the domain information of each entity pair into the task description to help the large language model better understand the background of the task. Then, the demonstration part provides a series of specific entity pairs and their label information. By learning the potential patterns in these data, the large language model deepens its understanding of the task. Finally, the task input part contains the specific information of the entity pairs to be judged, and the large language model will make judgments based on the knowledge learned from the given information.

[0113] In this section, we will evaluate the performance of the framework CiDER proposed in this paper. We first describe the experimental setup in detail, then conduct experiments, and focus on in-depth analysis of the following key issues:

[0114] Question 1: How effective is CiDER compared to deep learning-based and other large language model-based methods?

[0115] Question 2: In the candidate source data generation module, we select a subset of source data as candidate source data. Compared to the records in the source data, are the records in the candidate source data more helpful for the large language model to make decisions on the entity pairs in the target data?

[0116] Question 3: In the context demonstration selection module, we designed a new method to calculate the similarity between entity pairs. Can the demonstrations selected by this method better help the large language model make decisions on the records in the target data?

[0117] Question 4: In the prompt generation module, we inject the domain information of each entity pair into the task description. Is this effective?

[0118] The application uses Shenzhen chest X-ray set (2D X-Ray, 662 cases, 2 categories of pulmonary tuberculosis classification) and Abdomen Atlas 1.0 Mini abdominal multi-organ segmentation (3D CT, 5195 cases, 9 categories of abdominal organ segmentation) as the data set.

[0119] It is worth noting that although the label information of the target data is known, the application will not use it when the method being evaluated performs the cross-domain entity resolution task, and the application will only use the label information of the target data when evaluating the performance of the final method. In the following, in order to simplify the representation, the application can use the symbol , for example, RI→ WA, represents that the DS is the source data and the DT is the target data,

[0120] The application uses the F1 score to measure the performance of each method. The F1 score is the harmonic mean of accuracy and recall. The accuracy is calculated as , and TP represents true positive and FP represents false positive. The recall is calculated as , and FN represents false negative. The F1 score is calculated as , and it is noted that in this paper, the F1 score is expressed in percentage.

[0121] In order to comprehensively evaluate the performance of the method CiDER proposed by the application, the application considers two types of baselines. The first is a pre-trained model-based method, including the classic entity resolution method NoDA and DADER, which is the most advanced cross-domain entity resolution solution. The other type includes general entity resolution research based on large language models, which use entity resolution to perform entity resolution tasks. The main difference between these methods is the implementation of the demonstration selection. In the following, the application briefly describes the comparison methods:

[0122] (1) NoDA, a classic pre-trained model-based entity resolution method, does not consider the domain shift problem when dealing with cross-domain settings;

[0123] (2) DADER, the most advanced pre-trained model-based cross-domain entity resolution method. DADER consists of three modules, namely feature extractor, matcher and feature aligner. By aligning the distribution of source and target entity pairs, DADER effectively solves the domain shift problem in cross-domain scenarios;

[0124] (3) ZS, representing zero-shot prompting. Zero-shot prompting is widely used due to its strong understanding ability and rich prior knowledge. This method only constructs prompts based on entity pairs in the target data, without containing any information from the source data or other additional information;

[0125] (4) SDR-S, which randomly selects demonstrations from source data. This is a common approach for entity resolution

[31] ;

[0126] (5) KATE, a well-recognized entity resolution method, which intuitively selects demonstrations from source data based on semantic similarity;

[0127] (6) JAC, which treats entity pairs as a sequence and selects demonstrations by computing Jaccard similarity between entity pairs.

[0128] Regarding CiDER, the size of the candidate source data is fixed to 50, which balances between data quality and data quantity. The best 𝛾 is searched in [0.001, 0.005, 0.01, 0.1] and the best 𝛼 is searched in [0.1, 0.3, 0.5, 0.7, 0.9]. 10% of the target data is used as the validation set and 90% of the target data is used as the test set. For a fair comparison, two demonstrations are selected for each large language model-based method. GPT-3.5-Turbo-1106 is used, where 1106 means the model version is finalized on November 6. All experiments are repeated three times to obtain the average value of the results. The code of the four large language model-based baselines is implemented in the same environment. All experiments are conducted on Pytorch 2.0.1 and GeForce GTX 1660Ti (16GB RAM).

[0129] Table 1 shows the performance of all baseline methods and the proposed method CiDER. It is clear that CiDER achieves the best overall performance. Specifically, by comparing the performance of CiDER and other deep learning-based methods on different data sets, we find that CiDER achieves the best performance on most data set pairs. This proves the feasibility of large language models in solving cross-domain ER. By comparing the performance of CiDER and other large language model-based methods on different data sets, it is not difficult to see that CiDER performs better on each pair of data sets. This result proves that the demonstrations selected by CiDER are more conducive to the understanding of the task by the large language model and the reasoning of the target data compared to the demonstrations selected by other large language model-based methods.

[0130] Table 1 Main experimental results: f1 scores of CiDER and each large language model on data sets:

[0131] ;

[0132] By comparing the performance of CiDER and DADER, it can be found that CiDER performs better than DADER on most of the datasets. The overall performance of CiDER is better than DADER, and the average F1 score on all datasets is about 5.5% higher than DADER. Moreover, the overall performance of CiDER is also better than DADER on two types of datasets, and the average F1 score is increased by 5.7% and 4.5% respectively compared with DADER. This is mainly due to the strong semantic understanding and domain generalization ability of large language models, and the prompts constructed by CiDER maximize the potential of large language models. In addition, large language models also show excellent domain generalization ability in other application scenarios.

[0133] The present application explores the context learning ability of large language models in performing cross-domain entity resolution. The present application finds that although the source data and the target data come from different domains, the records in the source data still help the large language model to make decisions on the entity pairs in the target data. The present application also proposes a new framework CiDER, which greatly improves the performance of large language models in the cross-domain entity resolution task. CiDER consists of three modules, namely active candidate source data generation, context demonstration selection and prompt generation. Through these three modules, CiDER can select more valuable demonstrations and construct higher quality prompts, thereby releasing the power of large language models in the target domain.

[0134] The word "preferred" is used herein as a term of art to denote a choice that, while doing so, offers significant benefits, when viewed in the context of possible alternatives. Any aspect or design described herein as "preferred" is not necessarily to be construed as being more advantageous than other aspects or designs. The use of the word "preferred" is intended to present a concept in a particular way. The term "or" is intended to mean an inclusive "or" rather than an exclusive "or" unless specifically indicated otherwise or clear from context. That is, unless specified otherwise, or clear from context, "X employs A or B" is intended to mean any of the natural inclusive permutations. That is, if X employs A; X employs B; or X employs both A and B, then "X employs A or B" is satisfied under any of the foregoing instances.

[0135] Moreover, although the present disclosure has been illustrated and described with respect to one or more implementations, equivalent alterations and modifications will occur to others skilled in the art based on the foregoing description and accompanying drawings. The present disclosure includes all such modifications and alterations and is limited only by the scope of the following claims. In particular regard to the various functions performed by the above described components (e.g., elements, engines, modules, etc.), the terms used to describe such components are intended to correspond, where appropriate, to any component which performs the specified function (e.g., that is functionally equivalent), even though not structurally equivalent to the disclosed structure which performs the function in the herein illustrated exemplary implementations of the present disclosure. In addition, while a particular feature of the disclosure can have been disclosed with respect to only one of several implementations, such feature can be combined with one or other features of the other implementations as can be desired and advantageous for any given or particular application. Furthermore, to the extent that the terms "including", "includes", "having", "has", "contain", "contains", or variants thereof to be afforded similar meanings in the context of describing common ownership in the background portion of the detailed description are used, such terms are intended to be inclusive in a manner similar to the term "comprising" as comparable terms under the doctrine of equivalents in the field of patent law.

[0136] The various functional units in the embodiments of the present application can be integrated in one processing module, or each unit can exist physically, or a plurality of or more units can be integrated in one module. The integrated module can be realized in the form of hardware, or in the form of a software functional module. When the integrated module is realized in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer readable storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc. The above-mentioned devices or systems can execute the storage method in the corresponding method embodiments.

[0137] In summary, the above-mentioned embodiments are one embodiment of the present application, but the embodiments of the present application are not limited by the above-mentioned embodiments, and any changes, modifications, substitutions, combinations and simplifications made without departing from the spirit and principle of the present application are equivalent replacement methods and are included in the protection scope of the present application.

Claims

1. A method for cross-domain data matching based on a large language model, characterized in that, The method comprises the following steps: S1: obtaining source data and target data, the source data and the target data being from patient information and drug data respectively; S2: extracting entity pairs from the source data and the target data respectively, the entity pairs in the source data being labeled as matched or unmatched, and the entity pairs in the target data not being labeled; S3: using an active learning method to select a subset of the labeled source data as candidate source data, and preliminarily filtering the source data for learning by a large language model; Specifically, according to the uncertainty of each record in the source data and the relevance of the record to the target data, a record is selected from the source data as a candidate; S4: changing the demonstration selection range from the source data to the candidate source data; and combining the structural and semantic similarity to select more valuable demonstrations; S5: injecting the domain information of each entity pair into a predefined format, and sending a prompt to the large language model for processing, and the large language model returns the result of the specific entity pair according to the received prompt; S6: obtaining the label information of the target data entity pair and outputting; In step S3, the candidate source data is selected by using uncertainty sampling, and the strategy of the uncertainty sampling is entropy, and the calculation formula of the entropy is as follows: wherein is a semantic representation vector of the pair of entities , is a probability of a match of the two entities in the record, is a probability of a non-match of the two entities in the record, and ; calculating a probability of match for each pair of entities thereby calculating an uncertainty of the pair of entities; Transformed entity pairs Into d- semantic vector representation using pre-trained language model, using SBERT to generate entity pair embedding; calculating the relevance to the target data, i.e. calculating the distance between the representation vector of the entity pair in the source data and the average representation vector of all entity pairs in the target data, obtaining each record in the source data the value of the large language model in performing the cross-domain entity resolution task on the target data; According to the calculated value, all records in the source data set are sorted, and a plurality of records with the highest value are selected as candidate source data; the computing the match probability for each pair of entities thereby computing an uncertainty of the pair of entities, comprising: constructing a probabilistic classifier , for calculating the matching probability of entity pairs: dividing the source data evenly into folds; for the records in the th fold , using the entity pairs and their label pairs in the other folds to construct a classifier ; let the th fold judge the entity pairs in the th fold, thus obtaining the matching probability; repeating the iteration K rounds in this way to obtain the matching probability of all entity pairs in all source data; When the matching probability of the entity pair is calculated by using the Naive Bayes classifier, the calculation is based on the Bayes theorem and the feature conditional independence assumption, specifically, according to the feature values of each dimension in the vector representation of the entity pair, the conditional probability of these feature values under a given matching label is calculated; then, the conditional probability and the Bayes theorem are used to calculate the probability of matching the entity pair; the specific calculation method is as follows: wherein, is 0 or 1, 0 means no match, 1 means match, is the semantic vector representation of the pair of entities in the l-th dimension in the source data, is the dimension of the semantic vector representation of the pair of entities in the l-th dimension, represents is the dimension of the semantic vector representation of the pair of entities in the l-th dimension, is the prior probability, which is calculated according to the training data.

2. The cross-domain data matching method based on a large language model according to claim 1, characterized in that, The labeled source data is represented as , wherein is a set of all entity pairs in the source data, is a source data entity pair, is a label set of all entity pairs; and the unlabeled target data is represented as , wherein is a set of all entity pairs in the source data, is a target data entity pair, is label information to be judged.

3. The cross-domain data matching method based on a large language model according to claim 2, characterized in that, For a given pair of entities , the property value pairs of each entity are first serialized into a token sequence, denoted as: respectively represent the kth attribute and attribute value; The serialized token sequence of the two entities is connected with a special mark [SEP] to form a sequence of the entity pair; The purpose of using [SEP] is to help the model distinguish different entity information; and the entity pair (a, b) is converted into the following sequence: Finally, the SBERT encodes the sequence to convert the entity pair into a semantic vector representation: represents an encoder.

4. The cross-domain data matching method based on a large language model according to claim 3, characterized in that, Given source data and target data , define pairs of entities in the source data , and the target data , the value of large language models in performing cross-domain entity resolution tasks on target data is: where is the representation vector for the entity pair , is the average semantic representation vector for all entity pairs in the target data, is the distance function, is a hyperparameter, and when the value is equivalent to the entropy value.

5. The method of claim 4, wherein the method is performed by a large language model-based cross-domain data matching system. Step S4 aims to extract candidate source data. Select multiple records as The demonstration of each data record is then forwarded to the prompt generation module. The goal is to find the most useful example from the candidate source data, compared with demonstrations randomly sampled from the same pool, in order to improve the performance of the large language model on the target data. Generating a domain-invariant structure vector in a cross-domain environment, specifically, for a given pair of entities , the similarity under each attribute is calculated respectively, and the average value is taken as the similarity value: where n is the number of entity pairs the number of attributes included, denotes an index for computing the structural similarity, including three metrics; According to a three-dimensional structure feature vector is constructed for each entity pair wherein three dimensions correspond to each of the three measures: In this way, even if different entity pairs contain different numbers of attributes, the structure vector constructed has the same dimension; the structural similarity between the two entity pairs is the cosine similarity between the two structure feature vectors; and the final similarity between the two entity pairs is calculated by weighting the sum of the semantic similarity and the structural similarity, that is, wherein is a hyper-parameter; thus, for for each band judgment entity pair in the similarity between each record in and the highest similarity 6. The method of claim 5, wherein the method is performed by a large language model-based cross-domain data matching system. Φ includes the following three metrics: Jaccard similarity, defined as the ratio of the number of elements in the intersection of two sets to the number of elements in the concatenation of the two sets; Levenstein distance, the minimum number of operations required to convert one string into another by inserting, deleting and replacing operations; Jaro-Winkler distance, which calculates the number and order of matching characters between two strings, and then improves the score by adding a prefix ratio factor to the prefix.

7. The cross-domain data matching method based on a large language model according to claim 6, characterized in that, After obtaining the demonstrations of each target entity pair, they are organized according to a predefined format, and a piece of text, i.e., a prompt, is generated and provided to the large language model to generate the result; specifically, for m demonstration records, the domain information of each entity pair is injected into the task description, and the task description is provided to describe the overall requirements and background of the task, so that the large language model can clearly understand the task goal; then, the demonstration part provides a series of specific entity pairs and their label information, and through learning the potential patterns in these data, the large language model deepens the understanding of the task; finally, the task input part contains the specific information of the entity pairs to be judged, and the large language model judges them according to the knowledge learned from the given information.

Images