Cross-domain entity identification method and device for entity linking, equipment and medium

Abstract

The invention is suitable for the technical field of big data mining, and particularly relates to a cross-domain entity identification method and device for entity linking, equipment and a medium. According to the method, pre-training is carried out on embedded representation of source data and target data to obtain a representation model, a supervised training sample and a self-supervised training sample are constructed in a data expansion and enhancement mode, and supervised training and self-supervised training are respectively used for carrying out optimization training on the representation model. Therefore, an accurate expression model is provided for data identification of the target domain, the method can be applied to entity identification of data of multiple different domains, and the self-adaptive capacity of entity linking tasks is improved.

Description

Technical Field

[0001] The present application is applicable to the field of big data mining technology, and in particular, relates to a cross-domain entity recognition method, device, equipment and medium for entity linking. Background Art

[0002] With the continuous growth of data scale and the widespread promotion of machine learning applications, data deduplication (Entity Resolution, ER) has become an indispensable part of data processing, information integration and data cleaning. Especially when facing large-scale data sets, the deduplication task becomes particularly complex and the computational overhead is huge. In order to improve data processing efficiency, it is necessary to rely on entity resolution to cluster and classify data. Traditional entity resolution methods rely on entity linking (Entity Linking, EL), which reduces the number of entity tuple pairs that need to be matched later by dividing the data set into several smaller blocks. However, the existing entity linking technology still has the following challenges: 1. Insufficient matching recall rate: When performing entity linking, how to ensure that the tuple pairs after block division can cover all possible matching items? Therefore, a high recall rate is the key to ensuring the integrity of entity resolution. If there are too many missing matching tuple pairs, it will affect the subsequent entity matching steps and even the overall resolution accuracy. 2. Low computational efficiency: Existing entity linking usually relies on Cartesian product operations or predefined rules for matching. When the data set is huge, the computational overhead is extremely high. Especially for application scenarios that need to process data sets of tens of millions or more, how to efficiently screen candidate tuple pairs to avoid unnecessary calculations has become an urgent problem to be solved. 3. Poor domain adaptability: Existing entity linking focuses on optimizing datasets in specific fields and lacks cross-domain adaptability. Especially when integrating information from data sources in different fields, the general entity linking method is difficult to achieve its due effect due to the large differences in data distribution across fields.

[0003] With the rise of neural network-based representation learning, the research direction of entity linking has made significant progress. In this method, all tuples are converted into dense embedding representations, and the top K most similar tuples of each tuple are retrieved through K-Nearest Neighbor (KNN), where K is a predefined hyperparameter. The embedding representation relies on the representation model, which is usually implemented through a Transformer encoder, such as a BERT-based model, to convert tuples into vectors. In addition, a specific loss function, such as Contrastive Loss (CL), is used to bring the embedding representations of tuples pointing to the same entity closer and push the embedding representations of mismatched tuples farther away. Finally, after the tuples are converted into dense semantic vectors, efficient vector retrieval tools (such as Faiss retrieval tools) can be used to find the top K most similar results as candidate sets.

[0004] Due to the lack of manual annotation, existing entity linking models (such as DeepBlocker and Sudowoodo) automatically generate training tuple pairs through data augmentation technology, where positive samples are generated by randomly inserting, deleting or replacing the tags of a single tuple. However, the above models cannot accurately output results for cross-domain data and cannot adapt to the domain.

[0005] Therefore, how to accurately perform cross-domain entity recognition on cross-domain data in entity linking tasks, thereby improving the domain adaptation performance of entity linking tasks, becomes an urgent problem to be solved. Summary of the invention

[0006] In view of this, the embodiments of the present application provide a cross-domain entity recognition method, apparatus, device and medium for entity linking to solve the problem of how to accurately perform cross-domain entity recognition on cross-domain data in entity linking tasks, thereby improving the domain adaptation performance of entity linking tasks.

[0007] In a first aspect, an embodiment of the present application provides a cross-domain entity recognition method for entity linking, the cross-domain entity recognition method comprising: Acquire source data in a source domain and target data in a target domain, embed the source data and the target data using an initial representation model to obtain a source embedding representation and a target embedding representation, perform adversarial learning training on the representation model with the goal of aligning the source embedding representation and the target embedding representation, and obtain a pre-trained representation model; In the source domain, performing attribute extension on the source data to obtain source extended data corresponding to the source data, and associating the source extended data with a label corresponding to the source data to obtain a first supervised training sample; In the target domain, the target data is attribute-extended to obtain target extended data corresponding to the target data, the target extended data is enhanced to obtain enhanced target data, pseudo-labels are constructed for the enhanced target data to obtain corresponding pseudo-labels, and the enhanced target data is associated with the corresponding pseudo-labels to obtain a second supervised training sample; Using the first supervised training sample and the second supervised training sample, performing supervised contrastive learning training on the pre-trained representation model, and using the enhanced target data, performing self-supervised contrastive learning training on the pre-trained representation model to obtain a trained representation model; The trained representation model is used to embed the data to be identified in the target domain to obtain an embedded expression to be identified, and entity linking is performed on the data to be identified based on the embedded expression to be identified.

[0008] In a second aspect, an embodiment of the present application provides a cross-domain entity recognition device for entity linking, the cross-domain entity recognition device comprising: An alignment training module is used to obtain source data in a source domain and target data in a target domain, embed the source data and the target data using an initial representation model to obtain a source embedding representation and a target embedding representation, perform adversarial learning training on the representation model with the goal of aligning the source embedding representation and the target embedding representation, and obtain a pre-trained representation model; A source data extension module is used to perform attribute extension on the source data in the source domain to obtain source extension data corresponding to the source data, and associate the source extension data with a label corresponding to the source data to obtain a first supervised training sample; A target data expansion module is used to perform attribute expansion on the target data in the target domain to obtain target expansion data corresponding to the target data, enhance the target expansion data to obtain enhanced target data, construct pseudo labels on the enhanced target data to obtain corresponding pseudo labels, and associate the enhanced target data with the corresponding pseudo labels to obtain a second supervised training sample; A contrast training module, configured to perform supervised contrast learning training on the pre-trained representation model using the first supervised training sample and the second supervised training sample, and perform self-supervised contrast learning training on the pre-trained representation model using the enhanced target data, to obtain a trained representation model; The entity linking module is used to embed the data to be identified in the target domain using the trained representation model to obtain the embedded expression to be identified, and perform entity linking on the data to be identified based on the embedded expression to be identified.

[0009] In a third aspect, an embodiment of the present application provides a computer device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the cross-domain entity recognition method as described in the first aspect when executing the computer program.

[0010] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the cross-domain entity recognition method as described in the first aspect is implemented.

[0011] Compared with the prior art, the embodiments of the present application have the following beneficial effects: the present application obtains source data in a source domain and target data in a target domain, embeds the source data and the target data using an initial representation model to obtain a source embedding representation and a target embedding representation, performs adversarial learning training on the representation model with the goal of aligning the source embedding representation and the target embedding representation to obtain a pre-trained representation model, performs attribute extension on the source data in the source domain to obtain source extended data corresponding to the source data, associates the source extended data with a label corresponding to the source data to obtain a first supervised training sample, performs attribute extension on the target data in the target domain to obtain target extended data corresponding to the target data, and performs attribute extension on the target data in the target domain to obtain target extended data corresponding to the target data. The target extended data is enhanced to obtain enhanced target data, a pseudo-label is constructed for the enhanced target data to obtain a corresponding pseudo-label, the enhanced target data is associated with the corresponding pseudo-label to obtain a second supervised training sample, the first supervised training sample and the second supervised training sample are used to perform supervised contrastive learning training on the pre-trained representation model, and the enhanced target data is used to perform self-supervised contrastive learning training on the pre-trained representation model to obtain a trained representation model, the trained representation model is used to embed the data to be identified in the target domain to obtain an embedded expression to be identified, and entity linking is performed on the data to be identified according to the embedded expression to be identified.

[0012] The representation model is obtained by pre-training the embedded representation of the source data and the target data, and then the supervised training samples and the self-supervised training samples are constructed by data expansion and enhancement. The above representation model is optimized and trained using supervised training and self-supervised training respectively, thereby providing an accurate expression model for data recognition in the target field, which can be applied to entity recognition of data in different fields, thereby improving the adaptability of entity linking tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0014] Figure 1 This is a schematic diagram of an application environment of a cross-domain entity recognition method for entity linking provided in Embodiment 1 of the present application; Figure 2 It is a flowchart of a cross-domain entity recognition method for entity linking provided in Embodiment 2 of the present application; Figure 3 It is a flowchart of a cross-domain entity recognition method for entity linking provided in Embodiment 3 of the present application; Figure 4 It is a flowchart of a cross-domain entity identification method for entity linking provided in Embodiment 4 of the present application; Figure 5 It is a flowchart of a cross-domain entity recognition method for entity linking provided in Embodiment 5 of the present application; Figure 6 It is a structural diagram of a cross-domain entity identification device for entity linking provided in Embodiment 6 of the present application; Figure 7 It is a structural diagram of a computer device provided in Example 7 of the present application. DETAILED DESCRIPTION

[0015] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0016] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0017] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0018] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0019] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0020] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0021] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0022] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0023] It should be understood that the size of the serial numbers of the steps in the following embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0024] In order to illustrate the technical solution of the present application, a specific embodiment is provided below for illustration.

[0025] A cross-domain entity recognition method for entity linking provided in the first embodiment of the present application can be applied in the following aspects: Figure 1 In the application environment, the server connects the client and the corresponding database, etc., and the server obtains the data of the database and the client. The client may include but is not limited to a PDA, a desktop computer, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a cloud computer device, a personal digital assistant (PDA) and other computer devices. The server may be implemented as an independent server or an integrated server. The database may refer to a relational database such as Mysql and SqlServe, or a non-relational database.

[0026] See also Figure 2 , is a flow chart of a cross-domain entity recognition method for entity linking provided in Embodiment 2 of the present application. The cross-domain entity recognition method is applied to Figure 1 The server in . Figure 2 As shown, the cross-domain entity recognition method may include the following steps: Step S201, obtain source data in the source domain and target data in the target domain, use the initial representation model to embed the source data and the target data to obtain a source embedding representation and a target embedding representation, and perform adversarial learning training on the representation model with the goal of aligning the source embedding representation and the target embedding representation to obtain a pre-trained representation model. In this embodiment, the source domain and the target domain are different domains, where the domain may refer to a data acquisition scenario. For example, in a shopping scenario, the products on platform A and the products on platform B are data in two different domains. In a text classification scenario, language A and language B are data in two different domains.

[0027] For the data in the source domain, in order to realize the task model designed for tasks such as classification and clustering, the data in the source domain has been used to train the task model. In the target domain, the data in the target domain and the data in the source domain may have different distributions, which leads to the task model in the source domain being unable to accurately process the data in the target domain. Therefore, it is necessary to design a representation model so that the input to the task model has the same expression as the data in the source domain, thereby improving the accuracy of the task model in processing the data in the target domain.

[0028] The representation model can realize the embedded representation of data, so as to be used in the subsequent task model. The representation model can align the source embedded representation of the source data with the target embedded representation of the target data, so that when the target data to be identified is obtained, the target data can be embedded in the representation to obtain the content that the task model can recognize.

[0029] In this embodiment, the task model is a model corresponding to the entity linking task. Before inputting data into the model corresponding to the entity linking task, the representation model is used to perform embedded representation for cross-domain data, and then recognized when input into the task model. Among them, the entity is the object of data expression, and the data corresponding to the entity is the data expression and encapsulation of the object.

[0030] Step S202: In the source domain, attribute extension is performed on the source data to obtain source extended data corresponding to the source data, and the source extended data is associated with a label of the corresponding source data to obtain a first supervised training sample.

[0031] In this embodiment, in cross-domain tasks, there are usually differences in data distribution between the source domain and the target domain, which leads to a decrease in the performance of the model in the target domain. When aligning feature distribution to alleviate the above problem, the alignment effect is limited for scenes with complex data structures or lack of rich attributes. Therefore, attribute extension processing is performed on the source data and the target data to generate data that is conducive to improving the model alignment effect.

[0032] After the source data is expanded, source extended data is obtained. At this time, for each source extended data, the label of the corresponding source data is the label of the source extended data, thereby forming a first supervised training sample for subsequent supervised training of the representation model.

[0033] Step S203, in the target domain, attributes of the target data are extended to obtain target extended data corresponding to the target data, the target extended data is enhanced to obtain enhanced target data, pseudo-labels are constructed for the enhanced target data to obtain corresponding pseudo-labels, the enhanced target data is associated with the corresponding pseudo-labels to obtain a second supervised training sample. In this embodiment, the attribute extension method of the target data corresponds to the attribute extension method of the source data. The target extended data can retain the attribute characteristics of the target data while being closer to the distribution of the source data.

[0034] Since the target data in the target domain has no labels, enhancing the target extended data can improve the effect of using the target data for supervised training. At the same time, constructing pseudo-labels for the enhanced target data can obtain higher-quality pseudo-labels and improve the effect of using the target data for self-supervised training.

[0035] In one embodiment, a large language model (LLM) is used as a powerful generation and reasoning tool to generate more structured and domain-invariant attribute data, providing new possibilities for cross-domain adaptation.

[0036] Optionally, in the source field, attribute extension is performed on the source data to obtain source extended data corresponding to the source data, including: Inputting source data and a basic attribute set corresponding to the source data into a large language model, enabling the large language model to generate a first candidate attribute set according to a first preset instruction; Obtaining the attribute frequency of each attribute in the first candidate attribute set, filtering the attributes whose attribute frequency is greater than a preset frequency or whose attribute frequency is invalid from the first candidate attribute set, to obtain a first filtered attribute set; Based on the normalized mutual information evaluation index, a first extended attribute subset is screened out from the first filter attribute set, and the first extended attribute subset is used to perform attribute extension on the source data to obtain source extended data corresponding to the source data.

[0037] Optionally, in the target field, the target data is attribute-extended to obtain target extended data corresponding to the target data, including: Inputting the target data and the basic attribute set corresponding to the target data into the large language model, so that the large language model generates a second candidate attribute set according to a second preset instruction; Obtaining the attribute frequency of each attribute in the second candidate attribute set, filtering the attributes whose attribute frequency is greater than a preset frequency or whose attribute frequency is invalid from the second candidate attribute set, to obtain a second filtered attribute set; The second filtering attribute set is screened by using the maximum average difference optimization algorithm to obtain a second extended attribute subset, and the second extended attribute subset is used to extend the attributes of the target data to obtain target extended data corresponding to the target data.

[0038] First, a large language model is used in the source domain to generate a set of candidate attributes, and the Normalized Mutual Information (NMI) indicator is used to screen out an extended attribute subset that can significantly improve the performance of source domain tasks. Second, in the target domain, a large language model is used to generate a set of candidate attributes, and the Maximum Mean Discrepancy (MMD) optimization algorithm is used to select the attribute subset to ensure that the target domain data is closer to the distribution of the source domain data while retaining key features.

[0039] The specific implementation steps include: the first step is to input the source domain data and its basic attribute set into the large language model, generate a candidate attribute set according to the preset instructions, filter out low-frequency or invalid attributes by analyzing the attribute frequency, and select the attribute subset that can significantly improve the source domain task performance based on the normalized mutual information, and finally update the attribute set of the source domain data; the second step is to perform the same processing on the target domain data and the basic attribute set, and select the extended attribute set through the maximum average difference minimization algorithm, so that the target domain data is aligned with the source domain distribution while enhancing the representation ability, forming an optimized target domain attribute set.

[0040] This embodiment uses a large language model to automatically generate rich and discriminative attribute data, which reduces manual intervention and improves efficiency. Through dual optimization of normalized mutual information and maximum mean difference, the efficiency and robustness of extended attributes are ensured. It is suitable for different types of cross-domain tasks and data sets and has good versatility and scalability.

[0041] Step S204, using the first supervised training sample and the second supervised training sample, perform supervised contrastive learning training on the pre-trained representation model, and use the enhanced target data to perform self-supervised contrastive learning training on the pre-trained representation model to obtain a trained representation model.

[0042] In this embodiment, for the representation model used for data processing before the entity linking task, supervised contrastive learning and self-supervised contrastive learning are used for joint training to obtain a better representation model, thereby improving the accuracy of cross-domain data in the entity linking task.

[0043] Step S205, using the trained representation model, embeds the data to be identified in the target domain to obtain the embedded expression to be identified, and entity links the data to be identified based on the embedded expression to be identified.

[0044] The embodiment of the present application obtains a representation model by pre-training the embedded representation of the source data and the target data, and then constructs supervised training samples and self-supervised training samples by means of data expansion and enhancement, and optimizes and trains the above-mentioned representation model using supervised training and self-supervised training respectively, thereby providing an accurate expression model for data recognition in the target field, and can be applied to entity recognition of data in different fields, thereby improving the adaptability of entity linking tasks.

[0045] See also Figure 3 , is a flow chart of a cross-domain entity recognition method for entity linking provided in Embodiment 3 of the present application. Figure 3As shown, in the above step S201, the source data and the target data are embedded using the initial representation model to obtain a source embedding representation and a target embedding representation. With the goal of aligning the source embedding representation and the target embedding representation, the representation model is trained through adversarial learning to obtain a pre-trained representation model, including the following steps: Step S301: construct a training sample set based on source data and target data.

[0046] Each sample in the training sample set includes a source data tuple in the source data, a target data tuple matching the source data tuple, and a target data tuple not matching the source data tuple, and the target data tuple belongs to the target data.

[0047] Step S302: For any sample, use the initial representation model to embed the source data tuple and any target data tuple of the sample respectively to obtain the corresponding source embedding representation and target embedding representation, and use the initial feature extractor to extract the source embedding representation and the target embedding representation to obtain the similarity embedding representation.

[0048] Step S303: Use the initial classifier to predict the matching relationship between the source embedding representation and the target embedding representation to obtain a matching prediction result, and use the initial discriminator to predict the source of the similarity embedding representation to obtain a domain prediction result.

[0049] Step S304, with the matching prediction result being the true matching relationship between the source data tuple and any target data tuple and the domain prediction result not being the source domain as the goal, the initial representation model, the initial feature extractor and the initial discriminator are subjected to adversarial learning training to obtain a pre-trained representation model.

[0050] Among them, the goal of representation learning is to represent the input entity tuple as a high-dimensional vector embedding for subsequent tasks to calculate.

[0051] The specific implementation is as follows: Input serialization: Each tuple t contains multiple attributes A1, A2, ..., Am, which is serialized according to the following rule: serial(t) = [COL] A1 [VAL]t[A1] ... [COL]Am [VAL] t[Am], where [COL] and [VAL] are special separators used to structurally identify attributes and values; Embedding generation: embed the serialized tuple using the representation model M (e.g., SentenceBERT) to obtain a high-dimensional vector representation emb_t = M(serial(t)); Contrastive learning optimization: The labeled data DS of the source domain is used to generate a contrastive learning training sample set CL-S. Each sample contains a source data tuple and its positive example set (matching target data tuple) Pa and negative example set (unmatched target data tuple) Na, which are optimized by the cross entropy loss function.

[0052] In cross-domain scenarios, there are differences in data distribution between the source domain and the target domain. Representation learning that relies solely on the source domain is difficult to adapt to the characteristics of the target domain.

[0053] This application generates cross-domain similarity representations through the following methods: (a) Feature extractor design: A multi-layer perceptron (MLP) is used as the feature extractor F, which accepts the embedding representation of two tuples and generates their similarity embedding: emb(a, b) = ReLU(NN(emb(a)⊕emb(b))), where ⊕ is the row-by-row concatenation operation of the embedding and NN is the fully connected layer; (b) Classifier: Based on similarity embedding, design a binary classifier G to predict the matching relationship of tuple pairs: G(F(emb(a)⊕emb(b))); (c) Adversarial learning for cross-domain alignment: In order to align similarity representations in the source domain and the target domain, a binary discriminator D is used to predict whether the input similarity embedding comes from the source domain. Then, an optimization objective is constructed to allow the representation model M and the feature extractor F to learn cross-domain feature embeddings that can confuse the classification discriminator D. The overall loss function is L = L_CE +λ* L_adv, where L_CE is the cross entropy loss used to optimize the classifier G, and L_adv is the adversarial loss used to optimize M and F to confuse the discriminator D. (d) Optimization step: fix the classification discriminator D, optimize the representation model M and feature extractor F, so that the generated similarity embedding confuses the classification discriminator D as much as possible. Secondly, fix the representation model M and feature extractor F, optimize the classification discriminator D to improve the discrimination ability, and alternately train between the two steps to finally achieve cross-domain distribution alignment.

[0054] The embodiment of the present application efficiently captures the similarity information between tuples through contrastive learning and serialized embedding generation. Secondly, adversarial learning is introduced in the entity linking task, unsupervised domain adaptation is achieved through game optimization, and lightweight feature extractors and classification discriminators are used to significantly reduce the computational complexity while improving cross-domain adaptability.

[0055] See also Figure 4 , is a flow chart of a cross-domain entity recognition method for entity linking provided in Embodiment 4 of the present application. Figure 4As shown, in the above step S203, the target extended data is enhanced to obtain enhanced target data, which may include the following steps: Step S401 , performing data enhancement on each target data tuple in the target extended data to obtain a first enhancement result corresponding to the target data tuple.

[0056] Step S402: obtaining a preset enhanced attribute set, and for any target data tuple in the target extended data, randomly selecting N enhanced attribute subsets from the enhanced attribute set as a second enhanced result of the target data tuple.

[0057] Wherein, N is an integer greater than zero.

[0058] Step S403: Combine each target data tuple with the corresponding first enhancement result and second enhancement result to obtain enhanced target data.

[0059] In this embodiment, the training effect of the target field is improved by using data enhancement and LLM enhancement, as follows: Data augmentation: Extract all single tuples from the left and right tables (R^T_l and R^T_r) of the target data and apply data augmentation techniques to each tuple. Specifically, use data augmentation strategies (such as L_aug) including random insertion, deletion, and replacement operations to generate multiple augmented tuples. Through data augmentation, the data in the target domain becomes more diverse, which helps the model generalize better.

[0060] LLM enhancement: First, define the set of all enhanced attributes in the target domain (B^T_all), then randomly sample multiple subsets from it and combine these subsets with the attributes of the target tuple to generate new enhanced data in order to further enhance the robustness of the target domain data. For example, for a tuple t, if C_i and C_j are two different subsets sampled from the enhanced attribute set, then t[C_i] and t[C_j] will represent the same entity. Through LLM enhancement, the generated set of positive tuples (P_t^LLM) can provide more diverse training data.

[0061] Optionally, use the augmented target data to train the pre-trained representation model with self-supervised contrastive learning, including: For any target data tuple in the enhanced target data, a hard negative sample mining algorithm is used to construct a negative sample of the target data tuple to obtain a negative sample set corresponding to the target data tuple; The first enhancement result and the second enhancement result of the target data tuple are taken as a positive sample set; The target data tuple, the corresponding positive sample set and the negative sample set are taken as training triplets, and all the training triplets are used to perform self-supervised contrastive learning training on the pre-trained representation model.

[0062] Among them, by combining the results of data augmentation and LLM augmentation, a positive example set (P_t) of the target tuple is generated. For each augmented tuple, multiple augmented versions are generated, and a negative sample set (N_t) is generated through a hard negative sample mining strategy. These positive and negative samples will be input into self-supervised contrastive learning as training triplets (t, P_t, N_t), thereby optimizing the representation model so that different representations of the same entity are more similar and representations of different entities are more different.

[0063] See also Figure 5 , is a flow chart of a cross-domain entity recognition method for entity linking provided in Embodiment 5 of the present application. Figure 5 As shown, in the above step S204, pseudo labels are constructed for the enhanced target data to obtain corresponding pseudo labels, and the enhanced target data is associated with the corresponding pseudo labels to obtain a second supervised training sample, including the following steps: Step S501 : performing a K-nearest neighbor search on each target data tuple in the enhanced target data to obtain the most similar K target data tuples corresponding to each target data tuple.

[0064] Step S502: for any target data tuple, determine the allocation probability of the K target data tuples according to the similarity between the target data tuple and the K target data tuples, and randomly sample the K target data tuples according to the allocation probability of the K target data tuples to obtain a pseudo-label data set corresponding to the target data tuple.

[0065] Step S503: Use the pseudo-label data set as the pseudo-label of the corresponding target data tuple and associate it with the target data tuple, traverse all the target data tuples in the enhanced target data, and obtain a second supervised training sample.

[0066] In this embodiment, unlike self-supervised contrastive learning, supervised contrastive learning relies on training data generated by pseudo-labels. Pseudo-labels are generated by predicting data in the target domain through the model, thereby guiding the model to train.

[0067] First, for each tuple t in the target domain, the K tuples most similar to it are found through K nearest neighbor search, and then a high-quality pseudo-label training set is generated using these similar tuples. The pseudo-label dataset in this embodiment is a generated pseudo-positive tuple set, rather than directly predicting the pseudo-labels of tuple pairs, which can reduce noise interference in the training process.

[0068] In addition, in order to avoid over-reliance on the most similar tuples and avoid noise interference in the training data, this embodiment calculates the similarity between each pair of tuples (calculates the similarity of the embedded vectors through the inner product), assigns a probability p to each pair of tuples, and then randomly samples based on these probabilities to generate a high-quality pseudo-label data set (C^T). This method can reduce the impact of noise while ensuring data diversity and improve the performance of the model.

[0069] In supervised contrastive learning training, a high-quality training dataset C^T generated by pseudo labels is used together with the labeled data D^S of the source domain to perform supervised contrastive learning on the representation model. By optimizing the representation model, the entity representations of the target domain and the source domain are made more consistent in the embedding space, thereby improving the effect of cross-domain entity matching.

[0070] In the embodiment of the present application, a process of gradually optimizing the representation model M is designed by combining LLM enhancement, adversarial learning strategy and contrastive learning. The basic process is as follows: (a) Data preprocessing and initial training: First, labeled and unlabeled data are obtained from the source domain (D^S) and the target domain (D^T). The source domain data is converted into triplets (CL^S) and the model M is initially tuned using contrastive learning to adapt it to the source domain data. In this process, the MLP discriminator D and the adversarial learning strategy are used to further optimize the model M so that the similarity representations of the source and target domains are aligned; (b) Data augmentation and LLM augmentation: The tuples in the source and target domains are augmented by LLM to generate an augmented attribute set (B^S_all and B^T_all). Subsequently, the tuples in the target domain are filled with attributes by LLM to generate the final augmented features. (c) Iterative training and self-supervised learning: After data augmentation and LLM augmentation are completed, the iterative training phase begins. Through self-supervised contrastive learning, combined with the enhanced tuples and pseudo-label generation mechanism, the model M is gradually optimized. In the target domain (R^T_l and R^T_r), a K-nearest neighbor search is performed on each tuple, and high-quality pseudo-label tuple pairs are selected for training, and the model is further optimized through supervised contrastive learning; (d) Iterative optimization: Model M is continuously trained with labeled data from the source domain and pseudo-labeled data from the target domain in multiple iterations until the maximum number of iterations is reached. Each iteration generates a new training dataset and updates the model parameters, gradually improving the performance of the model in the target domain.

[0071] By comprehensively using technologies such as data augmentation, LLM enhancement, adversarial learning, contrastive learning and pseudo-label generation, the problem of data distribution differences in cross-domain entity linking is effectively solved, enabling the model to better match entities in the target domain, thereby improving the accuracy and robustness of the cross-domain entity linking task.

[0072] Corresponding to the cross-domain entity recognition method for entity linking in the above embodiment, Figure 6 The structure block diagram of the cross-domain entity recognition device for entity linking provided in the sixth embodiment of the present application is shown. The cross-domain entity recognition device is applied to Figure 1 For ease of description, only the parts related to the embodiment of the present application are shown.

[0073] See also Figure 6 , the cross-domain entity recognition device comprises: An alignment training module 61 is used to obtain source data in a source domain and target data in a target domain, embed the source data and the target data using an initial representation model to obtain a source embedding representation and a target embedding representation, perform adversarial learning training on the representation model with the goal of aligning the source embedding representation and the target embedding representation, and obtain a pre-trained representation model; A source data extension module 62 is used to perform attribute extension on the source data in the source domain to obtain source extended data corresponding to the source data, and associate the source extended data with a label corresponding to the source data to obtain a first supervised training sample; A target data expansion module 63 is used to perform attribute expansion on the target data in the target domain to obtain target expansion data corresponding to the target data, enhance the target expansion data to obtain enhanced target data, construct pseudo labels on the enhanced target data to obtain corresponding pseudo labels, and associate the enhanced target data with the corresponding pseudo labels to obtain a second supervised training sample; A contrast training module 64 is used to perform supervised contrast learning training on the pre-trained representation model using the first supervised training sample and the second supervised training sample, and to perform self-supervised contrast learning training on the pre-trained representation model using the enhanced target data to obtain a trained representation model; The entity linking module 65 is used to embed the data to be identified in the target domain using the trained representation model to obtain the embedded expression to be identified, and perform entity linking on the data to be identified based on the embedded expression to be identified.

[0074] Optionally, the alignment training module 61 includes: A training sample construction unit is used to construct a training sample set according to the source data and the target data, wherein each sample in the training sample set includes a source data tuple in the source data, a target data tuple matching the source data tuple, and a target data tuple not matching the source data tuple, and the target data tuple belongs to the target data; A similarity representation generation unit is used to embed the source data tuple and any target data tuple of the sample respectively using the initial representation model for any sample to obtain the corresponding source embedding representation and target embedding representation, and extract the source embedding representation and the target embedding representation using the initial feature extractor to obtain the similarity embedding representation; The target prediction unit is used to use the initial classifier to predict the matching relationship between the source embedding representation and the target embedding representation to obtain a matching prediction result, and use the initial discriminator to predict the source of the similarity embedding representation to obtain a domain prediction result; The adversarial training unit is used to perform adversarial learning training on the initial representation model, the initial feature extractor and the initial discriminator with the matching prediction result as the true matching relationship between the source data tuple and any target data tuple and the domain prediction result not being the source domain as the goal, so as to obtain a pre-trained representation model.

[0075] Optionally, the source data expansion module 62 includes: A first candidate unit, configured to input source data and a basic attribute set corresponding to the source data into a large language model, so that the large language model generates a first candidate attribute set according to a first preset instruction; A first filtering unit is used to obtain the attribute frequency of each attribute in the first candidate attribute set, and filter the attributes whose attribute frequency is greater than a preset frequency or whose attribute frequency is invalid from the first candidate attribute set to obtain a first filtered attribute set; The source extension unit is used to select a first extended attribute subset from the first filter attribute set based on the normalized mutual information evaluation index, and use the first extended attribute subset to perform attribute extension on the source data to obtain source extended data corresponding to the source data.

[0076] Optionally, the target data expansion module 63 includes: A second candidate unit, used for inputting the target data and the basic attribute set corresponding to the target data into the large language model, so that the large language model generates a second candidate attribute set according to a second preset instruction; A second filtering unit is used to obtain the attribute frequency of each attribute in the second candidate attribute set, and filter the attributes whose attribute frequency is greater than a preset frequency or whose attribute frequency is invalid from the second candidate attribute set to obtain a second filtered attribute set; The target extension unit is used to screen the second filtering attribute set using the maximum average difference optimization algorithm to obtain a second extended attribute subset, and use the second extended attribute subset to extend the attributes of the target data to obtain target extended data corresponding to the target data.

[0077] Optionally, the target data expansion module 63 includes: A first enhancement unit, configured to perform data enhancement on each target data tuple in the target extended data to obtain a first enhancement result corresponding to the target data tuple; The second enhancement unit is used to obtain a preset enhanced attribute set, and for any target data tuple in the target extended data, randomly select N enhanced attribute subsets from the enhanced attribute set as a second enhancement result of the target data tuple, where N is an integer greater than zero; The enhanced data unit is used to combine each target data tuple with the corresponding first enhancement result and second enhancement result to obtain enhanced target data.

[0078] Optionally, the target data expansion module 63 includes: A negative sample construction unit is used to construct a negative sample for any target data tuple in the enhanced target data using a hard negative sample mining algorithm to obtain a negative sample set corresponding to the target data tuple; A positive sample construction unit, used to take the first enhancement result and the second enhancement result of the target data tuple as a positive sample set; The self-supervised training unit is used to use the target data tuple, the corresponding positive sample set and the negative sample set as training triplets, and use all the training triplets to perform self-supervised comparative learning training on the pre-trained representation model.

[0079] Optionally, the comparison training module 64 includes: A similarity search unit, used for performing a K nearest neighbor search on each target data tuple in the enhanced target data to obtain the most similar K target data tuples corresponding to each target data tuple; A pseudo-label selection unit is used to determine, for any target data tuple, the allocation probability of the K target data tuples according to the similarity between the target data tuple and the K target data tuples, and randomly sample the K target data tuples according to the allocation probability of the K target data tuples to obtain a pseudo-label data set corresponding to the target data tuple; The supervised training sample construction unit is used to use the pseudo-label data set as the pseudo-label of the corresponding target data tuple and associate it with the target data tuple, traverse all the target data tuples in the enhanced target data, and obtain the second supervised training sample.

[0080] It should be noted that the information interaction, execution process and other contents between the above-mentioned modules are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0081] Figure 7 This is a schematic diagram of the structure of a computer device provided in Example 7 of this application. Figure 7 As shown, the computer device of this embodiment includes: at least one processor ( Figure 7 Only one is shown in the figure), a memory, and a computer program stored in the memory and executable on at least one processor, which implements the steps in any of the above-mentioned cross-domain entity recognition method embodiments for entity linking when the processor executes the computer program.

[0082] The computer device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that Figure 7 This is merely an example of a computer device and does not constitute a limitation on the computer device. The computer device may include more or fewer components than those shown in the figure, or a combination of certain components, or different components. For example, it may also include a network interface, a display screen, and an input device.

[0083] The processor may be a CPU, or other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0084] The memory includes a readable storage medium, an internal memory, etc., wherein the internal memory may be the memory of a computer device, and the internal memory provides an environment for the operation of the operating system and computer-readable instructions in the readable storage medium. The readable storage medium may be a hard disk of a computer device, and in other embodiments, it may also be an external storage device of the computer device, for example, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the computer device. Further, the memory may also include both an internal storage unit of the computer device and an external storage device. The memory is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of a computer program, etc. The memory may also be used to temporarily store data that has been output or is to be output.

[0085] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the above-mentioned method embodiment, which will not be repeated here. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned method embodiment can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device capable of carrying computer program code, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0086] The present application implements all or part of the processes in the above-mentioned embodiment method, and may also be completed through a computer program product. When the computer program product runs on a computer device, the computer device can implement the steps in the above-mentioned method embodiment when executing the computer program product.

[0087] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0088] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0089] In the embodiments provided in the present application, it should be understood that the disclosed devices / computer equipment and methods can be implemented in other ways. For example, the device / computer equipment embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0090] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0091] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A cross-domain entity recognition method for entity linking, characterized in that: The cross-domain entity recognition method comprises: Acquire source data in a source domain and target data in a target domain, embed the source data and the target data using an initial representation model to obtain a source embedding representation and a target embedding representation, perform adversarial learning training on the representation model with the goal of aligning the source embedding representation and the target embedding representation, and obtain a pre-trained representation model; In the source domain, performing attribute extension on the source data to obtain source extended data corresponding to the source data, and associating the source extended data with a label corresponding to the source data to obtain a first supervised training sample; In the target domain, the target data is attribute-extended to obtain target extended data corresponding to the target data, the target extended data is enhanced to obtain enhanced target data, pseudo-labels are constructed for the enhanced target data to obtain corresponding pseudo-labels, and the enhanced target data is associated with the corresponding pseudo-labels to obtain a second supervised training sample; Using the first supervised training sample and the second supervised training sample, performing supervised contrastive learning training on the pre-trained representation model, and using the enhanced target data, performing self-supervised contrastive learning training on the pre-trained representation model to obtain a trained representation model; The trained representation model is used to embed the data to be identified in the target domain to obtain an embedded expression to be identified, and entity linking is performed on the data to be identified based on the embedded expression to be identified.

2. The cross-domain entity recognition method according to claim 1, characterized in that: The method uses the initial representation model to embed the source data and the target data to obtain a source embedding representation and a target embedding representation, performs adversarial learning training on the representation model with the goal of aligning the source embedding representation and the target embedding representation, and obtains a pre-trained representation model, including: Constructing a training sample set according to the source data and the target data, wherein each sample in the training sample set includes a source data tuple in the source data, a target data tuple matching the source data tuple, and a target data tuple not matching the source data tuple, and the target data tuple belongs to the target data; For any sample, use the initial representation model to embed the source data tuple and any target data tuple of the sample respectively to obtain corresponding source embedding representation and target embedding representation, and use the initial feature extractor to extract the source embedding representation and the target embedding representation to obtain a similarity embedding representation; Using the initial classifier, predicting the matching relationship between the source embedding representation and the target embedding representation to obtain a matching prediction result, and using the initial discriminator, predicting the source of the similarity embedding representation to obtain a domain prediction result; With the matching prediction result being the true matching relationship between the source data tuple and any target data tuple and the domain prediction result being not the source domain as the goal, the initial representation model, the initial feature extractor and the initial discriminator are trained through adversarial learning to obtain a pre-trained representation model.

3. The cross-domain entity recognition method according to claim 1, characterized in that: In the source domain, the source data is attribute extended to obtain source extended data corresponding to the source data, including: Inputting the source data and a basic attribute set corresponding to the source data into a large language model, so that the large language model generates a first candidate attribute set according to a first preset instruction; Acquire the attribute frequency of each attribute in the first candidate attribute set, and filter out the attributes whose attribute frequency is greater than a preset frequency or whose attribute frequency is invalid from the first candidate attribute set to obtain a first filtered attribute set; Based on the normalized mutual information evaluation index, a first extended attribute subset is screened out from the first filter attribute set, and the first extended attribute subset is used to perform attribute extension on the source data to obtain source extended data corresponding to the source data.

4. The cross-domain entity recognition method according to claim 3, characterized in that: In the target field, the target data is attribute-extended to obtain target extended data corresponding to the target data, including: Inputting the target data and a basic attribute set corresponding to the target data into the large language model, so that the large language model generates a second candidate attribute set according to a second preset instruction; Acquire the attribute frequency of each attribute in the second candidate attribute set, and filter out the attributes whose attribute frequency is greater than a preset frequency or whose attribute frequency is invalid from the second candidate attribute set to obtain a second filtered attribute set; The second filtering attribute set is screened by using a maximum average difference optimization algorithm to obtain a second extended attribute subset, and the second extended attribute subset is used to perform attribute extension on the target data to obtain target extended data corresponding to the target data.

5. The cross-domain entity recognition method according to claim 1, characterized in that: The step of enhancing the target extended data to obtain enhanced target data includes: Performing data enhancement on each target data tuple in the target extended data to obtain a first enhancement result corresponding to the target data tuple; Obtaining a preset enhanced attribute set, and for any target data tuple in the target extended data, randomly selecting N enhanced attribute subsets from the enhanced attribute set as a second enhanced result of the target data tuple, where N is an integer greater than zero; Each target data tuple is combined with the corresponding first enhancement result and second enhancement result to obtain enhanced target data.

6. The cross-domain entity recognition method according to claim 5, characterized in that: The using the enhanced target data to perform self-supervised contrastive learning training on the pre-trained representation model includes: For any target data tuple in the enhanced target data, a hard negative sample mining algorithm is used to construct a negative sample for the target data tuple to obtain a negative sample set corresponding to the target data tuple; Taking the first enhancement result and the second enhancement result of the target data tuple as a positive sample set; The target data tuple, the corresponding positive sample set and the negative sample set are used as training triplets, and all the training triplets are used to perform self-supervised contrastive learning training on the pre-trained representation model.

7. The cross-domain entity recognition method according to any one of claims 1 to 6, characterized in that: The step of constructing a pseudo label for the enhanced target data to obtain a corresponding pseudo label, and associating the enhanced target data with the corresponding pseudo label to obtain a second supervised training sample includes: Performing a K nearest neighbor search on each target data tuple in the enhanced target data to obtain the most similar K target data tuples corresponding to each target data tuple; For any target data tuple, according to the similarity between the target data tuple and the K target data tuples, determine the allocation probability of the K target data tuples, and according to the allocation probability of the K target data tuples, randomly sample the K target data tuples to obtain a pseudo-label data set corresponding to the target data tuple; The pseudo-label data set is used as a pseudo-label corresponding to the target data tuple and associated with the target data tuple, and all target data tuples in the enhanced target data are traversed to obtain a second supervised training sample.

8. A cross-domain entity recognition device for entity linking, characterized in that: The cross-domain entity recognition device comprises: An alignment training module is used to obtain source data in a source domain and target data in a target domain, embed the source data and the target data using an initial representation model to obtain a source embedding representation and a target embedding representation, perform adversarial learning training on the representation model with the goal of aligning the source embedding representation and the target embedding representation, and obtain a pre-trained representation model; A source data extension module is used to perform attribute extension on the source data in the source domain to obtain source extension data corresponding to the source data, and associate the source extension data with a label corresponding to the source data to obtain a first supervised training sample; A target data expansion module is used to perform attribute expansion on the target data in the target domain to obtain target expansion data corresponding to the target data, enhance the target expansion data to obtain enhanced target data, construct pseudo labels on the enhanced target data to obtain corresponding pseudo labels, and associate the enhanced target data with the corresponding pseudo labels to obtain a second supervised training sample; A contrast training module, configured to perform supervised contrast learning training on the pre-trained representation model using the first supervised training sample and the second supervised training sample, and perform self-supervised contrast learning training on the pre-trained representation model using the enhanced target data, to obtain a trained representation model; The entity linking module is used to embed the data to be identified in the target domain using the trained representation model to obtain the embedded expression to be identified, and perform entity linking on the data to be identified based on the embedded expression to be identified.

9. A computer device, characterized in that: The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the cross-domain entity recognition method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the cross-domain entity recognition method according to any one of claims 1 to 7 is implemented.

Images