Named entity alignment method, device, electronic device and readable storage medium

Abstract

The present invention relates to a big data technology, and discloses a method for aligning named entities, including: standardizing the named entities to be aligned to obtain standard named entities to be aligned; sampling the test named entity set to obtain a test named entity subset; using each test named entity subset to train a preset neural network model to obtain a named entity alignment model set; and aligning the named entities to be aligned according to the named entity alignment model set to obtain an alignment result. The present invention also relates to a blockchain technology, and data used for model training can be stored in a blockchain. The present invention also proposes a named entity alignment device, an electronic device, and a computer-readable storage medium. The present invention can improve the accuracy of named entity alignment.

Description

Technical Field

[0001] The present invention relates to the field of big data, and in particular to a method, device, electronic device and readable storage medium for aligning named entities. Background Art

[0002] With the advent of the big data era, how to efficiently acquire and process the knowledge in it is an important research topic. The named entity alignment research in the field of natural language processing aims to standardize different expressions of the same concept, which can greatly facilitate users' understanding and application of knowledge.

[0003] Currently, there are two main types of named entity alignment methods. One is based on the morphological features of different entities for alignment, but some morphological feature alignments lose semantic features and have low accuracy. The other is based on semantic alignment of entities, which requires a large amount of training data for training. However, training data is not easy to obtain, resulting in low accuracy of this method. Summary of the invention

[0004] The present invention provides a method, device, electronic device and computer-readable storage medium for aligning named entities, the main purpose of which is to reduce the amount of data for model training and improve the accuracy of named entity alignment.

[0005] To achieve the above object, the present invention provides a method for aligning named entities, comprising:

[0006] Acquire named entities to be aligned, and perform standardization processing on the named entities to be aligned to obtain standard named entities to be aligned;

[0007] Acquire a test named entity set, perform sampling processing on the test named entity set, and obtain a test named entity subset;

[0008] Using each test named entity subset to train a preset neural network model, a named entity alignment model set is obtained;

[0009] Model alignment is performed on the named entities to be aligned according to the named entity alignment model set to obtain an alignment result.

[0010] Optionally, the using each of the test named entity subsets to train a preset neural network model to obtain a named entity alignment model set includes:

[0011] Convert each test named entity in the test named entity subset into a test named entity vector to obtain a test named entity vector subset;

[0012] Determining the test named entity vector subset as a training set;

[0013] Labeling the test named entity vector subset to obtain a label set;

[0014] Using the training set and the label set to train the neural network model to obtain a named entity alignment model;

[0015] All the named entity alignment models are aggregated to obtain the named entity alignment model set.

[0016] Optionally, before performing model alignment on the standard named entities to be aligned according to the named entity alignment model set, the method further includes:

[0017] Using the standard named entities to be aligned to perform morphological alignment in a pre-built standard named entity library, if the morphological alignment is successful, obtaining the alignment result;

[0018] If the morphological alignment is unsuccessful, model alignment is performed on the standard named entities to be aligned according to the named entity alignment model set.

[0019] Optionally, the using the standard named entities to be aligned to perform morphological alignment in a pre-built standard named entity library, and if the morphological alignment is successful, obtaining the alignment result, includes:

[0020] Calculating the edit distance between the standard named entity to be aligned and each standard named entity in the standard named entity library;

[0021] When a target edit distance equal to a preset edit distance value exists in the edit distances, it is determined that the alignment is successful, and a standard named entity corresponding to the target edit distance is selected as the alignment result.

[0022] Optionally, performing model alignment on the named entities to be aligned according to the named entity alignment model set to obtain an alignment result includes:

[0023] Convert each character in the standard named entity to be aligned into a character vector of a predetermined dimension, calculate the average value of the character vectors corresponding to all characters in the standard named entity to be aligned, and obtain a standard named entity vector;

[0024] Using each named entity alignment model in the named entity alignment model set to align the standard named entity vector to be aligned, to obtain a predicted aligned entity vector;

[0025] Convert each standard named entity in the standard named entity library into a standard named entity vector, and aggregate all the standard named entity vectors to obtain a standard named entity vector library;

[0026] A similarity calculation and analysis process is performed on the predicted aligned entity vector and each of the standard named entity vectors in the standard named entity vector library to obtain the alignment result.

[0027] Optionally, performing similarity calculation and analysis processing on the predicted aligned entity vector and each of the standard named entity vectors in the standard named entity vector library to obtain an alignment result includes:

[0028] Calculating a similarity value between the predicted aligned entity vector and each of the standard named entity vectors in the standard named entity vector library;

[0029] Aggregating all the similarity values ​​to obtain a similarity set;

[0030] Determine the maximum similarity value in the similarity set;

[0031] Selecting the standard named entity vector corresponding to the maximum similarity value in the standard named entity vector library as the target vector;

[0032] Selecting a standard named entity corresponding to the target vector in the standard named entity library as a result to be aligned;

[0033] Summarizing all the results to be aligned to obtain a set of results to be aligned;

[0034] The set of results to be aligned is screened using a majority voting mechanism to obtain the alignment result.

[0035] Optionally, the using a majority voting mechanism to screen the set of results to be aligned to obtain an alignment result includes:

[0036] Recording the number of occurrences of each result to be aligned in the set of results to be aligned;

[0037] Select the alignment result with the largest number of occurrences as the candidate alignment result;

[0038] Determining the number of candidate alignment results;

[0039] If the number is one, determining the candidate alignment result as the alignment result;

[0040] If the number is greater than one, the similarity values ​​corresponding to each result to be aligned in the result to be aligned set are summarized to obtain a similarity set of results to be aligned, and the result to be aligned corresponding to the maximum similarity value in the similarity set of results to be aligned is selected as the alignment result.

[0041] In order to solve the above problem, the present invention also provides a named entity alignment device, the device comprising:

[0042] A standardization module is used to obtain the named entities to be aligned, and perform standardization processing on the named entities to be aligned to obtain standard named entities to be aligned;

[0043] A model training module is used to obtain a test named entity set, perform sampling processing on the test named entity set, and obtain a test named entity subset; use each test named entity subset to train a preset neural network model to obtain a named entity alignment model set;

[0044] The model alignment module is used to train a preset neural network model using each test named entity subset to obtain a named entity alignment model set.

[0045] Optionally, the model training module uses each of the test named entity subsets to train a preset neural network model to obtain a named entity alignment model set, including:

[0046] Convert each test named entity in the test named entity subset into a test named entity vector to obtain a test named entity vector subset;

[0047] Determining the test named entity vector subset as a training set;

[0048] Labeling the test named entity vector subset to obtain a label set;

[0049] Using the training set and the label set to train the neural network model to obtain a named entity alignment model;

[0050] All the named entity alignment models are aggregated to obtain the named entity alignment model set.

[0051] Optionally, before the model alignment module performs model alignment on the standard named entities to be aligned according to the named entity alignment model set, it further includes:

[0052] Using the standard named entities to be aligned to perform morphological alignment in a pre-built standard named entity library, if the morphological alignment is successful, obtaining the alignment result;

[0053] If the morphological alignment is unsuccessful, model alignment is performed on the standard named entities to be aligned according to the named entity alignment model set.

[0054] Optionally, the model alignment module uses the standard named entities to be aligned to perform morphological alignment in a pre-built standard named entity library. If the morphological alignment is successful, the alignment result is obtained, including:

[0055] Calculating the edit distance between the standard named entity to be aligned and each standard named entity in the standard named entity library;

[0056] When a target edit distance equal to a preset edit distance value exists in the edit distances, it is determined that the alignment is successful, and a standard named entity corresponding to the target edit distance is selected as the alignment result.

[0057] Optionally, the model alignment module performs model alignment on the named entities to be aligned according to a named entity alignment model set to obtain an alignment result, including:

[0058] Convert each character in the standard named entity to be aligned into a character vector of a predetermined dimension, calculate the average value of the character vectors corresponding to all characters in the standard named entity to be aligned, and obtain a standard named entity vector;

[0059] Using each named entity alignment model in the named entity alignment model set to align the standard named entity vector to be aligned, to obtain a predicted aligned entity vector;

[0060] Convert each standard named entity in the standard named entity library into a standard named entity vector, and aggregate all the standard named entity vectors to obtain a standard named entity vector library;

[0061] A similarity calculation and analysis process is performed on the predicted aligned entity vector and each of the standard named entity vectors in the standard named entity vector library to obtain the alignment result.

[0062] Optionally, the model alignment module performs similarity calculation and analysis processing on the predicted aligned entity vector and each of the standard named entity vectors in the standard named entity vector library to obtain an alignment result, including:

[0063] Calculating a similarity value between the predicted aligned entity vector and each of the standard named entity vectors in the standard named entity vector library;

[0064] Aggregating all the similarity values ​​to obtain a similarity set;

[0065] Determine the maximum similarity value in the similarity set;

[0066] Selecting the standard named entity vector corresponding to the maximum similarity value in the standard named entity vector library as the target vector;

[0067] Selecting a standard named entity corresponding to the target vector in the standard named entity library as a result to be aligned;

[0068] Summarizing all the results to be aligned to obtain a set of results to be aligned;

[0069] The set of results to be aligned is screened using a majority voting mechanism to obtain the alignment result.

[0070] Optionally, the model alignment module uses a majority voting mechanism to screen the set of results to be aligned to obtain an alignment result, including:

[0071] Recording the number of occurrences of each result to be aligned in the set of results to be aligned;

[0072] Select the alignment result with the largest number of occurrences as the candidate alignment result;

[0073] Determining the number of candidate alignment results;

[0074] If the number is one, determining the candidate alignment result as the alignment result;

[0075] If the number is greater than one, the similarity values ​​corresponding to each result to be aligned in the result to be aligned set are summarized to obtain a similarity set of results to be aligned, and the result to be aligned corresponding to the maximum similarity value in the similarity set of results to be aligned is selected as the alignment result.

[0076] In order to solve the above problem, the present invention further provides an electronic device, the electronic device comprising:

[0077] a memory storing at least one instruction; and

[0078] The processor executes the instructions stored in the memory to implement the above-mentioned named entity alignment method.

[0079] In order to solve the above problems, the present invention also provides a computer-readable storage medium, including a data storage area and a program storage area, the data storage area stores data created according to the use of the blockchain node, the program storage area stores a computer program, and the computer-readable storage medium stores at least one instruction, and the at least one instruction is executed by a processor in an electronic device to implement the above-mentioned named entity alignment method.

[0080] In an embodiment of the present invention, the named entities to be aligned are standardized to obtain standard named entities to be aligned, and the influence of the alignment named entity format and irrelevant characters is eliminated; the test named entity set is sampled to obtain a test named entity subset, and the data for subsequent model training is expanded to reduce the total data volume for model training; each of the test named entity subsets is used to train a preset neural network model to obtain a named entity alignment model set; multiple test named entity subsets obtained by sampling are used to train a named entity alignment model set; the named entities to be aligned are model aligned according to the named entity alignment model set; multiple models are used to perform model alignment respectively, thereby improving the accuracy of named entity alignment. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] Figure 1A flowchart of a method for aligning named entities provided by an embodiment of the present invention;

[0082] Figure 2 A schematic diagram of a module of a named entity alignment device provided by an embodiment of the present invention;

[0083] Figure 3 A schematic diagram of the internal structure of an electronic device for implementing a method for named entity alignment provided by an embodiment of the present invention;

[0084] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0085] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0086] The present invention provides a method for aligning named entities. Figure 1 FIG. 1 is a flow chart of a method for aligning named entities according to an embodiment of the present invention. The method may be performed by a device, and the device may be implemented by software and / or hardware.

[0087] In this embodiment, the named entity alignment method includes:

[0088] S1, obtaining named entities to be aligned, and performing standardization processing on the named entities to be aligned to obtain standard named entities to be aligned;

[0089] In the embodiment of the present invention, the named entities are names of people, names of organizations, names of places, and all other entities identified by names, and the named entities to be aligned are named entities that do not use a unified identifier. For example, "Ali Group" and "Alibaba" are different entity identifier names, both of which represent the entity "Alibaba Network Technology Co., Ltd.", so "Ali Group" and "Alibaba" are the named entities to be aligned. The named entities to be aligned can be obtained from the Internet.

[0090] Furthermore, in the embodiment of the present invention, the named entities to be aligned are standardized to obtain the standard named entities to be aligned.

[0091] In detail, the standardization process includes: converting the named entities to be aligned into simplified and traditional Chinese characters, converting the named entities to upper and lower case characters, and deleting special characters to obtain the standard named entities to be aligned. The special characters refer to meaningless symbols in the named entities to be aligned, such as spaces, brackets, etc.

[0092] S2. Acquire a test named entity set, and perform sampling processing on the test named entity set to obtain a test named entity subset;

[0093] In the embodiment of the present invention, the test named entity set is a set of multiple named entities.

[0094] Preferably, in the embodiment of the present invention, the data in the test named entity set is less and difficult to obtain, and the test named entity set is sampled to obtain the test named entity subset to expand the data for subsequent model training.

[0095] S3, using each of the test named entity subsets to train a preset neural network model to obtain a named entity alignment model set;

[0096] An embodiment of the present invention converts each test named entity in the test named entity subset into a test named entity vector to obtain a test named entity vector subset, determines the test named entity vector subset as a training set, labels the test named entity vector subset to obtain a label set, and uses the training set and the label set to train the neural network model to obtain a named entity alignment model.

[0097] Specifically, in the embodiment of the present invention, marking the test named entity vector subset includes:

[0098] S11, converting the standard named entities in the pre-built standard named entity library into standard named entity vectors;

[0099] In detail, the standard named entity library is a collection of standard named entities, and the standard named entities are named entities with official unified identification.

[0100] S12. Use the standard named entity vector to mark the corresponding test named entity vector in the test named entity vector subset.

[0101] For example: the test named entity "Alibaba" in the test named entity subset corresponds to the standard named entity "Alibaba Network Technology Co., Ltd." in the pre-built standard named entity library, then the standard named entity vector converted from the standard named entity "Alibaba Network Technology Co., Ltd." is used to mark the test named entity vector converted from the test named entity "Alibaba" in the test named entity vector subset.

[0102] In detail, in an embodiment of the present invention, the above-mentioned sampling process can obtain multiple test named entity subsets, each of which can train a neural network model to obtain a named entity alignment model. For example: the test named entity set is sampled to obtain 5 test named entity subsets, each of which is trained with a preset neural network model to obtain a named entity alignment model, and a total of 5 named entity alignment models are obtained.

[0103] Preferably, there is less data in the test named entity subset described in the embodiment of the present invention, and an overly deep neural network will cause the model to overfit and the model effect to be poor. Therefore, the neural network model described in the embodiment of the present invention can be constructed using a shallow convolutional neural network.

[0104] In detail, the neural network model is trained using the training set and the label set, including:

[0105] A: performing convolution and pooling operations on the training set according to a preset number of convolution and pooling operations to obtain a dimension-reduced data set;

[0106] B: performing a deconvolution operation on the reduced-dimensionality data set according to a preset number of deconvolution operations to obtain an increased-dimensionality data set;

[0107] C: Calculate the dimension-increased data set using a preset activation function to obtain a predicted value, and use the predicted value and the label value included in the label set as input parameters of a pre-constructed loss function to calculate a loss value;

[0108] D: Compare the loss value with the preset loss threshold. If the loss value is greater than or equal to the loss threshold, return A; if the loss value is less than the loss threshold, obtain the named entity alignment model.

[0109] Furthermore, all the named entity alignment models are aggregated to obtain the named entity alignment model set.

[0110] In another embodiment of the present invention, the training data of each model in the named entity alignment model set can be stored in the blockchain.

[0111] S4. Perform model alignment on the named entities to be aligned according to the named entity alignment model set to obtain an alignment result.

[0112] In an embodiment of the present invention, in order to improve the efficiency of alignment, before the named entities to be aligned are model aligned according to a named entity alignment model set, the standard named entities to be aligned are used to perform morphological alignment in a pre-built standard named entity library, wherein the standard named entity library is a collection of standard named entities, and the standard named entities are named entities with official uniform identification.

[0113] In detail, the step of using the standard named entities to be aligned to perform morphological alignment in a pre-built standard named entity library includes:

[0114] S21, using the standard named entities to be aligned to perform morphological alignment in a pre-built standard named entity library, and if the morphological alignment is successful, obtaining the alignment result;

[0115] S22: If the morphological alignment is unsuccessful, performing model alignment on the standard named entities to be aligned according to the named entity alignment model set.

[0116] Preferably, the embodiment of the present invention utilizes the edit distance method to perform the morphological alignment.

[0117] In detail, the morphological alignment includes:

[0118] S211, calculating the edit distance between the standard named entity to be aligned and each standard named entity in the standard named entity library;

[0119] S212: When a target edit distance equal to a preset edit distance value exists in the edit distances, it is determined that the alignment is successful, and the standard named entity corresponding to the target edit distance is selected as the alignment result.

[0120] Preferably, the preset edit distance value is 0.

[0121] Specifically, the edit distance refers to the number of times a string is processed to transform it into another string. For example, if the standard named entity to be aligned is "Alibaba Network Technology Co., Ltd." and the standard named entity library contains "Alibaba Network Technology Co., Ltd.", then "Alibaba Network Technology Co., Ltd." needs 0 processing times to be transformed into "Alibaba Network Technology Co., Ltd.", so the edit distance between the two is 0.

[0122] Further, in an embodiment of the present invention, performing model alignment on the named entities to be aligned according to the named entity alignment model set includes:

[0123] S31, converting each character in the standard named entity to be aligned into a character vector of a predetermined dimension, calculating the average value of the character vectors corresponding to all characters in the standard named entity to be aligned, and obtaining a standard named entity vector to be aligned;

[0124] Preferably, in an embodiment of the present invention, an embedding method is used to convert each character in the standard named entity to be aligned into a character vector of a predetermined dimension.

[0125] S32, using each named entity alignment model in the named entity alignment model set to align the standard named entity vector to be aligned, to obtain a predicted aligned entity vector;

[0126] S33, converting each standard named entity in the standard named entity library into a standard named entity vector, and summarizing all the standard named entity vectors to obtain a standard named entity vector library;

[0127] In detail, in an embodiment of the present invention, converting each standard named entity in the standard named entity library into a standard named entity vector includes:

[0128] S331, converting each character in the standard named entity into a character vector of a predetermined dimension;

[0129] S332: Calculate the average value of the character vectors corresponding to all characters in the standard named entity to obtain a standard named entity vector.

[0130] S34, performing similarity calculation and analysis processing on the predicted aligned entity vector and each of the standard named entity vectors in the standard named entity vector library to obtain the alignment result.

[0131] Furthermore, in the embodiment of the present invention, the similarity calculation and analysis process includes:

[0132] S41, calculating the similarity value between the predicted aligned entity vector and each of the standard named entity vectors in the standard named entity vector library;

[0133] Preferably, the embodiment of the present invention uses cosine similarity to calculate the similarity value.

[0134] In detail, the similarity value can be calculated using the following formula:

[0135]

[0136] Where x represents the predicted aligned entity vector, y represents the standard named entity vector, and x i represents the i-th vector value of the predicted aligned entity vector, y i represents the i-th vector value of the standard named entity vector, i is a positive integer, and n represents the vector dimension of the predicted aligned entity vector and the standard named entity vector.

[0137] S42, summing up all the similarity values ​​to obtain a similarity set, and determining the maximum similarity value in the similarity set;

[0138] S43, selecting the standard named entity vector corresponding to the maximum similarity value in the standard named entity vector library as the target vector, and selecting the standard named entity corresponding to the target vector in the standard named entity library as the result to be aligned;

[0139] S44, summarizing all the results to be aligned to obtain the result set to be aligned;

[0140] S45. Filter the set of results to be aligned using a majority voting mechanism to obtain the alignment result.

[0141] Specifically, in the embodiment of the present invention, the majority voting mechanism is used to screen the to-be-aligned result entities in the to-be-aligned result set, including:

[0142] S51, recording the number of occurrences of each of the to-be-aligned results in the to-be-aligned result set; selecting the to-be-aligned result with the largest number of occurrences as the candidate alignment result, and determining the number of the candidate alignment results; if the number is one, determining the candidate alignment result as the alignment result.

[0143] In an embodiment of the present invention, for example, there are five results to be aligned in the set of results to be aligned, including three "Alibaba" and two "Ali Company", so "Alibaba" appears the most times, so there is only one alternative alignment result "Alibaba", so the number of alternative alignment results is one, and the alternative alignment result "Alibaba" is the alignment result.

[0144] S52: If the number is greater than one, sum up the similarity values ​​corresponding to each result to be aligned in the result to be aligned set to obtain a similarity set of results to be aligned, and select the result to be aligned corresponding to the maximum similarity value in the similarity set of results to be aligned as the alignment result.

[0145] In an embodiment of the present invention, for example: there are five results to be aligned in the set of results to be aligned, including 2 "Alibaba", 2 "Ali Company", and 1 "Ali Group", then "Alibaba" and "Ali Company" have the largest number of occurrences, so there are only two alternative alignment results "Alibaba" and "Ali Company", so the number of alternative alignment results is two greater than one, and the alignment result with the corresponding similarity among the five results to be aligned is selected as the alignment result.

[0146] In an embodiment of the present invention, the named entities to be aligned are standardized to obtain standard named entities to be aligned, and the influence of the alignment named entity format and irrelevant characters is eliminated; the test named entity set is sampled to obtain a test named entity subset, each of the test named entity subsets is used to train a preset neural network model to obtain a named entity alignment model set, and multiple test named entity subsets obtained by sampling are used to train a named entity alignment model set, and the named entities to be aligned are model aligned according to the named entity alignment model set, and multiple models are used to perform model alignment respectively, thereby improving the accuracy of named entity alignment.

[0147] like Figure 2 , which is a functional module diagram of the named entity alignment device of the present invention.

[0148] The named entity alignment device 100 of the present invention can be installed in an electronic device. According to the functions to be implemented, the named entity alignment device can include a standardization module 101, a model training module 102, and a model alignment module 103. The module described in the present invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0149] In this embodiment, the functions of each module / unit are as follows:

[0150] The standardization module 101 is used to obtain the named entities to be aligned, and perform standardization processing on the named entities to be aligned to obtain standard named entities to be aligned.

[0151] In the embodiment of the present invention, the named entities are names of people, names of organizations, names of places, and all other entities identified by names, and the named entities to be aligned are named entities that do not use a unified identifier. For example, "Ali Group" and "Alibaba" are different entity identifier names, both of which represent the entity "Alibaba Network Technology Co., Ltd.", so "Ali Group" and "Alibaba" are the named entities to be aligned. The named entities to be aligned can be obtained from the Internet.

[0152] Furthermore, in the embodiment of the present invention, the named entities to be aligned are standardized to obtain the standard named entities to be aligned.

[0153] Specifically, the standardization processing performed by the standardization module 101 includes: converting the named entities to be aligned into simplified and traditional Chinese characters, converting the named entities to be aligned into uppercase and lowercase characters, and deleting special characters to obtain the standard named entities to be aligned. The special characters refer to meaningless symbols in the named entities to be aligned, such as spaces, brackets, etc.

[0154] The model training module 102 is used to obtain a test named entity set, sample the test named entity set to obtain a test named entity subset; use each of the test named entity subsets to train a preset neural network model to obtain a named entity alignment model set.

[0155] In the embodiment of the present invention, the test named entity set is a set of multiple named entities.

[0156] Preferably, in the embodiment of the present invention, the data in the test named entity set is less and difficult to obtain, and the model training module 102 performs sampling processing on the test named entity set to obtain the test named entity subset to expand the data for subsequent model training.

[0157] The model training module 102 described in the embodiment of the present invention converts each test named entity in the test named entity subset into a test named entity vector to obtain a test named entity vector subset; the model training module 102 determines the test named entity vector subset as a training set; the model training module 102 labels the test named entity vector subset to obtain a label set; the model training module 102 uses the training set and the label set to train the neural network model to obtain a named entity alignment model.

[0158] Specifically, the model training module 102 in the embodiment of the present invention uses the following means to mark the test named entity vector subset:

[0159] Converting standard named entities in a pre-built standard named entity library into standard named entity vectors;

[0160] In detail, the standard named entity library is a collection of standard named entities, and the standard named entities are named entities with official unified identification.

[0161] The standard named entity vector is used to label the corresponding test named entity vector in the test named entity vector subset.

[0162] For example: the test named entity "Alibaba" in the test named entity subset corresponds to the standard named entity "Alibaba Network Technology Co., Ltd." in the pre-built standard named entity library, then the standard named entity vector converted from the standard named entity "Alibaba Network Technology Co., Ltd." is used to mark the test named entity vector converted from the test named entity "Alibaba" in the test named entity vector subset.

[0163] In detail, in an embodiment of the present invention, the above-mentioned sampling process can obtain multiple test named entity subsets, each of which can train a neural network model to obtain a named entity alignment model. For example: the test named entity set is sampled to obtain 5 test named entity subsets, each of which is trained with a preset neural network model to obtain a named entity alignment model, and a total of 5 named entity alignment models are obtained.

[0164] Preferably, there is less data in the test named entity subset described in the embodiment of the present invention, and an overly deep neural network will cause the model to overfit and the model effect to be poor. Therefore, the neural network model described in the embodiment of the present invention can be constructed using a shallow convolutional neural network.

[0165] In detail, the model training module 102 trains the neural network model using the following means:

[0166] A: performing convolution and pooling operations on the training set according to a preset number of convolution and pooling operations to obtain a dimension-reduced data set;

[0167] B: performing a deconvolution operation on the reduced-dimensionality data set according to a preset number of deconvolution operations to obtain an increased-dimensionality data set;

[0168] C: Calculate the dimension-increased data set using a preset activation function to obtain a predicted value, and use the predicted value and the label value included in the label set as input parameters of a pre-constructed loss function to calculate a loss value;

[0169] D: Compare the loss value with the preset loss threshold. If the loss value is greater than or equal to the loss threshold, return A; if the loss value is less than the loss threshold, obtain the named entity alignment model.

[0170] Furthermore, the model training module 102 aggregates all the named entity alignment models to obtain the named entity alignment model set.

[0171] In another embodiment of the present invention, data used for training each model in the named entity alignment model set may be stored in a blockchain.

[0172] The model alignment module 103 is used to perform model alignment on the named entities to be aligned according to a named entity alignment model set to obtain an alignment result.

[0173] In an embodiment of the present invention, in order to improve the efficiency of alignment, before performing model alignment on the named entities to be aligned according to a named entity alignment model set, the model alignment module 103 performs morphological alignment in a pre-built standard named entity library using the standard named entities to be aligned, wherein the standard named entity library is a collection of standard named entities, and the standard named entities are named entities with official uniform identification.

[0174] In detail, the model alignment module 103 performs morphological alignment in the pre-built standard named entity library using the following means:

[0175] Using the standard named entities to be aligned to perform morphological alignment in a pre-built standard named entity library, if the morphological alignment is successful, obtaining the alignment result;

[0176] If the morphological alignment is unsuccessful, model alignment is performed on the standard named entities to be aligned according to the named entity alignment model set.

[0177] Preferably, the embodiment of the present invention utilizes the edit distance method to perform the morphological alignment.

[0178] In detail, the model alignment module 103 performs morphology by:

[0179] Calculating the edit distance between the standard named entity to be aligned and each standard named entity in the standard named entity library;

[0180] When a target edit distance equal to a preset edit distance value exists in the edit distances, it is determined that the alignment is successful, and the standard named entity corresponding to the target edit distance is selected as the alignment result.

[0181] Preferably, the preset edit distance value is 0.

[0182] Specifically, the edit distance refers to the number of times a string is processed to transform it into another string. For example, if the standard named entity to be aligned is "Alibaba Network Technology Co., Ltd." and the standard named entity library contains "Alibaba Network Technology Co., Ltd.", then "Alibaba Network Technology Co., Ltd." needs 0 processing times to be transformed into "Alibaba Network Technology Co., Ltd.", so the edit distance between the two is 0.

[0183] Furthermore, in the embodiment of the present invention, the model alignment module 103 performs model alignment on the named entities to be aligned by using the following means:

[0184] Convert each character in the standard named entity to be aligned into a character vector of a predetermined dimension, calculate the average value of the character vectors corresponding to all characters in the standard named entity to be aligned, and obtain a standard named entity vector;

[0185] Preferably, in an embodiment of the present invention, an embedding method is used to convert each character in the standard named entity to be aligned into a character vector of a predetermined dimension.

[0186] Using each named entity alignment model in the named entity alignment model set to align the standard named entity vector to be aligned, to obtain a predicted aligned entity vector;

[0187] Convert each standard named entity in the standard named entity library into a standard named entity vector, and aggregate all the standard named entity vectors to obtain a standard named entity vector library;

[0188] In detail, in the embodiment of the present invention, the model alignment module 103 converts each standard named entity in the standard named entity library into a standard named entity vector by using the following means:

[0189] Convert each word in the standard named entity into a word vector of a predetermined dimension;

[0190] The standard named entity vector is obtained by calculating the average value of the word vectors corresponding to all characters in the standard named entity.

[0191] A similarity calculation and analysis process is performed on the predicted aligned entity vector and each of the standard named entity vectors in the standard named entity vector library to obtain the alignment result.

[0192] Furthermore, in the embodiment of the present invention, the model alignment module 103 performs similarity calculation and analysis processing by the following means:

[0193] Calculating a similarity value between the predicted aligned entity vector and each of the standard named entity vectors in the standard named entity vector library;

[0194] Preferably, the model alignment module 103 in the embodiment of the present invention calculates the similarity value using cosine similarity.

[0195] In detail, the model alignment module 103 calculates the similarity value using the following formula:

[0196]

[0197] Where x represents the predicted aligned entity vector, y represents the standard named entity vector, and x i represents the i-th vector value of the predicted aligned entity vector, y i represents the i-th vector value of the standard named entity vector, i is a positive integer, and n represents the vector dimension of the predicted aligned entity vector and the standard named entity vector.

[0198] Aggregating all the similarity values ​​to obtain a similarity set, and determining the maximum similarity value in the similarity set;

[0199] Selecting the standard named entity vector corresponding to the maximum similarity value in the standard named entity vector library as the target vector, and selecting the standard named entity corresponding to the target vector in the standard named entity library as the result to be aligned;

[0200] Summarizing all the results to be aligned to obtain the result set to be aligned;

[0201] The set of results to be aligned is screened using a majority voting mechanism to obtain the alignment result.

[0202] Specifically, in the embodiment of the present invention, the model alignment module 103 uses the following means to screen the to-be-aligned result entities in the to-be-aligned result set:

[0203] The number of occurrences of each of the to-be-aligned results in the to-be-aligned result set is recorded; the to-be-aligned result with the largest number of occurrences is selected as the candidate alignment result, and the number of the candidate alignment results is determined; if the number is one, the candidate alignment result is determined as the alignment result.

[0204] In an embodiment of the present invention, for example, there are five results to be aligned in the set of results to be aligned, including three "Alibaba" and two "Ali Company", so "Alibaba" appears the most times, so there is only one alternative alignment result "Alibaba", so the number of alternative alignment results is one, and the alternative alignment result "Alibaba" is the alignment result.

[0205] If the number is greater than one, the similarity values ​​corresponding to each result to be aligned in the result to be aligned set are summarized to obtain a similarity set of results to be aligned, and the result to be aligned corresponding to the maximum similarity value in the similarity set of results to be aligned is selected as the alignment result.

[0206] In an embodiment of the present invention, for example: there are five results to be aligned in the set of results to be aligned, including 2 "Alibaba", 2 "Ali Company", and 1 "Ali Group", then "Alibaba" and "Ali Company" have the largest number of occurrences, so there are only two alternative alignment results "Alibaba" and "Ali Company", so the number of alternative alignment results is two greater than one, and the alignment result with corresponding similarity among the five results to be aligned is selected as the alignment result.

[0207] like Figure 3 FIG. 1 is a schematic diagram of the structure of an electronic device for implementing the named entity alignment method of the present invention.

[0208] The electronic device 1 may include a processor 10, a memory 11 and a bus, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a named entity alignment program.

[0209] Among them, the memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of the electronic device 1, such as a mobile hard disk of the electronic device 1. In other embodiments, the memory 11 can also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 1. Further, the memory 11 can also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device 1, such as the code of the named entity alignment program, etc., but also can be used to temporarily store data that has been obtained or is to be obtained.

[0210] The processor 10 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The processor 10 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes or executes programs or modules (such as named entity alignment programs, etc.) stored in the memory 11, and calls data stored in the memory 11 to execute various functions of the electronic device 1 and process data.

[0211] The bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (FISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.

[0212] Figure 3 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 3The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0213] For example, although not shown, the electronic device 1 may also include a power source (such as a battery) for supplying power to each component. Preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so that the power management device can realize functions such as charging management, discharging management, and power consumption management. The power source may also include any components such as one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, etc. The electronic device 1 may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0214] Furthermore, the electronic device 1 may also include a network interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device 1 and other electronic devices.

[0215] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 1 and to display a visual user interface.

[0216] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0217] The named entity alignment program 12 stored in the memory 11 in the electronic device 1 is a combination of multiple instructions, and when executed in the processor 10, it can achieve:

[0218] Acquire named entities to be aligned, and perform standardization processing on the named entities to be aligned to obtain standard named entities to be aligned;

[0219] Acquire a test named entity set, perform sampling processing on the test named entity set, and obtain a test named entity subset;

[0220] Using each test named entity subset to train a preset neural network model, a named entity alignment model set is obtained;

[0221] Model alignment is performed on the named entities to be aligned according to the named entity alignment model set to obtain an alignment result.

[0222] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to Figure 1 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0223] Furthermore, if the module / unit integrated in the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0224] Furthermore, the computer-usable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0225] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0226] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0227] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0228] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0229] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0230] The blockchain referred to in the present invention is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the blockchain underlying platform, platform product service layer, and application service layer.

[0231] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the system claim can also be implemented by one unit or device through software or hardware. The second and other words are used to indicate names, but not to indicate any particular order.

[0232] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for named entity alignment, characterized in that: The method comprises: Acquire named entities to be aligned, and perform standardization processing on the named entities to be aligned to obtain standard named entities to be aligned; Acquire a test named entity set, perform sampling processing on the test named entity set, and obtain a test named entity subset; Using each test named entity subset to train a preset neural network model, a named entity alignment model set is obtained; Performing model alignment on the named entities to be aligned according to the named entity alignment model set to obtain an alignment result; Before performing model alignment on the standard named entities to be aligned according to the named entity alignment model set, the method further includes: performing morphological alignment on the standard named entities to be aligned in a pre-built standard named entity library using the standard named entities to be aligned, and obtaining the alignment result if the morphological alignment is successful; and performing model alignment on the standard named entities to be aligned according to the named entity alignment model set if the morphological alignment is unsuccessful; The method of performing model alignment on the named entities to be aligned according to the named entity alignment model set to obtain an alignment result includes: converting each character in the standard named entities to be aligned into a character vector of a predetermined dimension, calculating the average value of the character vectors corresponding to all characters in the standard named entities to be aligned, and obtaining a standard named entity vector to be aligned; using each named entity alignment model in the named entity alignment model set to perform alignment processing on the standard named entity vector to obtain a predicted aligned entity vector; converting each standard named entity in the standard named entity library into a standard named entity vector, aggregating all the standard named entity vectors, and obtaining a standard named entity vector library; performing similarity calculation and analysis processing on the predicted aligned entity vector and each standard named entity vector in the standard named entity vector library to obtain the alignment result.

2. The named entity alignment method according to claim 1, characterized in that: The method uses each test named entity subset to train a preset neural network model to obtain a named entity alignment model set, including: Convert each test named entity in the test named entity subset into a test named entity vector to obtain a test named entity vector subset; Determining the test named entity vector subset as a training set; Labeling the test named entity vector subset to obtain a label set; Using the training set and the label set to train the neural network model to obtain a named entity alignment model; All the named entity alignment models are aggregated to obtain the named entity alignment model set.

3. The named entity alignment method according to claim 1, characterized in that: The step of performing morphological alignment in a pre-built standard named entity library using the standard named entities to be aligned, and obtaining the alignment result if the morphological alignment is successful, includes: Calculating the edit distance between the standard named entity to be aligned and each standard named entity in the standard named entity library; When a target edit distance equal to a preset edit distance value exists in the edit distances, it is determined that the alignment is successful, and a standard named entity corresponding to the target edit distance is selected as the alignment result.

4. The named entity alignment method according to claim 1, characterized in that: The performing similarity calculation and analysis processing on the predicted aligned entity vector and each of the standard named entity vectors in the standard named entity vector library to obtain the alignment result includes: Calculating a similarity value between the predicted aligned entity vector and each of the standard named entity vectors in the standard named entity vector library; Aggregating all the similarity values ​​to obtain a similarity set; Determine the maximum similarity value in the similarity set; Selecting the standard named entity vector corresponding to the maximum similarity value in the standard named entity vector library as the target vector; Selecting a standard named entity corresponding to the target vector in the standard named entity library as a result to be aligned; Summarizing all the results to be aligned to obtain a set of results to be aligned; The set of results to be aligned is screened using a majority voting mechanism to obtain the alignment result.

5. The named entity alignment method according to claim 4, characterized in that: The using a majority voting mechanism to screen the set of results to be aligned to obtain the alignment result includes: Recording the number of occurrences of each result to be aligned in the set of results to be aligned; Select the alignment result with the largest number of occurrences as the candidate alignment result; Determining the number of candidate alignment results; If the number is one, determining the candidate alignment result as the alignment result; If the number is greater than one, the similarity values ​​corresponding to each result to be aligned in the result to be aligned set are summarized to obtain a similarity set of results to be aligned, and the result to be aligned corresponding to the maximum similarity value in the similarity set of results to be aligned is selected as the alignment result.

6. A named entity alignment device, used to implement the named entity alignment method according to any one of claims 1 to 5, characterized in that: The device comprises: A standardization module is used to obtain the named entities to be aligned, and perform standardization processing on the named entities to be aligned to obtain standard named entities to be aligned; A model training module is used to obtain a test named entity set, perform sampling processing on the test named entity set, and obtain a test named entity subset; use each test named entity subset to train a preset neural network model to obtain a named entity alignment model set; The model alignment module is used to train a preset neural network model using each test named entity subset to obtain a named entity alignment model set.

7. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the named entity alignment method according to any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the named entity alignment method according to any one of claims 1 to 5 is implemented.

Images