Method and apparatus for word sense disambiguation

Abstract

The application provides a word sense disambiguation method and device, wherein the method comprises the following steps: determining a to-be-disambiguated entity existing in to-be-disambiguated text and a corresponding candidate entity list based on an RPA knowledge graph; determining embedding features corresponding to each candidate entity in the candidate entity list and the to-be-disambiguated entity through RPA feature extraction based on each candidate entity and the to-be-disambiguated entity; and determining whether the to-be-disambiguated entity and the candidate entity are the same entity based on a word sense disambiguation model, the embedding features corresponding to each candidate entity, and the embedding features corresponding to the to-be-disambiguated entity. The application realizes the comparison of embedding features of the to-be-disambiguated text and the candidate text by comprehensively embedding entity features, entity context features and word features, determines whether the to-be-disambiguated entity and the candidate entity are the same entity, and obtains more rich and comprehensive text information, which is conducive to accurately analyzing the word sense and improving the word sense disambiguation accuracy.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method and apparatus for word sense disambiguation. Background Technology

[0002] With the rapid development of artificial intelligence technology, there is an increasing amount of research on enabling computers to more accurately recognize human language.

[0003] Current word sense disambiguation technologies primarily rely on semantic similarity between candidate and target texts, analyzing each character individually. This approach is limited in its analysis of the intrinsic features within both texts, and the feature extraction methods are relatively simplistic. Furthermore, these technologies require a large number of training samples, making them unsuitable for scenarios with small-scale labeled samples. Performance degrades significantly when the corpus is poorly configured or lacks sufficient information. Therefore, effectively extracting relevant information from the text and combining it to automatically and efficiently disambiguate word senses remains a critical challenge for the industry. Summary of the Invention

[0004] In view of the problems existing in the prior art, this application provides a method and apparatus for word sense disambiguation.

[0005] Firstly, this application provides a method for word meaning disambiguation, including:

[0006] Based on RPA knowledge graph, identify the entities to be disambiguated in the text to be disambiguated, and the corresponding candidate entity list;

[0007] Based on each candidate entity in the candidate entity list and the entity to be disambiguated, the embedded features corresponding to the candidate entity and the embedded features corresponding to the entity to be disambiguated are determined by RPA feature extraction.

[0008] Based on the word sense disambiguation model, the embedded features corresponding to each candidate entity and the embedded features corresponding to the entity to be disambiguated are used to determine whether the entity to be disambiguated and the candidate entity are the same entity.

[0009] The embedding features include: entity embedding features, entity context embedding features, and word embedding features.

[0010] Optionally, the step of determining the embedded features corresponding to each candidate entity and the entity to be disambiguated based on each candidate entity in the candidate entity list and through RPA feature extraction includes:

[0011] Based on the improved BERT dictionary, the index encoding sequence corresponding to the candidate entity text is determined as the word embedding feature of the candidate entity text, and the index encoding sequence corresponding to the text to be disambiguated is determined as the word embedding feature of the text to be disambiguated; the candidate entity text is determined based on any candidate entity in the candidate entity list and the RPA knowledge graph;

[0012] Based on the entity embedding model and RPA knowledge graph, determine the entity embedding features corresponding to each candidate entity, and / or the entity embedding features corresponding to the entities already determined in the text to be disambiguated;

[0013] Based on RPA knowledge graph and distance weight algorithm, determine the entity context embedding feature corresponding to each candidate entity, and / or the entity context embedding feature corresponding to the entity already determined in the text to be disambiguated.

[0014] Optionally, the step of determining the index encoding sequence corresponding to the candidate entity text as the word embedding feature of the candidate entity text and the index encoding sequence corresponding to the text to be disambiguated as the word embedding feature of the text to be disambiguated based on the improved BERT dictionary includes:

[0015] The candidate entity text and the text to be disambiguated are pre-segmented respectively to determine the first pre-segmentation set and the second pre-segmentation set;

[0016] Based on the improved BERT dictionary, the index encoding sequence corresponding to the first pre-segmented word set and the index encoding sequence corresponding to the first pre-segmented word set are determined.

[0017] Optionally, the entity embedding model is trained based on the following method:

[0018] Based on the TranSparse method, the first spatial vector and the second spatial vector in the first loss function are determined; the first spatial vector is the spatial vector of the head entity and the inter-entity relationship under each inter-entity relationship in the RPA knowledge graph, and the second spatial vector is the spatial vector of the tail entity and the inter-entity relationship under each inter-entity relationship in the RPA knowledge graph.

[0019] Based on the first loss function, and the constructed first sample data and second sample data, the value of the first loss function is determined;

[0020] The entity embedding model is obtained when the value of the first loss function is less than the first threshold.

[0021] Optionally, determining the entity embedding features corresponding to the candidate entity and / or the entity embedding features corresponding to the already determined entities in the text to be disambiguated, based on the entity embedding model and RPA knowledge graph, includes:

[0022] Based on RPA knowledge graph, a first set of corresponding inter-entity relationships is determined for the candidate entities, and / or a second set of corresponding inter-entity relationships is determined for the entities already identified in the text to be disambiguated; the first set includes all entities that have inter-entity relationships with the candidate entities and their corresponding inter-entity relationships, and the second set includes all entities that have inter-entity relationships with the identified entities in the text to be disambiguated and their corresponding inter-entity relationships;

[0023] Based on the entity embedding model, the values ​​of the first spatial vector and the second spatial vector corresponding to the first set are determined as entity embedding features corresponding to the candidate entities, and / or the values ​​of the first spatial vector and the second spatial vector corresponding to the second set are determined as entity embedding features corresponding to the entities already identified in the text to be disambiguated.

[0024] Optionally, the step of determining the entity context embedding features corresponding to each candidate entity based on RPA knowledge graph and distance weight algorithm, and / or the entity context embedding features corresponding to the entities already determined in the text to be disambiguated, includes:

[0025] Based on RPA knowledge graph, determine the first knowledge graph subgraph corresponding to each candidate entity, and / or the second knowledge graph subgraph corresponding to the entity already determined in the text to be disambiguated;

[0026] Based on the first knowledge graph subgraph and the distance weight algorithm, the entity context embedding features corresponding to the candidate entity are determined, and / or based on the second knowledge graph subgraph and the distance weight algorithm, the entity context embedding features corresponding to the entities already identified in the text to be disambiguated are determined.

[0027] Optionally, the step of determining whether the entity to be disambiguated and the candidate entity are the same entity based on the word sense disambiguation model, using the embedded features corresponding to each candidate entity and the embedded features corresponding to the entity to be disambiguated, includes:

[0028] Based on the word sense disambiguation model, the similarity between the candidate entity text and the text to be disambiguated is determined; the candidate entity text is determined based on any candidate entity in the candidate entity list and the RPA knowledge graph.

[0029] Based on pre-set confidence rules and the similarity between the candidate entity text and the text to be disambiguated, it is determined whether the entity to be disambiguated and the candidate entity are the same entity.

[0030] Optionally, the word sense disambiguation model is trained based on the following method:

[0031] Based on the preset masking rules, and the constructed first and second sample data, the third and fourth sample data are determined for training the word sense disambiguation model.

[0032] The value of the second loss function is determined based on the second loss function, as well as the third and fourth sample data; the second loss function is an improved cross-entropy loss function.

[0033] When the value of the second loss function is less than the second threshold, the word sense disambiguation model is obtained.

[0034] Optionally, the first sample data is obtained by labeling triples based on RPA knowledge graphs; the second sample data is obtained by replacing the head entity or tail entity corresponding to any entity relationship in the first sample with the head entity or tail entity corresponding to different entity relationships.

[0035] Optionally, the improved cross-entropy loss function is determined based on the cross-loss function in BERT.

[0036] Optionally, determining whether the entity to be disambiguated and the candidate entity are the same entity based on preset confidence rules and the similarity between the candidate entity text and the text to be disambiguated includes:

[0037] If the similarity satisfies the preset confidence rule, then it is determined that the entity to be disambiguated and the candidate entity are the same entity.

[0038] If the similarity does not meet the preset confidence rule, then it is determined that the entity to be disambiguated and the candidate entity are not the same entity.

[0039] Optionally, after determining whether the entity to be disambiguated and the candidate entity are the same entity based on preset confidence rules and the similarity between the candidate entity text and the text to be disambiguated, the process includes:

[0040] If the entity to be disambiguated and the candidate entity are the same entity, then the entity name of the entity to be disambiguated is used as an alias to update the RPA knowledge graph;

[0041] If the entity to be disambiguated and the candidate entity are not the same entity, then the entity to be disambiguated is added to the RPA knowledge graph, and the corresponding inter-entity relationship is updated.

[0042] Secondly, this application also provides a word meaning disambiguation device, comprising:

[0043] The candidate entity module is used to identify the entities to be disambiguated in the text to be disambiguated, as well as the corresponding candidate entity list, based on the RPA knowledge graph.

[0044] The embedding feature module is used to determine the embedding features corresponding to the candidate entities and the entity to be disambiguated based on each candidate entity in the candidate entity list and the entity to be disambiguated through RPA feature extraction.

[0045] The disambiguation module is used to determine whether the entity to be disambiguated and the candidate entity are the same entity based on the word sense disambiguation model, the embedded features corresponding to each candidate entity, and the embedded features corresponding to the entity to be disambiguated.

[0046] The embedding features include: entity embedding features, entity context embedding features, and word embedding features.

[0047] Thirdly, this application also provides an electronic device, including a processor and a memory storing a computer program, wherein the processor executes the program to implement the word meaning disambiguation method described in the first aspect.

[0048] Fourthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the word sense disambiguation method described in the first aspect above.

[0049] The word sense disambiguation method and apparatus provided in this application compare the embedding features of the text to be disambiguated and the candidate text by integrating entity embedding features, entity context embedding features and word embedding features, and determine whether the entity to be disambiguated and the candidate entity are the same entity. The obtained text information is richer and more comprehensive, which is conducive to accurately analyzing the word sense of the text and improving the accuracy of word sense disambiguation. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a flowchart illustrating the word sense disambiguation method provided in the embodiments of this application;

[0052] Figure 2 This is a flowchart illustrating the implementation of word sense disambiguation models in existing technologies;

[0053] Figure 3This is a schematic diagram illustrating the determination of RPA knowledge graph subgraphs provided in an embodiment of this application;

[0054] Figure 4 This is a schematic diagram of the MatchBERT network structure provided in the embodiments of this application;

[0055] Figure 5 This is a schematic diagram of the overall process of the word sense disambiguation method provided in the embodiments of this application;

[0056] Figure 6 This is a schematic diagram of the structure of the word sense disambiguation device provided in the embodiments of this application;

[0057] Figure 7 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0059] To help understand the solutions of the embodiments of this application, the following will briefly introduce the relevant concepts that may be involved in this application:

[0060] Robotic process automation (RPA) automates repetitive tasks and can be applied to industries such as finance, taxation, human resources, information technology, insurance, customer service, telecommunications, and manufacturing to upgrade business processes. When applied to question-answering systems, knowledge graph technology is used for semantic understanding and judgment. However, due to the poor quality of the question-answering corpus, the constructed knowledge graph contains many ambiguities. Therefore, it is necessary to combine language representation pre-training methods to train the RPA text corpus to obtain a word sense disambiguation model.

[0061] Traditional word sense disambiguation methods often employ the construction of entity alias dictionaries. These dictionaries use the entity's title, redirection name, and disambiguation name as a set of entity names, integrating the one-to-many mapping between names and entities to achieve referential recognition and disambiguation. With the development of deep learning, the focus now shifts to designing artificial features and complex similarity measures to achieve better performance. Current methods primarily use techniques such as CNN, Word2Vec, BERT, and MRC to learn word-by-word expressions and corresponding features in text. However, the resulting text contains relatively limited information and requires significant processing power. Similarity calculations are then performed using this acquired information, and finally, the results are ranked comprehensively to link and disambiguate different words.

[0062] To overcome the shortcomings of existing technologies, a word sense disambiguation method and apparatus are proposed. The RPA feature extraction model is used to obtain the word embedding features, entity embedding features, and entity context embedding features of the entities corresponding to the text to be disambiguated. The extracted features are then input into the word sense disambiguation model, and the features are fully connected to determine the word sense disambiguation result of the entities to be disambiguated in the text.

[0063] Figure 1 This is a flowchart illustrating the word sense disambiguation method provided in the embodiments of this application, as follows: Figure 1 As shown, the method includes:

[0064] Step 101: Based on the RPA knowledge graph, identify the entities to be disambiguated in the text to be disambiguated, and the corresponding candidate entity list;

[0065] Specifically, based on the RPA corpus, an RPA knowledge graph is constructed by designing a schema structure oriented towards the RPA domain. The triples constituting the RPA knowledge graph and the index corresponding to each entity within the triples are determined. The RPA knowledge graph triples can be represented as (head, relation, tail), where head is the head entity, tail is the tail entity, and relation represents the relationship between entities. Both the head and tail entities are word-based, each entity corresponds to an entity index, and each entity has corresponding entity attributes and / or tags, including entity description and entity name. This achieves the networking and vectorization of the RPA corpus into the RPA knowledge graph, providing the entities and relationships included in the RPA knowledge graph for subsequent word sense disambiguation. Compared to traditional character vector features, entity features at the word level offer richer feature information.

[0066] By pre-segmenting the text to be disambiguated and comparing it with the entity attributes of all entities included in the RPA knowledge graph, the entities to be disambiguated in the text and the candidate entity list corresponding to each entity are determined. These entities to be disambiguated mainly refer to entities whose corresponding entities cannot be clearly identified in the RPA knowledge graph. Typically, a word in the text to be disambiguated has a certain degree of similarity to one or more entities in the RPA knowledge graph, or several words have a certain degree of similarity to one or more entities in the RPA knowledge graph. For example, the entity name in the RPA knowledge graph may be exactly the same as the entity name of the entity to be disambiguated, but the entity description is different; or the entity name in the RPA knowledge graph and the entity name of the entity to be disambiguated may match to a certain degree, and the entity descriptions may also have a certain degree of similarity. In such cases, a candidate entity list corresponding to the entity to be disambiguated is determined. The candidate entity list may include one or more candidate entities.

[0067] Step 102: Based on each candidate entity in the candidate entity list and the entity to be disambiguated, determine the embedding features corresponding to the candidate entity and the entity to be disambiguated by RPA feature extraction; wherein, the embedding features include: entity embedding features, entity context embedding features and word embedding features.

[0068] Based on the candidate entity list determined in the above steps, any candidate entity is selected and combined with other entities in the RPA knowledge graph that have inter-entity relationships with that entity, as well as the entity descriptions of the aforementioned entities, to determine the candidate entity text corresponding to that candidate entity. Through RPA feature extraction, the entity embedding features, entity context embedding features, and word embedding features of the candidate entity text are determined.

[0069] Similarly, the text to be disambiguated is extracted using RPA features to determine its corresponding entity embedding features, entity context embedding features, and word embedding features. This yields richer and more comprehensive entity text embedding features, facilitating subsequent word sense disambiguation.

[0070] Step 103: Based on the word sense disambiguation model, the embedded features corresponding to each candidate entity and the embedded features corresponding to the entity to be disambiguated are used to determine whether the entity to be disambiguated and the candidate entity are the same entity.

[0071] After obtaining the entity embedding features, entity context embedding features, and word embedding features corresponding to each candidate entity, as well as the entity embedding features, entity context embedding features, and word embedding features corresponding to the entity to be disambiguated, it is necessary to merge these embedding features for each candidate entity and merge the embedding features corresponding to the entity to be disambiguated. Then, the two merged results are passed through a word sense disambiguation model to determine the word sense disambiguation result of the text to be disambiguated.

[0072] The word sense disambiguation method provided in this application integrates entity embedding features, entity context embedding features, and word embedding features to compare the embedding features of the text to be disambiguated and the candidate text, thereby determining whether the entity to be disambiguated and the candidate entity are the same entity. The obtained text information is richer and more comprehensive, which is conducive to accurately analyzing the word sense of the text and improving the accuracy of word sense disambiguation.

[0073] Optionally, the step of determining the embedded features corresponding to each candidate entity and the entity to be disambiguated based on each candidate entity in the candidate entity list and through RPA feature extraction includes:

[0074] Based on the improved BERT dictionary, the index encoding sequence corresponding to the candidate entity text is determined as the word embedding feature of the candidate entity text, and the index encoding sequence corresponding to the text to be disambiguated is determined as the word embedding feature of the text to be disambiguated; the candidate entity text is determined based on any candidate entity in the candidate entity list and the RPA knowledge graph;

[0075] Based on the entity embedding model and RPA knowledge graph, determine the entity embedding features corresponding to each candidate entity, and / or the entity embedding features corresponding to the entities already determined in the text to be disambiguated;

[0076] Based on RPA knowledge graph and distance weight algorithm, determine the entity context embedding feature corresponding to each candidate entity, and / or the entity context embedding feature corresponding to the entity already determined in the text to be disambiguated.

[0077] Specifically, the commonly used BERT dictionary is built on a character-by-character basis. This application improves the original BERT dictionary by using an RPA knowledge graph, adding all entities and their attributes from the RPA knowledge graph, i.e., adding word-based information to the BERT dictionary. By selecting a candidate entity from the aforementioned candidate entity list and, based on that candidate entity and its related descriptive information, determining the corresponding candidate entity text in the RPA knowledge graph, this candidate entity text is compared with the improved BERT dictionary to determine the index codes corresponding to words or characters in the candidate entity text. These word index codes or character abbreviation codes are then concatenated and merged to determine the index code sequence corresponding to the candidate entity text, which is used as the word embedding feature of the candidate entity text.

[0078] Similarly, the index encoding sequence corresponding to the text to be disambiguated can also be obtained.

[0079] Furthermore, in the RPA knowledge graph, all entities that have inter-entity relationships with the above candidate entities and the corresponding inter-entity relationships are identified; the entity embedding model is input, and the entity embedding features corresponding to each candidate entity are output; and the above processing is performed on each candidate entity in the candidate entity list.

[0080] If there are identified entities in the text to be disambiguated, meaning that one or more words in the text can be identified as corresponding entities in the RPA knowledge graph, then all entities with entity relationships with the identified entities in the text to be disambiguated, along with the corresponding entity relationships, are identified. The input is the entity embedding model, and the output is the entity embedding features corresponding to the identified entities in the text to be disambiguated.

[0081] If there are no identified entities in the text to be disambiguated, that is, if no entity with the same name, meaning, and description can be found in the RPA knowledge graph for all words in the text to be disambiguated, then there is no need to determine the corresponding entity embedding features.

[0082] Furthermore, by using the RPA knowledge graph, we can identify other entities within a certain range that have inter-entity relationships with each candidate entity in the RPA knowledge graph. Combined with the distance weight algorithm, we can determine the context embedding features corresponding to each candidate entity.

[0083] Similarly, if there are already identified entities in the above-mentioned text to be disambiguated, that is, if there is a word or several words in the text to be disambiguated that can be identified as corresponding entities in the RPA knowledge graph, then other entities that have already identified entities in the above-mentioned text to be disambiguated, have relationships between entities, and whose relationships are within a certain range are identified. Combined with the distance weight algorithm, the entity context embedding features corresponding to the identified entities in the text to be disambiguated are determined.

[0084] If there are no identified entities in the text to be disambiguated, that is, if no entity with the same name, meaning, and description can be found in the RPA knowledge graph for all words in the text to be disambiguated, then there is no need to determine the corresponding entity context embedding features.

[0085] The distance weighting algorithm described above can determine the distance between two entities based on the distance of their relationship, i.e. whether the two entities are connected by one entity relationship or multiple entity relationships. Different weight values ​​are assigned to different distances, which better reflects the actual relationship between entities and improves the accuracy of subsequent disambiguation.

[0086] The word sense disambiguation method provided in this application integrates entity embedding features, entity context embedding features, and word embedding features to compare the text to be disambiguated and candidate texts, determine the disambiguation result of the text to be disambiguated, and obtain richer and more comprehensive text information, which is conducive to accurately analyzing the word sense of the text and improving the accuracy of word sense disambiguation.

[0087] Optionally, the step of determining the index encoding sequence corresponding to the candidate entity text as the word embedding feature of the candidate entity text and the index encoding sequence corresponding to the text to be disambiguated as the word embedding feature of the text to be disambiguated based on the improved BERT dictionary includes:

[0088] The candidate entity text and the text to be disambiguated are pre-segmented respectively to determine the first pre-segmentation set and the second pre-segmentation set;

[0089] Based on the improved BERT dictionary, the index encoding sequence corresponding to the first pre-segmented word set and the index encoding sequence corresponding to the first pre-segmented word set are determined.

[0090] Specifically, based on the improved BERT dictionary, candidate entity text and disambiguation entity text are indexed and encoded, transforming them into multi-dimensional vectorized representations. Compared to traditional character embedding methods, word embedding methods consume fewer resources, reducing the number of input neuron parameters and lowering model complexity for training. The aforementioned candidate entity text, after pre-segmentation, is represented as [word1, word2, ..., word...]. n ]; Traverse each word i If a word does not exist in the improved BERT dictionary, it is considered an unregistered word and is segmented into characters; otherwise, its index number is directly taken. Finally, the sentence is converted into a hybrid index encoding sequence of words and characters, which serves as the word embedding feature corresponding to the candidate entity. Similarly, the text to be disambiguated is pre-segmented to determine its corresponding hybrid index encoding sequence, which serves as its word embedding feature.

[0091] The above approach results in shorter sentence encoding, faster processing speed, and improved word sense disambiguation efficiency in RPA; by using words as units, it reduces the semantic uncertainty when using characters as units and lowers the complexity of the word sense disambiguation model.

[0092] Optionally, the entity embedding model is trained based on the following method:

[0093] Based on the TranSparse method, the first spatial vector and the second spatial vector in the first loss function are determined; the first spatial vector is the spatial vector of the head entity and the inter-entity relationship under each inter-entity relationship in the RPA knowledge graph, and the second spatial vector is the spatial vector of the tail entity and the inter-entity relationship under each inter-entity relationship in the RPA knowledge graph.

[0094] Based on the first loss function, and the constructed first sample data and second sample data, the value of the first loss function is determined;

[0095] The entity embedding model is obtained when the value of the first loss function is less than the first threshold.

[0096] Specifically, after determining the word embedding features corresponding to the candidate entities and the entities to be disambiguated, it is also necessary to determine the entity embedding features corresponding to the candidate entities and the entities to be disambiguated. The entity embedding features are obtained through an entity embedding model. The specific methods for training this entity embedding model include:

[0097] 1. Determine the training samples

[0098] Based on RPA knowledge graph, all entities and relationships between entities are identified and labeled as positive samples;

[0099] Then, in the positive samples, the head entities are replaced with head entities or tail entities with different relationships between entities, or the tail entities are replaced with head entities or tail entities with different relationships between entities, or both head entities and tail entities are replaced with head entities and tail entities with different relationships between entities, to obtain negative samples. That is, all entities and relationships between entities in the negative samples are not corresponding or are not correct.

[0100] 2. Training of the entity embedding model constructed based on the TranSparse method

[0101] Figure 2 This is a flowchart illustrating the implementation of a word sense disambiguation model in existing technologies, such as... Figure 2 As shown, existing technologies typically employ methods including distance-based translational models and semantic-based matching models. Distance-based translational models, such as TransE, TransH, TransR, and TransD, generate entity feature embeddings. Semantic-based matching models map entities and relations to a latent semantic space, employing models such as the Semantic Matching Energy Model (SME), the Neural Tensor Network (NTN), the Multilayer Perceptron (MLP), and the Neural Association Model (NAM). However, these disambiguation models tend to consider only a limited number of factors.

[0102] Because RPA question-answering corpora contain both popular and unpopular questions, the triples (head entity, relation, tail entity) in the constructed RPA knowledge graph are imbalanced. When training the entity embedding model, traditional algorithms have a fixed number of model parameters. Therefore, entities or entity pairs with a small number of parameters are prone to overfitting, while entities or entity pairs with a large number of parameters are prone to underfitting. To establish a relationship between model parameters and the number of entity pairs, the TranSparse method is used for entity embedding. The specific implementation process is as follows:

[0103] The head and tail entities are mapped to different relation spaces, and r represents any entity relation in the knowledge graph. and Let represent the sparse mapping matrices corresponding to relation r in the head entity-relation and the sparse mapping matrices corresponding to relation r in the tail entity-relation, respectively. This indicates the number of head entities in the head entity-relation with relation r or the number of tail entities in the tail entity-relation with relation r. The value of l can be h or t, representing head entities or tail entities respectively. This indicates that the relation in the head entity-relation is r. * The maximum number of corresponding header entities. This indicates that the relationship in the tail entity-relation is r. * The maximum number of corresponding tail entities, r * This represents the relation when the number of sparse mapping matrices corresponding to relation r reaches its maximum value. Representing relation r * The maximum number of correspondence matrices sparsity hyperparameter Therefore, the sparsity factor of the "head-relation" sparse mapping matrix is ​​calculated. The sparsity factor of the "tail-relation" sparse mapping matrix Head entity mapping to relation space vector Tail entity mapping to relation space vector Its scoring function is: f r (h,t)=||h p +rt p || 2 Where h, r, and t are the vector values ​​of the original triples, the first loss function during training is calculated as follows:

[0104]

[0105] Where S and S′ represent the sets of positive and negative sample triples, respectively, and γ represents the distance between positive and negative samples. When γ+f r (h,t)-f r When (h′,t′)<0, the corresponding first loss function value is 0. And f r(h,t) represents the score function value after the head and tail entities are mapped to different relation spaces in the positive sample. The smaller the value, the closer the distance between them; f r (h′,t′) represents the score function value of the head and tail entities in the negative sample after they are mapped to different relation spaces. This value should be as large as possible to obtain a smaller loss value.

[0106] 3. Determining the training threshold for entity embedding models

[0107] The training time, the number of training samples, and the final accuracy of the model output can be statistically analyzed to determine a suitable threshold as a criterion for evaluating the completion of entity embedding model training. Alternatively, a preset value can be set and continuously adjusted during later training. The specific method is not limited. Typically, the first threshold is the first loss function value corresponding to an output accuracy greater than 90%.

[0108] If the obtained first loss function value is less than the first threshold, it indicates that the loss of the entity embedding model has met certain requirements, and the accuracy is greater than a certain proportion. That is, the smaller the first threshold, the higher the corresponding accuracy. Of course, the first threshold can also be set smaller according to the requirements, which will consume more training time and training samples to meet higher accuracy requirements.

[0109] Optionally, determining the entity embedding features corresponding to the candidate entity and / or the entity embedding features corresponding to the already determined entities in the text to be disambiguated, based on the entity embedding model and RPA knowledge graph, includes:

[0110] Based on RPA knowledge graph, a first set of corresponding inter-entity relationships is determined for the candidate entities, and / or a second set of corresponding inter-entity relationships is determined for the entities already identified in the text to be disambiguated; the first set includes all entities that have inter-entity relationships with the candidate entities and their corresponding inter-entity relationships, and the second set includes all entities that have inter-entity relationships with the identified entities in the text to be disambiguated and their corresponding inter-entity relationships;

[0111] Based on the entity embedding model, the values ​​of the first spatial vector and the second spatial vector corresponding to the first set are determined as entity embedding features corresponding to the candidate entities, and / or the values ​​of the first spatial vector and the second spatial vector corresponding to the second set are determined as entity embedding features corresponding to the entities already identified in the text to be disambiguated.

[0112] Specifically, in the RPA knowledge graph, all entities that have inter-entity relationships with the aforementioned candidate entities and their corresponding inter-entity relationships are identified as the first set. This first set is then input into the trained entity embedding model to obtain the first spatial vector h corresponding to the first set. pThe value of the second space vector t p The value of the first spatial vector h p The value of the second space vector t p The value is used as the entity embedding feature of the candidate entity. The above processing is then performed on each candidate entity in the candidate entity list.

[0113] If there are identified entities in the text to be disambiguated, that is, if there is a word or several words in the text that can be identified as corresponding entities in the RPA knowledge graph, then all entities that have entity relationships with the identified entities in the text to be disambiguated, as well as the corresponding entity relationships, are identified as the second set. This second set is then input into the trained entity embedding model to obtain the first spatial vector h corresponding to the second set. p The value of the second space vector t p The value of the first space vector h corresponding to the second set. p The value of the second space vector t p The value of is used as the entity embedding feature of the text to be disambiguated, that is, the entity embedding feature corresponding to the entity already determined in the text to be disambiguated.

[0114] If there are no identified entities in the text to be disambiguated, that is, if no entity with the same name, meaning, and description can be found in the RPA knowledge graph for all words in the text to be disambiguated, then there is no need to determine the corresponding entity embedding features.

[0115] Optionally, the step of determining the entity context embedding features corresponding to each candidate entity based on RPA knowledge graph and distance weight algorithm, and / or the entity context embedding features corresponding to the entities already determined in the text to be disambiguated, includes:

[0116] Based on RPA knowledge graph, determine the first knowledge graph subgraph corresponding to each candidate entity, and / or the second knowledge graph subgraph corresponding to the entity already determined in the text to be disambiguated;

[0117] Based on the first knowledge graph subgraph and the distance weight algorithm, the entity context embedding features corresponding to the candidate entity are determined, and / or based on the second knowledge graph subgraph and the distance weight algorithm, the entity context embedding features corresponding to the entities already identified in the text to be disambiguated are determined.

[0118] Specifically, after the word embedding features and entity embedding features of the candidate entities and the entities to be disambiguated are determined, it is also necessary to determine the corresponding entity context embedding features. By combining the above features, the information obtained is richer and more comprehensive.

[0119] The specific implementation method is as follows:

[0120] Based on the associations of each candidate entity in the RPA knowledge graph, a first knowledge graph subgraph corresponding to each candidate entity is determined. Entities in the first knowledge graph subgraph satisfy the condition that their distance from the candidate entity is less than or equal to a preset hop count threshold. Here, hop count refers to the minimum number of inter-entity relationships required to connect two entities; that is, if the minimum number of inter-entity relationships connecting two entities is less than or equal to this preset hop count threshold, then the entity is determined to belong to the first knowledge graph subgraph. Of course, the first knowledge graph subgraph includes all inter-entity relationships between all the entities determined above.

[0121] If there are identified entities in the text to be disambiguated, meaning that one or more words in the text can be identified as corresponding entities in the RPA knowledge graph, then the second knowledge graph subgraph corresponding to the identified entities in the text to be disambiguated is determined. Entities in the second knowledge graph subgraph satisfy the condition that their distance from the identified entities in the text to be disambiguated is less than or equal to a preset hop count threshold. Here, hop count refers to the minimum number of inter-entity relationships required to connect two entities; that is, the minimum number of inter-entity relationships required to establish a connection between two entities is less than or equal to this preset hop count threshold. The second knowledge graph subgraph includes all inter-entity relationships between the entities identified above.

[0122] If there are no identified entities in the text to be disambiguated, that is, if no entity with the same name, meaning, and description can be found in the RPA knowledge graph for all words in the text to be disambiguated, then there is no need to determine the corresponding entity context features.

[0123] For example, the preset hop count threshold is 1, meaning that two entities are connected through at least one inter-entity relationship. Figure 3 This is a schematic diagram of determining the subgraph of the RPA knowledge graph provided in the embodiments of this application, such as... Figure 3 As shown in the figure, the dashed lines represent either the first or second knowledge graph subgraph. Entities with diagonal backgrounds represent any entity in the candidate entity list, or a definite entity in the text to be disambiguated. Entities with dotted backgrounds represent other entities in the RPA knowledge graph that satisfy a preset hop count threshold along with the diagonal background entities. The diagonal background entities and dotted background entities, along with the corresponding relationships between entities, constitute the first or second knowledge graph subgraph.

[0124] First, determine the context of the entity, as follows:

[0125] C(e)={e i |(e,r,e i )∈G or(e i ,r,e)∈G}

[0126] Where e represents an entity, C(e) represents the list of entities in the context of the entity, r represents the relationship between any entities, and G represents a subgraph of the knowledge graph, which can be either the first or the second subgraph of the knowledge graph.

[0127] Then, for a given entity e's context, the context embedding is the weighted average of its context entities. The calculation formula is:

[0128]

[0129] Among them, e i e learned through knowledge graph embedding i The entity embedding corresponds to the vector value, w i It is the distance weight value between any entity in the subgraph and a given entity e. Since context entities are usually closely related to the current entity semantically and logically, introducing more contextual supplementary feature information helps improve the identifiability of the entity.

[0130] Based on the above method, the entity context embedding features corresponding to each candidate entity are determined, and / or the entity context embedding features corresponding to the entities already identified in the text to be disambiguated are determined.

[0131] The word sense disambiguation method provided in this application integrates entity embedding features, entity context embedding features, and word embedding features to compare the text to be disambiguated and candidate texts, determine the disambiguation result of the text to be disambiguated, and obtain richer and more comprehensive text information, which is conducive to accurately analyzing the word sense of the text and improving the accuracy of word sense disambiguation.

[0132] Optionally, the step of determining whether the entity to be disambiguated and the candidate entity are the same entity based on the word sense disambiguation model, using the embedded features corresponding to each candidate entity and the embedded features corresponding to the entity to be disambiguated, includes:

[0133] Based on the word sense disambiguation model, the similarity between the candidate entity text and the text to be disambiguated is determined; the candidate entity text is determined based on any candidate entity in the candidate entity list and the RPA knowledge graph.

[0134] Based on pre-set confidence rules and the similarity between the candidate entity text and the text to be disambiguated, it is determined whether the entity to be disambiguated and the candidate entity are the same entity.

[0135] Specifically, after confirming the word embedding features, entity embedding features, entity context embedding features, and / or the word embedding features, entity embedding features, and entity context embedding features corresponding to the entities identified in the text to be disambiguated, the above embedding features need to be compared to determine the corresponding disambiguation results.

[0136] The embedded features mentioned above are combined and disambiguated using a word sense disambiguation model. This model is based on the MatchBERT network, which is an improved version of the BERT network designed for RPA applications and is primarily used for word sense disambiguation in knowledge graph environments.

[0137] Before inputting the word sense disambiguation model, some preprocessing is required, such as:

[0138] Based on the entity connection dictionary and attribute dictionary, we find the most likely knowledge graph entities and their attributes. We then extract the core attributes or tags of all candidate entities from the knowledge graph to generate specific entity descriptions.

[0139] Figure 4 This is a schematic diagram of the MatchBERT network structure provided in the embodiments of this application, as shown below. Figure 4 As shown in the diagram, the small square with the letter E represents the smallest unit of any embedding information. Different rows represent different embedding information. For example, in word embedding, it represents the smallest unit of information included in the word embedding features extracted from the candidate text. The positional encoding represents the sequence number of each word (or character) in the candidate text or the text to be disambiguated. The above is merely illustrative and does not limit the number of words (or characters) in the text, or the number of smallest information units contained in each embedding information.

[0140] The input consists of two parts: first, the word embedding features, entity embedding features, and entity context embedding features corresponding to the candidate entities; second, the word embedding features, entity embedding features, and entity context embedding features corresponding to the entities identified in the text to be disambiguated.

[0141] The candidate entity list may contain multiple candidate entities. Each time, a candidate entity is selected, its corresponding embedding feature is determined, and word sense disambiguation is performed between this entity and the corresponding embedding feature determined based on the text to be disambiguated. This process continues until all candidate entities have completed word sense disambiguation with the text to be disambiguated.

[0142] 1) Feature encoding: Encoding the input text aims to capture lexical and semantic information. It combines (entity) context embedding, entity embedding, word embedding and positional encoding to generate the semantic encoding of the input text, where the positional encoding is the sequence number of each word (or character) in the sentence.

[0143] 2) Transformer Network: The encoding component consists of 6 stacked encoders. Each encoder is composed of a multi-head attention module and a feed-forward neural network, etc. It receives inputs such as context embeddings, entity embeddings, and word embeddings, and outputs a weighted feature vector to the decoder. The decoding component also consists of 6 decoders. Each decoder is composed of a multi-head attention module, a feed-forward neural network, and an encoder-decoder attention module, etc. Each decoder receives the feature vector output by the encoding component as input, and the output of the last decoder is passed to the fully connected layer.

[0144] 3) Fully connected layer: The output of the Transformer network is mapped to the sample label space through matrix transformation, and binary classification prediction is performed based on its output value.

[0145] 4) Binary classification using the Focal loss function: The output of the fully connected layer is classified according to the probability result. If the text to be disambiguated and the current candidate entity have a strong correlation or similarity, the output result is "yes," meaning the text to be disambiguated and the current candidate entity are the same entity. If they are not similar, the output result is "no," meaning the text to be disambiguated and the current candidate entity are not the same entity.

[0146] A word sense disambiguation model built on the MatchBERT network determines the similarity between the entity text corresponding to each candidate entity and the text to be disambiguated.

[0147] Then, it is determined which specific range of the preset confidence rule the similarity score falls within, thus determining whether the entity to be disambiguated and the candidate entity are the same entity. For example, in a three-class classification system, each class corresponds to a confidence score interval, and each confidence score interval corresponds to a decision result. Determining which interval the similarity score falls within corresponds to the decision result for that confidence score interval.

[0148] In this application, a binary word sense disambiguation model is established. The pre-set confidence rule corresponds to two intervals, which can also be determined by a confidence threshold. One interval corresponds to the result that the entity to be disambiguated and the candidate entity are the same entity, and the other interval corresponds to the result that the entity to be disambiguated and the candidate entity are not the same entity.

[0149] Optionally, the word sense disambiguation model is trained based on the following method:

[0150] Based on the preset masking rules, and the constructed first and second sample data, the third and fourth sample data are determined for training the word sense disambiguation model.

[0151] The value of the second loss function is determined based on the second loss function, as well as the third and fourth sample data; the second loss function is an improved cross-entropy loss function.

[0152] When the value of the second loss function is less than the second threshold, the word sense disambiguation model is obtained.

[0153] Specifically, the word sense disambiguation model built on the MatchBERT network requires corresponding training sample data. Unlike the training sample data for entity embedding models, the training sample data for entity embedding models uses word-level masking, transforming the traditional BERT token-level random masking into entity-level random masking. That is, while existing technologies use character-level random masking, this application uses word-level random masking. The random masking strategy determines the percentage of words in a sentence to mask, and within that percentage, which percentage is directly masked, which percentage is replaced, and which percentage remains unchanged. Statistical analysis yielded the following optimal allocation: 20% of the words in the sentence are masked, with an 80% probability of masking entity words, a 10% chance of replacing with an arbitrary word, and the remaining 10% leaving the original vocabulary unchanged. To improve model performance, dynamic masking is used during training to augment the sample size. Compared to the existing BERT network structure, the MatchBERT network structure used in this application has a higher degree of compatibility with RPA knowledge graphs, making it easier for it to learn entities in the knowledge graph.

[0154] In existing binary classification problems, the crossover loss function in BERT is:

[0155]

[0156] Where m is the number of positive samples, n is the number of negative samples, and N is the total number of samples, m + n = N.

[0157] Optionally, the improved cross-entropy loss function is determined based on the cross-loss function in BERT.

[0158] The masked training samples are then input into the word sense disambiguation model to determine the value of the improved cross-entropy loss function, which can be expressed as:

[0159] Loss(p t )=-α t(1-p t ) γ log(p t )

[0160] or,

[0161]

[0162]

[0163]

[0164] Where α is the weight parameter between the binary categories, and its value is the ratio of the number of negative samples to the total number of positive and negative samples. When the training samples are positive samples (y=1), α t The value of is α; in other cases (y = 0), α t The value is 1-α; while the class probability estimate p t When the training samples are positive samples (y=1), its value is p, which is the result of the output probability of the output layer after Softmax processing. In other cases (y=0), it is 1-p. t This is a modulation factor used to increase the weight of hard-to-classify samples. It is used when a sample is misclassified, and p... t When the value of p is very small, the modulation factor tends to 1, and the loss result is unaffected; when p t When the value is close to 1, the modulation factor is close to 0, and the weight of easily distinguishable samples is reduced. γ is a smoothing parameter, which takes a value of 2 during the training of the word sense disambiguation model, and is used to smoothly adjust the ratio of samples that are reduced in weight.

[0165] When the value of the improved cross-entropy loss function is less than the second threshold, the word sense disambiguation model is considered to have completed training. Using the cross-loss function in BERT is not conducive to setting a threshold to filter results when the labeled sample quality is low. However, using the improved cross-entropy loss function makes the distribution of the corresponding cross-entropy loss function smoother, improving the smoothness of the output confidence score.

[0166] Optionally, the first sample data is obtained by labeling triples based on RPA knowledge graphs; the second sample data is obtained by replacing the head entity or tail entity corresponding to any entity relationship in the first sample with the head entity or tail entity corresponding to different entity relationships.

[0167] Specifically, the training samples of the entity embedding model are positive samples obtained by labeling all head entities, tail entities, and inter-entity relationships of triples in the RPA knowledge graph, i.e., the first sample data; then, the head entities in the above positive samples are replaced with head entities or tail entities with different inter-entity relationships, or the tail entities in the positive samples are replaced with head entities or tail entities with different inter-entity relationships, or the head entities and tail entities in the positive samples are replaced with entity pairs with different inter-entity relationships, which are used as negative samples, i.e., the second sample data. In the second sample data, the inter-entity relationship between the two entities is not a correct inter-entity relationship.

[0168] Optionally, determining whether the entity to be disambiguated and the candidate entity are the same entity based on preset confidence rules and the similarity between the candidate entity text and the text to be disambiguated includes:

[0169] If the similarity satisfies the preset confidence rule, then it is determined that the entity to be disambiguated and the candidate entity are the same entity.

[0170] If the similarity does not meet the preset confidence rule, then it is determined that the entity to be disambiguated and the candidate entity are not the same entity.

[0171] Specifically, after determining the similarity between each candidate entity text and the entity text to be disambiguated based on the improved cross-entropy loss function, under the pre-set confidence rule, it is determined whether the similarity satisfies the pre-set confidence rule. If the pre-set confidence rule is determined by setting a confidence threshold, it is determined whether the similarity is greater than the confidence threshold. If it is greater than the confidence threshold, it is determined that the entity to be disambiguated and the candidate entity are not the same entity. If it is less than the confidence threshold, it is determined that the entity to be disambiguated and the candidate entity are the same entity.

[0172] If the pre-set confidence rules are determined by setting confidence ranges, and each confidence range corresponds to a different word sense disambiguation result, then determining which pre-set confidence range the similarity falls into corresponds to the word sense disambiguation result between the entity to be disambiguated and the candidate entity. For example, if the similarity is greater than 0.5, the corresponding word sense disambiguation result is the same entity, then the entity to be disambiguated and the candidate entity are the same entity; if the similarity is less than or equal to 0.5, the corresponding word sense disambiguation result is different entities, then the entity to be disambiguated and the candidate entity are not the same entity.

[0173] Optionally, after determining whether the entity to be disambiguated and the candidate entity are the same entity based on preset confidence rules and the similarity between the candidate entity text and the text to be disambiguated, the process includes:

[0174] If the entity to be disambiguated and the candidate entity are the same entity, then the entity name of the entity to be disambiguated is used as an alias to update the RPA knowledge graph;

[0175] If the entity to be disambiguated and the candidate entity are not the same entity, then the entity to be disambiguated is added to the RPA knowledge graph, and the corresponding inter-entity relationship is updated.

[0176] Specifically, after determining whether the entity to be disambiguated and the candidate entity are the same entity, the RPA knowledge graph needs to be updated accordingly based on different disambiguation results. For example, if it is determined that the entity to be disambiguated and the candidate entity are the same entity, the entity name of the entity to be disambiguated is used as an alias, and the RPA knowledge graph is updated; or if it is determined that the entity to be disambiguated and the candidate entity are not the same entity, the entity to be disambiguated is added to the RPA knowledge graph, and the corresponding inter-entity relationships are updated based on the attributes of the entity to be disambiguated, while supplementing the entity description information, etc.

[0177] The word sense disambiguation method provided in this application is illustrated below with specific examples. Figure 5 This is a schematic diagram of the overall flow of the word sense disambiguation method provided in the embodiments of this application, as follows: Figure 5 As shown:

[0178] (1) Construction of RPA knowledge graph: Entity extraction is performed based on the RPA corpus. A knowledge graph is constructed by designing a schema structure for the RPA domain. The knowledge graph generates triples representing RPA entity relationships and corresponding entity indexes. Based on the triples, entity vectors and relationships between entities in the graph network can be obtained. This part is an offline operation. It realizes the networking and vectorization transformation of RPA corpus to RPA entities, and provides entity context features based on knowledge graph for subsequent word sense disambiguation. Compared with traditional word (character) vector features, word sense features are richer.

[0179] Entities in the question-and-answer dialogue of the RPA knowledge question-answering system are obtained through entity extraction algorithms, and an RPA knowledge graph is constructed to generate entity indexes and triples containing technical terms and knowledge points. The pre-trained dictionary of the MatchBERT network is improved based on the index dictionary of RPA entities. By adding entity words, the semantic uncertainty of words is reduced when the word is used as a unit. Furthermore, because the input text is encoded as a unit of words, the number of input neuron parameters can be reduced, thereby reducing the complexity of the model and improving the inference speed after the model is deployed.

[0180] (2) Generation of candidate entities and text to be disambiguated: By querying the entity index in the RPA knowledge graph through the entity names included in the text to be disambiguated, the candidate entity list corresponding to the entity to be disambiguated is determined, and with the help of the entity description information in the RPA knowledge graph, each candidate entity is completed to form a set of text phrases.

[0181] (3) RPA feature extraction: Based on the RPA knowledge graph, entity embedding features, entity context embedding features, and word embedding features of the candidate entity, as well as entity embedding features, entity context embedding features, and word embedding features of the BERT dictionary, are extracted respectively.

[0182] By extracting and encoding entity features, we combine three parts of features: word embedding, entity embedding, and entity context embedding. When extracting entity embedding features, we use the TranSparse method to improve the common TransE equidistance calculation method to solve the problem of hot / cold topics in RPA question answering corpora. We establish the correspondence between model parameters and the number of entity pairs to improve the entity feature representation ability. We introduce the concept of graph subgraph to extract the context information of a certain entity in the RPA knowledge graph and provide richer semantic features with the help of knowledge graph network.

[0183] (4) Word sense disambiguation based on MatchBERT network: Query entities and entity relationships to generate entity embedding features and entity context embedding features, and mix them with BERT word embedding features. Compared with the traditional BERT network that only inputs word embedding features, the MatchBERT network makes full use of the various feature information generated by the RPA entity knowledge graph network. After full connection, it performs binary classification output to determine whether they are the same entity.

[0184] (5) RPA Knowledge Graph Update: Post-processing is performed based on the word sense disambiguation results of MatchBERT. If the result is yes, the entity to be disambiguated is used as an alias of the candidate entity; otherwise, it is treated as a new entity, and the new entity or alias is updated in the knowledge graph. Dynamic updates to the RPA graph are achieved by setting a confidence threshold, including updating entity vectors and relationships between entities, thus realizing automatic iterative upgrades during word sense disambiguation.

[0185] After text enhancement and feature extraction, candidate entities and entities to be disambiguated are fed into an entity disambiguation model based on the MatchBERT network. Entity-level masks are used to improve the learning ability for entities. To improve the smoothness of the output confidence and enhance the controllability of RPA graph updates, the Focal Loss loss function is used for optimization. By setting a confidence threshold, the automatic updating of the RPA graph is controlled, achieving high-quality iterative upgrades of the knowledge graph.

[0186] The word sense disambiguation method based on MatchBERT network provides intelligent reasoning support for graph construction in knowledge question answering systems in the RPA domain, ensuring the construction quality of RPA graph network, providing rich background knowledge for semantic understanding of questions and answers, realizing interpretability of the question answering process, and improving the accuracy of answers.

[0187] RPA feature extraction includes generating three parts of features: word embeddings, entity embeddings, and entity context embeddings.

[0188] 3.1 Word Embedding: Candidate entity text and the text to be disambiguated are indexed and encoded based on an improved BERT dictionary (vocab), and then transformed into a multi-dimensional vectorized representation. Compared to the traditional BERT word embedding method, this method introduces new words into the BERT dictionary on a word-by-word basis. Specific improvements include:

[0189] Assuming the input sentence is represented as [word1, word2, ..., word after pre-segmentation] n ]; Traverse each word i If a word does not exist in the vocab, it is considered an unregistered word and is segmented into characters; otherwise, its index number is directly taken. Finally, the sentence is converted into a mixed index coding sequence of words and characters.

[0190] The above approach results in shorter sentence encoding, faster processing speed, and improved word sense disambiguation efficiency in RPA; by using words as units, it reduces the word sense uncertainty when using characters as units and lowers the complexity of the MatchBERT model.

[0191] 3.2 Entity Embedding: Due to the presence of both popular and unpopular questions in the RPA question-answering corpus, the triples (head entity, relation, tail entity) in the constructed RPA knowledge graph are imbalanced. When training the entity embedding model, the number of model parameters in traditional algorithms remains constant. Therefore, entities or entity pairs with a small number of parameters are prone to overfitting, while entities or entity pairs with a large number of parameters are prone to underfitting. To establish a relationship between the model parameters and the number of entity pairs, the TranSparse method is used for entity embedding. The specific implementation process is as follows:

[0192] The head and tail entities are mapped to different relation spaces, and the sparse mapping matrix of the head entity-relation corresponding to each relation r between entities is determined. and the corresponding tail entity-relation sparse mapping matrix Head entity - relation r in the relation * Maximum number of corresponding header entities Tail entity - relation r in the relation * The maximum number of corresponding tail entities r *This represents the relation when the number of sparse mapping matrices corresponding to relation r reaches its maximum value. Relation r * The maximum number of correspondence matrices sparsity hyperparameter Therefore, the sparsity factor of the "head-relation" sparse mapping matrix is ​​calculated. The sparsity factor of the "tail-relation" sparse mapping matrix Head entity mapping to relation space vector Tail entity mapping to relation space vector Its scoring function is: f r (h,t)=||h p +rt p || 2 Where h, r, and t are the vector values ​​of the original triples, the first loss function during training is calculated as follows:

[0193]

[0194] Where S and S′ represent the sets of positive and negative sample triples, respectively, and γ represents the distance between positive and negative samples. When γ+f r (h,t)-f r When (h′,t′)<0, the corresponding first loss function value is 0. And f r (h,t) represents the score function value after the head and tail entities are mapped to different relation spaces in the positive sample. The smaller the value, the closer the distance between them; f r (h′,t′) represents the score function value of the head and tail entities in the negative sample after they are mapped to different relation spaces. This value should be as large as possible to obtain a smaller loss value.

[0195] After training, the head entity-relation sparse mapping matrix is ​​obtained. Sparse mapping matrix of entity-relationship Then through and The vector value of the entity embedding is calculated.

[0196] 3.3 Entity Context Embedding: The identified entity is placed into the knowledge graph, and n similar entities are found. These n entities and their corresponding relationships are used to determine the corresponding knowledge graph subgraph.

[0197] First, determine the context of the entity, as follows:

[0198] C(e)={e i |(e,r,e i )∈G or(e i ,r,e)∈G}

[0199] Where e represents an entity, C(e) represents the list of entities in the context of the entity, r represents the relationship between any entities, and G represents a subgraph of the knowledge graph, which can be either the first or the second subgraph of the knowledge graph.

[0200] Then, for a given entity e's context, the context embedding is the weighted average of its context entities. The calculation formula is:

[0201]

[0202] Among them, e i e learned through knowledge graph embedding i The entity embedding corresponds to the vector value, w i It is the distance weight value between any entity in the subgraph and a given entity e. Since context entities are usually closely related to the current entity semantically and logically, introducing more contextual supplementary feature information helps improve the identifiability of the entity.

[0203] (4) Entity disambiguation based on MatchBERT network

[0204] MatchBERT is an improved version of BERT designed for RPA applications, primarily used for word sense disambiguation in knowledge graph environments. Its input consists of two parts: a sentence containing the entity to be disambiguated, and candidate entities and descriptive sentences from the RPA knowledge graph. Both inputs combine entity features extracted from the RPA knowledge graph, including words, entities, and their context, and then feed them into the entity disambiguation network. To improve the network's ability to discriminate entities, the word masking in the original BERT is replaced with an entity-level mask. To improve the smoothness of the output confidence score and enhance the controllability of RPA graph updates, Focal Loss is used to calculate whether the entity to be disambiguated matches the current entity and outputs the matching degree (confidence). Based on the confidence threshold, it is determined whether to update the RPA graph.

[0205] 4.1 Feature Encoding: The input text is encoded to capture lexical and semantic information. It is generated by merging context embedding, entity embedding, word embedding and positional encoding. The positional encoding is the sequence number of each word (or character) in the sentence.

[0206] 4.2 Entity Masking in BERT: In the MatchBERT network, the traditional BERT's token-level (word-level) random masking is transformed into entity-level (word-level) random masking. When constructing training data, entities appearing in the knowledge graph are preferentially selected for random masking. The randomization strategy is as follows:

[0207] 20% of the words in a sentence are masked, with 80% of the masks targeting entity words, 10% replaced with an arbitrary word, and the remaining 10% leaving the original vocabulary unchanged. To improve model performance, dynamic masking is used during training to augment the sample size. Compared to the traditional BERT model, the Match BERT network has a higher compatibility with RPA knowledge graphs, making it easier for it to learn entities from the knowledge graph.

[0208] 4.3 The Focal Loss loss function is used to train the word sense disambiguation model. If the sentence to be disambiguated and the current candidate entity have a strong correlation or similarity, the judgment result is yes. If they are not similar, the judgment result is no.

[0209] Few-shot learning: MatchBERT's network structure simultaneously takes the text to be disambiguated and candidate entity text as input, and outputs a binary classification result. Therefore, after training on massive RPA corpora, a pre-trained model for RPA corpora can be obtained, which can be used to achieve real-time inference and prediction in cases with few samples. For example, with only 100 samples, by building...<Ei,Ej> With up to 10,000 training samples, the word sense disambiguation method provided in this application demonstrates superior performance compared to traditional deep learning methods. Furthermore, after training the word sense disambiguation model based on a massive RPA corpus using a MatchBERT network, it acquires the ability to learn contrastive entity features. When adding new entity content, there is no need to retrain the model; simply inputting the candidate entity and the entity to be disambiguated allows obtaining the result of whether they are the same entity, achieving the effect of continuous use after a single training.

[0210] Figure 6 This is a schematic diagram of the structure of the word sense disambiguation device provided in the embodiments of this application, as shown below. Figure 6 As shown, the word meaning disambiguation device includes:

[0211] The candidate entity module 601 is used to determine the entities to be disambiguated in the text to be disambiguated, as well as the corresponding candidate entity list, based on the RPA knowledge graph.

[0212] The embedding feature module 602 is used to determine the embedding feature corresponding to the candidate entity and the embedding feature corresponding to the entity to be disambiguated based on each candidate entity in the candidate entity list and the entity to be disambiguated through RPA feature extraction.

[0213] The disambiguation module 603 is used to determine whether the entity to be disambiguated and the candidate entity are the same entity based on the word sense disambiguation model, the embedded features corresponding to each candidate entity, and the embedded features corresponding to the entity to be disambiguated.

[0214] The embedding features include: entity embedding features, entity context embedding features, and word embedding features.

[0215] In a further description of the above apparatus, the embedding feature module 602, in the process of determining the embedding feature corresponding to each candidate entity and the entity to be disambiguated based on each candidate entity in the candidate entity list and the entity to be disambiguated through RPA feature extraction, is specifically used for:

[0216] Based on the improved BERT dictionary, the index encoding sequence corresponding to the candidate entity text is determined as the word embedding feature of the candidate entity text, and the index encoding sequence corresponding to the text to be disambiguated is determined as the word embedding feature of the text to be disambiguated; the candidate entity text is determined based on any candidate entity in the candidate entity list and the RPA knowledge graph;

[0217] Based on the entity embedding model and RPA knowledge graph, determine the entity embedding features corresponding to each candidate entity, and / or the entity embedding features corresponding to the entities already determined in the text to be disambiguated;

[0218] Based on RPA knowledge graph and distance weight algorithm, determine the entity context embedding feature corresponding to each candidate entity, and / or the entity context embedding feature corresponding to the entity already determined in the text to be disambiguated.

[0219] In a further description of the above device, the embedding feature module 602 also includes a word embedding unit. This word embedding unit is used to determine, based on the improved BERT dictionary, the index encoding sequence corresponding to the candidate entity text as the word embedding feature of the candidate entity text, and the index encoding sequence corresponding to the text to be disambiguated as the word embedding feature of the text to be disambiguated. Specifically, this process includes:

[0220] The candidate entity text and the text to be disambiguated are pre-segmented respectively to determine the first pre-segmentation set and the second pre-segmentation set;

[0221] Based on the improved BERT dictionary, the index encoding sequence corresponding to the first pre-segmented word set and the index encoding sequence corresponding to the first pre-segmented word set are determined.

[0222] In a further description of the above device, the embedding feature module 602 also includes an entity embedding unit. The entity embedding model used by the entity embedding unit is trained based on the following method, including:

[0223] Based on the TranSparse method, the first spatial vector and the second spatial vector in the first loss function are determined; the first spatial vector is the spatial vector of the head entity and the inter-entity relationship under each inter-entity relationship in the RPA knowledge graph, and the second spatial vector is the spatial vector of the tail entity and the inter-entity relationship under each inter-entity relationship in the RPA knowledge graph.

[0224] Based on the first loss function, and the constructed first sample data and second sample data, the value of the first loss function is determined;

[0225] The entity embedding model is obtained when the value of the first loss function is less than the first threshold.

[0226] In a further description of the above device, the entity embedding unit in the embedding feature module 602, in the process of determining the entity embedding features corresponding to the candidate entity and / or the entity embedding features corresponding to the entities already determined in the text to be disambiguated based on the entity embedding model and RPA knowledge graph, is specifically used for:

[0227] Based on RPA knowledge graph, a first set of corresponding inter-entity relationships is determined for the candidate entities, and / or a second set of corresponding inter-entity relationships is determined for the entities already identified in the text to be disambiguated; the first set includes all entities that have inter-entity relationships with the candidate entities and their corresponding inter-entity relationships, and the second set includes all entities that have inter-entity relationships with the identified entities in the text to be disambiguated and their corresponding inter-entity relationships;

[0228] Based on the entity embedding model, the values ​​of the first spatial vector and the second spatial vector corresponding to the first set are determined as entity embedding features corresponding to the candidate entities, and / or the values ​​of the first spatial vector and the second spatial vector corresponding to the second set are determined as entity embedding features corresponding to the entities already identified in the text to be disambiguated.

[0229] In a further description of the above device, the embedding feature module 602 also includes an entity context embedding unit for determining the entity context embedding feature corresponding to each candidate entity based on RPA knowledge graph and distance weight algorithm, and / or the entity context embedding feature corresponding to the entity already determined in the text to be disambiguated, specifically including:

[0230] Based on RPA knowledge graph, determine the first knowledge graph subgraph corresponding to each candidate entity, and / or the second knowledge graph subgraph corresponding to the entity already determined in the text to be disambiguated;

[0231] Based on the first knowledge graph subgraph and the distance weight algorithm, the entity context embedding features corresponding to the candidate entity are determined, and / or based on the second knowledge graph subgraph and the distance weight algorithm, the entity context embedding features corresponding to the entities already identified in the text to be disambiguated are determined.

[0232] In a further description of the above device, the disambiguation module 603, in determining whether the entity to be disambiguated and the candidate entity are the same entity based on the word sense disambiguation model, the embedding features corresponding to each candidate entity, and the embedding features corresponding to the entity to be disambiguated, is specifically used for:

[0233] Based on the word sense disambiguation model, the similarity between the candidate entity text and the text to be disambiguated is determined; the candidate entity text is determined based on any candidate entity in the candidate entity list and the RPA knowledge graph.

[0234] Based on pre-set confidence rules and the similarity between the candidate entity text and the text to be disambiguated, it is determined whether the entity to be disambiguated and the candidate entity are the same entity.

[0235] In a further description of the above device, the word sense disambiguation model is trained based on the following method:

[0236] Based on the preset masking rules, and the constructed first and second sample data, the third and fourth sample data are determined for training the word sense disambiguation model.

[0237] The value of the second loss function is determined based on the second loss function, as well as the third and fourth sample data; the second loss function is an improved cross-entropy loss function.

[0238] When the value of the second loss function is less than the second threshold, the word sense disambiguation model is obtained.

[0239] In a further description of the above device, the first sample data is obtained by labeling triples based on RPA knowledge graphs; the second sample data is obtained by replacing the head entity or tail entity corresponding to any entity relationship in the first sample with the head entity or tail entity corresponding to different entity relationships.

[0240] In a further description of the above device, the improved cross-entropy loss function is determined based on the cross-loss function in BERT.

[0241] In a further description of the above device, the disambiguation module 603 is used to determine whether the entity to be disambiguated and the candidate entity are the same entity based on preset confidence rules and the similarity between the candidate entity text and the text to be disambiguated, specifically including:

[0242] If the similarity satisfies the preset confidence rule, then it is determined that the entity to be disambiguated and the candidate entity are the same entity.

[0243] If the similarity does not meet the preset confidence rule, then it is determined that the entity to be disambiguated and the candidate entity are not the same entity.

[0244] In a further description of the above device, the disambiguation module 603 is also used to determine whether the entity to be disambiguated and the candidate entity are the same entity based on a preset confidence rule and the similarity between the candidate entity text and the text to be disambiguated, including:

[0245] If the entity to be disambiguated and the candidate entity are the same entity, then the entity name of the entity to be disambiguated is used as an alias to update the RPA knowledge graph;

[0246] If the entity to be disambiguated and the candidate entity are not the same entity, then the entity to be disambiguated is added to the RPA knowledge graph, and the corresponding inter-entity relationship is updated.

[0247] It should be noted that the word sense disambiguation apparatus provided in this application embodiment can implement all the methods implemented in the above-mentioned word sense disambiguation method embodiment and can achieve the same technical effect. Here, the parts and beneficial effects that are the same as or corresponding to the method embodiment in this embodiment will not be described in detail.

[0248] Figure 7 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application, such as... Figure 7 As shown, the electronic device may include: a processor 710, a communications interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communications interface 720, and the memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute word meaning disambiguation methods, such as:

[0249] Based on RPA knowledge graph, identify the entities to be disambiguated in the text to be disambiguated, and the corresponding candidate entity list;

[0250] Based on each candidate entity in the candidate entity list and the entity to be disambiguated, the embedded features corresponding to the candidate entity and the embedded features corresponding to the entity to be disambiguated are determined by RPA feature extraction.

[0251] Based on the word sense disambiguation model, the embedded features corresponding to each candidate entity and the embedded features corresponding to the entity to be disambiguated are used to determine whether the entity to be disambiguated and the candidate entity are the same entity.

[0252] The embedding features include: entity embedding features, entity context embedding features, and word embedding features.

[0253] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0254] It should be noted that the electronic device provided in this application embodiment can implement all the methods implemented in the above-mentioned word meaning disambiguation method embodiment and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0255] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to perform the word meaning disambiguation methods provided by the above methods.

[0256] Specifically, the computer program products provided in this application embodiment can implement all the methods implemented in the above method embodiments and can achieve the same technical effects. Here, the parts that are the same as those in the method embodiments and the beneficial effects will not be described in detail.

[0257] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0258] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0259] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for disambiguating word meanings, characterized in that, include: Based on RPA knowledge graph, identify the entities to be disambiguated in the text to be disambiguated, and the corresponding candidate entity list; Based on each candidate entity in the candidate entity list and the entity to be disambiguated, the embedded features corresponding to the candidate entity and the embedded features corresponding to the entity to be disambiguated are determined by RPA feature extraction. Based on the word sense disambiguation model, the embedded features corresponding to each candidate entity and the embedded features corresponding to the entity to be disambiguated are used to determine whether the entity to be disambiguated and the candidate entity are the same entity. The word sense disambiguation model is built on the MatchBERT network; The embedding features include: entity embedding features, entity context embedding features, and word embedding features; The step of determining the embedded features corresponding to the candidate entities and the entity to be disambiguated based on each candidate entity in the candidate entity list and the entity to be disambiguated through RPA feature extraction includes: Based on the improved BERT dictionary, the index encoding sequence corresponding to the candidate entity text is determined as the word embedding feature of the candidate entity text, and the index encoding sequence corresponding to the text to be disambiguated is determined as the word embedding feature of the text to be disambiguated; the candidate entity text is determined based on any candidate entity in the candidate entity list and the RPA knowledge graph; Based on the entity embedding model and RPA knowledge graph, determine the entity embedding features corresponding to each candidate entity, and / or the entity embedding features corresponding to the entities already determined in the text to be disambiguated; Based on RPA knowledge graph and distance weight algorithm, determine the entity context embedding feature corresponding to each candidate entity, and / or the entity context embedding feature corresponding to the entity already determined in the text to be disambiguated.

2. The word meaning disambiguation method according to claim 1, characterized in that, The improved BERT dictionary is used to determine the index encoding sequence corresponding to the candidate entity text as the word embedding feature of the candidate entity text, and the index encoding sequence corresponding to the text to be disambiguated as the word embedding feature of the text to be disambiguated, including: The candidate entity text and the text to be disambiguated are pre-segmented respectively to determine the first pre-segmentation set and the second pre-segmentation set; Based on the improved BERT dictionary, the index encoding sequence corresponding to the first pre-segmented word set and the index encoding sequence corresponding to the first pre-segmented word set are determined.

3. The word meaning disambiguation method according to claim 1, characterized in that, The entity embedding model is trained based on the following method: Based on the TranSparse method, the first spatial vector and the second spatial vector in the first loss function are determined; the first spatial vector is the spatial vector of the head entity and the inter-entity relationship under each inter-entity relationship in the RPA knowledge graph, and the second spatial vector is the spatial vector of the tail entity and the inter-entity relationship under each inter-entity relationship in the RPA knowledge graph. Based on the first loss function, and the constructed first sample data and second sample data, the value of the first loss function is determined; The entity embedding model is obtained when the value of the first loss function is less than the first threshold.

4. The word meaning disambiguation method according to claim 3, characterized in that, The determination of entity embedding features corresponding to the candidate entities based on entity embedding models and RPA knowledge graphs, and / or entity embedding features corresponding to entities already identified in the text to be disambiguated, includes: Based on RPA knowledge graph, a first set of corresponding inter-entity relationships is determined for the candidate entities, and / or a second set of corresponding inter-entity relationships is determined for the entities already identified in the text to be disambiguated; the first set includes all entities that have inter-entity relationships with the candidate entities and their corresponding inter-entity relationships, and the second set includes all entities that have inter-entity relationships with the identified entities in the text to be disambiguated and their corresponding inter-entity relationships; Based on the entity embedding model, the values ​​of the first spatial vector and the second spatial vector corresponding to the first set are determined as entity embedding features corresponding to the candidate entities, and / or the values ​​of the first spatial vector and the second spatial vector corresponding to the second set are determined as entity embedding features corresponding to the entities already identified in the text to be disambiguated.

5. The word meaning disambiguation method according to claim 1, characterized in that, The method based on RPA knowledge graph and distance weight algorithm determines the entity context embedding features corresponding to each candidate entity, and / or the entity context embedding features corresponding to the entities already identified in the text to be disambiguated, including: Based on RPA knowledge graph, determine the first knowledge graph subgraph corresponding to each candidate entity, and / or the second knowledge graph subgraph corresponding to the entity already determined in the text to be disambiguated; Based on the first knowledge graph subgraph and the distance weight algorithm, the entity context embedding features corresponding to the candidate entity are determined, and / or based on the second knowledge graph subgraph and the distance weight algorithm, the entity context embedding features corresponding to the entities already identified in the text to be disambiguated are determined.

6. The word meaning disambiguation method according to claim 1, characterized in that, The determination of whether the entity to be disambiguated and the candidate entity are the same entity based on the word sense disambiguation model, using the embedded features corresponding to each candidate entity and the embedded features corresponding to the entity to be disambiguated, includes: Based on the word sense disambiguation model, the similarity between the candidate entity text and the text to be disambiguated is determined; the candidate entity text is determined based on any candidate entity in the candidate entity list and the RPA knowledge graph. Based on pre-set confidence rules and the similarity between the candidate entity text and the text to be disambiguated, it is determined whether the entity to be disambiguated and the candidate entity are the same entity.

7. The word meaning disambiguation method according to claim 6, characterized in that, The word sense disambiguation model was trained based on the following method: Based on the preset masking rules, and the constructed first and second sample data, the third and fourth sample data are determined for training the word sense disambiguation model. The value of the second loss function is determined based on the second loss function, as well as the third and fourth sample data; the second loss function is an improved cross-entropy loss function. When the value of the second loss function is less than the second threshold, the word sense disambiguation model is obtained.

8. The word meaning disambiguation method according to claim 3 or 7, characterized in that, The first sample data is obtained by labeling triples based on RPA knowledge graphs; the second sample data is obtained by replacing the head entity or tail entity corresponding to any entity relationship in the first sample with the head entity or tail entity corresponding to different entity relationships.

9. The word meaning disambiguation method according to claim 7, characterized in that, The improved cross-entropy loss function is determined based on the cross-loss function in BERT.

10. The word meaning disambiguation method according to claim 6, characterized in that, The step of determining whether the entity to be disambiguated and the candidate entity are the same entity based on pre-set confidence rules and the similarity between the candidate entity text and the text to be disambiguated includes: If the similarity satisfies the preset confidence rule, then it is determined that the entity to be disambiguated and the candidate entity are the same entity. If the similarity does not meet the preset confidence rule, then it is determined that the entity to be disambiguated and the candidate entity are not the same entity.

11. The word meaning disambiguation method according to claim 6, characterized in that, After determining whether the entity to be disambiguated and the candidate entity are the same entity based on pre-set confidence rules and the similarity between the candidate entity text and the text to be disambiguated, the process includes: If the entity to be disambiguated and the candidate entity are the same entity, then the entity name of the entity to be disambiguated is used as an alias to update the RPA knowledge graph; If the entity to be disambiguated and the candidate entity are not the same entity, then the entity to be disambiguated is added to the RPA knowledge graph, and the corresponding inter-entity relationship is updated.

12. A word meaning disambiguation device, characterized in that, include: The candidate entity module is used to identify the entities to be disambiguated in the text to be disambiguated, as well as the corresponding candidate entity list, based on the RPA knowledge graph. The embedding feature module is used to determine the embedding features corresponding to the candidate entities and the entity to be disambiguated based on each candidate entity in the candidate entity list and the entity to be disambiguated through RPA feature extraction. The disambiguation module is used to determine whether the entity to be disambiguated and the candidate entity are the same entity based on the word sense disambiguation model, the embedded features corresponding to each candidate entity, and the embedded features corresponding to the entity to be disambiguated. The word sense disambiguation model is built on the MatchBERT network; The embedding features include: entity embedding features, entity context embedding features, and word embedding features; The step of determining the embedded features corresponding to the candidate entities and the entity to be disambiguated based on each candidate entity in the candidate entity list and the entity to be disambiguated through RPA feature extraction includes: Based on the improved BERT dictionary, the index encoding sequence corresponding to the candidate entity text is determined as the word embedding feature of the candidate entity text, and the index encoding sequence corresponding to the text to be disambiguated is determined as the word embedding feature of the text to be disambiguated; the candidate entity text is determined based on any candidate entity in the candidate entity list and the RPA knowledge graph; Based on the entity embedding model and RPA knowledge graph, determine the entity embedding features corresponding to each candidate entity, and / or the entity embedding features corresponding to the entities already determined in the text to be disambiguated; Based on RPA knowledge graph and distance weight algorithm, determine the entity context embedding feature corresponding to each candidate entity, and / or the entity context embedding feature corresponding to the entity already determined in the text to be disambiguated.

13. An electronic device comprising a processor and a memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the word sense disambiguation method according to any one of claims 1 to 11.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the word sense disambiguation method according to any one of claims 1 to 11.

Images