Full name information generation method and related device

Abstract

The invention discloses a full name information generation method and a related device, and relates to the technical field of natural language processing. According to the method, the pre-trained sequence generation model is utilized, and the intermediate vector after the vector is embedded based on the corresponding weight parameter fusion image is encoded layer by layer through the multi-layer encoder blocks in the encoder constructed based on the self-attention mechanism; decoding an intermediate vector after the vector is embedded on the basis of the weight parameter fusion graph layer by layer through a plurality of layers of decoder blocks in a decoder constructed on the basis of a self-attention mechanism to obtain a full-name embedding vector sequence formed by full-name embedding vectors of each target abbreviation sub-word; and generating full-name information of the target abbreviation information according to the full-name embedding vector sequence. Through the sequence generation model and based on graph embedded reinforcement learning, the purpose of improving the accuracy of the full name information of the target short name information is achieved.

Description

Technical Field

[0001] The present application relates to the technical field of natural language processing, and in particular to a method for generating full name information and a related device. Background Art

[0002] Based on the principle of easy identification and preventing ambiguity, the name of the institution recorded in the institution specification document should use the standard full name. The full name information of the institution is obtained by arranging the full name subwords according to the prescribed full name grammar. However, the bibliographic data in some databases often use abbreviation information to identify each institution. The abbreviation information of the institution is obtained by arranging the abbreviation subwords according to the prescribed abbreviation grammar. Therefore, in the process of constructing the specification document, it is necessary to generate the full name information based on the abbreviation information of the institution.

[0003] At present, the method of converting the abbreviation information of an organization to its full name information mainly uses the subword mapping method, that is, through the mapping relationship between the full name subwords and the abbreviation subwords, the abbreviation subwords are mapped to the full name subwords one by one, and then the full name subwords are combined to obtain the full name information. However, due to the incompleteness and non-uniqueness of the mapping relationship, it is easy to cause errors in the recognition of the full name information. Summary of the invention

[0004] In view of the above problems, the present application provides a method and a related device for generating full name information to achieve the purpose of improving the accuracy of generating full name information. The specific scheme is as follows:

[0005] The first aspect of the present application provides a method for generating full name information, comprising:

[0006] Acquire target abbreviation information, wherein the target abbreviation information is a subword sequence including a plurality of target abbreviation subwords;

[0007] Obtaining a graph embedding vector sequence and abbreviation Token encoding sequence of the target abbreviation information, wherein the graph embedding vector sequence includes graph embedding vectors of each abbreviation subword arranged in sequence, and the abbreviation Token encoding sequence includes Token encodings of each abbreviation subword arranged in sequence;

[0008] Inputting the graph embedding vector sequence and the abbreviation Token encoding sequence of the target abbreviation information into a pre-trained sequence generation model to obtain a full name embedding vector sequence output by the sequence generation model; the full name embedding vector sequence includes full name embedding vectors of multiple target full name subwords arranged in sequence;

[0009] The sequence generation model includes an encoder and a decoder constructed based on a neural network model of a self-attention mechanism, the encoder includes N encoder blocks connected in sequence, the decoder includes N decoder blocks connected in sequence, the input of the first encoder block is a fusion vector that performs weighted fusion of a graph embedding vector and a Token code based on a weight parameter, and the input of the nth encoder block is a fusion vector that performs weighted fusion of a graph embedding vector based on a weight parameter and an output vector of a previous encoder block; the input of the first decoder block is a fusion vector that performs weighted fusion of a graph embedding vector and an output vector of an Nth encoder block based on a weight parameter, and the input of the nth decoder block is a fusion vector that performs weighted fusion of a graph embedding vector and an output vector of a previous decoder block based on a weight parameter, and the output is the full embedding vector sequence; wherein, 1<n≤N; the weight parameters corresponding to each encoder block and decoder block are obtained through training based on training data;

[0010] Based on the full name embedding vector sequence, the full name information of the target abbreviation information is generated, and the full name information includes a plurality of the target full name subwords.

[0011] In a possible implementation, obtaining a graph embedding vector sequence of the target abbreviation information includes:

[0012] According to the graph embedding vector dictionary generated by training the graph neural network model, the graph embedding vector of each target abbreviation subword is obtained, and the graph embedding vector dictionary includes the graph embedding vector of each preset abbreviation subword; the graph neural network model is composed of multiple graph neural modules, and the graph neural module includes a graph attention network and a graph convolution network. The graph neural network model is obtained based on the relationship mapping graph training, and the relationship mapping graph has abbreviation subwords and full name subwords as nodes, and the mapping relationship from abbreviation subwords to full name subwords as directed edges;

[0013] The graph embedding vectors of the target abbreviation subwords are arranged in sequence to obtain a graph embedding vector sequence of the target abbreviation information.

[0014] In one possible implementation, the training method of the graph neural network model includes:

[0015] Based on the preset sub-word mapping relationship, construct the relationship mapping diagram, where the sub-word mapping relationship includes the mapping relationship between the abbreviation sub-word and the full name sub-word;

[0016] Based on the relationship map, two nodes with edges and an edge connecting the two nodes are selected from the relationship map as a positive sample; two nodes without edges are selected from the relationship map, and edges are constructed for the two nodes as a negative sample;

[0017] Generate graph training data, wherein the graph training data includes multiple positive samples and multiple negative samples;

[0018] The graph neural network model is trained based on the graph training data until a preset training completion condition is reached to obtain the graph embedding vector dictionary.

[0019] In a possible implementation, obtaining the abbreviated Token encoding sequence includes:

[0020] Input each of the target abbreviation subwords into a pre-trained Token encoding model to obtain a Token encoding of each of the target abbreviation subwords output by the Token encoding model, wherein the Token encoding model includes a causal language model;

[0021] Generate the Token code of each of the target abbreviation subwords in sequence to obtain the abbreviation Token code sequence of the target abbreviation information.

[0022] In one possible implementation, the encoder includes a Transformer encoder, and the decoder includes a Transformer decoder;

[0023] Each encoder block in the Transformer encoder includes a multi-head attention layer, a vector addition and normalization layer, a position element-by-position feedforward layer, and a vector addition and normalization layer connected in sequence, and each decoder block in the Transformer decoder includes a multi-head attention layer, a vector addition and normalization layer, a multi-head attention layer, a position element-by-position feedforward layer, and a vector addition and normalization layer connected in sequence;

[0024] The encoder further includes a first position encoding block arranged before the first encoder block and an encoding fusion block arranged before each encoder block, and the decoder further includes a decoding fusion block arranged before each decoder block.

[0025] In one possible implementation, the training method of the sequence generation model includes:

[0026] Constructing a parallel corpus, the parallel corpus including sample abbreviation information and corresponding sample full name information, the sample abbreviation information including multiple sample abbreviation subwords, and the sample full name information including multiple sample full name subwords;

[0027] Acquire sequence training data, wherein the sequence training data includes a graph embedding vector sequence and a short Token encoding sequence of each piece of the sample abbreviation information, and a full name Token encoding sequence of the sample full name information, wherein the full name Token encoding sequence of the sample full name information includes a Token encoding of each full name subword in the sample full name information;

[0028] The graph embedding vector sequence of the sample abbreviation information, the abbreviation Token encoding sequence, and the full name Token encoding sequence are input into the sequence generation model, and multiple iterations are performed, each of which includes:

[0029] Performing position encoding on the Token encoding of the sample abbreviation subword by the first position encoding block to obtain a position encoding vector of the sample abbreviation subword;

[0030] Performing weighted fusion on the position encoding vector of the sample abbreviation subword and the graph embedding vector of the sample abbreviation subword based on a weight parameter through a first encoding fusion block, and inputting the weighted fused intermediate vector into a first encoder block;

[0031] Performing weighted fusion on the output vector of the n-1th encoder block and the graph embedding vector of the sample abbreviation subword based on the weight parameter through each nth encoder fusion block, and inputting the weighted fused intermediate vector into the nth encoder block;

[0032] The weighted fused intermediate vectors are encoded and outputted through each encoder block;

[0033] Position encoding is performed on the Token encoding of the full name subword of the sample by a second position encoding block to obtain an embedding vector of the full name of the sample;

[0034] Performing weighted fusion on the output vector of the Nth encoder block, the graph embedding vector of the sample abbreviation subword, and the sample full name embedding vector based on the weight parameter through the first decoding fusion block, and inputting the weighted fused intermediate vector into the first decoder block;

[0035] Performing weighted fusion on the output vector of the n-1th decoder block and the graph embedding vector of the sample abbreviation subword based on the weight parameter through each nth decoding fusion block, and inputting the weighted fused intermediate vector into the nth encoder block;

[0036] The weighted fused intermediate vectors are encoded and outputted through each encoder block;

[0037] Performing mask loss calculation on the output of the Nth decoder block and the sample full name embedding vector according to the vector length through the loss calculation block, and updating the model parameters of the sequence generation model based on the mask loss calculation result, wherein the model parameters include weight parameters corresponding to each of the decoding fusion blocks;

[0038] Determine whether the preset training completion conditions are met. If so, obtain the trained sequence generation model. If not, execute the next iteration.

[0039] In a possible implementation, generating the full name information of the target abbreviation information based on the full name embedding vector sequence includes:

[0040] Obtaining the Token code of each full name embedding vector in the full name embedding vector sequence to obtain a target full name Token code sequence, wherein the target full name code sequence includes the Token codes of each full name embedding vector arranged in sequence,

[0041] Based on a pre-configured word dictionary, searching for a full name subword that matches the Token encoding of each of the full name embedding vectors as the target full name subword, the word dictionary including preset full name subwords and matching Token encodings;

[0042] The full name information of the target abbreviation information is obtained by sequentially combining the target full name subwords.

[0043] A second aspect of the present application provides a device for generating full name information, including:

[0044] an abbreviation information acquisition unit, used to acquire target abbreviation information, wherein the target abbreviation information is a subword sequence including a plurality of target abbreviation subwords;

[0045] An information representation unit, used to obtain a graph embedding vector sequence and abbreviation Token encoding sequence of the target abbreviation information, wherein the graph embedding vector sequence includes graph embedding vectors of each abbreviation subword arranged in sequence, and the abbreviation Token encoding sequence includes Token encodings of each abbreviation subword arranged in sequence;

[0046] A sequence generation unit, used to input the graph embedding vector sequence and the abbreviation Token encoding sequence of the target abbreviation information into a pre-trained sequence generation model to obtain a full name embedding vector sequence output by the sequence generation model; the full name embedding vector sequence includes full name embedding vectors of multiple target full name subwords arranged in sequence;

[0047] The sequence generation model includes an encoder and a decoder constructed based on a neural network model of a self-attention mechanism, the encoder includes N encoder blocks connected in sequence, the decoder includes N decoder blocks connected in sequence, the input of the first encoder block is a fusion vector that performs weighted fusion of a graph embedding vector and a Token code based on a weight parameter, and the input of the nth encoder block is a fusion vector that performs weighted fusion of a graph embedding vector based on a weight parameter and an output vector of a previous encoder block; the input of the first decoder block is a fusion vector that performs weighted fusion of a graph embedding vector and an output vector of an Nth encoder block based on a weight parameter, and the input of the nth decoder block is a fusion vector that performs weighted fusion of a graph embedding vector and an output vector of a previous decoder block based on a weight parameter, and the output is the full embedding vector sequence; wherein, 1<n≤N; the weight parameters corresponding to each encoder block and decoder block are obtained through training based on training data;

[0048] A full name matching unit is used to generate full name information of the target abbreviation information based on the full name embedding vector sequence, wherein the full name information includes a plurality of target full name subwords.

[0049] The third aspect of the present application provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the method for generating full name information of the above-mentioned first aspect or any implementation method of the first aspect.

[0050] A fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0051] The memory is used to store computer programs;

[0052] The processor is used to execute the computer program so that the electronic device can implement the method for generating full name information of the above-mentioned first aspect or any implementation manner of the first aspect.

[0053] By means of the above technical scheme, the present application provides a method for generating full name information and a related device, which inputs the graph embedding vector sequence and the abbreviation Token encoding sequence of the target abbreviation information into a pre-trained sequence generation model, and the multi-layer encoder blocks in the encoder constructed based on the self-attention mechanism in the sequence generation model encode the intermediate vectors after the graph embedding vectors are fused with the weight parameters corresponding to each layer layer by layer, and the multi-layer decoder blocks in the decoder constructed based on the self-attention mechanism in the sequence generation model decode the intermediate vectors after the graph embedding vectors are fused with the weight parameters corresponding to each layer layer by layer, and obtain a full name embedding vector sequence composed of the full name embedding vectors of the target full name sub-words corresponding to each target abbreviation sub-word, and further, the full name information of the target abbreviation information is generated through the full name embedding vector sequence. This scheme obtains the weight parameters of each layer through dynamic training, so that the graph embedding vectors obtained by graph embedded learning are fused with different weights to the encoder blocks and decoder blocks of each layer built based on the self-attention mechanism. The relationship between each target abbreviation sub-word is captured through the self-attention mechanism, and the relationship between each target abbreviation sub-word is reinforced through graph embedded learning to improve the expression accuracy of the target abbreviation sub-word, thereby improving the generation accuracy of the full name embedding vector sequence by the sequence generation model, and further improving the accuracy of the full name information of the target abbreviation information. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0055] Figure 1 A schematic diagram of a flow chart of a method for generating full name information provided in an embodiment of the present application;

[0056] Figure 2 A schematic diagram of the structure of a sequence generation model provided in an embodiment of the present application;

[0057] Figure 3 A schematic diagram of the structure of a full name translation model provided in an embodiment of the present application;

[0058] Figure 4 A schematic diagram of the specific structure of a graph neural network model provided in an embodiment of the present application;

[0059] Figure 5 A schematic diagram of the specific structure of a Transformer encoder block provided in an embodiment of the present application;

[0060] Figure 6 A schematic diagram of the specific structure of a Transformer decoder block provided in an embodiment of the present application;

[0061] Figure 7 A specific implementation flow chart of a classifier of a full name translation model is provided for the embodiment of the present application;

[0062] Figure 8 A specific implementation flow chart of a method for constructing a full name translation model is provided for an embodiment of the present application;

[0063] Fig. 9 A sub-graph schematic diagram of a mapping relationship graph is provided for an embodiment of the present application;

[0064] Fig.10 A schematic diagram of the structure of a full name conversion model provided in an embodiment of the present application;

[0065] Fig.11 A specific implementation process of a method for generating full name information is provided for an embodiment of the present application;

[0066] Fig.12 A schematic diagram of the structure of a device for generating full name information is provided for an embodiment of the present application;

[0067] Fig.13 A structural schematic diagram of an electronic device is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0068] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0069] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0070] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0071] This application can be applied in the field of natural language processing, and can be applied to the scenario of restoring the abbreviation information of an institution to its full name information. A specific application scenario is: in the process of constructing the institutional specification document, the abbreviation information of the institution is obtained from various data sources, and the abbreviation information is generated into the full name information. Among them, the data source of the abbreviation information includes the bibliographic data in the target database, and the target database includes the database provided by the global citation database WOS (Web of Science) information service platform and other open source databases.

[0072] Taking the target database WOS as an example, the bibliographic data provided by WOS uses abbreviations to display the abbreviation information of a large number of institutions. The information of the institution mainly includes the name and / or address of the institution, which is used to uniquely indicate the institution. After obtaining the abbreviation information of the institution, it is necessary to restore the information to obtain the complete and standard full name information of the institution. It should be noted that this application only takes the identification of the abbreviation information of WOS bibliographic data to obtain the full name information as an example. This application can be applied to the restoration of any abbreviation information obtained from other data sources, and the extraction of various information mentioned in this application (including but not limited to various information obtained from various databases) is authorized and is non-sensitive data.

[0073] In order to realize the conversion from abbreviation information to full name information, there are currently several generation technologies:

[0074] The first method is to use parallel database alignment. For example, the full text of the document can be queried from other databases according to the document's unique identifier, thereby obtaining the full name information of the institution. Obviously, due to the uniqueness of the document data in each database, there are a considerable number of documents that cannot be aligned with parallel databases, that is, the full name information of the institution cannot be identified through parallel database alignment.

[0075] The second method: Based on the pre-configured sub-word mapping relationship. WOS provides a mapping relationship from the full name sub-word to the abbreviation sub-word of the organization, address, and country information on its official website. The mapping relationship can be used to identify the abbreviation information of the organization into the full name information.

[0076] However, due to the incompleteness and non-uniqueness of the mapping relationship, it is easy to cause errors in the generation of full name information and the success rate is low. The specific analysis is as follows:

[0077] (1) The number of institutions is very large, and the mapping relationship cannot cover the mapping of the abbreviation sub-words and full name sub-words of all institutions, resulting in the risk of sub-word generation failure.

[0078] (2) The mapping relationship is not one-to-one, but there are cases where multiple full subwords are mapped to one abbreviation subword, or one full subword is mapped to multiple abbreviation subwords. For example, the mapping result of an abbreviation subword in the same data source includes multiple full subwords in the same language or different languages. For another example, in different data sources, a full subword may correspond to different abbreviation subwords. When using mapping relationships to map subwords, due to the many-to-many mapping relationship, it is easy to cause the search space to explode, and the generation efficiency and accuracy are low.

[0079] (3) The organizational hierarchical structures of organizational information from different data sources are different. Therefore, after obtaining the sub-word mapping results, when fusing the sub-word mapping results to obtain the complete full name information, there are difficulties in grammatical alignment. In this regard, Table 1 illustrates the comparison of organizational hierarchical structures between the WOS database and other databases.

[0080] Table 1 Example of institutional hierarchy

[0081]

[0082] As can be seen from Table 1, there are significant differences in the institutional hierarchical structure between WOS and other databases, that is, the different order of institutional levels leads to different grammatical rules for abbreviation information. In addition, as can be seen from Table 1, there are cases where a full name subword corresponds to different abbreviation subwords, that is, there are cases where some keywords are abbreviated. For example, the full name subword "University" corresponds to the abbreviation subword "Univ" in the title data of the data source WOS, and the corresponding abbreviation subword is "University" itself when the data source is other databases.

[0083] In summary, existing full name information recognition methods all have technical problems due to low recognition accuracy.

[0084] Usually, the abbreviation subword exists in the longer organization address information and contains contextual semantic information. Based on this, this application proposes to use the full name information and abbreviation information of the organization as two natural languages, and use the neural network model built based on deep learning to realize natural language recognition, thereby realizing the conversion of abbreviation information to full name information.

[0085] In order to solve the above technical problems, the embodiment of the present application provides a method for generating full name information. The method for generating full name information provided by the embodiment of the present application is described in detail below in conjunction with the accompanying drawings.

[0086] Reference Figure 1 , Figure 1 A flow chart of a method for generating full name information provided in an embodiment of the present application is shown as follows: Figure 1 As shown, a method for generating full name information provided in an embodiment of the present application may include S101 to S104, and these steps are described in detail below.

[0087] S101, obtaining target abbreviation information.

[0088] In this embodiment, the target abbreviation information is a subword sequence including a plurality of target abbreviation subwords.

[0089] S102, obtaining a graph embedding vector sequence and abbreviation Token encoding sequence of target abbreviation information.

[0090] In this embodiment, the graph embedding vector sequence includes graph embedding vectors of each abbreviation sub-word arranged in sequence, and the abbreviation Token encoding sequence includes Token encodings of each abbreviation sub-word arranged in sequence.

[0091] In this embodiment, by performing graph embedded reinforcement learning on each target abbreviation subword in the target abbreviation information to perform vector representation, the graph embedding vector of each target abbreviation subword is obtained, and the graph embedding vector of each target abbreviation subword is arranged according to the rank in the target abbreviation information to obtain the graph embedding vector sequence of the target abbreviation information. By performing Token encoding on each target abbreviation subword in the target abbreviation information, the Token encoding of each target abbreviation subword is obtained, and the Token encoding of each target abbreviation subword is arranged according to the rank in the target abbreviation information to obtain the Token encoding sequence of the target abbreviation information.

[0092] It should be noted that there are multiple methods for generating the graph embedding vector of the target abbreviation subword. In one possible implementation, the target abbreviation subword is input into the trained graph neural network model to obtain the graph embedding vector of the target abbreviation subword. There are multiple methods for generating the Token encoding of the target abbreviation subword. In one possible implementation, the target abbreviation subword is input into the trained Token encoding model to obtain the Token encoding of the target abbreviation subword. Among them, the graph neural network model is constructed based on the graph neural block, and the Token encoding model is constructed based on the autoregressive language model.

[0093] S103, inputting the graph embedding vector sequence of the target abbreviation information and the abbreviation Token encoding sequence into a pre-trained sequence generation model to obtain the full name embedding vector sequence output by the sequence generation model.

[0094] In this embodiment, the full name embedding vector sequence includes full name embedding vectors of multiple target full name subwords arranged in sequence. In this embodiment, each encoder block is connected end to end in the order of the first encoder block to the Nth encoder block, and the input of the first encoder block includes the input of the sequence generation model, and the output of the Nth encoder block is the output of the sequence generation model. The nth encoder block represents the nth encoder block, N represents the decoder block or the number of decoder blocks, 1<n≤N, and the weight parameters corresponding to each encoder block and decoder block are obtained based on training data.

[0095] Figure 2 A schematic diagram of a sequence generation model provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the sequence generation model includes an encoder and a decoder constructed based on a neural network model of a self-attention mechanism, the encoder includes N sequentially connected encoder blocks, respectively denoted as a first encoder block, a second encoder block, ..., an Nth encoder block, and the decoder includes N sequentially connected decoder blocks, respectively denoted as a first decoder block, a second decoder block, ..., an Nth decoder block.

[0096] The input of the encoder includes the graph embedding vector in the graph embedding vector sequence of the target abbreviation information and the token code in the abbreviation token code sequence of the target abbreviation information, wherein the graph embedding vector and the token code of the target abbreviation subword are input into the encoder simultaneously.

[0097] The input of the decoder includes the graph embedding vector in the graph embedding vector sequence of the target abbreviation information and the encoding vector output after the Token encoding of the target abbreviation information is encoded by the encoder, wherein the graph embedding vector and the encoding vector of the target abbreviation subword are input into the encoder simultaneously.

[0098] The input of the first encoder block is a fusion vector that is a weighted fusion of the graph embedding vector and the Token code based on the weight parameter, and the input of the nth encoder block is a fusion vector that is a weighted fusion of the graph embedding vector and the output vector of the previous encoder block based on the weight parameter.

[0099] Among them, the weight parameter corresponding to the input of the nth encoder block is expressed as Taking the nth encoder block as the second encoder block as an example, the input of the second encoder block is based on the weight parameter The fusion vector is a weighted fusion of the graph embedding vector and the output vector of the first encoder block, wherein the weighted fusion method can be expressed as: fusion vector ,in, represents the output vector of the first encoder block, Represents the graph embedding vector.

[0100] The input of the first decoder block is a fusion vector that is a weighted fusion of the graph embedding vector and the output vector of the Nth encoder block based on the weight parameter, and the input of the nth decoder block is a fusion vector that is a weighted fusion of the graph embedding vector and the output vector of the previous decoder block based on the weight parameter.

[0101] Among them, the weight parameter corresponding to the input of the nth decoder block is expressed as Taking the nth decoder block as the second decoder block as an example, the input of the second decoder block is based on the weight parameter

[0102] The fusion vector is a weighted fusion of the graph embedding vector and the output vector of the first decoder block, wherein the weighted fusion method can be expressed as: fusion vector ,in, represents the output vector of the first decoder block, Represents the graph embedding vector.

[0103] It should be noted that the sequence generation model is built based on the neural network model of the self-attention mechanism. In a possible implementation, the sequence generation model is built based on the Transformer model of the neural network model of the self-attention mechanism. The encoder and decoder built based on the neural network model of the self-attention mechanism are called Transformer encoder and Transformer decoder respectively. The core of the Transformer model is the self-attention mechanism, which can assign a weight to each position in each input sequence, and then use these weighted position vectors as output. The Transformer model can be constructed by continuously stacking multiple self-attention layers and feedforward neural network layers.

[0104] S104: Generate full name information of the target abbreviation information based on the full name embedding vector sequence.

[0105] In this embodiment, the full name information includes a plurality of target full name subwords.

[0106] It can be seen from the above technical scheme that an embodiment of the present application provides a method for generating full name information, which inputs a graph embedding vector sequence and abbreviation Token encoding sequence of the target abbreviation information into a pre-trained sequence generation model, and encodes the intermediate vectors after the graph embedding vectors are fused with the weight parameters corresponding to each layer through the multi-layer encoder blocks in the encoder constructed based on the self-attention mechanism in the sequence generation model layer by layer, and decodes the intermediate vectors after the graph embedding vectors are fused with the weight parameters corresponding to each layer through the multi-layer decoder blocks in the decoder constructed based on the self-attention mechanism in the sequence generation model layer by layer, and obtains a full name embedding vector sequence composed of the full name embedding vectors of the target full name sub-words corresponding to each target abbreviation sub-word, and further, generates the full name information of the target abbreviation information through the full name embedding vector sequence. This scheme obtains the weight parameters of each layer through dynamic training, so that the graph embedding vectors obtained by graph embedded learning are fused with different weights to the encoder blocks and decoder blocks of each layer built based on the self-attention mechanism. The relationship between each target abbreviation sub-word is captured through the self-attention mechanism, and the relationship between each target abbreviation sub-word is reinforced through graph embedded learning to improve the expression accuracy of the target abbreviation sub-word, thereby improving the generation accuracy of the full name embedding vector sequence by the sequence generation model, and further improving the accuracy of the full name information of the target abbreviation information.

[0107] Figure 3 A schematic diagram of the structure of a full name translation model provided in an embodiment of the present application is shown in FIG. Figure 3 As shown in the figure, the full information recognition model includes Token encoding model, graph neural network model, sequence generation model and classifier.

[0108] In this embodiment, the Token encoding model is used to perform Token encoding on the target abbreviation subwords in the target abbreviation information to obtain a Token encoding sequence, and the Token codes in the Token encoding sequence are sequentially input into the sequence generation model.

[0109] Specifically, the Token encoding model can be a causal language model CasualLM, for example, a GPT-o model.

[0110] In this embodiment, the graph neural network model is used to represent the target abbreviation subwords in the target abbreviation information with graph embedded vectors to obtain a graph embedding vector sequence, and the graph embedding vectors in the graph embedding vector sequence are sequentially input into the sequence generation model.

[0111] Specifically, the graph neural network model is composed of multiple graph neural modules, including graph attention network and graph convolution network. Different graph neural modules capture information at different levels in the mapping relationship graph. Figure 4 A specific structural diagram of a graph neural network model provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the graph neural network model includes a first graph attention convolution layer, a first connection layer, a graph convolution layer, a second connection layer, a graph sampling and aggregation convolution layer, a third connection layer, a second graph attention convolution layer, a fully connected layer and a feature fusion layer, which are sequentially connected. The third connection layer and the fully connected layer are connected to the feature fusion layer using a jump connection structure. Each connection layer includes a regularization layer and a dropout layer. A portion of samples are discarded with an appropriate probability by setting hierarchical regularization layers and dropout layers between each convolution layer. The residual network structure is added to prevent the gradient from disappearing during the model training process.

[0112] In this embodiment, the sequence generation model is used to fuse the graph embedding vector and Token encoding of the target abbreviation word, and generates the Token encoding of the target full name word based on the self-attention mechanism to obtain a full name embedding vector sequence.

[0113] Specifically, the sequence generation model includes an encoder and a decoder constructed based on a neural network model of a self-attention mechanism, wherein the encoder includes N sequentially connected encoder blocks, a first position encoding block configured before the first encoder block, and an encoding fusion block configured before each encoder block, and the decoder includes N sequentially connected decoder blocks and a decoding fusion block configured before each decoder block.

[0114] The input of the encoder includes the graph embedding vector in the graph embedding vector sequence of the target abbreviation information and the token code in the abbreviation token code sequence of the target abbreviation information, wherein the graph embedding vector and the token code of the target abbreviation subword are input into the encoder simultaneously.

[0115] The input of the first position coding block is the Token code in the Token code sequence, which is used to perform position coding on the Token code. The input of the first coding fusion block configured before the first encoder block is the position coding vector and the graph embedding vector after position coding on the Token code, which is used to perform weighted fusion on the graph embedding vector and the position coding vector based on the weight parameter to obtain the fusion vector. The input of the first encoder block is the fused vector , used to fusion vector based on self-attention mechanism Encode to get the encoding vector The input of the nth coding fusion block configured before the nth encoder block is the output coding vector of the n-1th encoder block, which is used to perform weighted fusion on the graph embedding vector and the output coding vector of the n-1th encoder block based on the weight parameter to obtain the fusion vector The input of the nth encoder block is the fused vector Among them, the weight parameter corresponding to the input of the nth encoder block is expressed as .

[0116] The input of the decoder includes the graph embedding vector in the graph embedding vector sequence of the target abbreviation information and the encoding vector output after the Token encoding of the target abbreviation information is encoded by the encoder, that is, the output encoding vector of the Nth encoder block. Among them, the graph embedding vector and encoding vector of the target abbreviation subword are input into the encoder simultaneously.

[0117] The first decoding fusion block configured before the first decoder block is inputted with the graph embedding vector of the target abbreviation subword, the output vector of the Nth encoder block, and the context-related attention vector, and is used to weightedly fuse the graph embedding vector of the target abbreviation subword, the output vector of the Nth encoder block, and the context-related attention vector based on the weight parameter to obtain a fusion vector The input of the first decoder block is the fused vector , used to fusion vector based on self-attention mechanism Decode to get the decoding vector The input of the nth decoding fusion block configured before the nth decoder block is the output decoding vector of the n-1th decoder block, which is used to perform weighted fusion on the graph embedding vector and the output decoding vector of the n-1th decoder block based on the weight parameter to obtain the fusion vector , the input of the nth decoder block is the fused vector , the output is the full embedding vector. Among them, the weight parameter corresponding to the input of the nth decoder block is expressed as The Nth decoder block, i.e., the last decoder block, also outputs the input context-related attention vector to the first decoding fusion block.

[0118] In this embodiment, the sequence generation model is constructed based on the Bahdanau attention mechanism or the Transformer attention mechanism. Taking the Transformer attention mechanism as an example, the sequence generation model is a Transformer model, wherein the Transformer model includes a Transformer encoder and a Transformer decoder, the Transformer encoder includes multiple Transformer encoder blocks, and the Transformer decoder includes multiple Transformer decoder blocks. The context-dependent attention vector is a vector sequence carrying context information generated based on the attention mechanism.

[0119] Figure 5 A specific structural diagram of a Transformer encoder block provided in an embodiment of the present application, such as Figure 5 As shown, the Transformer encoder block includes a multi-head attention layer, a vector addition and normalization layer, a position-wise element-wise feed-forward layer, and a vector addition and normalization layer connected sequentially.

[0120] Figure 6 A specific structural diagram of a Transformer decoder block provided in an embodiment of the present application, such as Figure 6 As shown, the Transformer decoder is a sequentially connected position encoder block and a Transformer decoding block, and the Transformer decoding block includes a sequentially connected multi-head attention layer, a vector addition and normalization layer, a multi-head attention layer, a position element-by-element feedforward layer, and a vector addition and normalization layer.

[0121] In this embodiment, the classifier includes a Token decoder and a word generator. Figure 7 A specific structural diagram of a classifier provided in an embodiment of the present application, wherein the input of the Token decoder is a sequence of full name embedding vectors output by the decoder, and the Token encoding of the full name subwords corresponding to each full name embedding vector is output to a word generator, and the output of the word generator is the full name information composed of the target full name subwords, that is, the full name information of the target abbreviation information.

[0122] based on Figure 3 The full name translation model shown, the embodiment of the present application provides a specific implementation process of a method for constructing a full name translation model, Figure 8 A specific implementation flow chart of a method for constructing a full name translation model is provided for the embodiment of the present application, such as Figure 8 As shown, this method specifically includes:

[0123] S801: Construct a relationship mapping diagram based on a preset sub-word mapping relationship.

[0124] In this embodiment, the subword mapping relationship includes the mapping relationship between the abbreviation subword and the full name subword. The abbreviation subword and the full name subword are used as nodes, and the mapping relationship is used as an edge. The direction of the edge is from the abbreviation subword to the full name subword, and a relationship mapping graph is constructed.

[0125] Taking WOS as an example, the list of abbreviation subwords provided by WOS contains the mapping relationship between abbreviation subwords and full name subwords related to addresses. The mapping relationship on the list is used to generate a mapping relationship diagram between abbreviation subwords and full name subwords in the form of a word-word correlation matrix. The abbreviation subwords and full name subwords with mapping relationships are connected by directed edges, and the direction of the directed edges is from the abbreviation subword to the full name subword.

[0126] Fig. 9 A sub-graph schematic diagram of a mapping relationship graph is provided for an embodiment of the present application. As shown in the sub-graph, a mapping relationship is represented by [abbreviated subword full name subword], and the mapping relationships include [Med Medizin], [Med Medicine], [Med Medicinal], [Med Medical], [Med Meditskkaya], [Medicine Medizin], [MedicineMeditskkaya], [Medicine Medical], and [Medicine Medicinal].

[0127] S802: Construct graph training data based on the relationship mapping graph.

[0128] In this embodiment, the graph training data includes positive samples and negative samples.

[0129] Select two nodes with edges and the edge connecting the two nodes from the relationship map as a positive sample. Select two nodes without edges from the relationship map and construct an edge for the two nodes as a negative sample.

[0130] It should be noted that due to the incompleteness of the mapping relationship, the mapping relationship graph is a sparse graph. Therefore, the number of positive samples is sparse. Therefore, data enhancement is performed on the number of samples. The enhancement method is to construct an enhanced edge for two unconnected nodes on the sparse graph. The negative sample is composed of the two nodes and the enhanced edge, thereby achieving training data enhancement.

[0131] Still taking WOS as an example, the mapping relationship graph constructed based on WOS is a sparse graph with 537 nodes and 473 edges. In order to enhance the robustness and adaptability of the graph, the method of data negative sampling is adopted. The first sampling node is randomly sampled from the sparse graph, and then a second sampling node that is not connected to the first sampling node is sampled, and the edge between the first sampling node and the second sampling node is constructed as a negative sample. Then, the loss of negative samples is subtracted when training the graph neural network model to improve the training effect.

[0132] S803: Train the graph neural network model based on the graph training data until the preset training completion condition is met to obtain a graph embedding vector dictionary.

[0133] In this embodiment, the graph embedding vector dictionary includes each abbreviation subword in the relationship mapping graph and the corresponding graph embedding vector. The graph neural network model is used to output the graph embedded vector of the abbreviation subword to be processed based on the trained graph embedding vector dictionary. The structure of the graph neural network model can be seen in Figure 4 .

[0134] It should be noted that S801~803 is a specific method provided in an embodiment of the present application for training a graph neural network to obtain a graph neural network model. After the graph neural network model is trained using the attention mechanism of the graph neural network and contrastive learning, the graph embedding vectors of each node on the mapping relationship graph are learned and a graph embedding vector dictionary is formed.

[0135] S804. Build a parallel corpus.

[0136] In this embodiment, the parallel corpus includes sample abbreviation information and corresponding sample full name information, the sample abbreviation information includes multiple sample abbreviation subwords, and the sample full name information includes multiple sample full name subwords.

[0137] In an optional embodiment, the method for constructing a parallel corpus includes:

[0138] 1. Retrieve bibliographic data from various databases and construct an institutional information set corresponding to each paper, wherein the institutional information set includes information of each institution, and the institutional information includes abbreviation information and / or full name information.

[0139] 2. Perform data cleaning on the institutional information sets of each paper, remove institutional information sets with only abbreviations or full names, remove institutional information with the number of subwords less than the preset threshold, remove institutional information containing special characters, and remove institutional information containing non-English characters.

[0140] 3. For each institutional information set of the paper, align the abbreviation information and full name information in the institutional information set to identify the full name information and corresponding abbreviation information of the same institution.

[0141] 4. Based on the alignment results, build a parallel corpus.

[0142] For example, by searching the Dimensions database according to the preset search conditions, 540,991 search data were obtained. In order to facilitate the construction of the parallel corpus, the bibliographic data with only one institution were retained, and the bibliographic data with only abbreviations or full names, the bibliographic data with too short institution information (less than 10 characters), the bibliographic data containing special characters, and the bibliographic data containing non-English characters were removed. Finally, a parallel corpus including 138,000 abbreviation information and corresponding full name information was obtained.

[0143] In this embodiment, the information format of the parallel corpus can be seen in Table 2.

[0144] Table 2 Parallel corpus table:

[0145]

[0146] S805. Use a pre-trained Token encoding model to obtain a short Token encoding sequence of the sample short name information and a full Token encoding sequence of the sample full name information.

[0147] In this embodiment, the Token encoding model includes a causal language model CasualLM.

[0148] Specifically, the abbreviation information and full name information in the parallel corpus are respectively input into the pre-trained Token encoding model, the Token encoding of each sample abbreviation subword and the Token encoding of each sample full name subword output by the Token encoding model are obtained, and the abbreviation Token encoding sequence of the sample abbreviation information and the full name embedding vector sequence of the sample full name information are obtained.

[0149] It should be noted that the method of using CasualLM to generate Token encoding of abbreviation and full name information in a parallel corpus can be referred to in the prior art.

[0150] S806. Use a graph neural network model to obtain a graph embedding vector of each sample abbreviation subword in the sample abbreviation information in the parallel corpus.

[0151] S807: Construct sequence training data.

[0152] In this embodiment, the sequence training data includes the graph embedding vector sequence and the abbreviation token encoding sequence of each sample abbreviation information, and the full name token encoding sequence of the sample full name information. Among them, the token encoding sequence of the sample abbreviation information includes the token encoding sequence of each sample abbreviation subword. , the graph embedding vector sequence of sample abbreviation information includes the graph embedding vector of each sample abbreviation subword The Token encoding sequence of the sample full name information includes the Token encoding of each sample full name subword .

[0153] S808. Input the graph embedding vector sequence, the abbreviation Token encoding sequence, and the full name Token encoding sequence of the sample abbreviation information into the sequence generation model, perform multiple iterations until the preset training completion condition is reached, and obtain the trained sequence generation model and the graph embedding vector dictionary.

[0154] In this embodiment, the sequence generation model is a neural network model based on a self-attention mechanism, wherein the model parameters of the sequence generation model include weight parameters of each encoding fusion layer and each decoding fusion layer.

[0155] Taking the sequence generation model as an example, the neural network model based on the Transformer self-attention mechanism, Fig.10 A schematic diagram of an iterative process provided in an embodiment of the present application, such as Fig.10 As shown, each iteration includes:

[0156] A1. Token encoding of the sample abbreviation subword through the first position encoding block Perform position encoding to obtain the position encoding vector of the sample abbreviation subword .

[0157] A2. Perform weighted fusion on the position encoding vector of the sample abbreviation subword and the graph embedding vector of the sample abbreviation subword based on the weight parameter through the first encoding fusion block, and input the weighted fused intermediate vector into the first encoder block.

[0158] A3. Sequentially execute the encoding of the weighted fused intermediate vector through the mth encoder block and output it to the m+1th encoding fusion block, perform weighted fusion on the encoded encoding vector and the graph embedding vector of the sample abbreviation subword based on the weight parameter through the m+1th encoding fusion block, and input the weighted fused intermediate vector to the m+1th encoder block until the m+1th encoder block is the Nth encoder block, that is, the last encoder block.

[0159] It should be noted that due to the short position encoding, the encoding embedding vector Scale by the square root of the dimension, increasing and The variance between .

[0160] A4, the Nth encoder block encodes the encoded code vector Output to the decoder block.

[0161] A5. Token encoding of the full subword of the sample through the second position encoding block Perform position encoding to obtain the full embedding vector of the sample.

[0162] A6, the output vector of the Nth encoder block is fused based on the weight parameter by the first decoding fusion block , the graph embedding vector of the sample abbreviation subword and the sample full name embedding vector are weighted fused, and the weighted fused intermediate vector is input into the first decoder block.

[0163] A7. Sequentially execute the encoding of the weighted fused intermediate vector through the mth decoder block and output it to the m+1th decoding fusion block, perform weighted fusion on the decoded decoding vector and the graph embedding vector of the sample abbreviation subword based on the weight parameter through the m+1th decoding fusion block, and input the weighted fused intermediate vector to the m+1th decoder block until the m+1th decoder block is the Nth decoder block, that is, the last decoder block.

[0164] A8. The loss calculation block (mask loss layer) performs mask loss calculation on the output of the Nth decoder block and the sample full name embedding vector according to the vector length to determine whether the preset training completion condition is met. If so, the trained sequence generation model is obtained. If not, the model parameters of the sequence generation model are updated using the back propagation algorithm based on the mask loss calculation result to execute the next iteration.

[0165] In this embodiment, the model parameters include weight parameters corresponding to each decoding fusion block. It should be noted that a gradient clipping layer is added before the loss calculation block to prevent gradient explosion and improve the model training effect.

[0166] It should be noted that in order to automatically add the graph embedding vectors of the abbreviation subwords in the abbreviation information to the model as needed, a learnable graph vector fusion algorithm is used to update the model parameters. The graph vector fusion algorithm is shown in Table 3, including:

[0167] Table 3 Weight learning algorithm

[0168]

[0169] S804~S808 are a specific training method for a sequence generation model provided in an embodiment of the present application.

[0170] S809: Construct a classifier based on the pre-constructed word dictionary.

[0171] In this embodiment, the word dictionary includes preset full-name subwords and matching Token codes.

[0172] In this embodiment, the Token decoder in the classifier is used to generate a full Token code sequence, which includes multiple Token codes.

[0173] In this embodiment, the word generator is used to search for full name subwords that match the Token encoding of each full name embedding vector based on a word dictionary as the target full name subword, and output the full name information of the target abbreviation information, that is, the target full name information.

[0174] based on Figure 3 The full name information recognition model shown in the embodiment of the present application provides a specific implementation method for generating a full name information. When converting the abbreviation information into the full name information, given an institution address information consisting of an institution abbreviation name, each institution abbreviation name is extracted to generate its corresponding institution full name, as shown in formula (1):

[0175] (1);

[0176] in, It is the abbreviation of information. It is the full subword of the full information, indicating the length of the full information, that is, the number of full subwords This application constructs a mapping through a full name information recognition model , so that the abbreviation information can be identified while maintaining the semantics, and the prediction result is the full name information.

[0177] Fig.11 A specific implementation process of a method for generating full name information is provided for the embodiment of the present application, such as Fig.11 As shown, this method specifically includes:

[0178] S1101. Obtain target abbreviation information.

[0179] In this embodiment, the target abbreviation information is any abbreviation information to be translated, and the target abbreviation information includes multiple target abbreviation subwords. For example, the target abbreviation information includes M abbreviation subwords, which are recorded as ,in, Indicates the mth abbreviation subword in the target abbreviation information.

[0180] S1102. Input each target abbreviation subword into a pre-trained graph neural network model to obtain a graph embedding vector of each target abbreviation subword output by the graph neural network model, arrange the graph embedding vectors of each target abbreviation subword in sequence, and obtain a graph embedding vector sequence of the target abbreviation information.

[0181] Specifically, each target abbreviation subword is input into the graph neural network model, and the graph neural network model outputs the graph embedding vector of each target abbreviation subword in sequence based on the trained graph embedding vector dictionary.

[0182] It should be noted that the target abbreviation subword is the sample abbreviation subword in the graph training data. Therefore, in the graph embedding vector dictionary, the embedded vector of the sample abbreviation subword has been generated. If the target abbreviation subword is not the sample abbreviation subword in the graph training data, the target abbreviation subword is added as a new node to the mapping relationship graph, and the graph embedding vector dictionary is retrained to obtain the embedded vector of the target abbreviation subword.

[0183] In this embodiment, the graph neural network model is composed of multiple graph neural modules, including a graph attention network and a graph convolutional network. The graph neural network model is trained based on a relationship mapping graph, and the relationship mapping graph has abbreviation subwords and full name subwords as nodes, and the mapping relationship from abbreviation subwords to full name subwords as directed edges. For the specific graph neural network model structure and training method, please refer to the above embodiment.

[0184] S1103, input each target abbreviation subword into a pre-trained Token encoding model, obtain the Token encoding of each target abbreviation subword output by the Token encoding model, arrange the Token encoding of each target abbreviation subword in sequence, and obtain the abbreviation Token encoding sequence of the target abbreviation information.

[0185] In this embodiment, the Token encoding model includes a causal language model.

[0186] S1104, input the abbreviation Token encoding sequence of the target abbreviation information into the trained sequence generation model, and obtain the full name embedding vector sequence of the target abbreviation information output by the sequence generation model.

[0187] Specifically, the model parameters of the trained sequence generation model include the weight parameters of each optimized encoding fusion block and each decoding fusion block. The process of the sequence generation model outputting the full name embedding vector sequence of the target abbreviation information includes multiple iterations, and each iteration outputs a full name embedding vector of a target abbreviation subword and outputs it in sequence until the length of the output full name embedding vector sequence reaches a preset length, or encounters an end symbol.

[0188] Taking one iteration as an example, the method of generating the full name embedding vector of the target abbreviation subword includes:

[0189] B1. Positionally encode the Token encoding of the target abbreviation subword through the first position encoding block to obtain the position encoding vector of the target abbreviation subword.

[0190] B2. Perform weighted fusion on the graph embedding vector of the target abbreviation subword and the position encoding vector of the target abbreviation subword based on the weight parameter through the first encoding fusion block, and input the weighted fused intermediate vector into the first encoder block.

[0191] B3. Sequentially execute the encoding of the weighted fused intermediate vector through the mth encoder block and output it to the m+1th encoding fusion block, perform weighted fusion on the graph embedding vector of the target abbreviation subword and the encoded encoding vector based on the weight parameter through the m+1th encoding fusion block, and input the weighted fused intermediate vector to the m+1th encoder block until the m+1th encoder block is the Nth encoder block, that is, the last encoder block.

[0192] B4. The Nth encoder block outputs the encoded code vector to the decoder block.

[0193] B5. The first decoding fusion block performs weighted fusion on the graph embedding vector of the target abbreviation subword, the output vector of the Nth encoder block, and the context-related attention vector based on the weight parameter, and inputs the weighted fused intermediate vector into the first decoder block. The context-related attention vector is returned by the last decoder block after the previous iteration.

[0194] B6. Sequentially execute the encoding of the weighted fused intermediate vector through the mth decoder block and output it to the m+1th decoding fusion block, perform weighted fusion on the graph embedding vector of the target abbreviation subword and the decoded decoding vector based on the weight parameter through the m+1th decoding fusion block, and input the weighted fused intermediate vector to the m+1th decoder block until the m+1th decoder block is the Nth decoder block, that is, the last decoder block.

[0195] B7. After decoding the weighted fused intermediate vector through the last decoder block, the full embedding vector is obtained and the full embedding vector is output to the classifier.

[0196] B9. If the length of the full name embedding vector sequence does not reach the preset length and the end symbol is not detected, the last decoder outputs the context-related attention vector to the first decoding fusion block, so that the first decoding fusion block performs weighted fusion on the graph embedding vector of the target abbreviation subword, the output vector of the Nth encoder block, and the context-related attention vector based on the weight parameter in the next iteration, and inputs the weighted fused intermediate vector to the first decoder block.

[0197] S1105. Input the full name embedding vector sequence of the target abbreviation information into the classifier one by one, and obtain the full name information of the target abbreviation information output by the classifier.

[0198] In this embodiment, the full name embedding vector sequence of the target abbreviation information includes the full name embedding vector output by the last decoder block. The Token decoder in the classifier obtains the Token encoding of each full name embedding vector in the full name embedding vector sequence to obtain the target full name Token encoding sequence.

[0199] Specifically, the method for the Token decoder to obtain the Token decoding of the target full name embedding vector (any full name embedding vector in the full name embedding vector sequence) includes:

[0200] Find the index with the maximum probability in the dictionary embedding dimension of the full name embedding vector, and use the index as the Token code of this full name embedding vector.

[0201] It should be noted that the specific method of decoding the embedded vector to obtain the Token code can be referred to the prior art.

[0202] In this embodiment, the word generator in the classifier searches for the corresponding subword in the subword dictionary based on the full name Token code, and generates the full name subword corresponding to the Token code of each full name embedding vector.

[0203] It should be noted that in an optional embodiment, the Bahdanau attention mechanism can be used to replace the Transformer attention mechanism. In the full name translation model based on the Bahdanau attention layer, at each time step, the decoder calculates an alignment distribution based on the previous output and all hidden states of the encoder, which indicates which positions in the encoder are most useful for the output of the current time step. The decoder then uses the weighted encoder hidden state to generate the output of the current time step. Thus, the dynamic alignment mechanism of the Bahdanau attention mechanism enables the model to better capture the complex dependencies between the source language and the target language, thereby improving the accuracy and fluency of the full name translation model.

[0204] In an optional embodiment, the loss value is calculated according to the actual length of the input vector to improve the accuracy of the loss value, improve the training effect, and thus improve the accuracy of the full name translation model.

[0205] In an optional embodiment, the full name translation model is tested using a test set, and the full name translation model is evaluated using indicators such as Rouge and BLUE. If the preset test conditions are not met, the full name translation model is retrained to improve the accuracy of the full name translation model.

[0206] In an optional embodiment, the method for identifying the full name information provided in the embodiment of the present application can be applied to solve the problem of low accuracy in restoring the abbreviation information to the full name information in the institutional information from WOS during the construction of institutional specifications. The present application uses a graph neural network to model and train the abbreviation subwords provided by WOS to map the full name subwords, obtain the graph embedding vectors of all abbreviation subwords and the trained graph neural network model, input the graph embedding vectors and Token vectors of the abbreviation information to be identified into the pre-trained full name translation model, and realize the recognition of the full name information through the encoder and decoder. Thus, the problem of restoring the institutional abbreviation information contained in the paper data in the WOS database to the full name information is converted into a sequence-to-sequence translation problem, and the institutional information provided in the WOS database is used to learn the embedding of each subword using the contrastive learning method in deep learning, and the learned graph embedding vector of each abbreviation subword and the Token encoding of each abbreviation subword are fused using a learnable dynamic fusion method to obtain the full name information. Furthermore, the Bahdanau attention mechanism is used to dynamically capture the attention within the input sequence and the target sequence, and the loss value is calculated according to the actual length of the input vector to improve the accuracy of the model in identifying the full name information.

[0207] A method for generating full name information provided in an embodiment of the present application is introduced above, and a device for executing the above method for generating full name information will be introduced below.

[0208] See also Fig.12 , Fig.12 This is a schematic diagram of the structure of a device for generating full name information provided in an embodiment of the present application. Fig.12 As shown, the device 1200 for generating the full name information includes:

[0209] The abbreviation information acquisition unit 1201 is used to acquire target abbreviation information, wherein the target abbreviation information is a subword sequence including a plurality of target abbreviation subwords;

[0210] The information representation unit 1202 is used to obtain a graph embedding vector sequence and abbreviated Token encoding sequence of the target abbreviated information, wherein the graph embedding vector sequence includes graph embedding vectors of each abbreviated subword arranged in sequence, and the abbreviated Token encoding sequence includes Token encodings of each abbreviated subword arranged in sequence;

[0211] A sequence generation unit 1203 is used to input the graph embedding vector sequence and the abbreviation Token encoding sequence of the target abbreviation information into a pre-trained sequence generation model to obtain a full name embedding vector sequence output by the sequence generation model; the full name embedding vector sequence includes full name embedding vectors of multiple target full name subwords arranged in sequence;

[0212] The sequence generation model includes an encoder and a decoder constructed based on a neural network model of a self-attention mechanism, the encoder includes N encoder blocks connected in sequence, the decoder includes N decoder blocks connected in sequence, the input of the first encoder block is a fusion vector that performs weighted fusion of a graph embedding vector and a Token code based on a weight parameter, and the input of the nth encoder block is a fusion vector that performs weighted fusion of a graph embedding vector based on a weight parameter and an output vector of a previous encoder block; the input of the first decoder block is a fusion vector that performs weighted fusion of a graph embedding vector and an output vector of an Nth encoder block based on a weight parameter, and the input of the nth decoder block is a fusion vector that performs weighted fusion of a graph embedding vector and an output vector of a previous decoder block based on a weight parameter, and the output is the full embedding vector sequence; wherein, 1<n≤N; the weight parameters corresponding to each encoder block and decoder block are obtained through training based on training data;

[0213] The full name matching unit 1204 is used to generate the full name information of the target abbreviation information based on the full name embedding vector sequence, and the full name information includes a plurality of the target full name subwords.

[0214] In a possible implementation, the information representation unit is used to obtain a graph embedding vector sequence of the target abbreviation information, specifically for:

[0215] According to the graph embedding vector dictionary generated by training the graph neural network model, the graph embedding vector of each target abbreviation subword is obtained, and the graph embedding vector dictionary includes the graph embedding vector of each preset abbreviation subword; the graph neural network model is composed of multiple graph neural modules, and the graph neural module includes a graph attention network and a graph convolution network. The graph neural network model is obtained based on the relationship mapping graph training, and the relationship mapping graph has abbreviation subwords and full name subwords as nodes, and the mapping relationship from abbreviation subwords to full name subwords as directed edges;

[0216] The graph embedding vectors of the target abbreviation subwords are arranged in sequence to obtain a graph embedding vector sequence of the target abbreviation information.

[0217] In a possible implementation, the device for generating the full name information further includes a training model of a graph neural network model, which is used to:

[0218] Based on the preset sub-word mapping relationship, construct the relationship mapping diagram, where the sub-word mapping relationship includes the mapping relationship between the abbreviation sub-word and the full name sub-word;

[0219] Based on the relationship map, two nodes with edges and an edge connecting the two nodes are selected from the relationship map as a positive sample; two nodes without edges are selected from the relationship map, and edges are constructed for the two nodes as a negative sample;

[0220] Generate graph training data, wherein the graph training data includes multiple positive samples and multiple negative samples;

[0221] The graph neural network model is trained based on the graph training data until a preset training completion condition is reached to obtain the graph embedding vector dictionary.

[0222] In a possible implementation, when the information representation unit is used to obtain the abbreviated Token encoding sequence, it is specifically used to:

[0223] Input each of the target abbreviation subwords into a pre-trained Token encoding model to obtain a Token encoding of each of the target abbreviation subwords output by the Token encoding model, wherein the Token encoding model includes a causal language model;

[0224] Generate the Token code of each of the target abbreviation subwords in sequence to obtain the abbreviation Token code sequence of the target abbreviation information.

[0225] In one possible implementation, the encoder includes a Transformer encoder, and the decoder includes a Transformer decoder;

[0226] Each encoder block in the Transformer encoder includes a multi-head attention layer, a vector addition and normalization layer, a position element-by-position feedforward layer, and a vector addition and normalization layer connected in sequence, and each decoder block in the Transformer decoder includes a multi-head attention layer, a vector addition and normalization layer, a multi-head attention layer, a position element-by-position feedforward layer, and a vector addition and normalization layer connected in sequence;

[0227] The encoder further includes a first position encoding block arranged before the first encoder block and an encoding fusion block arranged before each encoder block, and the decoder further includes a decoding fusion block arranged before each decoder block.

[0228] In a possible implementation, the device for generating the full name information further includes a training unit of a sequence generation model, which is used to:

[0229] Constructing a parallel corpus, the parallel corpus including sample abbreviation information and corresponding sample full name information, the sample abbreviation information including multiple sample abbreviation subwords, and the sample full name information including multiple sample full name subwords;

[0230] Acquire sequence training data, wherein the sequence training data includes a graph embedding vector sequence and a short Token encoding sequence of each piece of the sample abbreviation information, and a full name Token encoding sequence of the sample full name information, wherein the full name Token encoding sequence of the sample full name information includes a Token encoding of each full name subword in the sample full name information;

[0231] The graph embedding vector sequence of the sample abbreviation information, the abbreviation Token encoding sequence, and the full name Token encoding sequence are input into the sequence generation model, and multiple iterations are performed, each of which includes:

[0232] Performing position encoding on the Token encoding of the sample abbreviation subword by the first position encoding block to obtain a position encoding vector of the sample abbreviation subword;

[0233] Performing weighted fusion on the position encoding vector of the sample abbreviation subword and the graph embedding vector of the sample abbreviation subword based on a weight parameter through a first encoding fusion block, and inputting the weighted fused intermediate vector into a first encoder block;

[0234] Performing weighted fusion on the output vector of the n-1th encoder block and the graph embedding vector of the sample abbreviation subword based on the weight parameter through each nth encoder fusion block, and inputting the weighted fused intermediate vector into the nth encoder block;

[0235] The weighted fused intermediate vectors are encoded and outputted through each encoder block;

[0236] Position encoding is performed on the Token encoding of the full name subword of the sample by a second position encoding block to obtain an embedding vector of the full name of the sample;

[0237] Performing weighted fusion on the output vector of the Nth encoder block, the graph embedding vector of the sample abbreviation subword, and the sample full name embedding vector based on the weight parameter through the first decoding fusion block, and inputting the weighted fused intermediate vector into the first decoder block;

[0238] Performing weighted fusion on the output vector of the n-1th decoder block and the graph embedding vector of the sample abbreviation subword based on the weight parameter through each nth decoding fusion block, and inputting the weighted fused intermediate vector into the nth encoder block;

[0239] The weighted fused intermediate vectors are encoded and outputted through each encoder block;

[0240] Performing mask loss calculation on the output of the Nth decoder block and the sample full name embedding vector according to the vector length through the loss calculation block, and updating the model parameters of the sequence generation model based on the mask loss calculation result, wherein the model parameters include weight parameters corresponding to each of the decoding fusion blocks;

[0241] Determine whether the preset training completion conditions are met. If so, obtain the trained sequence generation model. If not, execute the next iteration.

[0242] In a possible implementation, when the full name matching unit is used to generate the full name information of the target abbreviation information based on the full name embedding vector sequence, it is specifically used to:

[0243] Obtaining the Token code of each full name embedding vector in the full name embedding vector sequence to obtain a target full name Token code sequence, wherein the target full name code sequence includes the Token codes of each full name embedding vector arranged in sequence,

[0244] Based on a pre-configured word dictionary, searching for a full name subword that matches the Token encoding of each of the full name embedding vectors as the target full name subword, the word dictionary including preset full name subwords and matching Token encodings;

[0245] The full name information of the target abbreviation information is obtained by sequentially combining the target full name subwords.

[0246] The present application also provides an electronic device in an embodiment. Fig.13 As shown, it shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present application. The electronic device in the embodiment of the present application may include but is not limited to fixed terminals such as mobile phones, laptops, PDAs (personal digital assistants), PADs (tablet computers), desktop computers, etc. Fig.13 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0247] like Fig.13 As shown, the electronic device may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1302 or a program loaded from a storage device 1308 to a random access memory (RAM) 1303. When the electronic device is powered on, various programs and data required for the operation of the electronic device are also stored in the RAM 1303. The processing device 1301, the ROM 1302, and the RAM 1303 are connected to each other via a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0248] Typically, the following devices may be connected to the I / O interface 1305: an input device 1306 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1308 including, for example, a memory card, a hard disk, etc.; and a communication device 1309. The communication device 1309 may allow the electronic device to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 An electronic device having various devices is shown, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0249] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any method for generating full name information provided in the embodiment of the present application.

[0250] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any method for generating full name information provided in an embodiment of the present application.

[0251] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0252] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0253] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0254] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A method for generating full name information, characterized in that: include: Acquire target abbreviation information, wherein the target abbreviation information is a subword sequence including a plurality of target abbreviation subwords; Obtaining a graph embedding vector sequence and abbreviated Token encoding sequence of the target abbreviated information, wherein the graph embedding vector sequence includes graph embedding vectors of each abbreviated subword arranged in sequence, and the abbreviated Token encoding sequence includes Token encodings of each abbreviated subword arranged in sequence; Inputting the graph embedding vector sequence and the abbreviation Token encoding sequence of the target abbreviation information into a pre-trained sequence generation model to obtain a full name embedding vector sequence output by the sequence generation model; the full name embedding vector sequence includes full name embedding vectors of multiple target full name subwords arranged in sequence; The sequence generation model includes an encoder and a decoder constructed based on a neural network model of a self-attention mechanism, the encoder includes N encoder blocks connected in sequence, the decoder includes N decoder blocks connected in sequence, the input of the first encoder block is a fusion vector that performs weighted fusion of the graph embedding vector and the Token code based on a weight parameter, and the input of the nth encoder block is a fusion vector that performs weighted fusion of the graph embedding vector and the output vector of the previous encoder block based on a weight parameter; the input of the first decoder block is a fusion vector that performs weighted fusion of the graph embedding vector and the output vector of the Nth encoder block based on a weight parameter, and the input of the nth decoder block is a fusion vector that performs weighted fusion of the graph embedding vector and the output vector of the previous decoder block based on a weight parameter, and the output is the full embedding vector sequence; wherein, 1<n≤N; the weight parameters corresponding to each encoder block and decoder block are obtained through training based on training data; Based on the full name embedding vector sequence, the full name information of the target abbreviation information is generated, and the full name information includes a plurality of the target full name subwords.

2. The method for generating full name information according to claim 1, characterized in that: Obtaining a graph embedding vector sequence of the target abbreviation information includes: According to the graph embedding vector dictionary generated by training the graph neural network model, the graph embedding vector of each target abbreviation subword is obtained, and the graph embedding vector dictionary includes the graph embedding vector of each preset abbreviation subword; the graph neural network model is composed of multiple graph neural modules, and the graph neural module includes a graph attention network and a graph convolution network. The graph neural network model is obtained based on the relationship mapping graph training, and the relationship mapping graph has abbreviation subwords and full name subwords as nodes, and the mapping relationship from abbreviation subwords to full name subwords as directed edges; The graph embedding vectors of the target abbreviation subwords are arranged in sequence to obtain a graph embedding vector sequence of the target abbreviation information.

3. The method for generating full name information according to claim 1, characterized in that: The training method of the graph neural network model includes: Based on the preset sub-word mapping relationship, construct the relationship mapping diagram, where the sub-word mapping relationship includes the mapping relationship between the abbreviation sub-word and the full name sub-word; Based on the relationship map, two nodes with edges and an edge connecting the two nodes are selected from the relationship map as a positive sample; two nodes without edges are selected from the relationship map, and edges are constructed for the two nodes as a negative sample; Generate graph training data, wherein the graph training data includes multiple positive samples and multiple negative samples; The graph neural network model is trained based on the graph training data until a preset training completion condition is reached to obtain the graph embedding vector dictionary.

4. The method for generating full name information according to claim 1, characterized in that: Obtaining the Token encoding sequence includes: Input each of the target abbreviation subwords into a pre-trained Token encoding model to obtain a Token encoding of each of the target abbreviation subwords output by the Token encoding model, wherein the Token encoding model includes a causal language model; Generate the Token code of each of the target abbreviation subwords in sequence to obtain the abbreviation Token code sequence of the target abbreviation information.

5. The method for generating full name information according to claim 1, characterized in that: The encoder includes a Transformer encoder, and the decoder includes a Transformer decoder; Each encoder block in the Transformer encoder includes a multi-head attention layer, a vector addition and normalization layer, a position element-by-position feedforward layer, and a vector addition and normalization layer connected in sequence, and each decoder block in the Transformer decoder includes a multi-head attention layer, a vector addition and normalization layer, a multi-head attention layer, a position element-by-position feedforward layer, and a vector addition and normalization layer connected in sequence; The encoder further includes a first position encoding block arranged before the first encoder block and an encoding fusion block arranged before each encoder block, and the decoder further includes a decoding fusion block arranged before each decoder block.

6. The method for generating full name information according to claim 5, characterized in that: The training method of the sequence generation model includes: Constructing a parallel corpus, the parallel corpus including sample abbreviation information and corresponding sample full name information, the sample abbreviation information including multiple sample abbreviation subwords, and the sample full name information including multiple sample full name subwords; Acquire sequence training data, wherein the sequence training data includes a graph embedding vector sequence and a short Token encoding sequence of each piece of the sample abbreviation information, and a full name Token encoding sequence of the sample full name information, wherein the full name Token encoding sequence of the sample full name information includes a Token encoding of each full name subword in the sample full name information; The graph embedding vector sequence of the sample abbreviation information, the abbreviation Token encoding sequence, and the full name Token encoding sequence are input into the sequence generation model, and multiple iterations are performed, each of which includes: Performing position encoding on the Token encoding of the sample abbreviation subword by the first position encoding block to obtain a position encoding vector of the sample abbreviation subword; Performing weighted fusion on the position encoding vector of the sample abbreviation subword and the graph embedding vector of the sample abbreviation subword based on a weight parameter through a first encoding fusion block, and inputting the weighted fused intermediate vector into a first encoder block; Performing weighted fusion on the output vector of the n-1th encoder block and the graph embedding vector of the sample abbreviation subword based on the weight parameter through each nth encoder fusion block, and inputting the weighted fused intermediate vector into the nth encoder block; The weighted fused intermediate vectors are encoded and outputted through each encoder block; Position encoding is performed on the Token encoding of the full name subword of the sample by a second position encoding block to obtain an embedding vector of the full name of the sample; Performing weighted fusion on the output vector of the Nth encoder block, the graph embedding vector of the sample abbreviation subword, and the sample full name embedding vector based on the weight parameter through the first decoding fusion block, and inputting the weighted fused intermediate vector into the first decoder block; Performing weighted fusion on the output vector of the n-1th decoder block and the graph embedding vector of the sample abbreviation subword based on the weight parameter through each nth decoding fusion block, and inputting the weighted fused intermediate vector into the nth encoder block; The weighted fused intermediate vectors are encoded and outputted through each encoder block; Performing mask loss calculation on the output of the Nth decoder block and the sample full name embedding vector according to the vector length through the loss calculation block, and updating the model parameters of the sequence generation model based on the mask loss calculation result, wherein the model parameters include weight parameters corresponding to each of the decoding fusion blocks; Determine whether the preset training completion conditions are met. If so, obtain the trained sequence generation model. If not, execute the next iteration.

7. The method for generating full name information according to claim 1, characterized in that: Based on the full name embedding vector sequence, generating the full name information of the target abbreviation information includes: Obtaining the Token code of each full name embedding vector in the full name embedding vector sequence to obtain a target full name Token code sequence, wherein the target full name code sequence includes the Token codes of each full name embedding vector arranged in sequence, Based on a pre-configured word dictionary, searching for a full name subword that matches the Token encoding of each of the full name embedding vectors as the target full name subword, the word dictionary including preset full name subwords and matching Token encodings; The full name information of the target abbreviation information is obtained by sequentially combining the target full name subwords.

8. A device for generating full name information, characterized in that: include: an abbreviation information acquisition unit, used to acquire target abbreviation information, wherein the target abbreviation information is a subword sequence including a plurality of target abbreviation subwords; An information representation unit, used to obtain a graph embedding vector sequence and abbreviation Token encoding sequence of the target abbreviation information, wherein the graph embedding vector sequence includes graph embedding vectors of each abbreviation subword arranged in sequence, and the abbreviation Token encoding sequence includes Token encodings of each abbreviation subword arranged in sequence; A sequence generation unit, used to input the graph embedding vector sequence and the abbreviation Token encoding sequence of the target abbreviation information into a pre-trained sequence generation model to obtain a full name embedding vector sequence output by the sequence generation model; the full name embedding vector sequence includes full name embedding vectors of multiple target full name subwords arranged in sequence; The sequence generation model includes an encoder and a decoder constructed based on a neural network model of a self-attention mechanism, the encoder includes N encoder blocks connected in sequence, the decoder includes N decoder blocks connected in sequence, the input of the first encoder block is a fusion vector that performs weighted fusion of the graph embedding vector and the Token code based on a weight parameter, and the input of the nth encoder block is a fusion vector that performs weighted fusion of the graph embedding vector and the output vector of the previous encoder block based on a weight parameter; the input of the first decoder block is a fusion vector that performs weighted fusion of the graph embedding vector and the output vector of the Nth encoder block based on a weight parameter, and the input of the nth decoder block is a fusion vector that performs weighted fusion of the graph embedding vector and the output vector of the previous decoder block based on a weight parameter, and the output is the full embedding vector sequence; wherein, 1<n≤N; the weight parameters corresponding to each encoder block and decoder block are obtained through training based on training data; A full name matching unit is used to generate full name information of the target abbreviation information based on the full name embedding vector sequence, wherein the full name information includes a plurality of target full name subwords.

9. A computer program product, characterized in that It includes computer-readable instructions, and when the computer-readable instructions are executed on an electronic device, the electronic device implements the method for generating full name information as described in any one of claims 1 to 7.

10. An electronic device, characterized in that: The method comprises at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the method for generating full name information as described in any one of claims 1 to 7.

Images