A method for generating a full name information and related apparatus

Abstract

The application discloses a full name information generation method and related device, and relates to the technical field of natural language processing. The application utilizes a pre-trained sequence generation model, encodes an intermediate vector after fusing a graph embedding vector based on corresponding weight parameters layer by layer through a plurality of encoder blocks in an encoder constructed based on a self-attention mechanism, decodes the intermediate vector after fusing the graph embedding vector based on the weight parameters layer by layer through a plurality of decoder blocks in a decoder constructed based on the self-attention mechanism, obtains a full name embedding vector sequence composed of full name embedding vectors of each target abbreviation subword, and generates full name information of target abbreviation information according to the full name embedding vector sequence. The application realizes the purpose of improving the accuracy of the full name information of the target abbreviation information through the sequence generation model and based on graph embedding reinforcement learning.

Description

Technical Field

[0001] This application relates to the field of natural language processing technology, and in particular to a method and related apparatus for generating full name information. Background Technology

[0002] Based on the principles of ease of identification and prevention of ambiguity, the names of institutions recorded in institutional normative documents should use the standard full name. The full name information of an institution is obtained by arranging the full name sub-words according to the prescribed full name syntax. However, some databases often use abbreviation information to identify institutions, and the abbreviation information of an institution is obtained by arranging the abbreviation sub-words according to the prescribed abbreviation syntax. Therefore, in the process of constructing normative documents, it is necessary to generate the full name information based on the abbreviation information of the institution.

[0003] Currently, the main method for converting abbreviations to full names of organizations is the sub-word mapping method. This involves mapping each abbreviation sub-word to its corresponding full name sub-word, and then combining these full name sub-words to obtain the full name. However, due to the incompleteness and non-uniqueness of the mapping relationship, this method is prone to errors in full name recognition. Summary of the Invention

[0004] In view of the above problems, this application provides a method and related apparatus for generating full name information, so as to improve the accuracy of full name information generation. The specific solution is as follows:

[0005] The first aspect of this application provides a method for generating full name information, including:

[0006] Obtain target abbreviation information, wherein the target abbreviation information is a sub-word sequence including multiple target abbreviation sub-words;

[0007] Obtain the graph embedding vector sequence and the abbreviation token encoding sequence of the target abbreviation information. The graph embedding vector sequence includes the graph embedding vectors of each abbreviation sub-word arranged in sequence, and the abbreviation token encoding sequence includes the token encoding of each abbreviation sub-word arranged in sequence.

[0008] The graph embedding vector sequence of the target abbreviation information and the abbreviation token encoding sequence are input into a pre-trained sequence generation model to obtain the full name embedding vector sequence output by the sequence generation model; the full name embedding vector sequence includes the full name embedding vectors of multiple target full name sub-words arranged in sequence;

[0009] The sequence generation model includes an encoder and a decoder constructed based on a neural network model of a self-attention mechanism, the encoder includes N sequentially connected encoder blocks, and the decoder includes N sequentially connected decoder blocks, an input of a first encoder block is a fusion vector obtained by weighting and fusing a graph embedding vector and a Token encoding based on weight parameters, an input of an n-th encoder block is a fusion vector obtained by weighting and fusing a graph embedding vector and an output vector of a previous encoder block based on weight parameters; an input of a first decoder block is a fusion vector obtained by weighting and fusing a graph embedding vector and an output vector of an Nth encoder block based on weight parameters, an input of an n-th decoder block is a fusion vector obtained by weighting and fusing a graph embedding vector and an output vector of a previous decoder block based on weight parameters, and an output is the sequence of the full name embedding vectors; wherein 1 < n ≤ N; the weight parameters corresponding to each encoder block and decoder block are obtained based on training data;

[0010] Based on the sequence of the full name embedding vectors, a full name information of the target abbreviation information is generated, and the full name information includes a plurality of target full name subwords.

[0011] In a possible implementation, the graph embedding vector sequence of the target abbreviation information is obtained, including:

[0012] According to a graph embedding vector dictionary generated by a training graph neural network model, a graph embedding vector of each target abbreviation subword is obtained, and the graph embedding vector dictionary includes a graph embedding vector of each preset abbreviation subword; the graph neural network model is composed of a plurality of graph neural modules, the graph neural module includes a graph attention network and a graph convolution network, and the graph neural network model is obtained based on a relationship mapping graph, the relationship mapping graph takes an abbreviation subword and a full name subword as a node, and takes a mapping relationship from an abbreviation subword to a full name subword as a directed edge;

[0013] The graph embedding vectors of each target abbreviation subword are sequentially arranged to obtain the graph embedding vector sequence of the target abbreviation information.

[0014] In a possible implementation, a training method of a graph neural network model includes:

[0015] Based on a preset subword mapping relationship, the relationship mapping graph is constructed, and the subword mapping relationship includes a mapping relationship between an abbreviation subword and a full name subword;

[0016] Based on the relationship mapping graph, two nodes with an edge and the edge connecting the two nodes are selected from the relationship mapping graph as a positive sample, and two nodes without an edge are selected from the relationship mapping graph, and an edge is constructed for the two nodes as a negative sample;

[0017] Graph training data is generated, and the graph training data includes a plurality of positive samples and a plurality of negative samples;

[0018] training the graph neural network model based on the graph training data until a preset training completion condition is reached, to obtain the graph embedding vector dictionary.

[0019] In a possible implementation, obtaining the Token encoding sequence includes:

[0020] inputting each target abbreviation subword into a pre-trained Token encoding model to obtain Token encoding of each target abbreviation subword output by the Token encoding model, the Token encoding model including a causal language model;

[0021] generating Token encoding of each target abbreviation subword in sequence to obtain the abbreviation Token encoding sequence of the target abbreviation information.

[0022] In a possible implementation, the encoder includes a Transformer encoder, and the decoder includes a Transformer decoder;

[0023] each encoder block in the Transformer encoder includes sequentially connected multi-head attention layers, vector addition and normalization layers, position element-wise feedforward layers, and vector addition and normalization layers, and each decoder block in the Transformer decoder includes sequentially connected multi-head attention layers, vector addition and normalization layers, multi-head attention layers, position element-wise feedforward layers, and vector addition and normalization layers;

[0024] the encoder further includes a first position encoding block configured before the first encoder block and an encoding fusion block configured before each encoder block, and the decoder further includes a decoding fusion block configured before each decoder block.

[0025] In a possible implementation, the training method of the sequence generation model includes:

[0026] constructing a parallel corpus, the parallel corpus including sample abbreviation information and corresponding sample full name information, the sample abbreviation information including a plurality of sample abbreviation subwords, and the sample full name information including a plurality of sample full name subwords;

[0027] obtaining sequence training data, the sequence training data including graph embedding vector sequences and abbreviation Token encoding sequences of each piece of the sample abbreviation information, and full name Token encoding sequences of the sample full name information, the full name Token encoding sequences of the sample full name information including Token encoding of each full name subword in the sample full name information;

[0028] inputting the graph embedding vector sequence of the sample abbreviation information, the token encoding sequence, and the full name token encoding sequence into a sequence generation model, performing multiple iterations, each iteration including:

[0029] performing position encoding on the token encoding of the sample abbreviation subword by the first position encoding block to obtain a position encoding vector of the sample abbreviation subword;

[0030] performing weighted fusion on the position encoding vector of the sample abbreviation subword and the graph embedding vector of the sample abbreviation subword based on a weight parameter by the first encoding fusion block, and inputting the intermediate vector after weighted fusion into the first encoder block;

[0031] performing weighted fusion on the output vector of the (n-1)th encoder block and the graph embedding vector of the sample abbreviation subword based on a weight parameter by each nth encoding fusion block, and inputting the intermediate vector after weighted fusion into the nth encoder block;

[0032] outputting the intermediate vector after weighted fusion by each encoder block after encoding;

[0033] performing position encoding on the token encoding of the sample full name subword by the second position encoding block to obtain a sample full name embedding vector;

[0034] performing weighted fusion on the output vector of the Nth encoder block, the graph embedding vector of the sample abbreviation subword, and the sample full name embedding vector based on a weight parameter by the first decoding fusion block, and inputting the intermediate vector after weighted fusion into the first decoder block;

[0035] performing weighted fusion on the output vector of the (n-1)th decoder block and the graph embedding vector of the sample abbreviation subword based on a weight parameter by each nth decoding fusion block, and inputting the intermediate vector after weighted fusion into the nth encoder block;

[0036] outputting the intermediate vector after weighted fusion by each encoder block after encoding;

[0037] performing mask loss calculation on the output of the Nth decoder block and the sample full name embedding vector according to the vector length by the loss calculation block, updating the model parameters of the sequence generation model based on the mask loss calculation result, the model parameters including the weight parameters corresponding to each decoding fusion block;

[0038] determining whether a preset training completion condition is reached, if yes, obtaining the trained sequence generation model, and if no, performing the next iteration.

[0039] In a possible implementation, based on the full name embedding vector sequence, the full name information of the target abbreviation information is generated, including:

[0040] obtaining Token encodings of each full name embedding vector in the full name embedding vector sequence to obtain a target full name Token encoding sequence, the target full name encoding sequence comprising Token encodings of each full name embedding vector arranged in sequence,

[0041] based on a pre-configured word dictionary, finding full name subwords matching the Token encodings of each full name embedding vector as the target full name subwords, the word dictionary comprising preset full name subwords and matching Token encodings;

[0042] sequentially combining each target full name subword to obtain full name information of the target short name information.

[0043] The second aspect of the application provides a full name information generation device, comprising:

[0044] a short name information acquisition unit configured to acquire target short name information, the target short name information being a subword sequence comprising a plurality of target short name subwords;

[0045] an information representation unit configured to acquire a graph embedding vector sequence and a short name Token encoding sequence of the target short name information, the graph embedding vector sequence comprising graph embedding vectors of each short name subword arranged in sequence, and the short name Token encoding sequence comprising Token encodings of each short name subword arranged in sequence;

[0046] a sequence generation unit configured to input the graph embedding vector sequence and the short name Token encoding sequence of the target short name information into a pre-trained sequence generation model to obtain a full name embedding vector sequence output by the sequence generation model, the full name embedding vector sequence comprising full name embedding vectors of a plurality of target full name subwords arranged in sequence;

[0047] The sequence generation model comprises an encoder and a decoder constructed based on a self-attention mechanism neural network model, the encoder comprises N encoder blocks connected in sequence, the decoder comprises N decoder blocks connected in sequence, the input of a first encoder block is a fusion vector obtained by weighting and fusing a graph embedding vector and a Token encoding based on weight parameters, the input of an nth encoder block is a fusion vector obtained by weighting and fusing a graph embedding vector and an output vector of a previous encoder block based on weight parameters; the input of a first decoder block is a fusion vector obtained by weighting and fusing a graph embedding vector and an output vector of an Nth encoder block based on weight parameters, the input of an nth decoder block is a fusion vector obtained by weighting and fusing a graph embedding vector and an output vector of a previous decoder block based on weight parameters, and the output is the full name embedding vector sequence; wherein 1

[0048] The full name matching unit is configured to generate full name information of the target abbreviation information based on the full name embedding vector sequence, wherein the full name information comprises a plurality of target full name subwords.

[0049] The third aspect of the present application provides a computer program product, comprising computer readable instructions, when the computer readable instructions run on an electronic device, make the electronic device realize the full name information generation method of the first aspect or any implementation manner of the first aspect.

[0050] The fourth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected with the processor, wherein:

[0051] The memory is configured to store a computer program;

[0052] The processor is configured to execute the computer program, so that the electronic device can realize the full name information generation method of the first aspect or any implementation manner of the first aspect.

[0053] By the above technical solution, the full name information generation method and related device provided by the present application input the graph embedding vector sequence and the abbreviation Token coding sequence of the target abbreviation information into the pre-trained sequence generation model, and through the multi-layer encoder block in the encoder constructed based on the self-attention mechanism in the sequence generation model, the intermediate vector after the graph embedding vector is fused based on the weight parameter corresponding to each layer is encoded layer by layer, and through the multi-layer decoder block in the decoder constructed based on the self-attention mechanism in the sequence generation model, the intermediate vector after the graph embedding vector is fused based on the weight parameter corresponding to each layer is decoded layer by layer, to obtain the full name embedding vector sequence composed of the full name embedding vector of each target abbreviation subword corresponding to the target full name subword. Further, the full name information of the target abbreviation information is generated through the full name embedding vector sequence. In this scheme, the weight parameters of each layer are obtained by dynamic training, so that the graph embedding vector obtained by graph embedding learning is fused into each layer of the encoder block and the decoder block constructed based on the self-attention mechanism with different weights. The relationship between each target abbreviation subword is captured through the self-attention mechanism, and the relationship between each target abbreviation subword is strengthened through graph embedding learning, which improves the expression accuracy of the target abbreviation subword, thereby improving the generation accuracy of the sequence generation model on the full name embedding vector sequence, and further improving the accuracy of the full name information of the target abbreviation information. BRIEF DESCRIPTION OF DRAWINGS

[0054] The above and other features, advantages, and aspects of the present disclosure will become more apparent by describing in detail the following specific embodiments with reference to the attached drawings. Throughout the drawings, the same or similar reference numerals refer to the same or similar elements. It should be understood that the drawings are schematic, and the original and elements are not necessarily drawn according to the scale.

[0055] Figure 1 A flowchart of a full name information generation method provided by an embodiment of the present application is shown in FIG. 1.

[0056] Figure 2 A structural diagram of a sequence generation model provided by an embodiment of the present application is shown in FIG. 2.

[0057] Figure 3 A structural diagram of a full name translation model provided by an embodiment of the present application is shown in FIG. 3.

[0058] Figure 4 A specific structural diagram of a graph neural network model provided by an embodiment of the present application is shown in FIG. 4.

[0059] Figure 5 A specific structural diagram of a Transformer encoder block provided by an embodiment of the present application is shown in FIG. 5.

[0060] Figure 6 A specific structural diagram of a Transformer decoder block provided by an embodiment of the present application is shown in FIG. 6.

[0061] Figure 7 A specific implementation flowchart of a classifier of a full name translation model provided by an embodiment of the present application is shown in FIG. 7.

[0062] Figure 8 A specific implementation flowchart of a construction method of a full name translation model provided by an embodiment of the present application is shown in FIG. 8.

[0063] Figure 9 A subgraph diagram of a mapping relationship graph provided by an embodiment of the present application is shown in FIG. 9.

[0064] Figure 10 A structural diagram of a full name conversion model provided by an embodiment of the present application is shown in FIG. 10.

[0065] Figure 11 A specific implementation flow of a full name information generation method provided by an embodiment of the present application is shown in FIG. 11.

[0066] Figure 12 A structural diagram of a full name information generation apparatus provided by an embodiment of the present application is shown in FIG. 12.

[0067] Figure 13 A structural diagram of an electronic device provided by an embodiment of the present application is shown in FIG. 13. DETAILED DESCRIPTION

[0068] The embodiments of the present application are described below with reference to the accompanying drawings. The terms used in the embodiment part of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0069] The embodiments of the present application will be described below in conjunction with the drawings. Those skilled in the art can know that, with the development of technology and the emergence of new scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0070] The terms "first", "second", etc. in the specification and claims of the present application and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the terms used in this way can be interchanged under appropriate circumstances, and this is only a way of distinguishing the objects with the same attributes in the description of the embodiments of the present application. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, so that the processes, methods, systems, products or equipment containing a series of units do not have to be limited to those units, but can include other units not clearly listed or inherent to these processes, methods, products or equipment.

[0071] The present application can be applied to the field of natural language processing, and can be applied to the scenario of restoring the abbreviation information of an organization to the full name information. A specific application scenario is that, in the construction process of an organization specification document, the abbreviation information of an organization is obtained from various data sources, and the full name information is generated from the abbreviation information. The data sources of the abbreviation information include bibliographic data in a target database, wherein the target database includes the Web of Science (WOS) information service platform and other open source databases.

[0072] Taking the target database WOS as an example, the bibliographic data provided by WOS displays the abbreviation information of a large number of organizations in the form of abbreviations. The information of an organization mainly includes the name of the organization and / or the address of the organization, which is used to uniquely indicate the organization. After obtaining the abbreviation information of an organization, information restoration is needed to obtain the complete and standard full name information of the organization. It should be noted that the present application only takes the example of identifying the abbreviation information of the bibliographic data of WOS to obtain the full name information, and the present application can be applied to the restoration of the abbreviation information obtained from any other data source. In addition, the extraction of each item of information (including but not limited to the extraction of each item of information from each database) mentioned in the present application is authorized and is non-sensitive data.

[0073] In order to realize the conversion of the abbreviation information to the full name information, the following generation technologies currently exist:

[0074] The first kind: parallel database alignment. For example, the full text of a document is obtained from other databases according to the unique identifier of the document, so as to obtain the full name information of the organization. Obviously, due to the uniqueness of the document data of each database, a considerable number of documents cannot be aligned by parallel databases, that is, the full name information of the organization cannot be identified by the parallel database alignment method.

[0075] Second: according to the pre-configured subword mapping relationship. WOS provides the mapping relationship from the full name subword to the abbreviation subword of the organization, address and country information on the official website, which can be used to identify the abbreviation information of the organization into the full name information.

[0076] However, due to the incompleteness and non-uniqueness of the mapping relationship, it is easy to cause the generation of full name information error, and the success rate is low. The specific analysis is as follows:

[0077] (1) The number of organizations is very large, and the mapping relationship cannot cover the mapping of the abbreviation subword and the full name subword of all organizations, and there is a risk of subword generation failure.

[0078] (2) The mapping relationship is not one-to-one, but there are multiple full name subwords mapping to one abbreviation subword, or one full name subword mapping to multiple abbreviation subwords. For example, the mapping results of one abbreviation subword in the same data source include multiple full name subwords in the same language or different languages, and for another example, for one full name subword, there may be different abbreviation subwords in different data sources. When using the mapping relationship to map the subword, due to the many-to-many mapping relationship, it is easy to cause the search space to explode, and the generation efficiency is low and the accuracy is low.

[0079] (3) The organization hierarchical structure of the organization information in different data sources is different, therefore, after obtaining the subword mapping result, it is difficult to align the grammar when fusing the subword mapping result to obtain the complete full name information. For this, Table 1 shows the comparison of the organization hierarchical structure of WOS database and other databases.

[0080] Table 1: Example of organization hierarchical structure

[0081]

[0082] As can be seen from Table 1, there is a big difference between the organization name recording methods of WOS and other databases in the organization hierarchical structure, that is, the order of the organization level is different, which leads to different grammar rules of the abbreviation information. And from Table 1, it can be seen that there are different abbreviation subwords corresponding to one full name subword, that is, there are some keyword abbreviations, for example, the full name subword "University" corresponds to the abbreviation subword "Univ" in the data source WOS, and the abbreviation subword corresponding to the data source is "University" itself. Other databases.

[0083] In summary, the existing full name information recognition method all has the technical problem of low recognition accuracy.

[0084] Generally, the abbreviation subword exists in the longer organization address information, and contains context semantic information. Based on this, the application proposes to take the full name information and the abbreviation information of the organization as two natural languages, to realize natural language recognition by using a neural network model constructed based on deep learning, so as to realize the conversion from the abbreviation information to the full name information.

[0085] To solve the above technical problems, the embodiment of the application provides a full name information generation method. The full name information generation method provided by the embodiment of the application will be described in detail below with reference to the accompanying drawings.

[0086] Referring to Figure 1 , Figure 1 The flowchart of the full name information generation method provided by the embodiment of the application is shown in Figure 1 The full name information generation method provided by the embodiment of the application can include S101 to S104, which will be described in detail below.

[0087] S101, obtaining target abbreviation information.

[0088] In this embodiment, the target abbreviation information is a subword sequence including multiple target abbreviation subwords.

[0089] S102, obtaining a graph embedding vector sequence and an abbreviation Token encoding sequence of the target abbreviation information.

[0090] In this embodiment, the graph embedding vector sequence includes graph embedding vectors of the target abbreviation subwords arranged in sequence, and the abbreviation Token encoding sequence includes Token encodings of the target abbreviation subwords arranged in sequence.

[0091] In this embodiment, the graph embedding vectors of the target abbreviation subwords are obtained by performing graph embedding type reinforcement learning on the target abbreviation subwords in the target abbreviation information, and the graph embedding vector sequence of the target abbreviation information is obtained by arranging the graph embedding vectors of the target abbreviation subwords according to the sequence positions in the target abbreviation information. The Token encodings of the target abbreviation subwords are obtained by performing Token encoding on the target abbreviation subwords in the target abbreviation information, and the Token encoding sequence of the target abbreviation information is obtained by arranging the Token encodings of the target abbreviation subwords according to the sequence positions in the target abbreviation information.

[0092] It should be noted that the generation method of the graph embedding vector of the target abbreviation subword includes multiple methods. In a possible implementation, the target abbreviation subword is input into a trained graph neural network model to obtain the graph embedding vector of the target abbreviation subword. The generation method of the Token encoding of the target abbreviation subword includes multiple methods. In a possible implementation, the target abbreviation subword is input into a trained Token encoding model to obtain the Token encoding of the target abbreviation subword. The graph neural network model is constructed based on a graph neural block, and the Token encoding model is constructed based on a self-attention language model.

[0093] S103, input the graph embedding vector sequence of the target abbreviation information and the abbreviation Token encoding sequence into a pre-trained sequence generation model to obtain a full name embedding vector sequence output by the sequence generation model.

[0094] In this embodiment, the full name embedding vector sequence includes a plurality of full name embedding vectors of target full name subwords arranged in sequence. In this embodiment, the encoder blocks are sequentially connected in the order of the first encoder block to the Nth encoder block, and the input of the first encoder block includes the input of the sequence generation model, and the output of the Nth encoder block is the output of the sequence generation model. The nth encoder block represents the nth encoder block, N represents the number of decoder blocks or decoder blocks, 1 < n ≤ N, and the weight parameters corresponding to each encoder block and decoder block are obtained based on training data.

[0095] Figure 2 A structural diagram of a sequence generation model provided by an embodiment of the present application is shown in FIG. 1. Figure 2 As shown in FIG. 1, the sequence generation model includes an encoder and a decoder constructed based on a neural network model of a self-attention mechanism. The encoder includes N encoder blocks connected in sequence, which are respectively denoted as a first encoder block, a second encoder block,..., and an Nth encoder block. The decoder includes N decoder blocks connected in sequence, which are respectively denoted as a first decoder block, a second decoder block,..., and an Nth decoder block.

[0096] The input of the encoder includes the graph embedding vectors in the graph embedding vector sequence of the target abbreviation information and the Token encodings in the abbreviation Token encoding sequence of the target abbreviation information, wherein the graph embedding vectors and the Token encodings of the target abbreviation subwords are input into the encoder at the same time.

[0097] The input of the decoder includes the graph embedding vectors in the graph embedding vector sequence of the target abbreviation information and the encoded vectors output by the encoder after encoding the Token encodings of the target abbreviation information, wherein the graph embedding vectors and the encoded vectors of the target abbreviation subwords are input into the encoder at the same time.

[0098] The input of the first encoder block is a fusion vector obtained by fusing the graph embedding vector and the Token encoding based on a weight parameter, and the input of the nth encoder block is a fusion vector obtained by fusing the graph embedding vector and the output vector of the previous encoder block based on a weight parameter.

[0099] The weight parameter corresponding to the input of the nth encoder block is represented as For example, taking the nth encoder block as the second encoder block, the input of the second encoder block is a fusion vector obtained by fusing the graph embedding vector and the output vector of the first encoder block based on a weight parameter The weight parameter corresponding to the input of the nth encoder block is represented as , wherein represents the output vector of the first encoder block, represents the graph embedding vector.

[0100] The input of the first decoder block is a fusion vector obtained by fusing the graph embedding vector and the output vector of the Nth encoder block based on a weight parameter, and the input of the nth decoder block is a fusion vector obtained by fusing the graph embedding vector and the output vector of the previous decoder block based on a weight parameter.

[0101] The weight parameter corresponding to the input of the nth decoder block is represented as For example, taking the nth decoder block as the second decoder block, the input of the second decoder block is a fusion vector obtained by fusing the graph embedding vector and the output vector of the first decoder block based on a weight parameter

[0102] The weight parameter corresponding to the input of the nth decoder block is represented as , wherein represents the output vector of the first decoder block, represents the graph embedding vector.

[0103] It should be noted that the sequence generation model is based on a neural network model constructed based on a self-attention mechanism. In one possible implementation, the sequence generation model is based on a Transformer model constructed based on a neural network model constructed based on a self-attention mechanism. The encoder and decoder of the neural network model constructed based on the self-attention mechanism are respectively referred to as a Transformer encoder and a Transformer decoder. The core of the Transformer model is the self-attention mechanism, which can assign a weight to each position in the input sequence, and then output these weighted position vectors. By continuously stacking multiple self-attention layers and feedforward neural network layers, a Transformer model can be constructed.

[0104] S104, generate the full name information of the target abbreviation information based on the full name embedding vector sequence.

[0105] In this embodiment, the full name information includes a plurality of target full name subwords.

[0106] As can be seen from the above technical solution, the full name information generation method provided in the embodiments of the present application inputs the graph embedding vector sequence and the abbreviation Token coding sequence of the target abbreviation information into the pre-trained sequence generation model, encodes the intermediate vector after fusing the graph embedding vector based on the weight parameters corresponding to each layer layer by layer through the multi-layer encoder block constructed based on the self-attention mechanism in the encoder of the sequence generation model, and decodes the intermediate vector after fusing the graph embedding vector based on the weight parameters corresponding to each layer layer by layer through the multi-layer decoder block constructed based on the self-attention mechanism in the decoder of the sequence generation model, to obtain the full name embedding vector sequence composed of the full name embedding vectors of the target full name subwords corresponding to each target abbreviation subword. Further, the full name information of the target abbreviation information is generated through the full name embedding vector sequence. In this scheme, the weight parameters of each layer are obtained by dynamic training, so that the graph embedding vectors obtained by graph embedding learning are fused into each layer of the encoder block and the decoder block constructed based on the self-attention mechanism with different weights. The relationship between each target abbreviation subword is captured through the self-attention mechanism, and the relationship between each target abbreviation subword is strengthened through graph embedding learning. The expression accuracy of the target abbreviation subword is improved, thereby improving the generation accuracy of the sequence generation model for the full name embedding vector sequence, and further improving the accuracy of the full name information of the target abbreviation information.

[0107] Figure 3 A structure diagram of a full name translation model provided in the embodiments of the present application is shown in FIG. 1, which includes a Token coding model, a graph neural network model, a sequence generation model, and a classifier. Figure 3

[0108] In this embodiment, the Token coding model is used to Token code the target abbreviation subwords in the target abbreviation information to obtain a Token coding sequence, and the Token codes in the Token coding sequence are sequentially input into the sequence generation model.

[0109] Specifically, the Token coding model can be a casual language model CasualLM, for example, a GPT-o model.

[0110] In this embodiment, the graph neural network model is used to perform graph embedding vector representation on the target abbreviation subwords in the target abbreviation information to obtain a graph embedding vector sequence, and the graph embedding vectors in the graph embedding vector sequence are sequentially input into the sequence generation model.

[0111] ​Specifically, the graph neural network model is composed of multiple graph neural modules, the graph neural modules include a graph attention network and a graph convolution network, and different graph neural modules capture information at different levels in the mapping relationship graph. Figure 4 A specific structure diagram of a graph neural network model provided by an embodiment of the present application is shown in the figure. Figure 4 As shown in the figure, the graph neural network model includes a first graph attention convolution layer, a first connection layer, a graph convolution layer, a second connection layer, a graph sampling and aggregation convolution layer, a third connection layer, a second graph attention convolution layer, a full connection layer, and a feature fusion layer connected in sequence, wherein the third connection layer and the full connection layer are connected to the feature fusion layer in a skip connection structure, each connection layer includes a regularization layer and a dropout layer, a part of samples are discarded at a proper probability by setting the hierarchical regularization layer and the dropout layer between the convolution layers, and the gradient vanishing in the model training process is prevented by adding a residual network structure.

[0112] In the embodiment, the sequence generation model is used for the fusion graph embedding vector of the target abbreviation word and the Token encoding, generates the Token encoding of the target full name word based on a self-attention mechanism, and obtains a full name embedding vector sequence.

[0113] Specifically, the sequence generation model includes an encoder and a decoder constructed based on a neural network model of a self-attention mechanism, wherein the encoder includes N encoder blocks connected in sequence, a first position encoding block arranged before the first encoder block, and an encoding fusion block arranged before each encoder block, and the decoder includes N decoder blocks connected in sequence, and a decoding fusion block arranged before each decoder block.

[0114] The input of the encoder includes a graph embedding vector in the graph embedding vector sequence of the target abbreviation information and a Token encoding in the abbreviation Token encoding sequence of the target abbreviation information, wherein the graph embedding vector and the Token encoding of the target abbreviation subword are input into the encoder at the same time.

[0115] The input of the first position encoding block is the Token encoding in the abbreviation Token encoding sequence, which is used for position encoding of the Token encoding. The input of the first encoding fusion block arranged before the first encoder block is the position encoding vector after position encoding of the Token encoding and the graph embedding vector, which is used for weighted fusion of the graph embedding vector and the position encoding vector based on a weight parameter to obtain a fusion vector The input of the first encoder block is the fusion vector , which is used for encoding the fusion vector based on a self-attention mechanism to obtain an encoding vector The input of the nth encoder fusion block, positioned before the nth encoder block, is the output encoded vector of the (n-1)th encoder block. This input is used to perform a weighted fusion of the graph embedding vector and the output encoded vector of the (n-1)th encoder block based on weight parameters, resulting in a fused vector. The input to the nth encoder block is the fusion vector. The weight parameters corresponding to the input of the nth encoder block are expressed as follows: .

[0116] The decoder's input includes the graph embedding vector from the graph embedding vector sequence of the target abbreviation information, and the encoded vector output by the encoder after the token encoding of the target abbreviation information is passed through the encoder, which is also the output encoded vector of the Nth encoder block. The graph embedding vector and encoded vector of the target abbreviation are simultaneously input to the encoder.

[0117] The first decoding fusion block, positioned before the first decoder block, takes as input the graph embedding vector of the target abbreviation, the output vector of the Nth encoder block, and the context-related attention vector. It is used to perform weighted fusion of the graph embedding vector of the target abbreviation, the output vector of the Nth encoder block, and the context-related attention vector based on weight parameters to obtain a fusion vector. The input to the first decoder block is the fused vector. Used for fusion vectors based on self-attention mechanism Decode to obtain the decoded vector The input of the nth decoder fusion block, positioned before the nth decoder block, is the output decoded vector of the (n-1)th decoder block. It is used to perform a weighted fusion of the graph embedding vector and the output decoded vector of the (n-1)th decoder block based on weight parameters, resulting in a fused vector. The input to the nth decoder block is the fusion vector. The output is a universal embedding vector. The weight parameters corresponding to the input of the nth decoder block are represented as follows: The Nth decoder block, also known as the last decoder block, outputs the input context-dependent attention vector to the first decoder fusion block.

[0118] In this embodiment, the sequence generation model is built based on either the Bahdanau attention mechanism or the Transformer attention mechanism. Taking the Transformer attention mechanism as an example, the sequence generation model is a Transformer model, which includes a Transformer encoder and a Transformer decoder. The Transformer encoder includes multiple Transformer encoder blocks, and the Transformer decoder includes multiple Transformer decoder blocks. The context-dependent attention vector is a sequence of vectors carrying contextual information generated based on the attention mechanism.

[0119] Figure 5 A schematic diagram of the specific structure of a Transformer encoder block provided in an embodiment of this application is shown below. Figure 5 As shown, the Transformer encoder block includes a multi-head attention layer, a vector summation and normalization layer, a position element-wise feedforward layer, and a vector summation and normalization layer connected in sequence.

[0120] Figure 6 A schematic diagram illustrating the specific structure of a Transformer decoder block provided in this application embodiment is shown below. Figure 6 As shown, the Transformer decoder sequentially connects the position encoder block and the Transformer decoder block. The Transformer decoder block includes a multi-head attention layer, a vector summation and normalization layer, a multi-head attention layer, a position element-wise feedforward layer, and a vector summation and normalization layer, which are sequentially connected.

[0121] In this embodiment, the classifier includes a token decoder and a word generator. Figure 7 This is a schematic diagram of the specific structure of a classifier provided in an embodiment of this application. The input of the Token decoder is the sequence of full name embedding vectors output by the decoder. The Token encoding of the full name sub-words corresponding to each full name embedding vector is output to the word generator. The output of the word generator is the full name information composed of the target full name sub-words, that is, the full name information of the target abbreviation information.

[0122] based on Figure 3 The present application provides a specific implementation flow for constructing a full-name translation model, as illustrated in the full-name translation model shown. Figure 8 This application provides a flowchart illustrating the specific implementation of a method for constructing a full-name translation model, as shown in the following embodiments. Figure 8 As shown, this method specifically includes:

[0123] S801. Construct a relation mapping graph based on the preset sub-word mapping relationship.

[0124] In this embodiment, the subword mapping relationship includes the mapping relationship between the abbreviated subword and the full subword, the abbreviated subword and the full subword are taken as nodes, the mapping relationship is taken as an edge, and the direction of the edge is from the abbreviated subword to the full subword, and a relationship mapping graph is constructed.

[0125] Taking WOS as an example, the list of abbreviated subwords provided by WOS contains the mapping relationship between the address-related abbreviated subword and the full subword. The mapping relationship on the list is generated in the form of a word-word correlation matrix to generate a mapping relationship graph of the abbreviated subword and the full subword. The abbreviated subword and the full subword with a mapping relationship are connected by a directed edge, and the direction of the directed edge is from the abbreviated subword to the full subword.

[0126] Figure 9 A subgraph schematic diagram of the mapping relationship graph is provided for the embodiments of the present application. As shown in the subgraph, [abbreviated subword full subword] represents a mapping relationship, and the mapping relationship includes [Med Medizin], [Med Medicine], [Med Medicinal], [Med Medical], [Med Meditskkaya], [Medicine Medizin], [Medicine Meditskkaya], [Medicine Medical], and [Medicine Medicinal].

[0127] S802, based on the relationship mapping graph, constructing a graph training data.

[0128] In this embodiment, the graph training data includes positive samples and negative samples.

[0129] Two nodes with an edge and the edge connecting the two nodes are selected from the relationship mapping graph as a positive sample. Two nodes without an edge are selected from the relationship mapping graph, and an edge is constructed for the two nodes as a negative sample.

[0130] It should be noted that, due to the incompleteness of the mapping relationship, the mapping relationship graph is a sparse graph, therefore, the number of positive samples is sparse, therefore, the number of samples is data enhanced, the enhancement method is to construct an enhanced edge for two unconnected nodes on the sparse graph, and the negative sample is composed of the two nodes and the enhanced edge, thereby realizing training data enhancement.

[0131] Still taking WOS as an example, the mapping relationship graph constructed based on WOS is a sparse graph with 537 nodes and 473 edges. In order to enhance the robustness and adaptability of the graph, a data negative sampling method is adopted. A first sampling node is randomly sampled from the sparse graph, and then a second sampling node not connected to the first sampling node is sampled. The edge between the first sampling node and the second sampling node is constructed as a negative sample. Then, the loss of the negative sample is subtracted when training the graph neural network model, so as to improve the training effect.

[0132] S803, training the graph neural network model based on the graph training data until a preset training completion condition is reached, and obtaining a graph embedding vector dictionary.

[0133] In this embodiment, the graph embedding vector dictionary includes each abbreviation subword in the relationship mapping graph and the corresponding graph embedding vector, and the graph neural network model is used to output the graph embedding vector of the abbreviation subword to be processed based on the graph embedding vector dictionary obtained by training. The structure of the graph neural network model can be referred to Figure 4 .

[0134] It should be noted that S801-S803 are a specific method for training a graph neural network to obtain a graph neural network model provided by the present application. After training the graph neural network model using the attention mechanism of the graph neural network and using contrastive learning, the graph embedding vector of each node on the mapping relationship graph is learned, and a graph embedding vector dictionary is formed.

[0135] S804, constructing a parallel corpus.

[0136] In this embodiment, the parallel corpus includes sample abbreviation information and corresponding sample full name information, the sample abbreviation information includes a plurality of sample abbreviation subwords, and the sample full name information includes a plurality of sample full name subwords.

[0137] In an optional embodiment, the method for constructing a parallel corpus includes:

[0138] 1. Retrieving bibliographic data from each database to construct an institution information set corresponding to each paper, wherein the institution information set includes each institution information, and the institution information includes abbreviation information and / or full name information.

[0139] 2. Data cleaning is performed on the institution information set of each paper to remove institution information sets with only abbreviations or full names, remove institution information with a number of subwords less than a preset number threshold, remove institution information containing special characters, and remove institution information containing non-English characters.

[0140] 3. For the institution information set of each paper, the abbreviation information and the full name information in the institution information set are aligned to identify the full name information and the corresponding abbreviation information of the same institution.

[0141] 4. Constructing a parallel corpus based on the alignment result.

[0142] For example, the Dimensions database is searched according to preset search conditions, and 540,991 pieces of search data are obtained. To facilitate the construction of the parallel corpus, the bibliographic data of only one organization is retained, and the bibliographic data with only an abbreviation or full name, the bibliographic data with too short (less than 10 characters) organization information, the bibliographic data containing special characters, and the bibliographic data containing non-English characters are removed. Finally, a parallel corpus including 138,000 pieces of abbreviation information and corresponding full name information is obtained.

[0143] In this embodiment, the information format of the parallel corpus can be seen from Table 2.

[0144] Table 2: Parallel corpus table

[0145]

[0146] S805. Obtaining an abbreviation Token encoding sequence of the sample abbreviation information and a full name Token encoding sequence of the sample full name information by using a pre-trained Token encoding model.

[0147] In this embodiment, the Token encoding model includes a CasualLM causal language model.

[0148] Specifically, the abbreviation information and the full name information in the parallel corpus are respectively input into the pre-trained Token encoding model, Token encoding of each sample abbreviation subword and Token encoding of each sample full name subword output by the Token encoding model are obtained, and an abbreviation Token encoding sequence of the sample abbreviation information and a full name Token embedding vector sequence of the sample full name information are obtained.

[0149] It should be noted that the method of generating Token encoding of the abbreviation and full name information in the parallel corpus by using the CasualLM can be seen from the prior art.

[0150] S806. Obtaining a graph embedding vector of each sample abbreviation subword in the sample abbreviation information in the parallel corpus by using a graph neural network model.

[0151] S807. Constructing sequence training data.

[0152] In this embodiment, the sequence training data includes a graph embedding vector sequence and an abbreviation Token encoding sequence of each piece of sample abbreviation information, and a full name Token encoding sequence of the sample full name information. The Token encoding sequence of the sample abbreviation information includes Token encoding of each sample abbreviation subword The graph embedding vector sequence of the sample short name information includes graph embedding vectors of sample short name subwords The Token encoding sequence of the sample full name information includes Token encodings of sample full name subwords .

[0153] S808, input the graph embedding vector sequence of the sample short name information, the short name Token encoding sequence, and the full name Token encoding sequence to the sequence generation model, perform multiple iterations until a preset training completion condition is reached, and obtain a trained sequence generation model and a graph embedding vector dictionary.

[0154] In this embodiment, the sequence generation model is a neural network model based on a self-attention mechanism, wherein the model parameters of the sequence generation model include weight parameters of each encoding fusion layer and each decoding fusion layer.

[0155] Taking the sequence generation model as a neural network model based on the Transformer self-attention mechanism as an example, Figure 10 An iterative process schematic diagram provided by an embodiment of the present application is shown in FIG. 8, and each iteration includes: Figure 10

[0156] A1, perform position encoding on the Token encoding of the sample short name subword by a first position encoding block to obtain a position encoding vector of the sample short name subword.

[0157] A2, perform weighted fusion on the position encoding vector of the sample short name subword and the graph embedding vector of the sample short name subword based on the weight parameters by a first encoding fusion block, and input the intermediate vector after the weighted fusion to a first encoder block.

[0158] A3, sequentially perform encoding on the intermediate vector after the weighted fusion by an mth encoder block and output to an m+1th encoding fusion block, perform weighted fusion on the encoded vector after the encoding and the graph embedding vector of the sample short name subword based on the weight parameters by the m+1th encoding fusion block, and input the intermediate vector after the weighted fusion to an m+1th encoder block until the m+1th encoder block is an Nth encoder block, that is, a last encoder block.

[0159] It should be noted that, since the position encoding is short, the encoded embedding vector is scaled according to the square root of the dimension, so as to increase the variance between and .

[0160] A4, output the encoded vector after the encoding of the Nth encoder block to a decoder block.

[0161] ​A5. Token encoding of the sample full name subword by the second position encoding block Position encoding is performed to obtain a sample full name embedding vector.

[0162] A6. Weighted fusion of the output vector of the Nth encoder block, the graph embedding vector of the sample short name subword, and the sample full name embedding vector based on the weight parameters by the first decoding fusion block.

[0163] A7. Sequentially performing encoding of the weighted fusion intermediate vector by the mth decoder block and outputting to the m+1th decoding fusion block, performing weighted fusion of the decoded vector and the graph embedding vector of the sample short name subword based on the weight parameters by the m+1th decoding fusion block, and inputting the weighted fusion intermediate vector to the m+1th decoder block until the m+1th decoder block is the Nth decoder block, i.e., the last decoder block.

[0164] A8. Mask loss calculation of the output of the Nth decoder block and the sample full name embedding vector according to the vector length by the loss calculation block (mask loss layer), and determining whether the preset training completion condition is reached, if yes, obtaining the trained sequence generation model, if no, updating the model parameters of the sequence generation model based on the mask loss calculation result using the back propagation algorithm, and performing the next iteration.

[0165] In this embodiment, the model parameters include the weight parameters corresponding to each decoding fusion block. It should be noted that a gradient clipping layer is added before the loss calculation block to prevent gradient explosion and improve the model training effect.

[0166] It should be noted that in order to automatically add the graph embedding vector of the short name subword in the short name information to the model as needed, a learnable graph vector fusion algorithm is used to update the model parameters, and the graph vector fusion algorithm is shown in Table 3, which includes:

[0167] Table 3 Weight learning algorithm

[0168]

[0169] S804~S808 is a specific training method of a sequence generation model provided by the embodiment of the application.

[0170] S809, constructing a classifier based on a pre-constructed word dictionary.

[0171] In this embodiment, the word dictionary includes a preset full name subword and a matching Token code.

[0172] ​In this embodiment, the Token decoder in the classifier is used to generate a full name Token encoding sequence, and the full name Token encoding sequence includes a plurality of Token encodings.

[0173] In this embodiment, the word generator is used to find, based on a word dictionary, full name subwords matching the Token encodings of the respective full name embedding vectors as target full name subwords, output full name information of the target abbreviation information, that is, target full name information.

[0174] Based on Figure 3 As shown in the full name information recognition model, the embodiment of the present application provides a specific implementation method of a full name information generation method. When converting abbreviation information into full name information, a given agency address information composed of agency abbreviations is extracted, and the corresponding agency full name is generated, as shown in formula (1):

[0175] (1);

[0176] Among them, is the abbreviation subword of the abbreviation information, is the full name subword of the full name information, indicating the length of the full name information, that is, the number of full name subwords , the present application constructs a mapping between the abbreviation information and the full name information through the full name information recognition model, so as to identify the abbreviation information while maintaining the semantics, and obtain the prediction result, that is, the full name information.

[0177] Figure 11 The embodiment of the present application provides a specific implementation process of a full name information generation method, as shown in Figure 11 The method specifically includes:

[0178] S1101, obtaining target abbreviation information.

[0179] In this embodiment, the target abbreviation information is any abbreviation information to be translated, and the target abbreviation information includes a plurality of target abbreviation subwords. For example, the target abbreviation information includes M abbreviation subwords, denoted as Among them, represents the mth abbreviation subword in the target abbreviation information.

[0180] S1102, inputting each target abbreviation subword into a pre-trained graph neural network model to obtain a graph embedding vector of each target abbreviation subword output by the graph neural network model, sequentially arranging the graph embedding vectors of each target abbreviation subword to obtain a graph embedding vector sequence of the target abbreviation information.

[0181] Specifically, each target abbreviation subword is input into the graph neural network model, and the graph neural network model sequentially outputs graph embedding vectors of each target abbreviation subword based on the graph embedding vector dictionary obtained through training.

[0182] It should be noted that the target abbreviation subword is a sample abbreviation subword in the graph training data, and therefore, in the graph embedding vector dictionary, an embedding vector of the sample abbreviation subword has been generated. If the target abbreviation subword is not a sample abbreviation subword in the graph training data, the target abbreviation subword is added as a new node to the mapping relationship graph, and the graph embedding vector dictionary is retrained to obtain an embedding vector of the target abbreviation subword.

[0183] In this embodiment, the graph neural network model is composed of multiple graph neural modules, the graph neural modules include a graph attention network and a graph convolution network, and the graph neural network model is trained based on a relationship mapping graph. The relationship mapping graph takes abbreviation subwords and full name subwords as nodes, and takes the mapping relationship from the abbreviation subword to the full name subword as a directed edge. For specific structures of the graph neural network model and training methods, refer to the above embodiments.

[0184] S1103, input each target abbreviation subword into the pre-trained Token encoding model to obtain Token encoding of each target abbreviation subword output by the Token encoding model, arrange the Token encoding of each target abbreviation subword in sequence to obtain a sequence of abbreviation Token encoding of the target abbreviation information.

[0185] In this embodiment, the Token encoding model includes a causal language model.

[0186] S1104, input the sequence of abbreviation Token encoding of the target abbreviation information into the trained sequence generation model to obtain a sequence of full name embedding vectors of the target abbreviation information output by the sequence generation model.

[0187] Specifically, the model parameters of the trained sequence generation model include the weight parameters of each encoding fusion block and each decoding fusion block optimized, and the process of outputting the sequence of full name embedding vectors of the target abbreviation information by the sequence generation model includes multiple iterations, and through each iteration, a full name embedding vector of a target abbreviation subword is output and sequentially output until the length of the output sequence of full name embedding vectors reaches a preset length or an end symbol is encountered.

[0188] Taking one iteration as an example, the method for generating the full name embedding vector of the target abbreviation subword includes:

[0189] B1, position encoding of the Token encoding of the target abbreviation subword is performed by the first position encoding block to obtain a position encoding vector of the target abbreviation subword.

[0190] B2, the first encoding fusion block is used for weighting and fusing the graph embedding vector of the target abbreviation subword and the position encoding vector of the target abbreviation subword based on a weight parameter, and the weighted and fused intermediate vector is input into the first encoder block.

[0191] B3, the weighted and fused intermediate vector is sequentially encoded by the mth encoder block and output to the m+1th encoding fusion block, the m+1th encoding fusion block is used for weighting and fusing the graph embedding vector of the target abbreviation subword and the encoded encoding vector based on a weight parameter, and the weighted and fused intermediate vector is input into the m+1th encoder block until the m+1th encoder block is the Nth encoder block, i.e. the last encoder block.

[0192] B4, the Nth encoder block outputs the encoded encoding vector to the decoder block.

[0193] B5, the first decoding fusion block is used for weighting and fusing the graph embedding vector of the target abbreviation subword, the output vector of the Nth encoder block and the context-related attention vector based on a weight parameter, and the weighted and fused intermediate vector is input into the first decoder block, wherein the context-related attention vector is returned by the last decoder block after the last iteration.

[0194] B6, the weighted and fused intermediate vector is sequentially encoded by the mth decoder block and output to the m+1th decoding fusion block, the m+1th decoding fusion block is used for weighting and fusing the graph embedding vector of the target abbreviation subword and the decoded decoding vector based on a weight parameter, and the weighted and fused intermediate vector is input into the m+1th decoder block until the m+1th decoder block is the Nth decoder block, i.e. the last decoder block.

[0195] B7, the weighted and fused intermediate vector is decoded by the last decoder block to obtain the full name embedding vector and output the full name embedding vector to the classifier.

[0196] B9, if the length of the full name embedding vector sequence does not reach the preset length and the end symbol is not detected, the last decoder outputs the context-related attention vector to the first decoding fusion block, so that the first decoding fusion block weights and fuses the graph embedding vector of the target abbreviation subword, the output vector of the Nth encoder block and the context-related attention vector based on a weight parameter in the next iteration, and inputs the weighted and fused intermediate vector into the first decoder.

[0197] S1105, the full name embedding vector sequence of the target abbreviation information is input into the classifier one by one to obtain the full name information of the target abbreviation information output by the classifier.

[0198] In this embodiment, the full name embedding vector sequence of the target abbreviation information includes the full name embedding vector output by the last position decoder block. The Token decoder in the classifier obtains the Token code of each full name embedding vector in the full name embedding vector sequence to obtain the target full name Token code sequence.

[0199] Specifically, the Token decoder obtains the Token decoding method of the target full name embedding vector (any full name embedding vector in the full name embedding vector sequence) includes:

[0200] Finding the index where the probability maximum value is located in the dictionary embedding dimension of the full name embedding vector, and taking the index as the Token code of the full name embedding vector.

[0201] It should be noted that the specific method of decoding the embedding vector to obtain the Token code can refer to the prior art.

[0202] In this embodiment, the word generator in the classifier finds the corresponding subword in the subword dictionary according to the full name Token code based on the subword dictionary to generate the full name subword corresponding to the Token code of each full name embedding vector.

[0203] It should be noted that in an optional embodiment, the Bahdanau attention mechanism can be used to replace the Transformer attention mechanism. Based on the full name translation model of the Bahdanau attention layer, at each time step, the decoder calculates an alignment distribution according to the previous output and all hidden states of the encoder, which indicates which positions in the encoder are most useful for the output of the current time step. Then, the decoder uses the weighted encoder hidden state to generate the output of the current time step. Therefore, the dynamic alignment mechanism of the Bahdanau attention mechanism makes the model better capture the complex dependency between the source language and the target language, thereby improving the accuracy and fluency of the full name translation model.

[0204] In an optional embodiment, the loss value is calculated according to the actual length of the input vector, which improves the accuracy of the loss value, improves the training effect, and thus improves the accuracy of the full name translation model.

[0205] In an optional embodiment, the full name translation model is tested by using a test set, and the full name translation model is evaluated by using indicators such as Rouge and BLUE. If the preset test condition is not met, the full name translation model is retrained to improve the accuracy of the full name translation model.

[0206] In an optional embodiment, the full name information recognition method provided by the embodiments of the present application can be applied to solve the problem of low accuracy of restoring the abbreviated information to the full name information in the institution information from WOS in the process of constructing the institution specification. The present application trains the mapping of the abbreviated subword to the full name subword provided by WOS by using the graph neural network to obtain the graph embedding vector of all abbreviated subwords and the trained graph neural network model. The graph embedding vector and the Token vector of the abbreviated information to be recognized are input into the pre-trained full name translation model, and the recognition of the full name information is realized through the encoder and the decoder. Thus, the problem of restoring the abbreviated information of the institutions in the paper data in the WOS database to the full name information is converted into a sequence-to-sequence translation problem. The institution information provided by the WOS database is learned by using the contrast learning method in deep learning to learn the embedding of each subword. The learned graph embedding vector of each abbreviated subword and the Token coding of each abbreviated subword are fused by using the learnable dynamic fusion method to obtain the full name information. Further, the Bahdanau attention mechanism is used to dynamically capture the attention inside the input sequence and the target sequence, and the loss value is calculated according to the actual length of the input vector to improve the accuracy of the model in recognizing the full name information.

[0207] The above introduces a full name information generation method provided by the embodiments of the present application. The device for executing the full name information generation method will be introduced below.

[0208] Please refer to Figure 12 , Figure 12 The structure diagram of the full name information generation device provided by the embodiments of the present application is shown in FIG. 12. Figure 12 As shown in FIG. 12, the full name information generation device 1200 comprises:

[0209] The abbreviated information acquisition unit 1201 is configured to acquire target abbreviated information, wherein the target abbreviated information is a subword sequence comprising a plurality of target abbreviated subwords.

[0210] The information representation unit 1202 is configured to acquire a graph embedding vector sequence and an abbreviated Token coding sequence of the target abbreviated information, wherein the graph embedding vector sequence comprises graph embedding vectors of the abbreviated subwords arranged in sequence, and the abbreviated Token coding sequence comprises Token coding of the abbreviated subwords arranged in sequence.

[0211] The sequence generation unit 1203 is configured to input the graph embedding vector sequence and the abbreviated Token coding sequence of the target abbreviated information into a pre-trained sequence generation model to obtain a full name embedding vector sequence output by the sequence generation model, wherein the full name embedding vector sequence comprises full name embedding vectors of a plurality of target full name subwords arranged in sequence.

[0212] The sequence generation model comprises an encoder and a decoder constructed based on a neural network model of a self-attention mechanism, the encoder comprises N sequentially connected encoder blocks, and the decoder comprises N sequentially connected decoder blocks; an input of a first encoder block is a fusion vector obtained by weighting and fusing a graph embedding vector and a Token encoding based on weight parameters, and an input of an nth encoder block is a fusion vector obtained by weighting and fusing a graph embedding vector and an output vector of a previous encoder block based on weight parameters; an input of a first decoder block is a fusion vector obtained by weighting and fusing a graph embedding vector and an output vector of the Nth encoder block based on weight parameters, an input of an nth decoder block is a fusion vector obtained by weighting and fusing a graph embedding vector and an output vector of a previous decoder block based on weight parameters, and an output is the universal embedding vector sequence; wherein 1 < n ≤ N; the weight parameters corresponding to each encoder block and decoder block are obtained based on training data.

[0213] The universal matching unit 1204 is configured to generate the universal information of the target abbreviation information based on the universal embedding vector sequence, and the universal information comprises a plurality of target universal subwords.

[0214] In a possible implementation, the information representation unit is configured to obtain a graph embedding vector sequence of the target abbreviation information, and specifically configured to:

[0215] The graph embedding vector dictionary is generated based on a trained graph neural network model, and the graph embedding vector dictionary comprises graph embedding vectors of each preset subword; the graph neural network model comprises a plurality of graph neural modules, and the graph neural module comprises a graph attention network and a graph convolution network; the graph neural network model is trained based on a relationship mapping graph, and the relationship mapping graph takes a subword and a universal subword as a node and takes a mapping relationship from a subword to a universal subword as a directed edge;

[0216] The graph embedding vector sequence of the target abbreviation information is obtained by sequentially arranging the graph embedding vectors of each target subword.

[0217] In a possible implementation, the universal information generation apparatus further comprises a training model of a graph neural network model, and the training model is configured to:

[0218] The relationship mapping graph is constructed based on a preset subword mapping relationship, and the subword mapping relationship comprises a mapping relationship between a subword and a universal subword;

[0219] Based on the relationship mapping graph, two nodes and an edge connecting the two nodes are selected from the relationship mapping graph as a positive sample if the two nodes have an edge, and two nodes are selected from the relationship mapping graph and an edge is constructed for the two nodes as a negative sample if the two nodes do not have an edge;

[0220] generate graph training data, the graph training data comprising a plurality of positive samples and a plurality of negative samples;

[0221] train the graph neural network model based on the graph training data until a preset training completion condition is reached, to obtain the graph embedding vector dictionary.

[0222] In a possible implementation, when the information representation unit is used to obtain the Token encoding sequence, the information representation unit is specifically configured to:

[0223] input each target abbreviation subword into a pre-trained Token encoding model to obtain Token encoding of each target abbreviation subword output by the Token encoding model, the Token encoding model comprising a causal language model;

[0224] generate Token encoding of each target abbreviation subword in sequence to obtain the Token encoding sequence of the target abbreviation information.

[0225] In a possible implementation, the encoder comprises a Transformer encoder, and the decoder comprises a Transformer decoder;

[0226] each encoder block in the Transformer encoder comprises, in sequence, a multi-head attention layer, a vector addition and normalization layer, a position element-wise feedforward layer, and a vector addition and normalization layer, and each decoder block in the Transformer decoder comprises, in sequence, a multi-head attention layer, a vector addition and normalization layer, a multi-head attention layer, a position element-wise feedforward layer, and a vector addition and normalization layer;

[0227] The encoder further comprises a first position encoding block configured before the first encoder block and an encoding fusion block configured before each encoder block, and the decoder further comprises a decoding fusion block configured before each decoder block.

[0228] In a possible implementation, the full name information generation apparatus further comprises a training unit of a sequence generation model, configured to:

[0229] construct a parallel corpus, the parallel corpus comprising sample abbreviation information and corresponding sample full name information, the sample abbreviation information comprising a plurality of sample abbreviation subwords, and the sample full name information comprising a plurality of sample full name subwords;

[0230] obtain sequence training data, the sequence training data comprising a graph embedding vector sequence and a short name Token encoding sequence of each piece of the sample short name information, and a full name Token encoding sequence of the sample full name information, the full name Token encoding sequence of the sample full name information comprising Token encodings of each full name subword in the sample full name information;

[0231] input the graph embedding vector sequence, the short name Token encoding sequence, and the full name Token encoding sequence of the sample short name information into a sequence generation model, and perform multiple iterations, each iteration comprising:

[0232] perform position encoding on the Token encoding of the sample short name subword through the first position encoding block to obtain a position encoding vector of the sample short name subword;

[0233] perform weighted fusion on the position encoding vector of the sample short name subword and the graph embedding vector of the sample short name subword based on a weight parameter through the first encoding fusion block, and input the intermediate vector after the weighted fusion into the first encoder block;

[0234] perform weighted fusion on the output vector of the (n-1)th encoder block and the graph embedding vector of the sample short name subword based on a weight parameter through the nth encoding fusion block, and input the intermediate vector after the weighted fusion into the nth encoder block;

[0235] output the intermediate vector after the weighted fusion through each encoder block;

[0236] perform position encoding on the Token encoding of the sample full name subword through the second position encoding block to obtain the sample full name embedding vector;

[0237] perform weighted fusion on the output vector of the Nth encoder block, the graph embedding vector of the sample short name subword, and the sample full name embedding vector based on a weight parameter through the first decoding fusion block, and input the intermediate vector after the weighted fusion into the first decoder block;

[0238] perform weighted fusion on the output vector of the (n-1)th decoder block and the graph embedding vector of the sample short name subword based on a weight parameter through the nth decoding fusion block, and input the intermediate vector after the weighted fusion into the nth encoder block;

[0239] output the intermediate vector after the weighted fusion through each encoder block;

[0240] perform mask loss calculation on the output of the Nth decoder block and the sample full name embedding vector according to the vector length through the loss calculation block, update the model parameters of the sequence generation model based on the mask loss calculation result, and the model parameters comprise the weight parameters corresponding to each decoding fusion block.

[0241] determining whether a preset training completion condition is reached, if yes, obtaining the trained sequence generation model, and if no, performing next iteration.

[0242] In a possible implementation, the full name matching unit is configured to generate the full name information of the target short name information based on the full name embedding vector sequence, and specifically configured to:

[0243] obtain Token encodings of each full name embedding vector in the full name embedding vector sequence to obtain a target full name Token encoding sequence, the target full name encoding sequence including Token encodings of each full name embedding vector arranged in sequence,

[0244] search for full name subwords matched with the Token encodings of each full name embedding vector as the target full name subwords based on a preconfigured word dictionary, the word dictionary including preset full name subwords and matched Token encodings;

[0245] sequentially combine each target full name subword to obtain the full name information of the target short name information.

[0246] The embodiments of the present application also provide an electronic device. Referring to FIG. 1, Figure 13 which shows a structural schematic diagram suitable for implementing the electronic device in the embodiments of the present application. The electronic device in the embodiments of the present application can include but is not limited to fixed terminals such as mobile phones, notebook computers, PDAs (Personal Digital Assistants), PADs (Tablet Personal Computers), desktop computers, and the like. Figure 13 The electronic device shown is only an example, and should not bring any limitation to the functions and use range of the embodiments of the present application.

[0247] As shown in FIG. 13, Figure 13 The electronic device can include a processing device (such as a central processing unit, a graphics processing unit, etc.) 1301, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 1302 or loaded from a storage device 1308 into a random access memory (RAM) 1303. In the state that the electronic device is powered on, the RAM 1303 also stores various programs and data required for the operation of the electronic device. The processing device 1301, the ROM 1302, and the RAM 1303 are connected to each other through a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.

[0248] In general, the following devices can be connected to the I / O interface 1305: input devices 1306, including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, and the like; output devices 1307, including, for example, a liquid crystal display (LCD), a speaker, a vibrator, and the like; storage devices 1308, including, for example, a memory card, a hard disk, and the like; and communication devices 1309. The communication devices 1309 can allow the electronic device to communicate wirelessly or through a wired connection with other devices to exchange data. Although Figure 6 An electronic device having various devices is shown, but it is understood that all of the shown devices are not required to be implemented or present. More or less devices can alternatively be implemented or present.

[0249] The embodiments of the present application also provide a computer program product comprising computer readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the full name information generation methods provided by the embodiments of the present application.

[0250] The embodiments of the present application also provide a computer readable storage medium carrying one or more computer programs, which, when executed by an electronic device, can cause the electronic device to implement any of the full name information generation methods provided by the embodiments of the present application.

[0251] In addition, it should be noted that the device embodiments described above are merely illustrative, and the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e. they can be located in one place or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments. In addition, the device embodiments provided in the present application, the connection relationship between the modules indicates that there is a communication connection between them, which can be implemented as one or more communication buses or signal lines.

[0252] Those skilled in the art can clearly understand that the application can be implemented by means of software plus necessary universal hardware, and of course can also be implemented by means of dedicated hardware including special integrated circuit, special CPU, special memory, special component, etc. Generally, any function completed by computer program can be easily implemented by corresponding hardware, and the specific hardware structure for implementing the same function can also be various, such as analog circuit, digital circuit or special circuit, etc. However, for the application, software program implementation is a better embodiment. Based on such understanding, the technical solution of the application or the part of the application which makes contribution to the prior art can be embodied in the form of software product, which is stored in readable storage medium, such as computer floppy disk, U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a plurality of instructions for making a computer device (which can be personal computer, training device or network device, etc.) execute the method described in various embodiments of the application.

[0253] In the above embodiments, the implementation can be achieved by software, hardware, firmware or any combination thereof, entirely or partially. When implemented by software, the implementation can be in the form of computer program product.

[0254] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the flow or function described in the embodiments of the application is generated entirely or partially. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another, for example, the computer instructions can be transmitted from one website, computer, training device or data center to another website, computer, training device or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that can be stored by the computer or a data storage device such as training device, data center, etc. integrated with one or more available media sets. The available medium can be magnetic medium (such as floppy disk, hard disk, magnetic tape), optical medium (such as DVD) or semiconductor medium (such as solid state disk (SSD)) etc.

Claims

1. A method of generating full name information, characterized by, The method comprises the following steps: obtaining target abbreviation information, the target abbreviation information being a subword sequence comprising a plurality of target subwords; obtaining a graph embedding vector sequence and an abbreviation Token coding sequence of the target abbreviation information, the graph embedding vector sequence comprising graph embedding vectors of the target subwords arranged in sequence, and the abbreviation Token coding sequence comprising Token codes of the target subwords arranged in sequence; inputting the graph embedding vector sequence and the abbreviation Token coding sequence of the target abbreviation information into a pre-trained sequence generation model to obtain a full name embedding vector sequence output by the sequence generation model, the full name embedding vector sequence comprising full name embedding vectors of a plurality of target full name subwords arranged in sequence; wherein the sequence generation model comprises an encoder and a decoder constructed based on a self-attention mechanism neural network model, the encoder comprises N sequentially connected encoder blocks, the decoder comprises N sequentially connected decoder blocks, the input of the first encoder block is a fusion vector obtained by weighting and fusing the graph embedding vector and the Token code based on weight parameters, the input of the nth encoder block is a fusion vector obtained by weighting and fusing the graph embedding vector and the output vector of the previous encoder block based on weight parameters, the input of the first decoder block is a fusion vector obtained by weighting and fusing the graph embedding vector and the output vector of the Nth encoder block based on weight parameters, the input of the nth decoder block is a fusion vector obtained by weighting and fusing the graph embedding vector and the output vector of the previous decoder block based on weight parameters, and the output is the full name embedding vector sequence; wherein 1 generating a full name information of the target abbreviation information based on the full name embedding vector sequence, the full name information comprising a plurality of target full name subwords.

2. The method of generating a full name information according to claim 1, wherein, The method comprises the following steps: obtaining a graph embedding vector sequence of the target abbreviation information, comprising: obtaining the graph embedding vector of each target subword according to a graph embedding vector dictionary generated by a training graph neural network model, the graph embedding vector dictionary comprising the graph embedding vector of each preset subword; the graph neural network model is composed of a plurality of graph neural modules, the graph neural module comprises a graph attention network and a graph convolution network, and the graph neural network model is trained based on a relationship mapping graph, the relationship mapping graph taking the subword and the full name subword as nodes and taking the mapping relationship from the subword to the full name subword as a directed edge; 3. The method of generating a full name information according to claim 2, wherein, arranging the graph embedding vectors of each target subword in sequence to obtain the graph embedding vector sequence of the target abbreviation information. The training method of the graph neural network model comprises the following steps: constructing the relationship mapping graph based on a preset subword mapping relationship, the subword mapping relationship comprising the mapping relationship between the subword and the full name subword; selecting two nodes and an edge connecting the two nodes from the relationship mapping graph as a positive sample, and selecting two nodes not having an edge from the relationship mapping graph and constructing an edge for the two nodes as a negative sample; Generate graph training data, the graph training data includes a plurality of positive samples and a plurality of negative samples; Train the graph neural network model based on the graph training data until a preset training completion condition is reached to obtain the graph embedding vector dictionary.

4. The method of claim 1, wherein, The sequence of Token encodings includes: Input each target abbreviation subword into a pre-trained Token encoding model to obtain Token encodings of each target abbreviation subword output by the Token encoding model, the Token encoding model including a causal language model; In sequence, generate Token encodings of each target abbreviation subword to obtain an abbreviation Token encoding sequence of the target abbreviation information.

5. The method of claim 1, wherein, The encoder includes a Transformer encoder, and the decoder includes a Transformer decoder; Each encoder block in the Transformer encoder includes a multi-head attention layer, a vector addition and normalization layer, a position element-wise feedforward layer, and a vector addition and normalization layer in sequence, and each decoder block in the Transformer decoder includes a multi-head attention layer, a vector addition and normalization layer, a multi-head attention layer, a position element-wise feedforward layer, and a vector addition and normalization layer in sequence; The encoder further includes a first position encoding block configured before the first encoder block and an encoding fusion block configured before each encoder block, and the decoder further includes a decoding fusion block configured before each decoder block.

6. The method of generating full name information according to claim 5, wherein, The training method of the sequence generation model includes: Construct a parallel corpus, the parallel corpus including sample abbreviation information and corresponding sample full name information, the sample abbreviation information including a plurality of sample abbreviation subwords, and the sample full name information including a plurality of sample full name subwords; Obtain sequence training data, the sequence training data including a graph embedding vector sequence and an abbreviation Token encoding sequence of each sample abbreviation information, and a full name Token encoding sequence of the sample full name information, the full name Token encoding sequence of the sample full name information including Token encodings of each full name subword in the sample full name information; Input the graph embedding vector sequence, the abbreviation Token encoding sequence, and the full name Token encoding sequence of the sample abbreviation information into a sequence generation model and perform multiple iterations, each iteration including: Positionally encode the Token encodings of the sample abbreviation subwords through the first position encoding block to obtain position encoding vectors of the sample abbreviation subwords; Weight fuse the position encoding vectors of the sample abbreviation subwords and the graph embedding vectors of the sample abbreviation subwords based on weight parameters through the first encoding fusion block, and input the weighted and fused intermediate vectors into the first encoder block; Weight fuse the output vectors of the n-1 encoder blocks and the graph embedding vectors of the sample abbreviation subwords based on weight parameters through each nth encoding fusion block, and input the weighted and fused intermediate vectors into the nth encoder block; Encode and output the weighted and fused intermediate vectors through each encoder block. perform position encoding on the Token encoding of the sample full name subword through a second position encoding block to obtain a sample full name embedding vector; perform weighted fusion on the output vector of the Nth encoder block, the graph embedding vector of the sample short name subword, and the sample full name embedding vector based on weight parameters through a first decoding fusion block, and input the intermediate vector after weighted fusion to a first decoder block; perform weighted fusion on the output vector of the n-1th decoder block and the graph embedding vector of the sample short name subword based on weight parameters through each n th decoding fusion block, and input the intermediate vector after weighted fusion to the n th encoder block; output the intermediate vector after weighted fusion after encoding through each encoder block; perform mask loss calculation on the output of the Nth decoder block and the sample full name embedding vector according to the vector length through a loss calculation block, update the model parameters of the sequence generation model based on the mask loss calculation result, and the model parameters include the weight parameters corresponding to each decoding fusion block; determine whether a preset training completion condition is reached, if yes, obtain the trained sequence generation model, and if no, perform the next iteration.

7. The generating method of the full name information according to claim 1, characterized by, based on the full name embedding vector sequence, generate the full name information of the target short name information, including: obtain the Token encoding of each full name embedding vector in the full name embedding vector sequence to obtain a target full name Token encoding sequence, the target full name encoding sequence includes the Token encoding of each full name embedding vector arranged in sequence, based on a pre-configured word dictionary, find the full name subword matching the Token encoding of each full name embedding vector as the target full name subword, and the word dictionary includes preset full name subwords and matching Token encodings; sequentially combine each target full name subword to obtain the full name information of the target short name information.

8. An apparatus for generating full name information, characterized by comprising: including: a short name information acquisition unit configured to acquire target short name information, the target short name information being a subword sequence including a plurality of target short name subwords; an information representation unit configured to acquire a graph embedding vector sequence and a short name Token encoding sequence of the target short name information, the graph embedding vector sequence including graph embedding vectors of each short name subword arranged in sequence, and the short name Token encoding sequence including Token encodings of each short name subword arranged in sequence; a sequence generation unit configured to input the graph embedding vector sequence and the short name Token encoding sequence of the target short name information into a pre-trained sequence generation model to obtain a full name embedding vector sequence output by the sequence generation model, the full name embedding vector sequence including full name embedding vectors of a plurality of target full name subwords arranged in sequence; The sequence generation model comprises an encoder and a decoder constructed based on a neural network model of a self-attention mechanism, the encoder comprises N sequentially connected encoder blocks, the decoder comprises N sequentially connected decoder blocks, an input of a first encoder block is a fusion vector obtained by weighting and fusing a graph embedding vector and a Token encoding based on weight parameters, an input of an nth encoder block is a fusion vector obtained by weighting and fusing a graph embedding vector and an output vector of a previous encoder block based on weight parameters; an input of a first decoder block is a fusion vector obtained by weighting and fusing a graph embedding vector and an output vector of the Nth encoder block based on weight parameters, an input of an nth decoder block is a fusion vector obtained by weighting and fusing a graph embedding vector and an output vector of a previous decoder block based on weight parameters, and an output is the full name embedding vector sequence; wherein 1 < n ≤ N; the weight parameters corresponding to each encoder block and decoder block are obtained by training based on training data; The full name matching unit is configured to generate full name information of the target abbreviated information based on the full name embedding vector sequence, and the full name information comprises a plurality of target full name subwords.

9. A computer program product, characterised in that, The computer readable instructions, when executed on an electronic device, cause the electronic device to implement the full name information generation method of any one of claims 1 to 7.

10. An electronic device, comprising: The memory is configured to store computer programs. The processor is configured to execute the computer programs to enable the electronic device to implement the full name information generation method of any one of claims 1 to 7. ​

Images