Translation model training method, device and electronic equipment

Abstract

The embodiments of the present disclosure provide a training method, device and electronic device for a translation model. The method includes: obtaining a first abstract semantic graph and a corresponding text sequence; obtaining a second abstract semantic graph according to the text sequence; obtaining a first representation vector of the first abstract semantic graph, and a second representation vector of the second abstract semantic graph; obtaining a text generation loss according to the first representation vector; calculating an information loss according to the first representation vector and the second representation vector; and updating the parameters of the translation model according to the text generation loss and the information loss. According to the embodiments of the present disclosure, the structural information of the abstract semantic graph can be well preserved, the global attention of the abstract semantic graph can be obtained, and the Smatch score model obtained by calculating the text generation loss and the information loss can be accurately evaluated and updated to improve the quality of text generation.

Description

Technical Field

[0001] The present disclosure relates to the technical field of machine translation, and more specifically, to a training method, device, electronic device, and computer-readable storage medium for a translation model. Background Art

[0002] An Abstract Meaning Representation (AMR) graph is a single-rooted directed graph that represents the meaning of a sentence. Nodes represent entities summarized by the sentence, and edges represent semantic relationships between entities. The AMR graph-to-text generation task involves translating an AMR graph into the natural language it represents. Because AMR is a widely used intermediate form of semantic representation, AMR graph-to-text generation has garnered increasing attention in recent years.

[0003] A drawback of existing techniques is that most require AMR graph nodes to be aligned with words in the original sentence, which is typically impossible to achieve using the Gold Standard Form and can only be achieved through automatic generation. This can propagate errors during the generation of the grammar. To better understand the structural information in the AMR graph, a graph-based model was proposed. This iteratively computes a sequence representation of each node based on its neighbors. However, the local propagation nature of this approach hinders the understanding of global information, affecting the quality of text generation.

[0004] Furthermore, for training evaluation, existing methods mostly use BLEU and METEOR scores, which compare the differences between the output text and the ground-truth text. Because the same AMR image can often be expressed in many different ways, while the dataset only provides a limited number of reference sentences for the same AMR image, even if the model outputs a sentence that accurately reflects the meaning of the AMR image, the differences between it and the reference sentences in the dataset can lead to incorrect feedback using BLEU and METEOR scores. Summary of the Invention

[0005] One purpose of the embodiments of the present disclosure is to provide a new technical solution for training a translation model.

[0006] According to a first aspect of the present disclosure, a method for training a translation model is provided, the method comprising:

[0007] Obtaining a first abstract semantic graph and a corresponding text sequence;

[0008] Obtaining a second abstract semantic graph according to the text sequence;

[0009] Obtaining a first representation vector of the first abstract semantic graph and a second representation vector of the second abstract semantic graph;

[0010] Obtaining a text generation loss based on the first representation vector;

[0011] Calculating information loss according to the first characterization vector and the second characterization vector;

[0012] Parameters of the translation model are updated according to the text generation loss and the information loss.

[0013] Optionally, the acquiring the first abstract semantic graph includes:

[0014] Obtain the abstract semantics to be translated;

[0015] The abstract semantics to be translated is represented as the first abstract semantic graph; wherein the first abstract semantic graph includes a node set, an edge set, and a relationship type set between two nodes.

[0016] Optionally, obtaining a first representation vector of the first abstract semantic graph includes:

[0017] Traversing the first abstract semantic graph using a depth-first algorithm to obtain an abstract semantic sequence;

[0018] Based on the abstract semantic sequence, obtaining a first entity vector representation and a first relationship connection vector representation of each node in the first abstract semantic graph;

[0019] Converting the first abstract semantic graph into an unlabeled connected bipartite graph to obtain a second entity vector representation and a second relationship connection vector representation of each node in the connected bipartite graph;

[0020] The first entity vector representation, the first relationship connection vector representation, the second entity vector representation, and the second relationship connection vector representation are determined as a first representation vector of the first abstract semantic graph.

[0021] Optionally, obtaining a text generation loss according to the first representation vector includes:

[0022] generating a context encoding for each node in the first abstract semantic graph based on a first representation vector of the first abstract semantic graph;

[0023] Based on the context encoding, the text generation loss is calculated.

[0024] Optionally, the generating of the context encoding of each node in the first abstract semantic graph based on the first representation vector of the first abstract semantic graph includes:

[0025] Calculating a path code for each path in the first abstract semantic graph based on the first representation vector of the first abstract semantic graph; the path code is used to represent a connection relationship between two connected nodes;

[0026] Based on the path encoding, the attention weight between each two connected nodes is calculated;

[0027] Based on the attention weights between each two connected nodes, the context encoding of each node is calculated.

[0028] Optionally, the calculating of the text generation loss based on the context encoding includes:

[0029] Calculating a hidden state for each node based on the context encoding;

[0030] Calculating a context vector for each node based on the hidden state of each node;

[0031] Calculating a probability of copying the input based on the hidden state of the node and the corresponding context vector;

[0032] Based on the probability of the copied input, a probability distribution of abstract semantic text generation is calculated;

[0033] The text generation loss is calculated based on the probability distribution of the abstract semantic text generation.

[0034] Optionally, the calculating information loss according to the first characterization vector and the second characterization vector includes:

[0035] The first characterization vector and the second characterization vector are calculated using a Smatch score algorithm to obtain the information loss.

[0036] Optionally, the updating of the parameters of the translation model according to the text generation loss and the information loss includes:

[0037] The parameters of the translation model are updated according to a result of weighted summation of the text generation loss and the information loss.

[0038] According to a second aspect of the present disclosure, a translation model training device is provided, the device comprising:

[0039] An acquisition module is configured to acquire a first abstract semantic graph and a corresponding text sequence; obtain a second abstract semantic graph based on the text sequence; and obtain a first representation vector of the first abstract semantic graph and a second representation vector of the second abstract semantic graph.

[0040] A generation module, configured to obtain a text generation loss based on the first representation vector;

[0041] a calculation module, configured to calculate information loss based on the first characterization vector and the second characterization vector;

[0042] An updating module is used to update parameters of the translation model according to the text generation loss and the information loss.

[0043] According to a third aspect of an embodiment of the present disclosure, an electronic device is also provided, comprising a processor and a memory; the memory stores machine-executable instructions that can be executed by the processor; the processor executes the machine-executable instructions to implement the training method of the translation model described in any one of the first aspects of the embodiments of the present disclosure.

[0044] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is further provided, storing executable instructions. When the executable instructions are executed by a processor, the training method of the translation model as described in any one of the first aspects of the embodiment of the present disclosure is performed.

[0045] According to one embodiment of the present disclosure, the structural information of the abstract semantic graph can be well preserved, the global attention of the abstract semantic graph can be obtained, and the Smatch score model obtained by calculating the text generation loss and the information loss can be accurately evaluated and updated to improve the text generation quality.

[0046] Other features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0048] Figure 1 Schematic diagram of the structure of an electronic device to which the training method of the translation model according to an embodiment of the present disclosure can be applied;

[0049] Figure 2 is a flowchart of a method for training a translation model according to an embodiment of the present disclosure;

[0050] Figure 3 is a schematic diagram of an abstract semantic graph according to an embodiment of the present disclosure;

[0051] Figure 4 Yes Figure 3 Schematic diagram of the connected bipartite graph after the abstract semantic graph is converted;

[0052] Figure 5is a structural diagram of a training device for a translation model according to an embodiment of the present disclosure;

[0053] Figure 6 A functional block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0054] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0055] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0056] Technologies, methods and equipment known to persons of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, such technologies, methods and equipment should be considered part of the specification.

[0057] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0058] It should be noted that like reference numerals and letters refer to like items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0059] <Hardware Configuration>

[0060] The AMR image-to-text generation task involves translating an AMR image into the natural language it represents. Because AMR is a widely used intermediate form of semantic representation, AMR image-to-text generation has received increasing attention in recent years. This is a challenging task, with the following difficulties: Providing corresponding natural language annotations for AMR images is costly, resulting in limited training data; the parsing process of AMR images loses a significant amount of information about the original sentences; and capturing complex structural information in AMR images is difficult. Currently, most neural network-based AMR image-to-text generation methods focus primarily on the nodes of the AMR image, while the relationships and relationship labels between nodes in the AMR image are also extremely important.

[0061] To address the shortcomings of existing methods in understanding the structural information of AMR graphs and the relationships between entity nodes, this paper proposes a graph path perception model based on reinforcement learning. First, the AMR graph is preprocessed to generate a dual representation of the AMR graph as the input of the model. Then, based on the standard Transformer model, the encoder is extended to calculate the path encoding of all entities. This adds graph path information to the calculation of the multi-head attention mechanism, gaining attention to the global structure of the AMR graph. The multi-head attention mechanism and copy mechanism are added to the output of the decoder.

[0062] During the evaluation process of model training, a reinforcement learning method was adopted. The AMR parsing module was used to re-parse the output sentences of the present disclosure into AMR graphs, and the same preprocessing method was used to generate a dual representation vector. The Smatch score was calculated for this representation and the AMR graph dual representation as input, and the loss function was calculated to obtain an accurate evaluation.

[0063] Figure 1 The figure is a schematic diagram of the composition structure of an electronic device to which the training method of the translation model according to the embodiment of the present disclosure can be applied.

[0064] like Figure 1 As shown, the electronic device 1000 of this embodiment may include a processor 1010, a memory 1020, an interface device 1030, a communication device 1040, a display device 1050, an input device 1060, a speaker 1070, a microphone 1080, and the like.

[0065] The processor 1010 may be a central processing unit (CPU), a microprocessor (MCU), or the like. The memory 1020 may include, for example, a ROM (read-only memory), a RAM (random access memory), or a non-volatile memory such as a hard disk. The interface device 1030 may include, for example, a USB interface, a headphone jack, or the like. The communication device 1040 may be capable of wired or wireless communication. The display device 1050 may be, for example, a liquid crystal display (LCD) or a touch screen display. The input device 1060 may include, for example, a touch screen or a keyboard.

[0066] The electronic device 1000 may output audio information through the speaker 1070. The electronic device 1000 may pick up voice information input by the user through the microphone 1080.

[0067] The electronic device 1000 may be a portable computer, a desktop computer, or the like.

[0068] In this embodiment, the electronic device 1000 can obtain a first abstract semantic graph and a corresponding text sequence; obtain a second abstract semantic graph based on the text sequence; obtain a first representation vector of the first abstract semantic graph and a second representation vector of the second abstract semantic graph; obtain a text generation loss based on the first representation vector; calculate the information loss based on the first representation vector and the second representation vector; and update the parameters of the translation model based on the text generation loss and the information loss.

[0069] In this embodiment, the memory 1020 of the electronic device 1000 is used to store instructions, which are used to control the processor 1010 to operate to support the training method of the translation model according to any embodiment of this specification.

[0070] It should be understood by those skilled in the art that although Figure 1 Multiple devices of the electronic device 1000 are shown in the figure, but the electronic device 1000 of the embodiment of this specification may only involve some of the devices, for example, only the processor 1010, the memory 1020, the display device 1050, the input device 1060, etc.

[0071] A person skilled in the art can design instructions according to the scheme disclosed in this disclosure. How instructions control the operation of a processor is well known in the art and will not be described in detail here.

[0072] <Method>

[0073] This embodiment provides a method for training a translation model. The method can be implemented by an electronic device, such as Figure 1 The electronic device 1000 is shown.

[0074] like Figure 2 As shown, the method includes the following steps 2100 to 2800:

[0075] Step 2100: Obtain a first abstract semantic graph and a corresponding text sequence.

[0076] Specifically, the electronic device 1000 may obtain abstract semantics to be translated, and represent the abstract semantics to be translated as the first abstract semantic graph; wherein the first abstract semantic graph includes a node set, an edge set, and a relationship type set between two nodes.

[0077] The first abstract semantic graph is a directed graph G = (V, E, R) with a root node, where V = v1, v2, ..., v n , represents the set of all nodes (including the root node) on the graph; E represents the edge set; R = r1, r2, ..., r m, represents a set of relationship types. n and m represent the number of nodes and edges in the abstract semantic graph. Each edge in the edge set can be represented by a triple, e = (i, label, j), where i and j ∈ V represent the nodes at the ends of the edge, and label ∈ R represents the type of relationship between the two nodes.

[0078] We view the task of translating abstract semantics into text as finding a parameter set θ G , for a given abstract semantic graph a, the model f can be used to predict the sentence but It can be expressed as Among them, S represents a parameter set θ G The sentence predicted by the model based on the abstract semantic graph a.

[0079] Step 2200: Obtain a second abstract semantic graph according to the text sequence.

[0080] Step 2300: Obtain a first representation vector of the first abstract semantic graph and a second representation vector of the second abstract semantic graph.

[0081] In this step, first, remove the wiki links and variables in the abstract semantic graph, and use the depth-first algorithm to traverse the first abstract semantic graph to obtain an abstract semantic sequence. In the example shown in Table 1, the abstract semantic sequence is:

[0082] (and:op1(specialize:ARG0(country:name(name:op1China):mod(modern)):mod(just)):op2(possible:domain(do:ARG 0(we):AGR 1(nothing)))).

[0083] Based on the abstract semantic sequence, the abstract semantic graph can be further converted into an entity sequence and a triple representing the relationship between entities. Each entity corresponds to a node in the abstract semantic graph, and the entity sequence can be expressed as v = v1, v2, ..., v n， v i ∈V, as shown in the third row of Table 1: and specialize country name china modern thisjust possibledo we nothing.

[0084] Each relation triple corresponds to an edge in the AMR graph, which can be expressed as e = e1, e2, ..., e m， e i∈E, as shown in the fourth row of Table 1: (0,1,:op1)(1,2,:AGR0)(2,3,:name)…(11,11,self). Through these two sequences, we obtain the first entity vector representation v and the first relationship connection vector representation e of each node in the first abstract semantic graph.

[0085] The first abstract semantic graph is converted into an unlabeled connected bipartite graph. Specifically, the label representing the relationship on each edge is regarded as a new entity node. These new entity nodes have connections with the original entity nodes in both forward and reverse directions, and each entity node also has a connection pointing to itself. Figure 3 and Figure 4 The example abstract semantic graph and the transformed connected bipartite graph are respectively represented, and the second entity vector representation and the second relationship connection vector representation of each node in the connected bipartite graph are obtained. Therefore, each node set in the connected bipartite graph can be represented as v'=v1', v2', ..., v n ',r1',…,r m ',v i '∈V', as shown in the fifth row of Table 1: and:op1 specialize:ARG0 country:name name:op1china Entities and relation nodes:AGR 1this:mod just:op2 possible:ARG1do:ARG0 we:ARG1 nothing. The new relation connection in the transformed abstract semantic graph can be expressed as e'=e1', e2', ..., e l ' ， r1'∈E', Here, direction indicates the number of directions of the connection.

[0086] We determine the first entity vector representation, the first relationship connection vector representation, the second entity vector representation, and the second relationship connection vector representation as the first representation vector G = (V, E, V', E') of the first abstract semantic graph.

[0087] Table 1

[0088]

[0089] Step 2400: Obtain text generation loss based on the first representation vector.

[0090] In this step, specifically, a context encoding of each node in the first abstract semantic graph may be generated based on the first representation vector of the first abstract semantic graph; and based on the context encoding, a text generation loss may be calculated.

[0091] Among them, when generating the context encoding of each node in the first abstract semantic graph based on the first representation vector of the first abstract semantic graph, a graph path perception model can be constructed on the basis of the linear Transformer model. The graph path perception model is composed of an encoder that introduces a multi-head attention mechanism and a decoder that adds a multi-head attention mechanism and a copy mechanism to the output part.

[0092] Specifically, the path-aware attention mechanism is used to implement the attention function and calculate the attention weight a between two nodes. ij , as an extension of the multi-head attention mechanism. The explicit relationship between two nodes is incorporated into their expression learning. In this step, based on the first representation vector of the first abstract semantic graph, the path encoding of each path in the first abstract semantic graph is calculated. The path encoding is used to represent the connection relationship between two connected nodes.

[0093] Specifically, each path in the abstract semantic graph is encoded to reflect the labels of the two entities it connects and the relationship between them. For each node v i ',v j 'The path between them is encoded as follows:

[0094] Among them, W Q , W K , W R Represents the parameter matrix. e' ij Node v i ',v j 'Relationship connection between. d z Indicates v' i W Q dimension.

[0095] Based on the path encoding, the attention weight α between each two connected nodes is calculated ij :

[0096]

[0097] Based on the attention weights between each connected node, the context encoding of each node is calculated. In the encoder, we implement a multi-head self-attention mechanism to obtain N independent attention weights, and then concatenate these independent attention weights to obtain the context-related encoding of each node in the graph:

[0098] in, Indicates connecting N attentions, | V |,| E| respectively represent the size of V and E in the input AMR dual representation, Represents node v i ' and v j 'The nth independent attention weight between, represents the parameter matrix.

[0099] In this embodiment, when calculating the text generation loss based on the context encoding, a graph attention network (GAT) framework is introduced into the model. By paying attention to the neighboring nodes of each point and following the self-attention strategy, the electronic device 1000 can calculate the hidden representation of each node in the graph based on the context encoding, as shown in the following formula: Where W represents the parameter matrix, represents the hidden state of the i-th node in the first layer, α i,i represents the attention weight of the i-th node to itself. n represents the number of nodes in V.

[0100] We added a multi-head attention mechanism and a copy mechanism to the output of the decoder. At each time step t in the decoding process, the context vector of each node is calculated based on the hidden state of each node. Among them, N represents N independent attentions, represents the nth attention of the jth node, represents the parameter matrix, represents the context-dependent encoding of the j-th point in V.

[0101] Then, based on the hidden state of the node and the corresponding context vector, the probability of copying the input is calculated as p=σ(W c [h t ||c t ]+b c ), where W c represents the parameter matrix, b c represents the parameter vector, σ represents the sigmoid activation function, and || represents h t and c t Connect them.

[0102] Based on the probability of the copied input, the probability distribution of abstract semantic text generation is calculated:

[0103] p GE =p×α copy +(1-p)×α vocab , where α copy represents the probability distribution of copy-generated words, α vocab Represents the probability distribution of directly generated words.

[0104] The text generation loss is calculated based on the probability distribution of the abstract semantic text generation. Among them, X represents the abstract semantic graph of the input, θ G Represents model parameters. Indicates that the parameter set is θ G , the input is X, and the context vector generated before t-1 is In the case of , the tth generated context vector is probability.

[0105] Step 2500: Calculate information loss based on the first characterization vector and the second characterization vector.

[0106] In this embodiment, a reinforcement learning approach is employed, introducing a parsing module to optimize the training process and improve the robustness of the model. Specifically, an AMR parsing module is used to evaluate the correctness of the model's output sequence. Specifically, the AMR parsing module considers AMR parsing as a process of converting a sequence into an AMR graph. This module is implemented using an attention-based model and consists of two components: node prediction via an extended pointer generation network and edge prediction via a deep biaffine classifier network.

[0107] Specifically, the trained parsing module is used to parse the text sequence generated by the model of the present invention, and the corresponding AMR image is predicted. The predicted AMR is then processed using the processing method in S2 to obtain the dual representation GH of the AMR. By comparing the true value G with the dual representation GH of the AMR corresponding to the sentence predicted by the present invention, the first representation vector and the second representation vector are calculated using the Smatch score algorithm to obtain the information loss. The calculation formula is:

[0108] Step 2600: Update the parameters of the translation model according to the text generation loss and the information loss.

[0109] The parameters of the translation model are updated according to the result of weighted summation of the text generation loss and the information loss. The specific formula is L=L GE +λL RE , where λ represents the weight parameter.

[0110] The technical solution of the disclosed embodiment comprises obtaining a first abstract semantic graph and a corresponding text sequence; obtaining a second abstract semantic graph based on the text sequence; obtaining a first representation vector of the first abstract semantic graph and a second representation vector of the second abstract semantic graph; obtaining a text generation loss based on the first representation vector; calculating an information loss based on the first and second representation vectors; and updating the parameters of the translation model based on the text generation loss and the information loss. A new point set is formed by adding the label on each edge in the abstract semantic graph as a new entity node to the original point set. In the new point set, each associated entity node has connections in both forward and reverse directions, as well as connections pointing back to itself. These connections form a new edge set. The point set and edge set of the converted abstract semantic graph are used as model inputs along with the point set and edge set of the original abstract semantic graph, effectively preserving the structural information of the abstract semantic graph. By calculating the path encodings of all nodes, graph path information is added to the calculation of the multi-head attention mechanism, achieving global attention of the abstract semantic graph. Simultaneously, the Smatch score model obtained from the calculation of the text generation loss and the information loss is accurately evaluated and updated to improve text generation quality.

[0111] <Device>

[0112] This embodiment provides a translation model training device, which is, for example, Figure 5 The training apparatus 5000 of the translation model is shown.

[0113] like Figure 5 As shown, the training device 5000 for the translation model may include an acquisition module 5100 , a generation module 5200 , a calculation module 5300 and an update module 5400 .

[0114] Among them, the acquisition module 5100 is used to obtain a first abstract semantic graph and a corresponding text sequence; obtain a second abstract semantic graph according to the text sequence; obtain a first representation vector of the first abstract semantic graph, and a second representation vector of the second abstract semantic graph.

[0115] The generation module 5200 is configured to obtain a text generation loss according to the first representation vector.

[0116] The calculation module 5300 is configured to calculate information loss according to the first characterization vector and the second characterization vector.

[0117] The updating module 5400 is configured to update the parameters of the translation model according to the text generation loss and the information loss.

[0118] Specifically, the acquisition module 5100 is specifically used to obtain abstract semantics to be translated; represent the abstract semantics to be translated as the first abstract semantic graph; wherein the first abstract semantic graph includes a node set, an edge set, and a relationship type set between two nodes.

[0119] When obtaining the first representation vector of the first abstract semantic graph, the processing module 5100 specifically uses a depth-first algorithm to traverse the first abstract semantic graph to obtain an abstract semantic sequence; based on the abstract semantic sequence, obtains a first entity vector representation and a first relationship connection vector representation of each node in the first abstract semantic graph; converts the first abstract semantic graph into an unlabeled connected bipartite graph to obtain a second entity vector representation and a second relationship connection vector representation of each node in the connected bipartite graph; and determines the first entity vector representation, the first relationship connection vector representation, the second entity vector representation, and the second relationship connection vector representation as the first representation vector of the first abstract semantic graph.

[0120] The generation module 5200 is specifically used to generate a context encoding of each node in the first abstract semantic graph based on the first representation vector of the first abstract semantic graph; and calculate the text generation loss based on the context encoding.

[0121] In which, the generation module 5200 can calculate the path encoding of each path in the first abstract semantic graph based on the first representation vector of the first abstract semantic graph; the path encoding is used to represent the connection relationship between two connected nodes; based on the path encoding, the attention weight between each two connected nodes is calculated; based on the attention weight between each two connected nodes, the context encoding of each node is calculated.

[0122] The calculation module 5300 is specifically used to calculate the hidden state of each node based on the context encoding; calculate the context vector of each node based on the hidden state of each node; calculate the probability of copying the input based on the hidden state of the node and the corresponding context vector; calculate the probability distribution of abstract semantic text generation based on the probability of copying the input; calculate the text generation loss based on the probability distribution of abstract semantic text generation.

[0123] Furthermore, the calculation module 5300 may also be configured to calculate the first characterization vector and the second characterization vector using an Smatch score algorithm to obtain the information loss.

[0124] The updating module 5400 is specifically configured to update the parameters of the translation model according to a result of weighted summation of the text generation loss and the information loss.

[0125] The translation model training device of this embodiment can be used to implement the method and technical solution of this embodiment. Its implementation principle and technical effects are similar and will not be described in detail here.

[0126] <device>

[0127] In this embodiment, an electronic device is further provided, which includes the training device 5000 of the translation model described in the device embodiment of the present disclosure; or, the electronic device is Figure 6 The electronic device 6000 shown includes a processor 6200 and a memory 6100 .

[0128] The memory 6100 stores machine-executable instructions that can be executed by the processor; the processor 6200 executes the machine-executable instructions to implement the training method of the translation model as described in any one of the embodiments.

[0129] <Computer-readable storage medium embodiment>

[0130] This embodiment provides a computer-readable storage medium, which stores executable commands. When the executable commands are executed by a processor, the method described in any method embodiment of the present disclosure is executed.

[0131] The present disclosure may be a system, method and / or computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0132] A computer-readable storage medium can be a tangible device that can hold and store instructions for use by an instruction execution device. A computer-readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination thereof. As used herein, a computer-readable storage medium is not to be construed as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through an electrical wire.

[0133] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in the computer-readable storage medium in each computing / processing device.

[0134] The computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, and conventional procedural programming languages ​​such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., utilizing an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions. The electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0135] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0136] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0137] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0138] The flowcharts and block diagrams in the accompanying drawings show the possible implementation architectures, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment or part of an instruction, and a module, program segment or part of an instruction contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that implementation by hardware, implementation by software, and implementation by a combination of software and hardware are all equivalent.

[0139] The embodiments of the present disclosure have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terms used herein are selected to best explain the principles of the embodiments, their practical applications, or technical improvements in the marketplace, or to enable other persons skilled in the art to understand the embodiments disclosed herein. The scope of the present disclosure is defined by the appended claims.

Claims

1. A method for training a translation model, the method comprising: Obtaining a first abstract semantic graph and a corresponding text sequence; Obtaining a second abstract semantic graph according to the text sequence; Obtaining a first representation vector of the first abstract semantic graph and a second representation vector of the second abstract semantic graph; Obtaining a text generation loss based on the first representation vector; Calculating information loss according to the first characterization vector and the second characterization vector; updating parameters of the translation model according to the text generation loss and the information loss; Among them, obtaining the first representation vector of the first abstract semantic graph includes: traversing the first abstract semantic graph using a depth-first algorithm to obtain an abstract semantic sequence; obtaining a first entity vector representation and a first relationship connection vector representation of each node in the first abstract semantic graph based on the abstract semantic sequence; converting the first abstract semantic graph into an unlabeled connected bipartite graph to obtain a second entity vector representation and a second relationship connection vector representation of each node in the connected bipartite graph; and determining the first entity vector representation, the first relationship connection vector representation, the second entity vector representation, and the second relationship connection vector representation as the first representation vector of the first abstract semantic graph.

2. The method according to claim 1, wherein The obtaining of the first abstract semantic graph includes: Obtain the abstract semantics to be translated; The abstract semantics to be translated is represented as the first abstract semantic graph; wherein the first abstract semantic graph includes a node set, an edge set, and a relationship type set between two nodes.

3. The method according to claim 1, wherein Obtaining a text generation loss according to the first representation vector includes: generating a context encoding for each node in the first abstract semantic graph based on a first representation vector of the first abstract semantic graph; Based on the context encoding, the text generation loss is calculated.

4. The method according to claim 3, wherein: The generating, based on the first representation vector of the first abstract semantic graph, a context encoding of each node in the first abstract semantic graph includes: Calculating a path code for each path in the first abstract semantic graph based on the first representation vector of the first abstract semantic graph; the path code is used to represent a connection relationship between two connected nodes; Based on the path encoding, the attention weight between each two connected nodes is calculated; Based on the attention weights between each two connected nodes, the context encoding of each node is calculated.

5. The method according to claim 4, wherein The text generation loss is calculated based on the context encoding, including: Calculating a hidden state for each node based on the context encoding; Calculating a context vector for each node based on the hidden state of each node; Calculating a probability of copying an input based on the hidden state of the node and the corresponding context vector; Based on the probability of the copied input, a probability distribution of abstract semantic text generation is calculated; The text generation loss is calculated based on the probability distribution of the abstract semantic text generation.

6. The method according to claim 5, wherein: The calculating information loss according to the first characterization vector and the second characterization vector includes: The first characterization vector and the second characterization vector are calculated using a Smatch score algorithm to obtain the information loss.

7. The method according to claim 6, wherein: The updating of the parameters of the translation model according to the text generation loss and the information loss includes: The parameters of the translation model are updated according to a result of weighted summation of the text generation loss and the information loss.

8. A translation model training device, comprising: An acquisition module, configured to acquire a first abstract semantic graph and a corresponding text sequence; Obtaining a second abstract semantic graph according to the text sequence; Obtaining a first representation vector of the first abstract semantic graph and a second representation vector of the second abstract semantic graph; A generation module, configured to obtain a text generation loss based on the first representation vector; a calculation module, configured to calculate information loss based on the first characterization vector and the second characterization vector; An updating module, configured to update parameters of the translation model according to the text generation loss and the information loss; The acquisition module is further configured to: traverse the first abstract semantic graph using a depth-first algorithm to obtain an abstract semantic sequence; and obtain a first entity vector representation and a first relationship connection vector representation of each node in the first abstract semantic graph based on the abstract semantic sequence; The first abstract semantic graph is converted into an unlabeled connected bipartite graph to obtain a second entity vector representation and a second relationship connection vector representation of each node in the connected bipartite graph; and the first entity vector representation, the first relationship connection vector representation, the second entity vector representation, and the second relationship connection vector representation are determined as a first representation vector of the first abstract semantic graph.

9. An electronic device, characterized in that: The invention comprises a processor and a memory; the memory stores machine-executable instructions that can be executed by the processor; the processor executes the machine-executable instructions to implement the training method of the translation model according to any one of claims 1 to 7.

10. A computer-readable storage medium storing executable instructions, wherein when the executable instructions are executed by a processor, the method for training a translation model according to any one of claims 1 to 7 is executed.

Images