Generating rich descriptor frameworks for text using graphs and structured neural encoders

Abstract

Techniques for converting Resource Description Framework (RDF) data into natural language text using a bidirectional graph convolutional neural network ("BGCNN"). Some embodiments perform RDF-to-text generation by learning a graph-enhanced structural neural encoder that includes: (a) a bidirectional graph-based meta-path encoder; (b) a bidirectional graph convolutional network encoder; and (c) a separate attention mechanism for combining the encoder and a decoder to translate RDF triples into natural language descriptions.

Description

BACKGROUND

[0001] The present invention relates generally to the field of converting rich descriptor framework (RDF) data to text data using machine logic (e.g., software).

[0002] The Wikipedia entry “Convolutional neural network” (as of June 16, 2020) states in part as follows: “In deep learning, a convolutional neural network (CNN or ConvNet) is a class of deep neural networks, most commonly applied to analyzing visual imagery. Based on their shared- weight architecture and shift-invariance features, they are also known as shift-invariant or spatially invariant artificial neural networks (SIANN). They have applications in natural language processing, and CNNs are regularized versions of multilayer perceptrons.

[0003] Multilayer perceptrons generally refer to fully connected networks, i.e., each neuron in one layer is connected to all neurons in the next layer. The“full connectivity” of these networks makes them prone to overfitting the data. Typical regularization methods include adding some form of weight magnitude measure to the loss function. CNNs take a different regularization approach: they exploit hierarchical patterns in the data and use smaller and simpler patterns to assemble more complex patterns. Thus, CNNs are at the lower end of the spectrum in terms of connectivity and complexity. The name“convolutional neural network” indicates that the network employs a mathematical operation called convolution. Convolution is a special linear operation. Convolutional networks are simple neural networks that use convolution in at least one of their layers instead of the general matrix multiplication. CNNs include an input layer and an output layer and multiple hidden layers. The hidden layers of a CNN typically include a series of convolutional layers that convolve with multiplication or other dot products. The activation function is typically a RELU layer followed by additional convolutions such as pooling layers, fully connected layers, and normalization layers, which are called hidden layers because their inputs and outputs are masked by activation functions and final convolutions. While these layers are colloquially called convolutions, this is only by convention. Mathematically, it is a sliding dot product or cross-correlation. This has important implications for the indices in the matrix because it affects how the weights are determined at a particular index point. Convolutional layers in neural networks should have the following properties: a convolution kernel defined by a width and a height (hyperparameters). The number of input channels and output channels (hyperparameters). The depth of the convolutional filter (input channels) must be equal to the number of channels (depth) of the input feature map. The weight vector and bias are called a filter and represent a particular feature (e.g., a particular shape) of the input. A notable feature of CNNs is that many neurons can share the same filter. This reduces the memory footprint because a single bias and a single weight vector are used across all receptive fields that share that filter, rather than each receptive field having its own bias and vector weight.(footnote omitted)

[0004] The Wikipedia entry for "resource description framework" (as of June 16, 2020) states in part as follows: "The Resource Description Framework (RDF) is a family of W3C specifications originally designed as a metadata data model. It has come to be used as a general method of conceptual description or modeling information resources, with various syntaxs and data serialization formats, and to build vocabularies to share and link these data using Web technologies. It is also used in knowledge management applications. The RDF data model is similar to classical conceptual modeling methods such as entity-relationship or class diagrams. It is based on the idea that information about resources (especially Web resources) is stated in the form of triples, so-called triples. The subject denotes a resource, the predicate denotes a property or aspect of the resource, and the object expresses a relationship between the subject and the object. For example, one way to express the concept "The sky has the color blue" in RDF is the triple: the subject denotes "the sky", the predicate denotes "has the color", and the object denotes "blue". Thus, RDF uses subjects rather than objects (or entities), as opposed to the typical approach of entity-attribute-value models in object-oriented design: entity (sky), attribute (color), and value (blue). RDF is an abstract model with several serializations (i.e., file formats), so the specific encoding of a resource or triple varies by format. A collection of RDF statements represents essentially a labeled, directed multigraph. RDF expresses information in a formal way that machines can understand. The purpose of RDF is to provide encoding and interpretation mechanisms so that resources can be described in a way that particular software can understand; in other words, so that software can access and use information it could not otherwise use." (footnotes omitted)

[0005] The Wikipedia entry for "bidirected graph" (as of June 16, 2020) states in part: "In the mathematical realm of graph theory, a bidirected graph is a graph in which each edge is given an independent orientation (or direction, or arrow) at each end. Therefore, there are three types of bidirected edges: those with arrows pointing outwards and both ends pointing to vertices; those with both arrows pointing inwards, away from vertices; and those with one arrow pointing away from its vertex to the opposite end, while the other arrow points in the same direction as the first arrow, away from the opposite end, to its own vertex. These three types of edges can be called extraverted, introverted, and directed, respectively. A 'directed' edge is the same as a regular directed edge in a directed graph; therefore, a directed graph is a special type of bidirected graph. Sometimes it is desirable for an edge to have only one end (half an edge); these have only one arrow. Edges without ends (loose edges) have no arrows. An edge that is neither a half nor a loose edge can be called a regular edge. Antisymmetric graphs are double-covered graphs of bidirected graphs." (Footnotes omitted)

[0006] Terminology Note: As is the current convention in this field, the terms “bidirected graph” and “bidirectional graph” refer to the same thing and are used interchangeably. Summary of the Invention

[0007] According to one aspect of the invention, there is a method, computer program product, and / or system capable of performing the following operations (not necessarily in the following order): (i) training a bidirectional graph convolutional neural network (BGCNN) using multiple training datasets; (ii) receiving a computer-readable Resource Description Frame (RDF) dataset comprising multiple Resource Description Frame (RDF) triples; (iii) creating a bidirectional graph dataset comprising bidirectional graphs by machine logic, the bidirectional graphs containing all the information of the multiple RDF triples; and (iv) translating the bidirectional graphs into a natural language text using BGCNN.

[0008] According to another aspect of the present invention, a computer-implemented method (CIM) is provided, comprising: training a bidirectional graph convolutional neural network (BGCNN) using multiple training datasets; receiving a computer-readable Resource Description Frame (RDF) dataset including multiple Resource Description Frame (RDF) triples; creating a bidirectional graph dataset including a bidirectional graph by machine logic, the bidirectional graph containing all the information of the multiple RDF triples; and translating the bidirectional graph into a natural language text using BGCNN.

[0009] Preferably, the present application provides a computer-implemented method, wherein the training comprises at least one of: supervised learning and / or unsupervised learning.

[0010] Preferably, the present application provides a computer-implemented method, wherein translating the bi-directional graph into a piece of natural language text comprises the sub-operation of encoding meta-paths associated with the bi-directional graph by a meta-path encoder module of the BGCNN.

[0011] Preferably, the present application provides a computer-implemented method, wherein translating the bi-directional graph into a piece of natural language text comprises the sub-operation of encoding the bi-directional graph by a convolutional network (CN) encoder portion of the BGCNN.

[0012] Preferably, the present application provides a computer-implemented method, wherein translating the bi-directional graph into a piece of natural language text comprises the sub-operation of combining the encoder and the decoder by an independent attention mechanism portion of the BGCNN to translate the RDF dataset into a piece of natural language text.

[0013] Preferably, the present application provides a computer-implemented method, wherein the BGCNN comprises: a meta-path encoder based on the bi-graph; a graph convolutional network encoder input; and a bi-graph convolutional network encoder.

[0014] According to another aspect of the present application, there is provided a computer program product (CPP) comprising: a set of storage devices; and computer code collectively stored in the set of storage devices, the computer code comprising data and instructions that cause a set of processors to perform at least the following: training a bi-directional graph convolutional neural network (BGCNN) using a plurality of training datasets, receiving a computer-readable resource description framework (RDF) dataset comprising a plurality of resource description framework (RDF) triples, creating, by machine logic, a bi-directional graph dataset comprising a bi-directional graph that contains all of the information of the plurality of RDF triples, and translating, using the BGCNN, the bi-directional graph into a piece of natural language text.

[0015] Preferably, the present application provides a computer program product, wherein the training comprises at least one of: supervised learning and / or unsupervised learning.

[0016] Preferably, the present application provides a computer program product, wherein translating the bi-directional graph into a piece of natural language text comprises the sub-operation of encoding meta-paths associated with the bi-directional graph by a meta-path encoder module of the BGCNN.

[0017] Preferably, the present application provides a computer program product, wherein translating the bi-directional graph into a piece of natural language text comprises the following sub-operations: encoding the bi-directional graph by a convolutional network (CN) encoder portion of the BGCNN.

[0018] Preferably, the present application provides a computer program product, wherein translating the bi-directional graph into a piece of natural language text comprises the following sub-operations: combining the encoder and the decoder by an independent attention mechanism portion of the BGCNN to translate the RDF dataset into a piece of natural language text.

[0019] Preferably, the present application provides a computer program product, wherein the BGCNN comprises: a meta-path encoder based on the bi-graph; a graph convolutional network encoder input; and a bi-graph convolutional network encoder.

[0020] According to another aspect of the present application, there is provided a computer system (CS) comprising: a processor group; a set of storage devices; and computer code collectively stored in the set of storage devices, the computer code comprising data and instructions that cause the processor group to perform at least the following: training a bi-directional graph convolutional neural network (BGCNN) using a plurality of training datasets, receiving a computer-readable resource description framework (RDF) dataset comprising a plurality of resource description framework (RDF) triples, creating, by machine logic, a bi-directional graph dataset comprising a bi-directional graph that contains all of the information of the plurality of RDF triples, and translating, using the BGCNN, the bi-directional graph into a piece of natural language text.

[0021] Preferably, the present application provides a computer system, wherein the training comprises at least one of: supervised learning and / or unsupervised learning.

[0022] Preferably, the present application provides a computer system, wherein translating the bi-directional graph into a piece of natural language text comprises the following sub-operations: encoding, by a meta-path encoder module of the BGCNN, a meta-path associated with the bi-directional graph.

[0023] Preferably, the present application provides a computer system, wherein translating the bi-directional graph into a piece of natural language text comprises the following sub-operations: encoding the bi-directional graph by a convolutional network (CN) encoder portion of the BGCNN.

[0024] Preferably, the present application provides a computer system, wherein translating the bi-directional graph into a piece of natural language text comprises the following sub-operations: combining the encoder and the decoder by an independent attention mechanism portion of the BGCNN to translate the RDF dataset into a piece of natural language text.

[0025] Preferably, the present application provides a computer system, wherein the BGCNN comprises: a bi-patter-based meta-path encoder; a graph convolution network encoder input; and a bi-patter convolution network encoder. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 is a block diagram of a first embodiment of a system according to the present application;

[0027] Figure 2 is a flowchart of a first embodiment method performed at least in part by the first embodiment system;

[0028] Figure 3 is a block diagram of a machine logic (e.g., software) portion of the first embodiment system;

[0029] Figure 4 is a screenshot view generated for the first embodiment system;

[0030] Figure 5A is a block diagram of a second embodiment of a system according to the present application;

[0031] Figure 5B is a block diagram of a portion of the second embodiment system;

[0032] Figure 5C is a block diagram of another portion of the second embodiment system;

[0033] Figure 5D is a block diagram of another portion of the second embodiment system; and

[0034] Figure 6 is a bi-patter graph generated for the second embodiment system. DETAILED DESCRIPTION

[0035] Some embodiments of the present application relate to converting RDF data to natural language text using a bi-patter graph convolutional neural network (“BGCNN”). Some embodiments perform RDF-to-text generation by learning a graph-enhanced structural neural encoder that includes: (a) a bi-patter-based meta-path encoder; (b) a bi-patter graph convolution network encoder, and (c) an independent attention mechanism for combining the encoder and a decoder to translate RDF triples into natural language descriptions.

[0036] This DETAILED DESCRIPTION section is divided into the following subsections: (i) Hardware and Software Environment; (ii) Example Embodiments; (iii) Further Comments and / or Embodiments; and (iv) Definitions.

[0037] I. Hardware and Software Environment

[0038] The present application can be a system, a method, and / or a computer program product at any possible technical detail of integration. A computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present application. The computer readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, semiconductor system, or any suitable combination of the foregoing. More specific examples (a non- exhaustive list) of the computer readable storage medium include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0039] Computer readable storage media can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0040] A "storage device" is defined herein as any thing that stores computer code in a manner that the computer code is accessible by a computer processor. A storage device typically includes a storage medium, which is the material that stores the computer code data. A single "storage device" can have (i) multiple discrete portions spaced apart or distributed (e.g., a set of six solid state storage devices, each located in a separate laptop computer, that collectively store a single computer program); and / or (ii) can use multiple storage media (e.g., a set of computer code that is stored partly in magnetic domains in the non-volatile memory of a computer, and partly in a set of semiconductor switches in the volatile memory of the computer). The term "storage medium" should be interpreted to cover use of multiple different types of storage media.

[0041] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions to a computer readable storage medium within the respective computing / processing device for storage and / or execution.

[0042] Computer readable program instructions for carrying out operations of the present application can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or any combination of one or more of the above, written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present application.

[0043] The computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions to a computer readable storage medium within the respective computing / processing device for storage and / or execution.

[0044] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including

[0045] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0046] The flow and block diagrams in the figures illustrate the architecture, functionality, and operations of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flow and block diagrams can represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical functions (‘instructions’). In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or combinations of special purpose hardware and computer instructions.

[0047] As shown in FIG. 1, the networked computer system 100 is an embodiment of a hardware and software environment for various embodiments of the present application. The networked computer system 100 includes: a server subsystem 102 (sometimes referred to herein simply as subsystem 102); client subsystems 104, 106, 108, 110, 112; and a communications network 114. The server subsystem 102 includes: a server computer 200; a communications unit 202; a processor set 204; an input / output (I / O) interface set 206; a memory 208; a persistent storage 210; a display 212; external device(s) 214; a random access memory (RAM) 230; a cache 232; and a program 300. Figure 1 As shown in FIG. 1, the networked computer system 100 is an embodiment of a hardware and software environment for various embodiments of the present application. The networked computer system 100 includes: a server subsystem 102 (sometimes referred to herein simply as subsystem 102); client subsystems 104, 106, 108, 110, 112; and a communications network 114. The server subsystem 102 includes: a server computer 200; a communications unit 202; a processor set 204; an input / output (I / O) interface set 206; a memory 208; a persistent storage 210; a display 212; external device(s) 214; a random access memory (RAM) 230; a cache 232; and a program 300.

[0048] Subsystem 102 can be a notebook computer, a tablet computer, a netbook computer, a personal computer (PC), a desktop computer, a personal digital assistant (PDA), a smartphone, or any other type of computer (see the definition of "computer" in the Definitions section below). Program 300 is a collection of machine-readable instructions and / or data used to create, manage, and control certain software functionality, which will be discussed in detail in the Example Embodiments section below.

[0049] Subsystem 102 is capable of communicating with other computer subsystems via communication network 114. Network 114 can be, for example, a local area network (LAN), a wide area network (WAN), such as the Internet, or a combination of the two, and can include wired, wireless, or fiber optic connections. In general, network 114 can be any combination of connections and protocols that will support communications between servers and client subsystems.

[0050] Subsystem 102 is shown as a block diagram with many double arrows. These double arrows (without separate reference numerals) represent a communication fabric that provides for communication between the various components of subsystem 102. This communication fabric can be implemented with any architecture designed to transfer data and / or control information between processors (such as microprocessors, communication and network processors, etc.), system memory, peripheral devices, and any other hardware components within a computer system. For example, the communication fabric can be implemented at least partially with one or more buses.

[0051] Memory 208 and persistent storage 210 are computer readable storage media. In general, memory 208 can include any suitable volatile or non-volatile computer readable storage media. Note also that, in some embodiments, external device 214 can be capable of providing some or all of memory for subsystem 102, and / or a device external to subsystem 102 can be capable of providing memory for subsystem 102. Memory 208 and persistent storage 210: (i) store data in a non-transitory manner; and (ii) store data on a tangible medium (e.g., a magnetic or optical domain). In this embodiment, memory 208 is a volatile memory and persistent storage 210 provides non-volatile storage. The media used by persistent storage 210 can also be removable. For example, a removable hard drive can be used for persistent storage 210. Other examples include optical and magnetic disks, thumb drives, and smart cards that are inserted into a drive, for transfer onto another computer readable storage medium, also a part of persistent storage 210.

[0052] The communication units 202 provide communications with other data processing systems or devices external to the subsystem 102. In these examples, the communication units 202 include one or more network interface cards. The communication units 202 can provide communications through the use of either or both physical and wireless communications links. Any of the software modules discussed herein can be downloaded to persistent storage (e.g., persistent storage 210) through a communication unit, such as the communication units 202.

[0053] The I / O interface set 206 allows for input and output of data with other devices that can be in local data communication with the server computer 200. For example, the I / O interface set 206 provides a connection to the external device set 214. The external device set 214 generally includes devices such as keyboards, keypads, touch screens, and / or some other suitable input devices. The external device set 214 can also include portable computer-readable storage media such as, for example, thumb drives, portable optical or magnetic disks, and memory cards. Software and data used to practice embodiments of the present application, such as the program 300, can be stored on such portable computer-readable storage media. The I / O interface set 206 also is in data communication with the display 212. The display 212 is a display device that provides a mechanism to show data to a user and can be, for example, a computer monitor or a smartphone display screen.

[0054] In this embodiment, the program 300 is stored in the persistent storage 210 for access and / or execution by one or more computer processors of the processor set 204, typically through one or more memories of the memory 208. Those skilled in the art will appreciate that the program 300 can be stored in a more highly distributed fashion at its runtime and / or when not running. The program 300 can include machine-readable and executable instructions and / or entity data (i.e., the type of data stored in a database). In this particular embodiment, the persistent storage 210 includes a magnetic hard disk drive. The persistent storage 210 can include a solid state disk drive, a semiconductor memory device, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, or any other computer-readable storage medium capable of storing program instructions or digital information, to name a few possible variations.

[0055] The programs described herein are identified based upon the application for which they are implemented in a particular embodiment of the present application. It should be understood, however, that any particular program nomenclature that is used herein is used solely for the purpose of facilitating this discussion and thus the present application should not be limited to usage by only the described program nomenclature.

[0056] The description of various embodiments of the application is merely intended to be illustrative and not exhaustive or limiting to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art, without departing from the scope of the described embodiments. The terminology used here is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The use of the terms "example" and "exemplary" is intended to be illustrative and not exhaustive or limiting to the disclosure.

[0057] II. Example Embodiments

[0058] As shown in Figure 1 , network computer system 100 is an environment in which example methods of the present application can be performed. As shown in Figure 2 , flowchart 250 illustrates an example method according to the present application. As shown in Figure 3 , program 300 performs or controls performance of at least some of the method operations of flowchart 250. The method and related software will now be discussed extensively in the following paragraphs with reference to blocks of Figure 1 , Figure 2 and Figure 3 .

[0059] The process begins with operation S255, in which RDF data module ("mod") 302 receives a computer-readable RDF data set, and includes information indicative of a set of RDF triples (or "triples"). As shown in region 402 of screenshot 400, Figure 4 , in this example there are three triples as follows: (i) water includes atom(s) of oxygen; (ii) water is a species of molecule; and (iii) water includes atom(s) of hydrogen. The purpose of the method of flowchart 250 is to generate, by machine logic, a natural language piece of text that accurately conveys all of the information of these triples.

[0060] The process proceeds to operation S260, in which bidirectional graph creation module 304 creates a bidirectional graph based on the RDF data set. This is illustrated by region 404 of screenshot 400. Figure 4 More specifically, the single bidirectional graph of region 404 is created by combining the three independent triadic graphs of region 402.

[0061] Processing proceeds to operation S265, where the training module 308 trains a bidirectional graph convolutional neural network (BGCNN) 320 using a training dataset (not shown separately). As shown, the BGCNN 320 includes a meta-path encoder 322, a convolutional network encoder 324, and an independent attention mechanism 326. In the following subsections of this DETAILED DESCRIPTION section, the BGCNN will be discussed, which also includes: (i) a bi-GMP (bidirectional graph-based meta-path) encoder; (ii) a GCN (graph convolutional network) encoder input; and (iii) a bi-GCN (bidirectional graph convolutional network) encoder. Figure 3

[0062] Processing proceeds to operation S270, where the translation module 310 translates the bidirectional graph (shown in area 404 of screenshot 400) into a piece of natural language text (shown in area 406 of screenshot 400) using the bidirectional graph CNN 320. More specifically, in this example, the translation operation includes the following three (3) sub-operations: (i) encoding the meta-path using the meta-path encoder 322 (this will be discussed in more detail in the following subsections of this DETAILED DESCRIPTION); (ii) encoding the bidirectional graph using the convolutional network (CN) encoder 324 (this will be discussed in more detail in the following subsections of this DETAILED DESCRIPTION); and (iii) combining the encoder and decoder using the independent attention mechanism 326 to translate the RDF triples into a natural language description (this will be discussed in more detail in the following subsections of this DETAILED DESCRIPTION).

[0063] Processing proceeds to operation S275, where the output module 312 outputs the natural language text obtained at operation S265. In this example, the text is sent as a text message over the communication network 114 to the client subsystem 104 (in this example, the client subsystem 104 is a smartphone of a chief chemist of the Advanced Chemistry Institute). Alternatively or additionally, the natural language text can be sent as computer-readable data to another computer device for storage and / or further processing.

[0064] III. FURTHER COMMENTS AND / OR EMBODIMENTS

[0065] Some embodiments of the present application recognize that natural language generation, business use can be applied in many domains, such as: machine translation sequences, abstracted summary sequences, image processing related domains, such as image captioning, table related domains, such as Wiki-infoboxes for biographical and sports report generation, and graph related domains, such as knowledge-based QA and entity summarization.

[0066] ​Some embodiments of the invention recognize one or more of the following facts, potential problems, and / or potential areas of improvement over the prior art: (i) a current conventional solution for converting RDF data to text data is referred to as GTR-LSTM1 (Graph-based TRiple-Long Short Term Memory); (ii) GTR-LSTM1 encodes global information by preserving the graph structure; (iii) GTR-LSTM1 fails to capture rich local complex structure information between entities and relations; (iv) another current conventional solution for converting RDF data to text data is referred to as Modified Graph Convolutional Network (DGCN)2; (v) Modified Graph Convolutional Network (DGCN)2 performs better in capturing local structure information; (vi) Modified Graph Convolutional Network (DGCN)2 is prone to overfitting when the number of GCN layers is greater than or equal to three (3); and / or (vii) Modified Graph Convolutional Network (DGCN)2 fails to capture long-range dependency (global) information among RDF triples.

[0067] Some embodiments of the invention can include one or more of the following operations, features, characteristics and / or advantages: (i) using a graph-enhanced structural neural encoder for RDF-to-text generation; (ii) combining a graph encoder plus a graph-based meta-path encoder; (iii) a graph structure; (iv) a bidirectional graph-based meta-path encoder; (v) a bidirectional graph convolutional network encoder; and / or (vi) a combined encoder and decoder.

[0068] Some embodiments of the invention can include one or more of the following operations, features, characteristics and / or advantages: (i) RDF-to-text generation by learning a graph-enhanced structural neural encoder; (ii) including operations performed by a bidirectional graph-based meta-path encoder; (iii) including operations performed by a bidirectional graph convolutional network encoder; and / or (iv) combining an encoder and a decoder with an independent attention mechanism for translating RDF triples into a corresponding natural language description, respectively.

[0069] According to embodiments of the invention, a method utilizes a computing device to generate a natural language description from resource description framework data. The method includes the following operations (not necessarily in the following order): (i) receiving, by the computing device, resource description framework data, the resource description framework data being represented as a directed graph; (ii) encoding, by the computing device, a bidirectional graph based on the resource description framework data; (iii) training, by the computing device, a bidirectional graph convolutional neural network; and (iv) translating, by the computing device, the resource description framework data into the natural language description utilizing the bidirectional graph convolutional neural network.

[0070] Some embodiments of the invention can include one or more of the following operations, features, characteristics and / or advantages: (i) jointly learn local and global structural information by machine learning via the combination of two new graph-enhanced structural neural encoders for input triples (i.e., a bi-directional graph encoder and a bi-directional graph-based meta-path encoder); (ii) the graph-based meta-path encoder captures global structural information; (iii) generate natural language expressions (meanings) faithfully from subgraphs of a knowledge graph; (iv) use a more advanced technique by jointly learning local and global structural information via the combination of two new graph-enhanced structural neural encoders; (v) perform RDF-to-text generation directly in a knowledge graph; and / or (vi) use an advanced deep learning-based approach that can be used for any system.

[0071] For purposes of this document, "RDF-to-text generation" is defined herein as: generating, by machine logic running on a set of computers, a descriptive text for a given set of RDF triples, where: (i) the input is computer code that includes information indicative of a set of RDF triples; and (ii) the output is computer code indicative of a piece of descriptive natural language text.

[0072] An example input set of triples is as follows:

[0073]

[0074]

[0075] An example output of a piece of descriptive natural language text produced by processing the input described in the preceding paragraph is as follows: "The school of business and social sciences at the Aarhus University in Denmark is affiliated with the European University Association. The European University Association is headquartered in Brussels. Brussels is the capital of Belgium. Denmark has a monarch; its religion is the church of Denmark; and its leader is Lars Lokke Rasmussen".

[0076] Formulas for RDF-to-text generation will now be discussed. A set of RDF triples is mathematically represented as follows: S = {t_1, t_(2),... t_n} where t_i = <s_i, r_i, O_i>. S is represented as a directed graph, mathematically expressed as follows: G = <V, E>, and the true target text is mathematically represented as follows: y = {w_1, w_(2),... w_T}. In performing the RDF-to-text generation task, given a set of RDF triples S or its corresponding graph G, a machine logic running on a computer generates the descriptive text Y^.

[0077] Challenges in RDF-to-text generation and motivation for RDF-to-text generation will now be discussed. In some embodiments, the structural information formed in a set of RDF triples S is better utilized by using a reasonable model with the following properties: (i) accurately predict local entities and their attributes (within triples / local); and (ii) completely cover global input information (between triples / global). Some embodiments combine graph-enhanced structural neural encoders (specifically, a graph encoder and a graph-based meta-path encoder) to jointly learn local and global structural information. As shown in Figures 5A to 5D Graph 500 (including meta-path input block 502, RDF triples block 504, GCN (graph convolutional network) encoder input 506, dual GMP encoder 512, dual GCN encoder 514, and decoder 530) represents the framework of the combined graph-enhanced structural neural encoder model, as shown.

[0078] Graph construction process will now be discussed in accordance with embodiments of the present invention. Entity masking improves the generalization capability of the model. In the example: (i) “Bakewell pudding” (FOOD-1) leads to “ENTITY-1 FOOD”: and (ii) “Bakewell tart” (FOOD-2) leads to “entity-2 FOOD”. “Relation” is treated as an additional node. The received RDF triples are converted into a graph data structure. In this example, the dual GCN input of (FOOD-1, region, PLACE) leads to region being associated with FOOD-1, and region is also associated with PLACE. In this example, topological sorting and single-source shortest path analysis are performed on the dual GMP input to produce the following three results: (i) FOOD1 -> region -> PLACE -> leaderName -> PERSON; (ii) FOOD-1 -> region -> PLACE -> county -> COUNTY; and (iii) FOOD-1 -> dishVariation -> FOOD-2 -> ingredient -> INGREDIENT. The graph jointly constructed by the dual GCN and dual GMP encoders is shown in Figure 6is shown in the graph 600.

[0079] In some embodiments, the bidirectional graph-based meta-path encoder has one or more of the following properties: (i) the input graph G1 is transformed into a sequence of a set of meta-paths (ii) the representation r_i of each token in each meta-path is computed according to mathematical expression (1) (see below); (iii) the hidden state masking between different meta-paths; (iv) r_(i-1) is either an all-zero vector or the previous time step hidden state in the meta-path; (v) f(·) and g(·) are single LSTM (Long Short-Term Memory) units; (vi) a copy mechanism is introduced; (vii) the output: a set of representations R_1 = {r_1, r_2...}; and / or (viii) the graph embedding of the graph G1 is given by mathematical expression (2) (see below).

[0080] Mathematical expressions (1) and (2) (see above) are as follows:

[0081]

[0082] (Expression 1)

[0083] (Expression 2)

[0084] In some embodiments, the bidirectional graph-based convolutional network encoder has one or more of the following properties: (i) the vector representation at layer l is (see mathematical expression (3) below); (ii) mathematical expression (4) (see below) represents the source-to-target and target-to-source adjacency matrices of the directed graph G2 with inserted self-loops given by mathematical expression (5) (see below); (iii) I is the identity matrix given by mathematical expression (6); (iv) the output: a set of entity and relation node representations; (v) the graph embedding of the graph G2 is shown by mathematical expression (7) (see below); and / or is a single layer perceptron.

[0085] Mathematical expressions (3) to (7) (see above) are as follows:

[0086]

[0087]

[0088] (Expression 3)

[0089] (Expression 4)

[0090] (R2= H) (L)

[0091] (R2= H)

[0092] (R2= H)

[0093] Now the combined encoder and decoder will be discussed: the combined graph embedding is represented by mathematical expression (8):

[0094] (R2= H)

[0095] At time step t, R_1 and R_2 are concerned respectively, given by the following mathematical expressions (9) and (10):

[0096] (R2= H)

[0097]

[0098] (R2= H)

[0099] The dual GMP-level context vector c_u and the dual GCN-level context vector c_v are given by the following mathematical expression (11):

[0100] (R2= H)

[0101] The final attention hidden state is given by the following mathematical expression (12):

[0102] (R2= H)

[0103] The objective function is given by the following mathematical expression (13):

[0104] (R2= H)

[0105] IV. DEFINITIONS

[0106] The present invention: should not be taken as an absolute indication of what is covered by the claims as filed or the claims as ultimately issued following prosecution of the patent application; while the term "the invention" is used to help the reader get a general sense of what is disclosed herein that is believed to be potentially new, such understanding as indicated by the use of the term "the invention" is tentative and provisional and subject to change during the patent application process as relevant information develops and the claims are likely to be modified.

[0107] EMBODIMENT: See the definition of "INVENTION" above - similar caveats apply to the term "EMBODIMENT".

[0108] AND / OR: inclusive or; for example, A, B "AND / OR" C means at least one of A or B or C is true and applicable.

[0109] INCLUDES / CONTAINS / COMPRISING: means "including but not necessarily limited to", unless explicitly specified otherwise.

[0110] MODULE / SUB-MODULE: any group of hardware, firmware, and / or software that is operable to function to achieve a certain functionality, regardless of whether the module is: (i) in a single local proximity; (ii) distributed across a wide area; (iii) in a single proximity within a larger piece of software code; (iv) located in a single piece of software code; (v) located in a single storage device, memory, or medium; (vi) mechanically connected; (vii) electrically connected; and / or (viii) connected in data communication.

[0111] COMPUTER: any device having significant data processing and / or machine-readable instruction reading capability, including but not limited to: desktop computers, mainframe computers, laptop computers, field programmable gate array (FPGA)-based devices, smartphones, personal digital assistants (PDAs), body-worn or embedded computers, embedded device-type computers, application-specific integrated circuit (ASIC)-based devices.

Claims

1. A computer-implemented method, comprising: The bidirectional graph convolutional neural network BGCNN was trained using multiple training datasets. Receives a computer-readable Resource Description Framework (RDF) dataset that corresponds to multiple Resource Description Framework (RDF) triples; A bidirectional graph dataset is created by machine logic, comprising a bidirectional graph containing all the information of the plurality of RDF triples; and The bidirectional graph was translated into a natural language text using BGCNN.

2. The method according to claim 1, wherein, The training includes at least one of the following: supervised learning and / or unsupervised learning.

3. The method according to claim 1, wherein, Translating the bidirectional graph into a natural language text segment includes the following sub-operations: The metapaths associated with the bidirectional graph are encoded by the metapath encoder module of BGCNN.

4. The method according to claim 3, wherein, Translating the bidirectional graph into a natural language text segment includes the following sub-operations: The bidirectional graph is encoded by the encoder portion of the convolutional network in BGCNN.

5. The method according to claim 4, wherein, Translating the bidirectional graph into a natural language text segment includes the following sub-operations: The encoder and decoder are partially combined by the independent attention mechanism of BGCNN to translate the RDF dataset into the natural language text.

6. The method according to claim 1, wherein, BGCNN includes: A dual-graphics-based meta-path encoder; Input to the graph convolutional network encoder; and Dual-graphic convolutional network encoder.

7. A computer program product, comprising: A set of storage devices; as well as Computer code co-located in the set of storage devices, the computer code including data and instructions that cause the processor group to perform at least the following operations: The bidirectional graph convolutional neural network BGCNN was trained using multiple training datasets. Receives a computer-readable RDF dataset containing multiple RDF triples. A bidirectional graph dataset, including bidirectional graphs, is created by machine logic. The bidirectional graphs contain all the information from the plurality of RDF triples, and... The bidirectional graph was translated into a natural language text using BGCNN.

8. The computer program product according to claim 7, wherein, The training includes at least one of the following: supervised learning and / or unsupervised learning.

9. The computer program product according to claim 7, wherein, Translating the bidirectional graph into a natural language text segment includes the following sub-operations: The metapaths associated with the bidirectional graph are encoded by the metapath encoder module of BGCNN.

10. The computer program product according to claim 9, wherein, Translating the bidirectional graph into a natural language text segment includes the following sub-operations: The bidirectional graph is encoded by the encoder portion of the convolutional network in BGCNN.

11. The computer program product according to claim 10, wherein, Translating the bidirectional graph into a natural language text segment includes the following sub-operations: The encoder and decoder are partially combined by the independent attention mechanism of BGCNN to translate the RDF dataset into the natural language text.

12. The computer program product according to claim 7, wherein, BGCNN includes: A dual-graphics-based meta-path encoder; Input to the graph convolutional network encoder; and Dual-graphic convolutional network encoder.

13. A computer system, comprising: Processor group; A set of storage devices; as well as Computer code co-located in the set of storage devices, the computer code including data and instructions that cause the processor group to perform at least the following operations: The bidirectional graph convolutional neural network BGCNN was trained using multiple training datasets. Receives a computer-readable RDF dataset containing multiple RDF triples. A bidirectional graph dataset, including bidirectional graphs, is created by machine logic. The bidirectional graphs contain all the information from the plurality of RDF triples, and... The bidirectional graph was translated into a natural language text using BGCNN.

14. The computer system according to claim 13, wherein, The training includes at least one of the following: supervised learning and / or unsupervised learning.

15. The computer system according to claim 13, wherein, Translating the bidirectional graph into a natural language text segment includes the following sub-operations: The metapaths associated with the bidirectional graph are encoded by the metapath encoder module of BGCNN.

16. The computer system according to claim 15, wherein, Translating the bidirectional graph into a natural language text segment includes the following sub-operations: The bidirectional graph is encoded by the encoder portion of the convolutional network in BGCNN.

17. The computer system according to claim 16, wherein, Translating the bidirectional graph into a natural language text segment includes the following sub-operations: The encoder and decoder are partially combined by the independent attention mechanism of BGCNN to translate the RDF dataset into a piece of natural language text.

18. The computer system according to claim 13, wherein, BGCNN includes: A dual-graphics-based meta-path encoder; Input to the graph convolutional network encoder; and Dual-graphic convolutional network encoder.

Images