Hypernym Detection Using Strict Partial Order Networks

Abstract

One embodiment of the present invention provides a method, including receiving a text corpus, and generating a first list of triples based on the text corpus. Each triple of the first list includes a first word representing a candidate hyponym, a second word representing a candidate hypernym, and a frequency value indicating the number of times a hyponym-hypernym relationship is observed between the candidate hyponym and the candidate hypernym in the text corpus. The method also includes training a neural network for hypernym induction based on the first list. The trained neural network is a strictly partial order network (SPON) model.

Description

Technical Field

[0001] The present invention generally relates to the field of hypernym induction, and more particularly, to a method and system for hypernym induction based on neural network.

[0002] invention

[0003] One embodiment of the present invention provides a method, including receiving a text corpus, and generating a first list of triples based on the text corpus. Each triple of the first list includes a first term representing a candidate hyponym, a second term representing a candidate hypernym, and a frequency value indicating the number of times a hypernym relationship is observed between the candidate hyponym and the candidate hypernym in the text corpus. The method also includes training a neural network based on the first list for hypernym induction. The trained neural network is a strictly partial order network (SPON) model.

[0004] Another embodiment of the present invention provides a system, including at least one processor and a non-transitory processor-readable memory device storing instructions, which, when executed by the at least one processor, causes the at least one processor to perform operations. The operations include receiving a text corpus, and generating a first list of triples based on the text corpus. Each triple of the first list includes a first word representing a candidate hyponym, a second word representing a candidate hypernym, and a frequency value indicating the number of times a hypernym relationship is observed between a candidate hyponym and a candidate hypernym in the text corpus. The operations also include training a neural network based on the first list for hypernym induction. The trained neural network is a SPON model.

[0005] One embodiment of the present invention provides a computer program product, including a computer-readable storage medium having program instructions embodied therein. The program instructions are executable by a processor to cause the processor to receive a text corpus and generate a first list of triples based on the text corpus. Each triple of the first list includes a first word representing a candidate hyponym, a second word representing a candidate hypernym, and a frequency value indicating the number of times a hypernym relationship is observed between a candidate hyponym and a candidate hypernym in the text corpus. The program instructions are further executable by the processor to cause the processor to train a neural network based on the first list for hypernym induction. The trained neural network is a SPON model.

[0006] These and other aspects, features and advantages of the present invention will be understood with reference to the accompanying drawings and detailed description herein, and will be realized by means of the various elements and combinations particularly pointed out in the appended claims. It should be understood that the above general description and the following brief description of the drawings and detailed description of the present invention are exemplary and illustrative preferred embodiments of the present invention, rather than limitations of the invention as claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The subject matter which is regarded as the present invention is particularly pointed out and distinctly claimed in the claims at the conclusion of the specification. The foregoing and other objects, features and advantages of the present invention will become apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:

[0008] Figure 1 An example computing architecture for implementing neural network-based hypernym induction according to an embodiment of the present invention is shown;

[0009] Figure 2 An example of a hypernym induction system based on a neural network according to an embodiment of the present invention is shown;

[0010] Figure 3 An exemplary strictly partial order network (SPON) model according to an embodiment of the present invention is shown;

[0011] Figure 4 is a flowchart of an example process for neural network-based hypernym induction according to an embodiment of the present invention; and

[0012] Figure 5 is a high-level block diagram illustrating an information processing system for implementing an embodiment of the present invention.

[0013] The detailed description explains the preferred embodiments of the invention, together with advantages and features, by way of example with reference to the drawings. DETAILED DESCRIPTION

[0014] The present invention generally relates to the field of hypernym induction, and more specifically, to a method and system for hypernym induction based on a neural network. One embodiment of the present invention provides a method, including receiving a text corpus, and generating a first list of triples based on the text corpus. Each triple of the first list includes a first word representing a candidate hyponym, a second word representing a candidate hypernym, and a frequency value indicating the number of times a hypernym relationship is observed between a candidate hyponym and a candidate hypernym in the text corpus. The method also includes training a neural network based on the first list for hypernym induction. The trained neural network is a strictly partial order network (SPON) model.

[0015] Another embodiment of the present invention provides a system, including at least one processor and a non-transitory processor-readable memory device storing instructions, which when executed by the at least one processor causes the at least one processor to perform operations. The operations include receiving a text corpus, and generating a first list of triples based on the text corpus. Each triple of the first list includes a first word representing a candidate hyponym, a second word representing a candidate hypernym, and a frequency value indicating the number of times a hypernym relationship is observed between a candidate hyponym and a candidate hypernym in the text corpus. The operations also include training a neural network based on the first list for hypernym induction. The trained neural network is a SPON model.

[0016] One embodiment of the present invention provides a computer program product, including a computer-readable storage medium having program instructions embodied therein. The program instructions are executable by a processor to cause the processor to receive a text corpus, and to generate a first list of triples based on the text corpus. Each triple of the first list includes a first word representing a candidate hyponym, a second word representing a candidate hypernym, and a frequency value indicating the number of times a hypernym relationship is observed between a candidate hyponym and a candidate hypernym in the text corpus. The program instructions are further executable by the processor to cause the processor to train a neural network based on the first list for hypernym induction. The trained neural network is a SPON model.

[0017] A hyponym is a term that represents an abstract concept such as a type or class. A hypernym is a term that represents a more general abstract concept such as a type or class than the hyponym.

[0018] Named entities (NEs) are proper nouns. Available NE types are limited and predefined, ranging from tens to hundreds at most. By comparison, hypernyms are broader than NE types.

[0019] Hyponym and hyponym are the building blocks for taxonomy construction. Hyponym and hyponym relationships (i.e., relations) include is-a relationships between hyponyms and hypernyms (i.e., they are relationships between hyponyms and hypernyms). For example, potato pancakes are a pancake is a hyponym relationship between the hyponym potato pancakes and the hypernym pancakes.

[0020] The hierarchical relationship in the classification satisfies the following properties: reflexivity, transitivity, and asymmetry (i.e., asymmetry). For example, if Wittgenstein is a philosopher, and a philosopher is a person, then by transitivity we get that Wittgenstein is a person. Similarly, a philosopher is not Wittgenstein, but by asymmetry we get that a person is not a philosopher.

[0021] For the purpose of explanation, the term "hyponym-hypernym pair" generally refers to a pair of words including a hyponym and a hypernym that exhibit a hyponymous relationship.

[0022] For the purposes of this description, the term "hypernym detection" generally refers to the task of classifying or predicting whether a given pair of words (x, y) (eg, (robin, passerine)) has a hyponymous relationship.

[0023] For purposes of this description, the term "hypernym targeting" generally refers to the task of identifying or predicting which word of a given pair of words (x, y) is the hypernym (ie, which of (x, y) or (y, x) is the hyponym).

[0024] For purposes of this description, the term "hypernym discovery" generally refers to the task of predicting, for a given hyponym x, ranked predictions of the hypernyms of the given hyponym x.

[0025] The ability to generalize the meaning of domain-specific terms is essential for many natural language processing (NLP) applications. However, manually constructing a taxonomy for a new domain is time-consuming. This drives the need to develop automatic systems that can identify hyponyms and hyponyms from text.

[0026] Traditionally, supervised machine learning systems can be trained to predict hyponymy if there is sufficient training data. However, training data may not be equally available across all domains.

[0027] Embodiments of the present invention provide a method and system for automatically extracting hyponym-hypernym pairs from unstructured free text using a deep neural network model that learns how to encode hyponym and hyponym relationships. Embodiments of the present invention can be used to perform tasks such as, but not limited to, hypernym detection, hypernym orientation, and hypernym discovery. Each task can be performed in an unsupervised or supervised manner.

[0028] In order theory, a strict partial order is a binary relation that is transitive, non-reflexive, and asymmetric. A strict partial order relation corresponds more directly to a directed acyclic graph (DAG).

[0029] Embodiments of the present invention provide a kind of SPON, this SPON is designed to learn the hyponym-hypernym pair of hyponym relation in representation taxonomy and model it.SPON is the neural network framework including non-negative activation and residual connection, and it is designed to enforce the asymmetry and transitivity requirements of strict partial order relation as soft constraint.One embodiment provides the SPON model that is trained to enforce the asymmetry and transitivity requirements of strict partial order relation as soft constraint.In the learning process, adopt hyponym-hypernym pair to train SPON model.In one embodiment, the enhanced variant of SPON model can summarize the type information learned for the word in vocabulary to previously unseen information, to identify / predict the hyponym relation of out-of-vocabulary (OOV) word (that is, SPON model does not observe / see word during the learning process); The enhanced variant uses the normalized distribution similarity value as weight to complete this in the probability model.

[0030] Figure 1 An example computing architecture 100 for implementing neural network-based hypernym induction according to an embodiment of the present invention is shown. In one embodiment, the computing architecture 100 is a centralized computing architecture. In another embodiment, the computing architecture 100 is a distributed computing architecture.

[0031] In one embodiment, the computing architecture 100 includes computing resources, such as but not limited to one or more processor units 110 and one or more storage units 120. One or more applications utilize the computing resources of the computing architecture 100 to execute / operate on the computing architecture 100. In one embodiment, the applications on the computing architecture 100 include but are not limited to a neural network-based hypernym induction system 200, which is configured to automatically extract hyponym-hypernym pairs from a text corpus (e.g., unstructured free text) using a deep neural network that learns to encode hyponym-hypernym relationships. For example, in one embodiment, the neural network-based hypernym induction system 200 is used to automatically extract hyponym-hypernym pairs from a large unannotated text corpus.

[0032] As described in detail later herein, in one embodiment, the neural network utilized by the neural network-based hypernym induction system 200 is the SPON model 250 ( Figure 2 ), which is trained to enforce the asymmetry and transitivity requirements of strict partial order relations as soft constraints in the context of hypernym induction.

[0033] In one embodiment, the neural network-based hypernym induction system 200 is configured to exchange data with one or more electronic devices 400 and / or one or more remote server devices 20 via a connection (e.g., a wireless connection such as a Wi-Fi connection or a cellular data connection, a wired connection, or a combination of both).

[0034] In one embodiment, the electronic device 400 includes one or more computing resources, such as but not limited to one or more processor units 410 and one or more storage units 420. One or more applications are executed / operated on the electronic device 400 using one or more computing resources of the electronic device 400, such as but not limited to one or more software applications 450 loaded or downloaded to the electronic device 400. Examples of software applications 450 include, but are not limited to, artificial intelligence (AI) applications, NLP applications.

[0035] Examples of electronic device 400 include, but are not limited to, desktop computers, mobile electronic devices (e.g., tablet computers, smart phones, laptop computers, etc.), wearable devices (e.g., smart watches, etc.), Internet of Things (IoT) devices, smart appliances such as smart TVs, etc.

[0036] In one embodiment, the electronic device 400 includes one or more input / output (I / O) units 430 integrated in or coupled to the electronic device 400, such as a keyboard, a keypad, a touch interface, a display screen, etc. The user utilizes the I / O module 430 of the electronic device 400 to configure one or more user preferences, configure one or more parameters (e.g., hyperparameters for the SPON model 250, etc.), provide input (e.g., manually labeled training data, such as a list of manually verified relationships), etc.

[0037] In one embodiment, the electronic device 400 and / or the remote server device 20 is a source of training data and / or a text corpus.

[0038] In one embodiment, the neural network-based hypernym induction system 200 is accessed or used by one or more online services (e.g., AI services, NLP services) hosted on a remote server device 20 and / or one or more software applications 450 (e.g., AI applications, NLP applications) running on an electronic device 400. For example, in one embodiment, a virtual assistant, a search engine, or another type of software application 450 running on an electronic device 400 calls the system 200 to perform hypernym detection, hypernym orientation, and / or hypernym discovery to generate broader or narrower search terms, answer questions by finding more general categories, build taxonomies and knowledge bases, etc.

[0039] Figure 2An example of a neural network-based hypernym induction system 200 according to an embodiment of the present invention is shown. In one embodiment, the system 200 includes a linguistic model-based unit 210, which is configured to: (1) receive a text corpus (e.g., unstructured free text) as input, and (2) generate a list of candidate hyponym-hypernym pairs based on the text corpus. It is generally assumed that C represents the text corpus received as input.

[0040] In one embodiment, the linguistic pattern-based unit 210 is configured to generate a list of candidate hyponym-hypernym pairs by applying one or more linguistic patterns to a text corpus C to extract one or more candidate hyponym-hypernym pairs, wherein the list includes one or more extracted candidate hyponym-hypernym pairs. For example, in one embodiment, the one or more linguistic patterns applied include hearst-like patterns. In another embodiment, the linguistic pattern-based unit 210 is configured to generate a list of candidate hyponym-hypernym pairs based on a manually labeled training set (e.g., a list of manually verified is-a relationships) provided to the system 200.

[0041] In one embodiment, the unit 210 based on linguistic patterns is configured to remove the candidate hyponym-hypernym pairs (if any) that may be wrong (i.e., incorrect or false) from the list of candidate hyponym-hypernym pairs based on one or more statistical significance and indirect evidence. For example, in one embodiment, if the candidate hypernym (of the candidate hyponym-hypernym pair) included in the list of candidate hyponym-hypernym pairs is a subword (rather than a vocabulary head) of several other candidate hypernyms included in the list, the unit 210 based on linguistic patterns marks / identifies the candidate hypernym and several other candidate hypernyms as being possibly wrong so as to be removed. As another example, if "X is-a Y" and "Y is-a Z" are hypernym relations verified, then "Z is-not-a X" is determined by asymmetry; the candidate hyponym-hypernym pairs representing "Z is-a X" are marked / identified as being possibly wrong so as to be removed by the unit 210 based on linguistic patterns.

[0042] In one embodiment, system 200 includes a filtering and expansion unit 220 configured to: (1) receive a list of candidate hyponym-hypernym pairs (e.g., from linguistic pattern based unit 210), and (2) improve the quality of the received list by applying one or more filtering and / or expansion techniques to the received list to filter and / or expand the received list.

[0043] In one embodiment, the filtering and expansion unit 220 automatically filters the list of candidate hyponym-hypernym pairs based on heuristics to remove one or more candidate hyponym-hypernym pairs that may be erroneous (if there are candidate hyponym-hypernym pairs that may be erroneous). For example, in one embodiment, the filtering and expansion unit 220 filters the list based on a heuristic defined as follows: if the candidate hypernym of the candidate hyponym-hypernym pair contained in the list is a proper noun, then the pair is filtered out (i.e., removed) from the list because the pair is erroneous (e.g., the candidate hyponym-hypernym pair (x, North America) is filtered out). As another example, in one embodiment, the filtering and expansion unit 220 filters the list based on a heuristic defined as follows: if there is a sequence of words x1, x2, and x3 such that (x1, x2), (x2, x3), and (x3, x1) are candidate hyponym-hypernym pairs of the list, then one of these pairs is filtered out from the list to interrupt the loop formed between the words.

[0044] In one embodiment, the filtering and expansion unit 220 automatically expands the list of candidate hyponym-hypernym pairs by utilizing superwords and subwords to include one or more additional candidate hyponym-hypernym pairs. The one or more additional candidate hyponym-hypernym pairs are not extracted from the text corpus C and are not included in the manually labeled training set. For example, (Riverdale Bank, Bank) is a candidate hyponym-hypernym pair obtained from utilizing the words Riverdale Bank and Bank, where Riverdale Bank is a superword of Bank and Bank is a subword of Riverdale Bank.

[0045] In one embodiment, the system 200 includes an initialization unit 230 configured to control how the SPON model 250 is initialized. The initialization unit 230 is configured to: (1) receive a list of candidate hyponym-hypernym pairs (e.g., from the filtering and expansion unit 220 or the linguistic pattern-based unit 210), and (2) generate a list of candidate hyponym-hypernym-count triples, wherein each triple includes a candidate hyponym, a candidate hypernym, and a frequency (i.e., a count) indicating the number of times a hyponym-hypernym relationship is observed between the candidate hyponym and the candidate hypernym in the text corpus C. For example, (Ireland, Country, 795) and (Store, Public Area, 78) are some examples of triples extracted from the corpus.

[0046] In one embodiment, the initialization unit 230 is configured to encode each word (i.e., candidate hyponyms, candidate hypernyms) of the list of candidate hyponym-hypernym-count triples based on pre-initialized word / character embeddings or word / character embeddings trained from a text corpus C, so that the resulting list includes an initial (i.e., initialized) vector for each word of the list. For example, if each word of the list is encoded based on pre-initialized word / character embeddings, the SPON model 250 is initialized using an external pre-trained vector. As another example, if each word of the list is encoded based on word / character embeddings trained from a text corpus C, the SPON model 250 is initialized using the embedding from the text corpus C.

[0047] In one embodiment, during a learning process (i.e., a learning or training period / stage), the system 200 is configured to train the SPON model 250 based on training data including is-a relationships. Generally, it represents the training data used to train the SPON model 250. In one embodiment, the training data Including candidate hyponym-hypernym pairs. For example, in one embodiment, the system 200 uses a list of candidate hyponym-hypernym-count triples (e.g., from the initialization unit 230) as training data is fed to the SPON model 250. As described in detail later herein, during the learning process, the SPON model 250 is trained for a loss function that satisfies the asymmetry and transitivity requirements of strict partial order relations as soft constraints.

[0048] The trained SPON model 250 obtained from the learning process is configured to summarize the hyponymy. In one embodiment, during deployment / in the stage, the trained SPON model 250 is used to perform tasks, such as but not limited to hypernym detection, hypernym orientation and hypernym discovery. For example, in one embodiment, the trained SPON model 250 is applied to whether the unseen word pair (that is, unseen word pair in training data) of classification / prediction is a hyponymy.

[0049] Figure 3 An example SPON model 250 according to an embodiment of the present invention is shown. In one embodiment, the SPON model 250 is a neural network architecture trained to enforce the asymmetry and transitivity requirements of strict partial order relations as soft constraints.

[0050] Assume that ε generally represents the vocabulary of words (i.e., (the set of all words in the candidate hyponym-hypernym pairs observed / seen in ), and assume that φ generally denotes a triple in the list of candidate hyponym-hypernym-count triples, where φ = (x, y, s), x∈ε, y∈ε, and s is the frequency (i.e., count) of the candidate hyponym-hypernym pair (x, y), indicating the number of times the hyponymy relationship between x and y is observed.

[0051] In one embodiment, during the learning process, the training data is used The derived positive examples (i.e., positive instances) and negative examples (i.e., negative instances) are used to train the SPON model 250. Assume Generally speaking, in the training data The set of all hypernyms observed / seen in . Each positive instance consists of a pair of words in Is the training data A set of candidate hypernyms for a given candidate hyponym x in . Each negation instance consists of a pair of words in Is the training data In one embodiment, the system 200 is configured to generate negative instances of a given candidate hyponym x (i.e., from Uniformly select words at random, where the selected words appear to be hypernyms, but are not hypernyms of a given candidate hyponym x).

[0052] Assumptions represents the input vector, where And each word x∈ε is passed through the input vector The SPON model 250 includes a multiplication unit 260, which is configured to perform an input vector Element-wise multiplication with the weight vector w1 to obtain the weighted vector in is an element-by-element multiplication. The SPON model 250 includes a Add to the bias vector b1 to obtain a biased and weighted vector The adding unit 265.

[0053] Assume that f and g represent functions, where as well as In one embodiment, functions f and g satisfy two constraints: First, functions f and g satisfy the constraint that function g is a nonlinear non-negative activation function. In one embodiment, the SPON model 250 includes a rectified linear unit (ReLU) layer 270, which is configured to satisfy the constraint by implementing function g as a smooth function represented by equation (1) provided below:

[0054]

[0055] Second, functions f and g satisfy the constraints expressed by equation (2) provided below:

[0056]

[0057] In one embodiment, the SPON model 250 includes an addition unit 280 configured to add from the ReLU layer 270 according to equation (2) Add to input vector To obtain

[0058] The two constraints on the functions f and g ensure that component-wise f(x) ≥ x. General Assumptions represents the input vector, where And each word y∈ε is passed through the input vector During the learning process, the SPON model 250 is trained to ensure that each component The appropriate model parameters θ in terms of components are used to model the hierarchical relationship “xis-a y”. The condition is imposed as a soft constraint, and the system 200 ensures that violations of this constraint (encoded via the loss function) are minimized as much as possible.

[0059] General Assumptions represents the input vector, where And each word z∈ε is passed through the input vector If "x is-a y" means component by component If the model parameters θ are learned in such a way that θ=0.05 (or vice versa) (i.e., the violation of the inequality is zero), the SPON model 250 satisfies the asymmetry and transitivity requirements of the strict partial order relation.

[0060] For example, if "x is-a y" and "y is-a z", then the component-by-component and component by component It can be clearly seen from equation (1) that for all have The expression Applying to equation (2) yields The expression of Therefore, in fact Therefore, component by component Or in other words “x is-a z.” This demonstrates the transfer property of the SPON model 250.

[0061] As another example, if "x is-a y", then it follows component-wise that Using the previous observation, for all There is a component-by-component From these two points of view, each component Or in other words, “y is-not-a x.” This demonstrates the asymmetry of the SPON model 250.

[0062] Assume that ψ(x,y) denotes the loss value that captures the satisfactory distance of a candidate hyponym-hypernym pair (x,y) (i.e., satisfies the inequality component by component In one embodiment, to ensure that the condition is violated as little as possible during the learning process, the components are The SPON model 250 includes a loss layer 290, which is configured to calculate a loss value according to equation (3) provided below:

[0063]

[0064] Wherein, the sum is taken over all components of the participating dimension d, and ∈ is a scalar hyperparameter. In one embodiment, the SPON model 250 is configured to calculate and minimize the cross entropy loss of each candidate hyponym-hypernym pair (x, y), wherein the cross entropy loss is weighted, and the weights of the factors in the cross entropy loss are calculated as a function of the frequency parameter.

[0065] For tasks that require automatic induction of hyponymy and hyponymy from a text corpus C, applying linguistic patterns such as acoustically similar patterns to the text corpus C may not be exhaustive. Some applications have a practical requirement to assign OOV hyponyms outside the vocabulary to their most likely correct types. In one embodiment, the system 200 meets this requirement through hypernym discovery, thereby allowing hyponymy and hyponymy to be created from the text corpus C, and avoiding the sparsity problem that usually characterizes most knowledge bases. In one embodiment, the system 200 implements hypernym discovery by utilizing a distribution similarity measure between words in the same text corpus C.

[0066] For example, in one embodiment providing an augmented variant of the SPON model 250, during the learning process, the SPON model 250 is trained to learn The embeddings of words / characters (i.e., terms) that appear in . The general assumption is In one embodiment, the SPON model 250 is configured to generate The top k ranked hypernyms of each hyponym in , where k is a positive integer. Generally, assume that Y x Indicates the training data A random variable for the distribution of hypernyms of OOV hyponyms x that are not seen in , where Y x Get training data The value in the set of known hypernyms in The SPON model 250 is configured to calculate the random variable Y according to equation (4) provided below: x Value Probability:

[0067]

[0068] where Y x ⊥x|h (i.e., given h, Y x is independent of x), m is a scoring function that provides a score between [0, 1] (i.e., m is a (normalized) similarity function, where ), Contains p words from the ε that provide the top p maximum values ​​of the scoring function m, and p is a hyperparameter. In one embodiment, the SPON model 250 first uses a softmax operation to normalize the value of m(h|x) before calculating the weighted sum according to equation (4), where

[0069] Table 1 below provides an example of ranked predictions (from left to right) of hypernyms produced by the SPON model 250 performing hypernym discovery on a set of eight randomly selected test queries from the English dataset.

[0070] Table 1

[0071]

[0072] As shown in Table 1, the top four query terms are OOV hyponyms, while the bottom four query terms are in-vocabulary hyponyms. Predicted hypernyms that match the expected hypernyms are bold, and predicted hypernyms that are not included in the expected hypernyms are underlined.

[0073] Figure 45 is a flowchart of an example process 500 for hypernym induction based on a neural network according to an embodiment of the present invention. Process block 501 includes receiving a text corpus. Process block 502 includes generating a first list of triples based on the text corpus, wherein each triple of the first list includes a first word representing a candidate hyponym, a second word representing a candidate hypernym, and a frequency value indicating the number of times a hyponym relationship is observed between a candidate hyponym and a candidate hypernym in the text corpus. Process block 503 includes training a neural network for hypernym induction based on the first list, wherein the trained neural network is a strictly partial order network (SPON) model. Process block 504 includes applying the SPON model to perform a task (e.g., hypernym detection, hypernym orientation, or hypernym discovery).

[0074] In one embodiment, processing blocks 501 - 504 are performed by one or more components of system 200 .

[0075] Figure 5 is a high-level block diagram illustrating an information handling system 300 for implementing one embodiment of the present invention. The computer system includes one or more processors, such as processor 302. Processor 302 is connected to a communication fabric 304 (eg, a communication bus, crossbar, or network).

[0076] The computer system may include a display interface 306 that forwards graphics, text, and other data from the voice communication architecture 304 (or from a frame buffer not shown) for display on a display unit 308. In one embodiment, the computer system also includes a main memory 310, preferably a random access memory (RAM), and also includes an auxiliary memory 312. In one embodiment, the auxiliary memory 312 includes, for example, a hard disk drive 314 and / or a removable storage drive 316, which represents, for example, a floppy disk drive, a tape drive, or an optical drive. The removable storage drive 316 reads from and / or writes to a removable storage unit 318 in a manner known to those of ordinary skill in the art. The removable storage unit 318 represents, for example, a floppy disk, a compact disk, a magnetic tape, or an optical disk, etc., which is read and written by the removable storage drive 316. As will be appreciated, the removable storage unit 318 includes a computer-readable medium having computer software and / or data stored therein.

[0077] In alternative embodiments, secondary storage 312 includes other similar components for allowing computer programs or other instructions to be loaded into the computer system. Such components include, for example, a removable storage unit 320 and an interface 322. Examples of such components include program packaging and packaging interfaces (such as found in video game devices), removable memory chips (such as EPROM or PROM) and associated slots, and other removable storage units 320 and interfaces 322 that allow software and data to be transferred from the removable storage unit 320 to the computer system.

[0078] In one embodiment, the computer system also includes a communication interface 324. The communication interface 324 allows software and data to be transferred between the computer system and external devices. In one embodiment, examples of the communication interface 324 include a modem, a network interface (e.g., an Ethernet card), a communication port or PCMCIA slot and card, etc. In one embodiment, the software and data transmitted via the communication interface 324 are in the form of signals, such as electronic, electromagnetic, optical, or other signals that can be received by the communication interface 324. These signals are provided to the communication interface 324 via a communication path (i.e., channel) 326. In one embodiment, the communication path 326 carries signals and is implemented using wires or cables, optical fibers, telephone lines, cellular phone links, RF links, and / or other communication channels.

[0079] The present invention may be a system, method and / or computer program product at any possible level of integration of technical details. The computer program product may include a computer-readable storage medium (or multiple media) having computer-readable program instructions thereon, and the computer-readable program instructions are used to cause a processor to perform various aspects of the present invention.

[0080] A computer-readable storage medium may be a tangible device capable of retaining and storing instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device such as a punch card or a raised structure in a groove on which instructions are recorded, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium should not be interpreted as a temporary signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (e.g., a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0081] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in a computer-readable storage medium within the corresponding computing / processing device.

[0082] The computer-readable program instructions for performing the operation of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, configuration data of an integrated circuit, or source code or object code written in any combination of one or more programming languages ​​(including object-oriented programming languages, such as Smalltalk, C++, etc.) and process programming languages ​​(such as "C" programming language or similar programming languages). The computer-readable program instructions may be executed completely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider through the Internet). In some embodiments, in order to perform various aspects of the present invention, an electronic circuit including, for example, a programmable logic circuit, a field programmable gate array (FPGA) or a programmable logic array (PLA) may execute a computer-readable program instruction to personalize the electronic circuit by utilizing the state information of the computer-readable program instructions.

[0083] Various aspects of the present invention are described herein with reference to the flow chart and / or block diagram of the method, device (system) and computer program product according to embodiments of the present invention. It will be understood that each frame of the flow chart and / or block diagram and the combination of frames in the flow chart and / or block diagram can be implemented by computer-readable program instructions.

[0084] These computer-readable program instructions can be provided to a processor of a computer or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device create components for implementing the functions / actions specified in one or more boxes of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, which can guide the computer, programmable data processing device and / or other equipment to work in a specific manner, so that the computer-readable storage medium having the instructions stored therein includes an article of manufacture, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0085] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, so that a series of operational steps are performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable apparatus, or other device implement the functions / actions specified in one or more boxes of the flowchart and / or block diagram.

[0086] The flow chart and block diagram in the accompanying drawings show the architecture, function and operation of the possible implementation of the system, method and computer program product according to various embodiments of the present invention.In this regard, each frame in the flow chart or block diagram can represent the module, segment or part of the instruction, which includes one or more executable instructions for realizing the specified logical function.In some alternative embodiments, the function noted in the frame may not occur in the order noted in the figure.For example, two frames performed while being shown continuously, substantially at the same time, in a mode of partial or full time overlap can actually be realized as a step, or these frames can sometimes be performed in reverse order, depending on the function involved.It will also be noted that the combination of the frames in each frame of the block diagram and / or flow chart illustration and the block diagram and / or flow chart illustration can be realized by a dedicated hardware-based system that performs a specified function or action or performs a combination of special hardware and computer instructions.

[0087] From the above description, it can be seen that the embodiments of the present invention provide systems, computer program products and methods for implementing the embodiments of the present invention. The embodiments of the present invention also provide non-transitory computer-usable storage media for implementing the embodiments of the present invention. The non-transitory computer-usable storage medium has a computer-readable program, wherein the program causes the computer to implement the steps of the embodiments of the present invention described herein when it is processed on the computer. Unless explicitly stated in this way, the reference to the singular element in the claim is not intended to mean "one and only one", but to mean "one or more". All structural and functional equivalents of the elements of the above-mentioned exemplary embodiments that are currently known or will be known to ordinary technicians in the field are intended to be covered by this claim. The claim elements of this article should not be interpreted according to the provisions of the sixth paragraph of Section 112 of Chapter 35 of the United States Code unless the element is explicitly stated using the phrase "components for..." or "steps for..."

[0088] The terms used herein are intended only to describe specific embodiments of the present invention and are not intended to be limiting. As used herein, the singular forms "a", "an", and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise. It will also be understood that the terms "comprises" and / or "comprising" when used in this specification specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0089] The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the claims below are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed.

[0090] The description of various embodiments of the present invention has been given for the purpose of illustration, but it is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein are selected to best explain the principles of the embodiments, practical applications, or technical improvements existing in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method comprising: receiving a text corpus; generating a first list of triples based on the text corpus, wherein each triple of the first list includes a first word representing a candidate hyponym, a second word representing a candidate hypernym, and a frequency value indicating the number of times a hyponymy relationship is observed between the candidate hyponym and the candidate hypernym in the text corpus; training a neural network for hypernym induction via machine learning based on the first list, wherein the trained neural network is a strict partial order network (SPON), the SPON comprising activation and connection layers that enforce asymmetry and transitivity requirements of strict partial order relations as soft constraints; as well as The trained neural network is applied to perform at least one of hypernym detection, hypernym orientation, and hypernym discovery.

2. The method of claim 1 , wherein generating a first list of triples based on the text corpus comprises: extracting one or more candidate hyponym-hypernym pairs from the text corpus by applying a linguistic model to the text corpus, wherein each candidate hyponym-hypernym pair comprises a pair of words representing a candidate hyponym and a candidate hypernym; generating a second list comprising each candidate hyponym-hypernym pair extracted from the text corpus; applying at least one of a filtering technique and an expansion technique to the second list; generating the first list based on the second list, wherein the first list includes each candidate hyponym-hypernym pair of the second list and a corresponding frequency value indicating the number of times a hyponym-hypernym relationship is observed between a candidate hyponym and a candidate hypernym of the candidate hyponym-hypernym pair; as well as Each word of the first list is encoded.

3. A method according to claim 2, wherein applying at least one of a filtering technique and an expansion technique to the second list includes one or more of the following: filtering the second list to remove at least one erroneous candidate hyponym-hypernym pair, and expanding the second list to include at least one candidate hyponym-hypernym pair that was not extracted from the text corpus. The method according to claim 2 , wherein the linguistic pattern is an auditory similarity pattern.

5. The method according to claim 1, wherein the training comprises: The neural network is trained against a loss function that enforces the asymmetry and transitivity requirements of the strict partial order relation as the soft constraints.

6. The method according to claim 5, wherein the training further comprises: The neural network is trained to learn embeddings for words that appear in the first list.

7. The method according to claim 6, further comprising: Ranked predictions of hypernyms of out-of-vocabulary hyponyms are generated via the SPON by utilizing a distributional similarity measure between the words.

8. A system comprising: at least one processor; as well as a non-transitory processor-readable memory device storing instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising: receiving a text corpus; generating a first list of triples based on the text corpus, wherein each triple of the first list includes a first word representing a candidate hyponym, a second word representing a candidate hypernym, and a frequency value indicating the number of times a hyponymy relationship is observed between the candidate hyponym and the candidate hypernym in the text corpus; training a neural network for hypernym induction via machine learning based on the first list, wherein the trained neural network is a strict partial order network (SPON), the SPON comprising activation and connected layers that enforce asymmetry and transitivity requirements of strict partial order relations as soft constraints; as well as The trained neural network is applied to perform at least one of hypernym detection, hypernym orientation, and hypernym discovery.

9. The system of claim 8, wherein generating a first list of triples based on the text corpus comprises: extracting one or more candidate hyponym-hypernym pairs from the text corpus by applying a linguistic model to the text corpus, wherein each candidate hyponym-hypernym pair comprises a pair of words representing a candidate hyponym and a candidate hypernym; generating a second list comprising each candidate hyponym-hypernym pair extracted from the text corpus; applying at least one of a filtering technique and an expansion technique to the second list; generating the first list based on the second list, wherein the first list includes each candidate hyponym-hypernym pair of the second list and a corresponding frequency value indicating the number of times a hyponym-hypernym relationship is observed between a candidate hyponym and a candidate hypernym of the candidate hyponym-hypernym pair; as well as Each word of the first list is encoded.

10. The system of claim 9, wherein applying at least one of a filtering technique and an expansion technique to the second list comprises one or more of: filtering the second list to remove at least one erroneous candidate hyponym-hypernym pair, and expanding the second list to include at least one candidate hyponym-hypernym pair that was not extracted from the text corpus.

11. The system of claim 9, wherein the linguistic pattern is an auditory similarity pattern.

12. The system of claim 8, wherein the training comprises: The neural network is trained against a loss function that enforces the asymmetry and transitivity requirements of the strict partial order relation as the soft constraints.

13. The system of claim 12, wherein the training further comprises: The neural network is trained to learn embeddings for words that appear in the first list.

14. The system of claim 13, wherein the operations further comprise: Ranked predictions of hypernyms of out-of-vocabulary hyponyms are generated via the SPON by utilizing a distributional similarity measure between the words.

15. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions being executable by a processor to cause the processor to: receiving a text corpus; generating a first list of triples based on the text corpus, wherein each triple of the first list includes a first word representing a candidate hyponym, a second word representing a candidate hypernym, and a frequency value indicating the number of times a hyponymy relationship is observed between the candidate hyponym and the candidate hypernym in the text corpus; A neural network for hypernym induction is trained via machine learning based on the first list, wherein: The trained neural network is a strictly partial order network (SPON), comprising activation and connected layers that enforce asymmetry and transitivity requirements of a strict partial order relation as soft constraints; as well as The trained neural network is applied to perform at least one of hypernym detection, hypernym orientation, and hypernym discovery.

16. The computer program product of claim 15, wherein generating a first list of triples based on the text corpus comprises: extracting one or more candidate hyponym-hypernym pairs from the text corpus by applying a linguistic model to the text corpus, wherein each candidate hyponym-hypernym pair comprises a pair of words representing a candidate hyponym and a candidate hypernym; generating a second list comprising each candidate hyponym-hypernym pair extracted from the text corpus; applying at least one of a filtering technique and an expansion technique to the second list; generating the first list based on the second list, wherein the first list includes each candidate hyponym-hypernym pair of the second list and a corresponding frequency value indicating the number of times a hyponym-hypernym relationship is observed between a candidate hyponym and a candidate hypernym of the candidate hyponym-hypernym pair; as well as Each word of the first list is encoded.

17. A computer program product according to claim 16, wherein applying at least one of a filtering technique and an expansion technique to the second list includes one or more of the following: filtering the second list to remove at least one erroneous candidate hyponym-hypernym pair, and expanding the second list to include at least one candidate hyponym-hypernym pair that was not extracted from the text corpus.

18. The computer program product of claim 16, wherein the linguistic pattern is an auditory similarity pattern.

19. The computer program product of claim 15, wherein the training comprises: The neural network is trained against a loss function that enforces the asymmetry and transitivity requirements of the strict partial order relation as the soft constraints.

20. The computer program product of claim 19, wherein the training further comprises: The neural network is trained to learn embeddings for words appearing in the first list, wherein the SPON model is configured to generate ranked predictions of hypernyms of out-of-vocabulary hyponyms by utilizing a distributional similarity measure between the words.

21. The computer program product of claim 20, wherein the program instructions are executable by a processor to further cause the processor to: Ranked predictions of hypernyms of out-of-vocabulary hyponyms are generated via the SPON by utilizing a distributional similarity measure between the words.

Images