Semantic element prediction method, computer device and storage medium

Abstract

A method and device for automatically predicting lexical semantic elements using a lexical dictionary, comprising: inputting a word, retrieving the semantic definition of the word and multiple semantic elements corresponding to the word from an online dictionary, setting each retrieved semantic element as a candidate semantic element, inputting the semantic definition of the word and the candidate semantic elements, and estimating the probability that the candidate semantic element can be inferred from the semantic definition of the word.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Application No. 16 / 364,777, filed on March 26, 2019, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0003] The method and device relate to information technology, and in particular to a semantic element extraction method, a computer device and a storage medium. Background Art

[0004] Sememes can be generally defined as the smallest semantic unit. Existing sememe prediction methods can use word embeddings to predict sememes. In this context, word embeddings can generally refer to a collection of language modeling and feature learning techniques in natural language processing (NLP), where words or phrases from a vocabulary are mapped to real number vectors.

[0005] However, these existing semantic feature prediction methods using word embeddings not only fail to provide accurate predictions based on low-frequency words (they do not learn embeddings for low-frequency words), but may also be sensitive to the quality of embeddings.

[0006] For example, Xie et al. (2017) described a semantic feature prediction method that predicts the semantic features of a word by measuring the similarity between jointly learned word embeddings and semantic feature embeddings. Specifically, the reference discusses semantic feature prediction based on word embeddings (SPWE) and semantic feature prediction based on semantic feature embeddings (SPSE), which are inspired by collaborative filtering and matrix decomposition, respectively. The described SPWE predicts semantic features of words that are close to unlabeled words in the embedding space. The disclosed SPSE learns semantic feature embeddings by matrix decomposition in the same embedding space of words, and recommends the most relevant semantic features to unlabeled words in the embedding space.

[0007] In addition, Jin et al. (2018) described a method that additionally considers character information but uses a similarity method based on similar embeddings to predict lexical senses.

[0008] The prediction methods proposed by Xie et al. (2017) and Jin et al. (2018) are relatively simple and easy. However, when these methods are applied to real-word applications, users usually do not have access to accurately learned word embeddings. This is especially true for new words. This is because it is difficult to collect enough contextual data to learn embeddings for new words, and in most deep learning applications, word embeddings are fixed after training, making it difficult to learn embeddings for new words and fix them to systems that use the methods proposed by Xie et al. (2017) and Jin et al. (2018).

[0009] Effects and advantages of certain embodiments

[0010] In view of the above technical problems, the inventors propose a novel semantic component prediction method, a computer device and a storage medium. In some embodiments, the semantic component prediction method uses an online dictionary. Some embodiments of the prediction method, computer device and storage medium are inspired by how humans understand words, that is, reading the semantic definition of words from a dictionary.

[0011] Compared to word embeddings, dictionary entries can provide more accurate semantic definitions and have wider coverage for low-frequency words. Therefore, embodiments of semantic element prediction methods, computer devices, and storage media use multi-label text classification to formulate semantic element predictions. Embodiments take the definition of a word as input and predict multiple semantic elements of the word. Here, in some embodiments, the prediction of each semantic element label is a binary classification task.

[0012] In some embodiments, regarding text classification tasks, since the target semantic element label itself is a meaningful text, the embodiments are able to fully utilize the valuable information in the semantic element label, rather than treating each semantic element label as an abstract symbol. Therefore, some embodiments achieve the benefit of representing each binary semantic element classification as a text match and making predictions based on the semantic relevance between the input word definition and the semantic element.

[0013] The vocabulary knowledge base may contain a large number (eg, thousands) of semes. Generally, based on known techniques, it is relatively time-consuming to process all incorrect semes of a training instance as its negative labels.

[0014] Certain embodiments of the present disclosure may address this problem using a negative sampling strategy that samples negative prime labels from a noise distribution.

[0015] For example, in some embodiments, the two sampling strategies establish the above noise distribution based on embedding similarity or based on the confusion matrix between semantic elements to select the most competitive negative labels to accelerate model training.

[0016] That is, some embodiments of the present disclosure use dictionary entries of words with accurate semantic definitions and wider coverage to predict sememes. Some embodiments use text matching models to capture the semantic relevance between word definitions and sememes, and two negative sampling methods can be used to improve the robustness of the prediction model.

[0017] Experimental results, such as Figure 5 The experimental results shown indicate that the performance of certain embodiments of the present disclosure is significantly better than the existing baselines. Furthermore, the negative sampling methods of certain embodiments significantly improve their performance. Summary of the invention

[0018] One or more embodiments provide a semantic element prediction method, a computer device, and a storage medium.

[0019] According to one aspect of the embodiment, a semantic element prediction method executed by at least one computer processor is provided, the method comprising: inputting a word, retrieving the semantic definition of the word and multiple semantic elements corresponding to the word from an online dictionary, setting each retrieved semantic element as a candidate semantic element, for each candidate semantic element, inputting the semantic definition of the word and the candidate semantic element, and for each candidate semantic element, estimating the probability that the candidate semantic element can be inferred from the semantic definition of the word.

[0020] According to one aspect of the embodiment, a semantic element prediction device is provided, the device comprising: at least one memory configured to store program code; at least one processor configured to read the program code and operate according to the instructions of the program code, the program code comprising: a first input code configured to cause the at least one processor to input a word; a retrieval code configured to cause the at least one processor to retrieve the semantic definition of the word and multiple semantic elements corresponding to the word from an online dictionary; a setting code configured to cause the at least one processor to set each retrieved semantic element as a candidate semantic element; a second input code configured to cause the at least one processor to input the semantic definition of the word and the candidate semantic element for each candidate semantic element; and a probability estimation code configured to cause the at least one processor to estimate, for each candidate semantic element, the probability that the candidate semantic element can be inferred from the semantic definition of the word.

[0021] According to one aspect of the embodiment, a non-volatile computer-readable medium for semantic element prediction is provided, which stores instructions, the instructions including: one or more instructions, which, when executed by one or more processors of a device, cause the one or more processors to: input a word, retrieve a semantic definition of the word and multiple semantic elements corresponding to the word from an online dictionary, set each retrieved semantic element as a candidate semantic element, for each candidate semantic element, input the semantic definition of the word and the candidate semantic element, and for each candidate semantic element, estimate the probability that the candidate semantic element can be inferred from the semantic definition of the word. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a diagram of an environment in which the methods, apparatuses, and systems described herein may be implemented, according to an embodiment.

[0023] Figure 2 yes Figure 1 A diagram of example components of one or more devices.

[0024] Figure 3 is a diagram of a semantic component prediction method according to an embodiment.

[0025] Figure 4 is a diagram of a prediction model according to an embodiment.

[0026] Figure 5 is a graph of experimental test results according to an embodiment. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present application clearer and more understandable, the embodiments will be further described in detail with reference to the accompanying drawings. It should be understood that the specific implementation described herein is only used to explain the present application, rather than to limit the present application.

[0028] Figure 1 is a schematic diagram of an application environment of a semantic element prediction method according to an embodiment. Figure 1 As shown, the application environment includes a user terminal 110 and a server 120, and the user terminal 110 communicates with the server 120. The user inputs text, such as Chinese characters, through one user terminal 110, and the input text is sent to the server 120 through a communication network. The server 120 processes the input text and predicts corresponding semantic elements for the input text. Alternatively, the user inputs text through one user terminal 110, the user terminal 110 processes the input text, predicts corresponding semantic elements for the input text, and sends the predicted semantic elements to the server 120 through a communication network, and the communication network can send the predicted semantic elements to another user terminal 110.

[0029] Figure 2 Schematic diagram of the internal structure of a computer device according to an embodiment. The computer device may be a user terminal or a server. Figure 2 As shown, the computer device includes a processor, a memory and a network interface connected by a system bus. The processor is configured to provide computing and control capabilities to support the operation of the computer device. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium can store an operating system and computer-readable instructions, and the internal memory provides an environment for running the operating system and computer-readable instructions in the non-volatile storage medium. When the processor executes the computer-readable instructions, the processor can execute the semantic element prediction method. The network interface is configured to perform network communication with an external terminal.

[0030] The embodiments are not limited to Figure 2 The structure shown in the figure may be modified in various ways without departing from the scope of the present disclosure. Specifically, for example, a computer device may include Figure 2 The components in the component list may include more or fewer components, or may include a combination of two or more components, or may include different component layouts.

[0031] Reference Figure 3 In one embodiment, a semantic component prediction method is provided. Based on text received from, for example, a user terminal 110, Figure 1 The semantic element prediction method is executed in the server 120 shown. The semantic element prediction method comprises the following steps:

[0032] S310: Obtain a word.

[0033] Generally, the word is a Chinese character, such as "zhé luàn qí mǐ". However, the word is not limited to Chinese characters and can be words in different languages.

[0034] In one implementation, before step 310, the method further includes the step of obtaining a word from a user who inputs text through a user terminal. The user terminal sends the text through a communication network, and the server obtains the text input by the user from the user terminal through the communication network.

[0035] S320: Retrieve the semantic definition D of the word from an online dictionary.

[0036] The online dictionary and the semantic definition D can be accessed in various ways. For example, the online dictionary and the semantic definition D can be automatically accessed, or the online dictionary and the semantic definition D can be accessed through user interaction with the user terminal. In addition, although an online dictionary is used in this embodiment, the present application is not limited thereto. For example, a non-online dictionary can be used to retrieve the semantic definition of a word. In addition, other types of definitions can be retrieved.

[0037] S330: Apply a prediction model to estimate the probability of being able to infer semantic elements from the semantic definition D.

[0038] The semantic element prediction model according to the embodiment will be described below.

[0039] S340: Normalize the probability.

[0040] Here, the probability can be normalized in any way.

[0041] S350: Use a threshold to select the predicted semantic elements.

[0042] Here, a predetermined probability threshold can be selected based on the selected probability, and the predicted semantic elements can be presented to the user through, for example, the user terminal.

[0043] In some embodiments, multiple thresholds can be used or no threshold can be used.

[0044] S360: Present the selected semantic elements to the user.

[0045] Here, the selected semantic elements can be presented to the user through, for example, the user terminal 110. However, the selected semantic elements can be presented to the user in different ways.

[0046] Now, the prediction model mentioned in Figure 3 will be described. The prediction model can be the Dict2Sem model 400.

[0047] The Dict2Sem model 400 can be applied to each candidate sememe 420. For example, for the example Chinese character 410, such as "zhé luàn qí mǐ" (in disorder), the candidate sememe 420 can be, for example, "lose" (represented in English as "lose"). The Dict2Sem model 400 can be used to estimate the probability that the sememe S can be inferred from the semantic definition D. In this case, the semantic definition D corresponds to the example Chinese character 410, such as "zhé luàn qí mǐ".

[0048] Figure 4 The illustrated Dict2Sem model 400 can include the following three layers.

[0049] 430: Input representation layer. The input reaching the Dict2Sem model 400 can include two parts. For the first part, the semantic definition D and the candidate sememe 420 can be input as (e.g., Chinese) character sequences, rather than as segmented words. For the second part, segmentation of the semantic definition D and the candidate sememe 420 can be performed using, for example, Stanford CoreNLP, resulting in a sequence of segmented words.

[0050] The input representation layer 430 can also include constructing vectors for each character in the semantic dictionary D and the candidate sememe 420 by concatenating the character and sememe embeddings of the candidate sememe 420, where the sememe embedding averages the embeddings of all assigned sememes. In some embodiments, the character and sememe embeddings can be randomly initialized. Additionally, the model can include more or fewer than two inputs. That is, the input representation layer 430 can be configured to have more than two inputs or fewer than two inputs.

[0051] 440: Matching layer. The matching layer 440 can use a bidirectional long short-term memory (Bi-LSTM) model to encode the context embeddings for each time step of the semantic definition D and the candidate sememe 420. Additionally, the matching layer 440 can compare each context embedding of one sentence with all context embeddings of the other sentence. However, the matching layer 440 is not necessarily so restricted.

[0052] 450 Prediction layer. In the prediction layer 450, another Bi-LSTM model can be applied to the two sequences of matching vectors respectively. Then, the final fixed-length matching vector can be constructed by concatenating multiple vectors from the last time step of the Bi-LSTM model. The prediction layer 450 can also use a two-layer feedforward neural network to consume the fixed-length matching vector and apply the Softmax function in the output layer.

[0053] Although in Figure 4It is not shown in the figure, but the Dict2Sem model 400 can be used after training. The training may include any of the following four training methods: random sampling, frequency-based sampling, embedding-based sampling, and confusion matrix sampling. Of course, these methods can also be combined. In addition, other training methods can also be used.

[0054] In short, for embedding-based sampling, the method can build a noise distribution from the semantic embeddings at each training stage. For confusion matrix-based sampling, the method can build a noise distribution from the confusion matrix, which is built by analyzing the prediction results of the training data at each training stage.

[0055] Although the above model is described with detailed specificity, other prediction models may also be used. That is, the present application is not limited to the Dict2Sem model.

[0056] The semantic component prediction device corresponds to the semantic component prediction method, and the specific technical features of the device corresponding to the method are not repeated here.

[0057] It will be appreciated by those skilled in the art that all or some modules, units, components and processes of the aforementioned embodiments may be implemented by providing a computer program that provides instructions to the relevant hardware. The program may be stored in a non-volatile computer-readable storage medium. When the program is executed, the program may control the hardware to execute the process of the embodiment of each of the aforementioned methods. In each embodiment of the present application, any use of a memory, a storage device, a database or other medium may include a non-volatile memory and / or a volatile memory. The non-volatile memory may include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM) or a flash memory. The volatile memory may include a random access memory (RAM) or an external cache memory. For description, rather than for limitation, RAM may be in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM) and memory bus dynamic RAM (RDRAM).

[0058] Each technical feature in the aforementioned embodiment can be randomly combined. In order to simplify the description, all possible combinations of each technical feature in the aforementioned embodiment are not described. However, as long as there is no contradiction in the combination of technical features, it should be considered that such a combination falls within the scope of the specification. The aforementioned embodiments only describe several implementations of the present application, and the descriptions of these implementations are specific and detailed, but cannot be interpreted as limiting the scope of the patent disclosed herein. It should be noted that those of ordinary skill in the art can further make changes and improvements without departing from the concept of the present application, and all these changes and improvements fall within the scope of protection of the present application. Therefore, the scope of patent protection of the present application should be based on the attached claims.

Claims

1. A semantic component prediction method, comprising: Input word, Retrieving a semantic definition of the word and a plurality of sememes corresponding to the word from an online dictionary, Set each retrieved sememe as a candidate sememe, For each of the candidate semantic elements, the semantic definition of the word and the candidate semantic element are input as a sequence of Chinese characters, and For each of the candidate semantic elements, estimating the probability that the candidate semantic element can be inferred from the semantic definition of the word; The estimation of the probability of inferring the candidate sememe includes: For each of the candidate semantic elements, segmenting the semantic definition of the word and the candidate semantic element to obtain a sequence of segmented words; and By cascading the characters, semantic embeddings and average embeddings of all candidate semantics, a vector is constructed for the semantic definition of the word and each character in the candidate semantics, wherein the constructed vector is used to estimate the probability of inferring the candidate semantics.

2. The method according to claim 1, wherein: The estimating of the probability of inferring the candidate sememe further comprises performing the construction of the vector by randomly initializing the character and sememe embeddings.

3. The method according to claim 2, wherein: The estimation of the probability of inferring the candidate sememe further comprises: Using a bidirectional long short-term memory (Bi-LSTM) model to encode the semantic definition of the word and the contextual embedding of the candidate sememes at each time step, and Compare each context embedding of one sentence with all context embeddings of the other sentence.

4. The method according to claim 3, wherein: The estimation of the probability of inferring the candidate sememe further comprises: Another bidirectional long short-term memory Bi-LSTM model is applied to the two matching vector sequences respectively. constructing a final fixed-length matching vector by concatenating multiple vectors from the last time step of the bidirectional long short-term memory Bi-LSTM model, and A two-layer feed-forward neural network is used to consume the final fixed-length matching vector.

5. The method according to claim 4, wherein: The estimating of the probability of inferring the candidate sememe further comprises: applying a Softmax function.

6. The method according to claim 1, wherein: The estimating of the probability of inferring the candidate sememe includes: training a model.

7. The method according to claim 6, wherein: The training includes adopting at least one of random sampling, frequency-based sampling, embedding-based sampling, and confusion matrix sampling.

8. The method according to claim 1, further comprising: The candidate semantic element with the highest probability inferred from the semantic definition of the word is output.

9. A device comprising: at least one memory configured to store program code; At least one processor is configured to read the program code and operate according to the instructions of the program code, wherein the program code includes: a first input code configured to cause the at least one processor to input a word, a retrieval code configured to cause the at least one processor to retrieve a semantic definition of the word and a plurality of semes corresponding to the word from an online dictionary, setting code configured to cause the at least one processor to set each retrieved sememe as a candidate sememe, a second input code configured to cause the at least one processor to input the semantic definition and candidate semantic elements of the word as a Chinese character sequence, and A probability estimation code configured to cause the at least one processor to estimate, for each candidate semantic element, a probability that the candidate semantic element can be inferred from the semantic definition of the word; The estimation of the probability of inferring the candidate sememe includes: For each of the candidate semantic elements, segmenting the semantic definition of the word and the candidate semantic element to obtain a sequence of segmented words; and By cascading the characters, semantic embeddings and average embeddings of all candidate semantics, a vector is constructed for the semantic definition of the word and each character in the candidate semantics, wherein the constructed vector is used to estimate the probability of inferring the candidate semantics.

10. The device according to claim 9, wherein: The code for estimating the probability of inferring the candidate sememe is further configured to cause the at least one processor to construct the vector by randomly initializing the character and sememe embeddings.

11. The device according to claim 10, wherein: The code for estimating the probability of inferring the candidate sememe is further configured to cause the at least one processor to: Using a bidirectional long short-term memory (Bi-LSTM) model to encode the semantic definition of the word and the contextual embedding of the candidate sememes at each time step, and Compare each context embedding of one sentence with all context embeddings of the other sentence.

12. The device according to claim 11, wherein The code for estimating the probability of inferring the candidate sememe is further configured to cause the at least one processor to: Another bidirectional long short-term memory Bi-LSTM model is applied to the two matching vector sequences respectively. Constructing a final fixed-length matching vector by concatenating multiple vectors from the last time step of the bidirectional long short-term memory Bi-LSTM model, and A two-layer feed-forward neural network is used to consume the final fixed-length matching vector.

13. The device according to claim 12, wherein: The code for estimating the probability of inferring the candidate sememe is further configured to cause the at least one processor to apply a Softmax function.

14. The apparatus according to claim 9, wherein: The code for estimating the probability of inferring the candidate sememe is further configured to cause the at least one processor to train a model for estimating the probability that the candidate sememe can be inferred from the semantic definition of the word.

15. The apparatus according to claim 9, wherein: The code for estimating the probability of inferring the candidate sememe is further configured to cause the at least one processor to employ at least one of random sampling, frequency-based sampling, embedding-based sampling, and confusion matrix sampling.

16. A non-volatile computer-readable medium storing instructions, wherein the instructions include one or more instructions, and the one or more instructions implement the semantic element prediction method described in any one of claims 1 to 15 when executed by one or more processors of a device.

17. A semantic element prediction device, comprising: A first input module configured to input words; A retrieval module, configured to retrieve a semantic definition of the word and a plurality of semantic elements corresponding to the word from an online dictionary; A setting module configured to set each retrieved sememe as a candidate sememe; A second input module is configured to input the semantic definition of the word and the candidate semantic element as a Chinese character sequence for each candidate semantic element; as well as A probability estimation module, configured to estimate, for each candidate semantic element, the probability that the candidate semantic element can be inferred from the semantic definition of the word; The estimation of the probability of inferring the candidate sememe includes: For each of the candidate semantic elements, segmenting the semantic definition of the word and the candidate semantic element to obtain a sequence of segmented words; and By cascading the characters, semantic embeddings and average embeddings of all candidate semantics, a vector is constructed for the semantic definition of the word and each character in the candidate semantics, wherein the constructed vector is used to estimate the probability of inferring the candidate semantics.

Images