Natural language processing system, program, and natural language processing method

Abstract

To provide an information processing system, a program and an information processing method which create a synonym group from a word embedding model using a corpus in an object field.SOLUTION: An information processing method comprises: constructing an embedding model, by using a corpus specified by a user S102; extracting noun phrases, with respect to the whole sentences of the corpus S103; calculating similarity degrees of the noun phrases S104; determining those equal to or higher than a preset threshold value, of the noun phrases to be synonymy phrases S105; taking out words which are not shared between pairs of the synonymy phrases which are not determined S106; calculating the similarity degree again with respect to the words and, when the calculation result similarly exceeds the preset threshold value S105, determining those equal to or higher than the threshold value, to be a synonym, gathering vertexes of respective connected components of a graph in which each element of the synonym pair is defined to be a side; and creating a synonym group S109.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a natural language processing system, a program, and a natural language processing method.

Background Art

[0002] Inquiries from users from web pages, etc., generally use an automatic conversation program that utilizes artificial intelligence such as a chatbot. At that time, methods such as creating a word embedding model using a corpus related to the target field, forming a group of synonymous sentences, and improving the conversation response accuracy are taken.

[0003] Patent Document 1 describes a system that extracts time expressions from input sentences in natural language and automatically classifies their types to improve the response ability of an automatic conversation program that utilizes artificial intelligence.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, a system that extracts time expressions as disclosed in Patent Document 1 and automatically classifies their types has an advantage in grasping the time series, but it is not a system suitable for improving the conversation response accuracy by itself. Also, in natural language processing, it is common to use a word embedding model based on a specified corpus, but the extraction of the above time expressions and the embedding model do not improve the conversation response accuracy in relation to each other.

[0006] In view of the above circumstances, the present invention aims to provide an information processing system, program, and information processing method for selecting and creating synonym groups from a word embedding model using a corpus related to the target field, in order to improve the accuracy of conversational responses in automated conversation programs that utilize artificial intelligence, such as chatbots. [Means for solving the problem]

[0007] The first aspect of the present invention is, A system for creating synonym groups in natural language processing, It is configured to perform the following steps: In the build step, the user but designation do Using a corpus, we build an embedding model, In the extraction step, the corpus of Extract noun phrases from the entire text, In calculation step 1, the similarity of the noun phrases is calculated using the embedding model, In the determination step 1, the calculation Similarity calculated in Step 1 If the value exceeds a pre-set threshold, it is determined to be a synonym. In the removal step, multiple The aforementioned synonyms Composed of Extract words that are not shared between pairs, In calculation step 2, the similarity of the aforementioned words is calculated using the embedding model. In the determination step 2, the calculation Similarity calculated in Step 2 but so that it becomes greater than the threshold in the determination step 1. If the value exceeds a pre-set threshold, it is determined to be a synonym. In the set step, multiple The above synonyms Composed of This method involves grouping the vertices of each connected component of a graph, where each element of a pair is an edge, to form synonym groups.

[0008] A second aspect of the present invention is, A method for creating synonym groups in natural language processing, Server It is configured to perform the following steps: In the construction step, the user but specifies do to construct an embedding model using a corpus, In the extraction step, for the entire text of the corpus of extract noun phrases, In calculation step 1, calculate the similarity of the noun phrases using the embedding model, In determination step 1, when the calculation Similarity calculated in Step 1 is greater than or equal to a preset threshold, determine them as synonyms, In the extraction step, multiple for the synonyms Composed of extract words not shared between pairs, In calculation step 2, calculate the similarity of the words using the embedding model, In determination step 2, when the calculation Similarity calculated in Step 2 is so that it becomes greater than the threshold in the determination step 1. greater than a preset threshold, determine them as synonyms, In the aggregation step, multiple for the synonyms Composed of aggregate the vertices of each connected component of a graph with each element of the pair as an edge to form a synonym group. This is the method.

[0009] According to this, an information processing system, program, and information processing method in natural language can be provided that can create synonym groups for each user's individual settings and for each target, thereby improving the accuracy of conversation responses.

Brief Description of Drawings

[0010] [Figure 1] It is an overall view of the information processing system 1 according to this embodiment. [Figure 2] It is a block diagram showing the hardware configuration of the server 2. [Figure 3] It is a block diagram showing the functions realized by the server 2. [Figure 4] It is a flowchart diagram of this embodiment.

Modes for Carrying Out the Invention

[0011] Embodiments of the present invention will be described below with reference to the drawings. The various features shown in the embodiments below can be combined with each other.

[0012] By the way, the program for realizing the software appearing in this embodiment is a computer The software may be provided as a non-transitory computer-readable medium, or it may be provided as a downloadable file from an external server, or it may be provided so that the program is launched on an external computer and its functions are realized on the client terminal (so-called cloud computing).

[0013] Furthermore, in this embodiment, "part" refers to, for example, hardware implemented by a circuit in a broad sense. Wearable resources and the software that can be specifically realized by these hardware resources. This may also include a combination of information processing and other related technologies. Furthermore, this embodiment handles various types of information, which can be represented, for example, by the physical values ​​of signal values ​​representing voltage and current, the high or low values ​​of signal values ​​as a set of binary bits consisting of 0s or 1s, or by quantum superposition (so-called qubits), and communication and calculations can be performed on a circuit in a broad sense.

[0014] Furthermore, a circuit in a broad sense is a circuit realized by combining at least a suitable combination of circuits, circuits, processors, and memory. In other words, it includes application-specific integrated circuits (ASICs), programmable logic devices (for example, simple programmable logic devices (SPDs), complex programmable logic devices (CPLDs), and field programmable gate arrays (FPGAs)), etc.

[0015] 1. Hardware Configuration Section 1 describes the hardware configuration of this embodiment.

[0016] 1.1 Information Processing System 1 Figure 1 is an overall diagram of the information processing system 1 according to this embodiment. The information processing system 1 comprises other user terminals 3 (for example, other user terminals 3-1, 3-2, ..., 3-n) and its own server 2, which are connected via a network. These components will be described further. Here, the system exemplified in the information processing system 1 consists of one or more devices or components.

[0017] 1.2 Server 2 Server 2 is connected to the network and is configured to share corpora and embedded models with other user terminals 3 through selection, etc. The shared corpora may include, but are not limited to, email records or product manuals. Similarly, the embedded models may include, but are not limited to, word2vec or fastText. In other words, the extraction unit 232, which is one of the functions of the control unit 23 of Server 2, extracts noun phrases from the entire corpus, but this extraction of noun phrases is unrelated to the embedded model.

[0018] Figure 2 is a block diagram showing the hardware configuration of Server 2. Server 2 has a communication unit 21, a storage unit 22, and a control unit 23, and these components are electrically connected within Server 2 via a communication bus 20. Each component will be described further.

[0019] The communication unit 21 preferably uses wired communication methods such as USB, IEEE1394, Thunderbolt, and wired LAN network communication, but may also include wireless LAN network communication, mobile communication such as 3G / LTE / 5G, and Bluetooth® communication as needed. In other words, the communication unit 21 is a so-called network interface and is more preferably implemented as a collection of the above-mentioned communication methods. The server 2 can share corpora and embedded models with other user terminals 3 via the communication unit 21, and the control unit 23 creates synonym groups. Further details will be described later.

[0020] The memory unit 22 stores corpora, embedded models, etc., that can be shared with other user terminals 3. This can be implemented, for example, as a storage device such as a solid-state drive (SSD) that stores various programs stored in the server 2 executed by the control unit 23, or as memory such as random access memory (RAM) that stores temporarily necessary information (arguments, arrays, etc.) related to program calculations. A combination of these may also be used. Further details will be described later.

[0021] The control unit 23 performs processing and control of the overall operation related to the server 2. The control unit 23 is, for example, a central processing unit (CPU) not shown. The control unit 23 recognizes a corpus specified by the user by reading a predetermined program stored in the memory unit 22. That is, information processing by software stored in the memory unit 22 can be concretely realized by the control unit 23, which is an example of hardware, and can be executed as a functional unit included in the control unit 23. These will be described in more detail in the next section. Note that the control unit 23 is not limited to being a single unit, and may be implemented with multiple control units 23 for each function, or a combination thereof.

[0022] The control unit 23 also reads a predetermined program stored in the memory unit 22, constructs an embedding model using a user-specified corpus, and extracts noun phrases. It then performs similarity assessments of these noun phrases, creates synonym groups, and temporarily stores them in a separate area of ​​the memory unit 22. Further details will be described later.

[0023] Regarding synonym pairs, it is not necessary to include all of them in the above synonym groups; only a portion of the pairs may be included, and the user may be able to select which ones to include. After this selection, the selected synonyms are not stored immediately, but are linked to an embedded model or a user-specified corpus and stored in the memory unit 22. Further details will be described later.

[0024] 1.3 Other user terminals 3 Other user terminals 3 can provide or be configured to provide corpora and embedded models to server 2 via the network. The corpora and embedded models may also be created by an automated response program stored in server 2. The terminals themselves are not particularly limited; various devices such as edge devices and notebook PCs are compatible. Furthermore, the network connection can be wired or wireless.

[0025] 2. Functional Configuration This section describes the functional configuration of this embodiment. As mentioned above, the information processing performed by the software stored in the memory unit 22 is specifically realized by the control unit 23, which is an example of hardware, and can be executed as a functional unit included in the control unit 23.

[0026] Figure 3 is a block diagram showing the functions implemented by Server 2 (Control Unit 23). Specifically, Server 2 (Control Unit 23) comprises a Construction Unit 231, an Extraction Unit 232, a Calculation Unit 233, a Determination Unit 234, an Extraction Unit 235, and an Aggregation Unit 236.

[0027] The construction unit 231 is configured to build an embedded model using a user-specified corpus. For example, the construction unit 231 creates an embedded model from a corpus that can be obtained from another user terminal 3, or from a corpus stored on the server 2. Here, the corpus is the foundation for creating synonym groups, and the user specifies the target field to limit the scope of the analysis.

[0028] While the embedding model will be created using a corpus, preprocessing appropriate to the performance of the user-specified corpus may be performed to improve the model's accuracy. Naturally, the corpus that can be shared with other user terminals 3 may also be one that has undergone the above preprocessing.

[0029] The above embedding models include, but are not limited to, word2vec and fastText. Since the above corpus is related to the target field and specified by the user, it is preferable to be able to select a suitable embedding model based on it. Therefore, although the direction vectors described below are specific to particular methods and learning processes, they can be used in combination as long as there are no differences in the direction vectors or the interaction between the models.

[0030] The extraction unit 232 performs part-of-speech tagging and dependency parsing on the entire corpus text and extracts noun phrases based on this. Specifically, for Japanese, it is envisioned that it extracts consecutive parts of words determined to be nouns or adjectival nouns using the morphological analysis software MeCab based on the UniDic dictionary, and for English, it extracts using the dependency parsing results.

[0031] The calculation unit 233 is configured to calculate the similarity of the extracted noun phrases using the embedding model described above. The determination unit 234 measures the similarity and is configured to determine if the similarity exceeds a preset threshold. For example, it is assumed that the similarity is calculated using the cosine similarity of the vectors for the noun phrases. In the case of word2vec, it is calculated using the sum of the word vectors that make up the phrase, while in the case of fastText, in addition to the sum of the word vectors, a vector for the entire phrase may be calculated.

[0032] The extraction unit 235 is configured to extract words whose calculated value exceeds a preset threshold and which are not shared between the synonym pairs. The extracted words are then used to calculate similarity in the calculation unit 233, and the determination unit 234 determines them to be synonyms if the similarity exceeds a preset threshold.

[0033] The aggregation unit 236 is configured to aggregate the vertices of each connected component of a graph whose edges are each element of the above-mentioned synonym pair to form a synonym group. For example, if there are words "leakage" and "leak" that are not shared between the synonym pair (fuel leak amount, fuel leak amount), the similarity is calculated, and if it is above a threshold, it is recognized as a synonym pair (leakage, leak), and a category is created as a synonym group.

[0034] The resulting synonym group is stored in the memory unit 22. It may also be configured to send it to other user terminals 3 via the network. The system may be configured so that each user of the other user terminals 3 can choose, as appropriate, whether or not to incorporate the above synonym group into the corpus and embedding model related to the target field at each terminal.

[0035] Furthermore, it is also possible to decide whether or not to include each of the above synonym pairs in a synonym group, and the pre-set thresholds can also be programmed to be set independently.

[0036] 3. Information Processing Methods This section describes the information processing method of the aforementioned natural language information processing system 1. This information processing method comprises the following steps: In the construction step, an embedding model is constructed using a user-specified corpus. In the extraction step, noun phrases are extracted from the entire corpus. In calculation step 1, the similarity of the noun phrases is calculated using the embedding model. In judgment step 1, if the calculated value is greater than or equal to a predetermined threshold, the phrases are determined to be synonyms. In the extraction step, words that are not shared between the synonym pairs are extracted. In calculation step 2, the similarity of these words is calculated using the embedding model. In judgment step 2, if the calculated value exceeds a predetermined threshold, the words are determined to be synonyms. In the aggregation step, the vertices of each connected component of a graph, where each element of the synonym pair is an edge, are aggregated to form a synonym group.

[0037] Figure 4 is a flowchart illustrating the flow of information processing performed by the information processing system 1 in natural language. The following explanation will follow each step in this flowchart.

[0038] In automated conversation programs that utilize artificial intelligence, such as chatbots, it is useful to create a word embedding model in advance using a corpus related to the target field and form synonym groups in order to improve the accuracy of the response. Therefore, the user managing Server 2 must first select and specify the corpus of the target field in advance (Step S101). Furthermore, in order to improve the accuracy of the model, it is preferable to perform preprocessing appropriate to the characteristics of the corpus, such as removing email headers and quotation marks, on the corpus specified by the user.

[0039] Subsequently, an embedding model will be constructed using word2vec or fastText (step S102), but it is not limited to these. Here, the corpus is related to the target field and is specified by the user, but it may also be possible to select the corpus to be used as the embedding model as needed. Furthermore, if there are no differences in the interaction between the models or in the direction vectors described below, a combination may be constructed as the embedding model.

[0040] Then, noun phrases are extracted from the corpus (step S103), and the similarity between noun phrases is calculated (step S104). A typical example of this similarity is the cosine similarity of the vectors for the noun phrases. Cosine similarity is almost equivalent to normalizing the noun phrases and calculating the dot product of the normalized vectors for the pair of noun phrases. The narrower the angle between the two normalized vectors, the larger the dot product, and the more similar the directions of the noun phrase vectors are.

[0041] In this way, it is determined whether the calculated similarity exceeds a predetermined threshold (step S105), and if it exceeds the threshold, words that are not shared between the synonym pairs are extracted (step S106).

[0042] If N is the set of all noun phrases in the corpus, and A and B are pairs of noun phrases, then the set of synonymous phrase pairs Sp can be expressed by the following equation (1-1). TIFF0007865896000001.tif444…(1-1) sim(A,B) is the similarity between words A and B, t p This is an arbitrary threshold that is set in advance when creating the above synonym group.

[0043] When the threshold determination is performed in step S105, the above threshold t p The threshold t described later w If the number exceeds a certain threshold, it is created as a synonym group (step S109). This allows feedback to be provided to the embedding model without creating the synonym groups described later.

[0044] The similarity of synonym pairs is based on a threshold t. p and the threshold t described later w If the words are between the specified range, as described above, words that are not shared between the synonym pairs will be extracted, and the similarity will be recalculated among the extracted non-shared words (step S107), followed by a threshold determination (step S108).

[0045] As described above, let's denote pairs of noun phrases as A and B, and let's represent words that are in A but not in B as a. Similarly, let's represent words that are in B but not in A as b. Then the set of synonym pairs Sw can be represented by the following equation (1-2). JPEG0007865896000002.jpg773…(1-2) sim(a,b) is the similarity between words a and b, t w This is an arbitrary threshold that is set in advance when creating the above synonym group.

[0046] For example, if there are words "leakage" and "leak" that are not shared among the synonym pairs (fuel leakage amount, fuel leak amount) belonging to Sp, their similarity is calculated, and if it is above a threshold, they are recognized as a synonym pair (leakage, leak), and the vertices of each connected component of the graph, which has each element of the synonym pair as an edge, are set together to create a synonym group (step S109).

[0047] In the above example, t p The value is set to 0.6, and the phrase pair (fuel leakage amount, fuel leakage amount) is a synonymous phrase pair belonging to Sp. w The value is set to 0.8, and "leakage" and "leak" are considered a synonymous pair.

[0048] This synonym group may then be saved as training data for machine learning, linked to a user-specified corpus (step S110). It is preferable to be able to select whether to use the entire synonym group as training data, or only the user-specified synonym pairs, as appropriate. Using machine learning is preferable because it can shorten the time required to build an embedding model, extract noun phrases, and perform two similarity assessments on the corpus.

[0049] Subsequently, all synonym pairs and synonym groups temporarily stored in the memory unit 22 are sorted according to the user's specifications, only the sorted ones are stored, and the rest are erased, thus completing the series of steps.

[0050] The synonym groups created in this way will be shared with other user terminals 3 via the network, linked to the specified corpus and embedded model. Furthermore, synonym groups created by other user terminals 3 can also be imported into server 2 via the network, provided they share the same data format. Needless to say, users can freely configure whether or not to import these into server 2.

[0051] 4. Others Regarding the information processing system 1 according to this embodiment, the following configuration may be adopted. In the information processing system 1, the creation of the synonym group in the set step also uses pre-trained data that has been trained in advance as training data. Using machine learning allows us to determine the likelihood of the created synonym groups, and by effectively utilizing pre-trained data, we can also partially perform similarity determination, thus saving time, which is preferable.

[0052] In the information processing system 1 using machine learning described above, the above-mentioned training data is generated or updated. Updating the training data is even more preferable because it further improves accuracy. The same applies to time reduction.

[0053] In Information Processing System 1, the above corpus consists of emails and product manuals from which non-textual material has been removed beforehand. When receiving questions from product users and others via the web, it is common to use automated conversation programs utilizing artificial intelligence, such as chatbots, to avoid causing waiting times for the other party. Therefore, when creating the above synonym groups, emails and product manuals written in a conversational style that make it easy to extract product keywords and noun phrases are useful. Furthermore, when extracting noun phrases, the presence of header information and quotation marks reduces the accuracy of the extraction, so removing them beforehand makes the data more useful.

[0054] In the information processing system 1, the part that extracts the noun phrase is a sequence of words that have been determined to be nouns or adjectival nouns by the dictionary selected by the user. In Japanese, in particular, many verb phrases are formed by adding "suru" to a kanji character, and similarly, adjectival nouns are also components of compound words made up of kanji characters. Therefore, it is preferable to extract the consecutive parts of these words as phrases in order to include them in the analysis, as this further improves the accuracy of the resulting synonym group. For example, the word "high" in "high economic growth" is classified as an adjectival noun (or ideographic word within the UniDic dictionary).

[0055] Finally, various embodiments of the present invention have been described, but these are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of Symbols]

[0056] 1: Information Processing System 2: Server 20: Communications bus 21: Communications Department 22: Storage section 23: Control Unit 231: Construction Department 232:Extraction part 233: Calculation Unit 234: Judgment section 235: Removal section 236: Gathering section 3: Other user terminals

Claims

1. A system for creating synonym groups in natural language processing, It is configured to perform the following steps: In the build step, an embedding model is constructed using a corpus specified by the user. In the extraction step, noun phrases are extracted from the entire text of the corpus, In calculation step 1, the similarity of the noun phrases is calculated using the embedding model, In the determination step 1, if the similarity calculated in the calculation step 1 is equal to or greater than a predetermined threshold, it is determined to be a synonym. In the extraction step, words that are not shared among pairs composed of multiple synonyms are extracted. In calculation step 2, the similarity of the aforementioned words is calculated using the embedding model. In determination step 2, if the similarity calculated in calculation step 2 exceeds a threshold set in advance so that it is greater than the threshold in determination step 1, the words are determined to be synonyms. In the set step, the vertices of each connected component of a graph, where each element of a pair composed of multiple synonyms is an edge, are set together to form a synonym group.

2. In the system for creating synonym groups in natural language processing according to claim 1, In the aforementioned set step, the creation of the synonym group also utilizes pre-trained data that has been used as training data.

3. In the system for creating synonym groups in natural language processing according to claim 2, The aforementioned set step further comprises a learning step, In the aforementioned learning step, the created synonym group is further added as training data to generate or update the trained data.

4. In a system for creating synonym groups in natural language processing according to any one of claims 1 to 3, The aforementioned corpus consists of emails and product manuals from which non-text content has been removed beforehand.

5. A system for creating synonym groups in natural language processing according to any one of claims 1 to 3, The portion of the extraction step in the aforementioned extraction step that extracts the noun phrase is a sequence of words that have been determined to be nouns or adjectival nouns by the dictionary selected by the user.

6. A system for creating synonym groups in natural language processing according to claim 4, The portion of the extraction step in the aforementioned extraction step that extracts the noun phrase is a sequence of words that have been determined to be nouns or adjectival nouns by the dictionary selected by the user.

7. A method for creating synonym groups in natural language processing, The server is configured to perform the following steps: In the build step, an embedding model is constructed using a corpus specified by the user. In the extraction step, noun phrases are extracted from the entire text of the corpus, In calculation step 1, the similarity of the noun phrases is calculated using the embedding model, In the determination step 1, if the similarity calculated in the calculation step 1 is equal to or greater than a predetermined threshold, it is determined to be a synonym. In the extraction step, words that are not shared among pairs composed of multiple synonyms are extracted. In calculation step 2, the similarity of the aforementioned words is calculated using the embedding model. In determination step 2, if the similarity calculated in calculation step 2 exceeds a threshold set in advance so that it is greater than the threshold in determination step 1, the words are determined to be synonyms. The method involves a set step in which the vertices of each connected component of a graph, where each element of a pair composed of multiple synonyms is an edge, are set together to form a synonym group.

8. In the method for creating synonym groups in natural language processing according to claim 7, A method for creating the synonym groups in the aforementioned set step, which also uses pre-trained data that has been used as training data.

9. In the method for creating synonym groups in natural language processing according to claim 8, The aforementioned set step further comprises a learning step, A method for generating or updating the trained data by further adding the created synonym group as training data in the learning step.

10. In a method for creating synonym groups in natural language processing according to any one of claims 7 to 9, The method wherein the aforementioned corpus consists of emails and product manuals from which non-textual material has been removed beforehand.

11. A method for creating synonym groups in natural language processing according to any one of claims 7 to 9, A method wherein the portion of the extraction step in the extraction step that extracts the noun phrase is a sequence of words that have been determined to be nouns or adjectival nouns by the dictionary selected by the user.

12. A method for creating synonym groups in natural language processing according to claim 10, A method wherein the portion of the extraction step in the extraction step that extracts the noun phrase is a sequence of words that have been determined to be nouns or adjectival nouns by the dictionary selected by the user.

13. It is a program, A device that causes a computer to perform each step of the natural language information processing system described in any one of claims 1 to 3.

Images