Language processing method and language processing system using language model based on multi text analysis algorithm

Abstract

To provide a language processing method and system using a language model based on multiple text analysis algorithms, which extracts training text from source text information using a text extraction model generated based on multiple text analysis algorithms and trains a language model based on the extracted training text to perform language processing specialized for the source domain, the language processing method using a language model based on multiple text analysis algorithms according to the present invention comprises the steps of: generating a text extraction model based on multiple text analysis algorithms that extract text into different morphemes from text information; applying source text information provided from the source to the text extraction model to extract training text for training the language model; refining the extracted training text; and training a language model based on the training data and performing language processing for the source domain using the trained language model. Accordingly, the present invention can obtain a text extraction model suitable for the source domain, and thereby includes the effect that the accuracy of language processing in the source domain of the language model can be improved.

Description

Technology Field

[0001] The present invention relates to a language processing method and processing system using a language model, and more specifically, to a language processing method and processing system using a language model based on a multi-text analysis algorithm that performs language processing specialized for the original text domain. Background Technology

[0002] Generally, nuclear power plants store information generated during operations in a nuclear power plant database. The plant can then utilize this information to provide technical support to workers currently performing tasks.

[0003] Technology utilizing such nuclear power plant information has already been disclosed in Korean Patent Publication No. 10-0991688 (Standardization method for planned preventive maintenance process management according to characteristics of nuclear power plant type and capacity, Oct. 28, 2010). The aforementioned patent includes a concept of systematic work management by standardizing various work management items required for performing planned preventive maintenance of nuclear power plants.

[0004] Meanwhile, in the field of original texts, research is being conducted on technologies that apply original text information to language models for technical support. Conventional language models are trained based on natural language, but unlike natural language, original text information includes specialized terminology and expressions. Consequently, language models trained on natural language have suffered from low accuracy in the original text domain. Therefore, in the field of original texts, methods for training language models on original text information are being devised to achieve language processing specialized for the original text domain. Prior art literature

[0005] Republic of Korea Registered Patent Publication No. 10-0991688 (Standardization method for planned preventive maintenance process management according to characteristics of nuclear power plant type and capacity, Oct. 28, 2010) The problem to be solved

[0006] The present invention aims to provide a language processing method and a processing system using a language model based on multiple text analysis algorithms, which extracts training text from source text information using a text extraction model generated based on a plurality of text analysis algorithms, and trains a language model based on the extracted training text to perform language processing specialized for the source domain. means of solving the problem

[0007] A language processing method using a language model based on a multiple text analysis algorithm according to the present invention comprises the steps of: generating a text extraction model based on a plurality of text analysis algorithms that extract text into different morphemes from text information; applying source text information provided from a source text to the text extraction model to extract training text for training the language model; refining the extracted training text; and training the language model based on the training data and performing language processing on the source domain using the trained language model.

[0008] In the step of generating the text extraction model above, at least two of the plurality of text analysis algorithms may be arbitrarily selected, and a text extraction model may be generated based on the selected text analysis algorithms.

[0009] In the step of generating the text extraction model above, the selected text analysis algorithm is combined to generate multiple different algorithm combinations, one of the multiple algorithm combinations is selected, and a text extraction model can be generated based on the selected algorithm combination.

[0010] In the step of extracting the above training text, the original text information is individually applied to each of the text analysis algorithms of the selected algorithm combination, so that each of the text analysis algorithms can extract a first training text and a second training text, which are extracted as different morphemes.

[0011] In the step of refining the extracted training text, the first and second training texts are merged to generate merged data, and text that is unnecessary for training among the texts of the merged data can be removed based on the frequency of appearance of the text in the merged data.

[0012] In the step of generating the text extraction model, one of the plurality of algorithm combinations is selected, and another text extraction model is generated based on the selected other algorithm combination. In the step of extracting the training text, the original text information is input into the text extraction model to generate training data, and the original text information is input into the other text extraction model to generate other training data.

[0013] A language model is trained based on the above training data, another language model is trained based on the above other training data, the language model with higher performance among the above language model and the above other language model is selected as the final language model, and language processing can be performed using the final language model if the accuracy of the above final language model exceeds a threshold value when compared with the threshold value.

[0014] The above text analysis algorithm may include Hannanum, Kkma, Komoran, Mecab, and Okt.

[0015] The above language model may include a Recurrent Neural Network-Language Model (RNN-LM).

[0016] A language processing system using a language model based on a multiple text analysis algorithm according to the present invention generates a text extraction model based on multiple text analysis algorithms that extract text into different morphemes from text information, applies source text information provided from a source text to the text extraction model to extract training text for training the language model, refines the extracted training text, trains the language model based on the training data, and performs language processing on the source domain using the trained language model. Effects of the invention

[0017] The language processing method and processing system using a language model based on a multi-text analysis algorithm according to the present invention can obtain a text extraction model suitable for the original text domain, and accordingly, the language model includes the effect of improving the accuracy of language processing in the original text domain.

[0018] The technical effects of the present invention as described above are not limited to those mentioned above, and other unmentioned technical effects will be clearly understood by those skilled in the art from the description below. Brief explanation of the drawing

[0019] FIG. 1 is a schematic diagram showing a language processing system using a language model based on a multi-text analysis algorithm according to the present embodiment. FIG. 2 is a flowchart illustrating a language processing method using a language model based on a multi-text analysis algorithm according to the present embodiment. FIG. 3 is a flowchart illustrating the step of training a language model among a language processing method using a language model based on a multi-text analysis algorithm according to the present embodiment. Specific details for implementing the invention

[0020] Embodiments of the present invention will be described in detail below with reference to the attached drawings. However, the embodiments disclosed below are not limited to those disclosed below and may be implemented in various forms; the embodiments provided are merely intended to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention. The shapes of elements in the drawings may be exaggerated for clearer explanation, and elements indicated by the same reference numeral in the drawings represent the same element.

[0021] FIG. 1 is a schematic diagram showing a language processing system using a language model based on a multi-text analysis algorithm according to the present embodiment.

[0022] As illustrated in FIG. 1, a language processing system (100, hereinafter referred to as the 'processing system') using a language model based on a multiple text analysis algorithm according to the present embodiment generates a text extraction model based on a plurality of text analysis algorithms and extracts training text for language model training from original text information using the text extraction model. Then, the processing system (100) trains a language model based on the training text and performs language processing using the trained language model.

[0023] The processing system (100) may receive nuclear power plant text information from the nuclear power plant (10). Here, the nuclear power plant text information includes various historical information generated during work by a worker at the nuclear power plant (10). At this time, the historical information may include a notification provided to the worker when a defect in the nuclear power plant equipment occurs, information on the response to the defect, and information on the work performed by the worker.

[0024] The processing system (100) performs data processing on original text information to generate training data and uses it to train a language model. Then, the processing system (100) performs language processing on the original text information using the language model. Accordingly, the processing system (100) can provide information generated through language processing to a worker, and thereby provide various technical support to the worker.

[0025] Meanwhile, below, the configuration of the processing system (100) according to the present embodiment will be described with reference to FIG. 1.

[0026] The processing system (100) according to the present embodiment includes a storage unit (110) and a computation processing unit (120).

[0027] The storage unit (110) stores various types of information and provides it to the computation processing unit (120). Here, the storage unit (110) stores original text information, text analysis algorithms, and language models.

[0028] Nuclear text information can be obtained from a nuclear power plant (10).

[0029] Text analysis algorithms include Hannanum, Kkma, Komoran, Mecab, and Okt. However, there is no limitation on the types of text analysis algorithms used by the processing system (100). Furthermore, the language dictionary and the learning method for text analysis for each text analysis algorithm may differ from one another. Accordingly, each text analysis algorithm extracts text of different morphemes from text information.

[0030] The language model includes a Recurrent Neural Network-Language Model (RNN-LM). However, there is no limitation on the type of language model used by the processing system (100). Furthermore, the Recurrent Neural Network-Language Model introduces the concept of a sentence time step, and thus is not constrained by the length of the input word. The Recurrent Neural Network-Language Model recognizes the input word as the current time step and predicts the word that may follow the input word. Here, the Recurrent Neural Network-Language Model can predict words based on learned text information and can predict a predetermined sentence by repeatedly performing this process.

[0031] In addition, certain parameters can be pre-set and stored for the language model for training. For example, the configuration of language processing layers and the weights applied to the layers can be pre-set. Accordingly, when text information is input, the language model learns based on that text information, and the parameters can be adjusted accordingly.

[0032] Meanwhile, the computation processing unit (120) can retrieve data from the storage unit (110) for the training of the language model. The computation processing unit (120) can perform extraction and refinement of training text through data processing, and accordingly, can generate training data. Then, the computation processing unit (120) trains the language model based on the training data and can perform language processing using the trained language model.

[0033] Meanwhile, below, a language processing method using a language model of a processing system according to the present embodiment will be described with reference to FIGS. 2 and 3. However, the description of the components described above will be omitted and will be described by assigning the same reference numerals.

[0034] FIG. 2 is a flowchart illustrating a language processing method using a language model based on a multi-text analysis algorithm according to the present embodiment, and FIG. 3 is a flowchart illustrating a step of training a language model among the language processing methods using a language model based on a multi-text analysis algorithm according to the present embodiment.

[0035] As illustrated in FIGS. 2 and 3, the processing system (100) according to the present embodiment performs the steps of generating a text extraction model (S100), extracting training text (S200), refining training text (S300), training a language model (S400), and performing language processing (S500).

[0036] First, in the step (S100) of generating a text extraction model, the processing system (100) generates a text extraction model using a plurality of text analysis algorithms. The processing system (100) combines a plurality of text analysis algorithms to generate a plurality of algorithm combinations that include at least two or more text analysis algorithms. Then, the processing system (100) selects one of the plurality of text analysis algorithm combinations to generate a text extraction model.

[0037] Then, in the step of extracting training text (S200), the processing system (100) applies information from the source text to the generated text extraction model to extract training text. At this time, the information from the source text is applied to each text analysis algorithm of the text extraction model, and each text analysis algorithm extracts training text of different morphemes from the information from the source text.

[0038] In the step of refining training data (S300), the processing system (100) generates training data based on the training text. The processing system (100) merges the training text to generate merged data. Then, the processing system (100) removes text that is unnecessary for the training of the language model from the merged data. At this time, the processing system (100) can remove text based on the frequency of occurrence of the text included in the merged data. For example, the processing system (100) can remove particles and prepositions, etc., by removing texts with a relatively high frequency of occurrence among the texts included in the merged data. In addition, the processing system (100) can perform tagging on the parts of speech of the text included in the training text, thereby enabling the language model trained based on the training text to distinguish homonyms. In addition, the processing system (100) can convert the training text into vector values. Then, the processing system (100) can derive similarity between words by applying Euclidean distance or cosine similarity based on the vectorized training text. Accordingly, the processing system (100) performs refinement of the training text through this process and can generate training data accordingly.

[0039] In the step of training a language model (S400), the processing system (100) trains a language model based on training data. At this time, the processing system (100) trains a language model (S410), trains another language model (S420), selects a language model (S430), and evaluates a language model (S400).

[0040] In the language model training stage (S410), the processing system (100) trains the language model based on the training data. At this time, the processing system (100) adjusts the parameters of the language model to increase the accuracy of the language model, and accordingly, the processing system (100) can train the optimal language model for the training data.

[0041] In the other language model learning stage (S420), the processing system (100) may re-select a different text analysis algorithm among multiple text analysis algorithms to train a different language model. The processing system (100) applies original text information to the re-selected different text analysis algorithm to extract a different training text. Then, the processing system (100) performs refinement on the different training text as described above to generate different training data. Then, the processing system (100) trains a language model based on the different training data, thereby training it into the optimal language model for the different training data, that is, the different language model.

[0042] In the language model selection step (S430), the processing system (100) selects a language model and another language model. At this time, the processing system (100) calculates the accuracy of each of the language model and the other language model, and selects one of them based on the calculated accuracy. Accordingly, the processing system (100) can select a language model and obtain a final language model.

[0043] In the language model evaluation stage (S400), the processing system (100) performs an evaluation of the final language model. The processing system (100) performs an evaluation of the final language model based on a pre-established standard value. At this time, the processing system (100) allows the final language model to be used if the accuracy of the final language model exceeds the standard value. However, if the accuracy of the final language model does not reach the standard value, the processing system (100) learns another language model and repeats the language model selection (S430). The processing system (100) repeats the process until a language model is obtained in which the accuracy of the final language model exceeds the standard value.

[0044] And in the step of performing language processing (S500), the processing system (100) performs language processing on the original text information using the final language model that has been evaluated.

[0045] Accordingly, a language processing system using a language model according to the present invention generates a text extraction model using a plurality of text analysis algorithms and obtains training data by applying the generated text extraction model to source text information. Furthermore, the present invention can obtain an optimal language model by repeatedly performing language model training until the accuracy of the language model exceeds a threshold.

[0046] Accordingly, the language processing method and processing system using a language model based on a multi-text analysis algorithm according to the present invention can obtain a text extraction model suitable for the original text domain, and thus includes the effect that the accuracy of language processing in the original text domain can be improved by the language model.

[0047] An embodiment of the present invention described above and illustrated in the drawings should not be interpreted as limiting the technical scope of the present invention. The scope of protection of the present invention is limited only by the matters described in the claims, and a person skilled in the art may modify or change the technical scope of the present invention in various forms. Accordingly, such modifications and changes will fall within the scope of protection of the present invention insofar as they are obvious to a person skilled in the art.

Claims

Claim 1 A processing unit arbitrarily selects at least two of a plurality of text analysis algorithms that extract text into different morphemes from text information, combines the selected text analysis algorithms to generate a plurality of different algorithm combinations, and generates a text extraction model that extracts text using a text analysis algorithm included in the algorithm combination; the processing unit applies original text information provided from the original text to the text extraction model to extract a plurality of training texts through each of the text analysis algorithms of the algorithm combination; the processing unit merges each of the training texts to generate merged data, and removes unnecessary text from the merged data based on the frequency of occurrence of the text to generate training data for training a language model. and the above-mentioned computational processing unit includes the step of training a language model based on the training data and performing language processing for a nuclear power plant domain using the trained language model, wherein the nuclear power plant text information includes historical information generated during a worker's work on nuclear power plant facilities, the text analysis algorithm includes Hannanum, Kkma, Komoran, Mecab, and Okt, and the language model is a Recurrent Neural Network-Language Model;A language processing method using a language model based on a multi-text analysis algorithm, comprising: a multi-text analysis algorithm-based language model; wherein, in the step of generating the text extraction model, the processing unit selects two algorithm combinations from among a plurality of algorithm combinations and generates a first text extraction model and a second text extraction model by individually utilizing each algorithm combination; wherein, in the step of performing the language processing, the processing unit refines each training text by removing unnecessary text from each training text extracted from each text extraction model, trains the language model by individually utilizing the two generated training data, selects the language model with higher performance among the trained language models as the final language model, and performs language processing using the final language model when the accuracy of the final language model exceeds a reference value compared to the reference value; wherein the language model has parameters including a layer for language processing and weights of said layer, and when training the language model, the processing unit inputs each training data into the language model so that said parameters are adjusted to train the language model. Claim 2 delete Claim 3 delete Claim 4 delete Claim 5 delete Claim 6 delete Claim 7 delete Claim 8 delete Claim 9 delete Claim 10 delete

Images