Language processing method using language model learned with recombination data

Abstract

To provide a language processing method using a language model trained on recombinant data, which performs language processing specialized for the source domain using a language model trained on processing data that merges source text information and natural language information, the language processing method using a language model trained on recombinant data according to the present invention comprises the steps of: generating a processing dataset by merging source text information and pre-constructed natural language text information; extracting training text for language model training by natural language processing the processing dataset; training a language model based on the training text; and performing language processing specialized for the source domain using the trained language model. Accordingly, the present invention includes the effect of facilitating the training of a language model specialized for the source domain by merging source text information and natural language text information, even in the absence of source text information.

Description

Technology Field

[0001] The present invention relates to a language processing method using a language model, and more specifically, to a language processing method using a language model trained with recombinant data that performs language processing specialized for the source domain. Background Technology

[0002] Generally, nuclear power plants store information generated during operations in a nuclear power plant database. The plant can then utilize this information to provide technical support to workers currently performing tasks.

[0003] Technology utilizing such nuclear power plant information has already been disclosed in Korean Patent Publication No. 10-0991688 (Standardization method for planned preventive maintenance process management according to characteristics of nuclear power plant type and capacity, Oct. 28, 2010). The aforementioned patent includes a concept of systematic work management by standardizing various work management items required for performing planned preventive maintenance of nuclear power plants.

[0004] Meanwhile, in the field of nuclear power, research is being conducted on technologies that apply source information to language models for technical support. Conventional language models are trained based on natural language, but unlike natural language, source information includes specialized terminology and expressions. Consequently, when source information is applied to conventional language models, the accuracy of the model may suffer. Furthermore, the amount of data in source databases is insufficient to achieve the required performance of language models. Therefore, in the field of nuclear power, there is a demand for language model training methods that utilize source information to perform language processing specialized for the source domain. Prior art literature

[0005] Republic of Korea Registered Patent Publication No. 10-0991688 (Standardization Method for Planned Preventive Maintenance Process Management According to Characteristics by Nuclear Power Plant Type and Capacity, Oct. 28, 2010) The problem to be solved

[0006] The present invention aims to provide a language processing method using a language model trained on recombinant data, which performs language processing specialized for the original text domain using a language model trained on processing data that merges original text information and natural language information. means of solving the problem

[0007] A language processing method using a language model trained with recombinant data according to the present invention comprises the steps of: generating a processing dataset by merging source text information and pre-constructed natural language text information; extracting training text for language model training by natural language processing of the processing dataset; training a language model based on the training text; and performing language processing specialized for the source domain using the trained language model.

[0008] The above nuclear power plant text information can be obtained from each of a plurality of nuclear power plants containing different nuclear power plant facilities.

[0009] In the step of generating the processing data above, the original text information is integrated to generate integrated data, and natural language text information is merged into the integrated data to generate the processing dataset.

[0010] The above nuclear power plant text information includes first and second nuclear power plant text information obtained from first and second nuclear power plants including nuclear power plant facility a, and third nuclear power plant text information obtained from third nuclear power plants including nuclear power plant facility b. In the step of generating the above processing dataset, the first to third nuclear power plant text information is integrated to generate integrated data, and the data included in the integrated data is classified based on the type of facility to generate multiple first classification data for nuclear power plant facility a and multiple second classification data for nuclear power plant facility b, one of the first and second classification data is selected, and natural language information is merged with the selected classification data to generate the above processing dataset.

[0011] The above nuclear power plant text information includes first and second nuclear power plant text information obtained from first and second nuclear power plants including nuclear power plant facility a, third nuclear power plant text information obtained from third nuclear power plants including nuclear power plant facility b, and fourth nuclear power plant text information obtained from fourth nuclear power plants including nuclear power plant facility c. In the step of generating the above processing dataset, the first to fourth nuclear power plant text information is integrated to generate integrated data, and the data included in the integrated data is classified based on the type of facility to generate first classification data for a plurality of nuclear power plant facilities a, second classification data for nuclear power plant facilities b, and third classification data for nuclear power plant facilities c. The first to third classification data are arbitrarily combined to generate first aggregate data and second aggregate data in which at least two or more classification data among the nuclear power plants are aggregated, and natural language text information is merged into each of the first and second aggregate data to generate first and second processing datasets.

[0012] In the step of extracting the training text, a first training text is extracted from the first processing dataset and a second training text is extracted from the second processing dataset; in the step of performing the language processing, a first language model is trained based on the first training text and a second language model is trained based on the second training text, and language processing can be performed using either the first or second language model.

[0013] In the step of performing the above language processing, in the language processing for the fifth nuclear power plant including nuclear power plant facility d, the nuclear power plant facility d is compared with nuclear power plant facilities a to c to search for nuclear power plant facilities similar to nuclear power plant facility d among nuclear power plant facilities a to c, similar classification data for the similar nuclear power plant facility is searched among the plurality of classification data, a processing dataset containing the similar classification data is searched among the plurality of processing datasets, a language model trained based on the training text extracted from the searched processing dataset is searched among the trained language models, and the fifth nuclear power plant text information obtained from the fifth nuclear power plant is input into the model having the highest performance among the searched language models to perform language processing.

[0014] In the step of extracting the training text, the processing dataset is applied to a text analysis algorithm to extract the training text, and the text analysis algorithm may include any one of Hannanum, Kkma, Komoran, Mecab, and Okt.

[0015] The above language model may include a Recurrent Neural Network-Language Model (RNN-LM). Effects of the invention

[0016] A language processing method using a language model trained with recombinant data according to the present invention includes the following effects.

[0017] First, the present invention includes the effect of improving accuracy so that the language model can be applied to nuclear power plant sites.

[0018] Second, the present invention includes the effect of facilitating the training of a language model specialized for the source domain by merging source text information and natural language text information, even in the event of a lack of source text information.

[0019] The technical effects of the present invention as described above are not limited to those mentioned above, and other unmentioned technical effects will be clearly understood by those skilled in the art from the description below. Brief explanation of the drawing

[0020] FIG. 1 is a schematic diagram showing a language processing system using a language model trained with recombinant data according to the present embodiment. FIG. 2 is a flowchart illustrating a language processing method using a language model trained with recombinant data according to the present embodiment. FIG. 3 is a flowchart illustrating the step of generating a processing dataset among a language processing method using a language model trained with recombinant data according to the present embodiment. FIG. 4 is a flowchart showing the steps of performing language processing among a language processing method using a language model trained with recombinant data according to the present embodiment. Specific details for implementing the invention

[0021] Embodiments of the present invention will be described in detail below with reference to the attached drawings. However, the embodiments disclosed below are not limited to those disclosed below and may be implemented in various forms; the embodiments provided are merely intended to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention. The shapes of elements in the drawings may be exaggerated for clearer explanation, and elements indicated by the same reference numeral in the drawings represent the same element.

[0022] FIG. 1 is a schematic diagram showing a language processing system using a language model trained with recombinant data according to the present embodiment.

[0023] As illustrated in FIG. 1, a language processing method using a language model trained with recombinant data according to the present embodiment can be applied to a processing system (100) provided in a source environment. The processing system (100) is connected to a source server (10) and can receive various source information from the source server (10). The processing system (100) trains a language model based on source information and performs language processing specialized for the source domain using the trained language model.

[0024] The nuclear power plant server (10) may receive nuclear power plant information from a plurality of nuclear power plants (30). The nuclear power plant information includes information on nuclear power plant facilities and nuclear power plant text information. Here, nuclear power plant facilities may include various facilities constituting the nuclear power plant (30), but as an example to explain this embodiment, it will be described that the nuclear power plant facilities include reactor type, type, and unit. Furthermore, the information on nuclear power plant facilities, that is, nuclear power plant facility information, includes information on the reactor type, type, and unit of each nuclear power plant (30). The nuclear power plant text information includes history information and work information generated during work, such as inspection of nuclear power plant facilities, by a worker at the nuclear power plant (30). Accordingly, the nuclear power plant information may be provided to the processing system (100) with the nuclear power plant text information labeled as nuclear power plant facility information.

[0025] Additionally, the source server (10) can receive information from the processing system (100), and accordingly, the processing system (100) can provide data to the source server (10), such as a learned language model.

[0026] Meanwhile, below, the configuration of the processing system (100) according to the present embodiment will be described with reference to FIG. 1.

[0027] As illustrated in FIG. 1, the processing system (100) according to the present embodiment includes a storage unit (110) and a computation processing unit (120).

[0028] First, the storage unit (110) stores various types of information and can provide information stored in the computation processing unit (120). The storage unit (110) can store nuclear power plant equipment information, nuclear power plant text information, and information for learning a language model provided from the nuclear power plant server (10).

[0029] Here, nuclear power plant equipment information and nuclear power plant text information may be provided from and stored by the nuclear power plant server (10). And information for learning the language model includes natural language text information, a text analysis algorithm, and a language model.

[0030] Natural language text information includes commonly used languages, such as everyday conversations and general documents. This natural language text information may differ from the specialized terminology, expressions, and terms included in the source domain.

[0031] Text analysis algorithms include Hannanum, Kkma, Komoran, Mecab, and Okt. However, there are no limitations on the text analysis algorithms used by the processing system (100). Furthermore, each text analysis algorithm may have different learned language dictionaries and learning methods for text analysis. Accordingly, the processing system (100) can extract text into different morphemes from the same text information using each text analysis algorithm.

[0032] The language model includes a Recurrent Neural Network-Language Model (RNNLM). However, the processing system (100) is not limited to the language model used for language processing. Furthermore, the Recurrent Neural Network-Language Model may not fix the length of the input word by introducing the concept of a time step into the sentence structure. The Recurrent Neural Network-Language Model recognizes the input word as the current time step and predicts the word for the next, or subsequent, time step after the input word. Then, the Recurrent Neural Network-Language Model recognizes the predicted word again as the current time step and, accordingly, predicts the word for the time step after the predicted word. The Recurrent Neural Network-Language Model repeatedly performs the word prediction process based on learned text information, and thereby can predict a predetermined sentence.

[0033] Such a language model may have specific parameters pre-set and stored for training. For example, the language model may have the reading order of text information, the configuration of layers for text prediction, and the weights of each layer pre-set. Accordingly, the language model can perform training when training text is input, and the trained language model can predict words or sentences.

[0034] Meanwhile, the computation processing unit (120) can retrieve various information from the source server (10) or storage unit (110) to perform learning of the language model. The computation processing unit (120) performs data processing for each piece of information and can extract training text for learning the language model. In addition, the computation processing unit (120) can perform learning of the language model based on the training text. In addition, the computation processing unit (120) performs language processing using the learned language model.

[0035] Meanwhile, below, a language processing method of a processing system according to the present embodiment will be described with reference to FIGS. 2 to 4. However, detailed descriptions of the components described above will be omitted, and they will be described by assigning the same reference numerals.

[0036] FIG. 2 is a flowchart illustrating a language processing method using a language model trained with recombinant data according to the present embodiment, and FIG. 3 is a flowchart illustrating the step of generating a processing dataset among the language processing method using a language model trained with recombinant data according to the present embodiment. FIG. 4 is a flowchart illustrating the step of performing language processing among the language processing method using a language model trained with recombinant data according to the present embodiment.

[0037] As illustrated in FIGS. 2 to 4, the processing system (100) according to the present embodiment performs the steps of generating a processing dataset (S100), extracting training text (S200), training a language model (S300), and performing language processing (S400).

[0038] First, in the step of generating a processing dataset (S100), the processing system (100) performs integrated data generation (S110), classification data generation (S120), summation data generation (S130), and natural language text information merging (S140).

[0039] In the integrated data generation (S110), the processing system (100) integrates nuclear power plant text information to generate integrated data. The processing system (100) integrates nuclear power plant text information for each nuclear power plant (30) provided by the nuclear power plant server (10). At this time, the processing system (100) also integrates the nuclear power plant equipment information of the nuclear power plant (30) that provided the nuclear power plant text information.

[0040] Then, in the generation of classification data (S120), the processing system (100) classifies the integrated data to generate classification data. The processing system (100) can classify the integrated data based on the type of nuclear power plant equipment. For example, the processing system (100) can classify the integrated data based on one of the following: by reactor type, by type of nuclear power plant, and by unit. Accordingly, the processing system (100) can generate multiple classification data classified by nuclear power plant equipment from the integrated data.

[0041] In the generation of aggregated data (S130), the processing system (100) generates aggregated data based on the classification data. The processing system (100) arbitrarily combines multiple classification data to generate multiple aggregated data in which at least two or more classification data among the nuclear power plant facilities are aggregated. At this time, the classification data combined and aggregated for each aggregated data may be different from one another.

[0042] And in the step of merging natural language text information (S140), the processing system (100) merges natural language text information into each aggregated data. Accordingly, the processing system (100) can generate multiple different processing datasets.

[0043] In the step of extracting training text (S200), the processing system (100) inputs a plurality of processing datasets individually into a text analysis algorithm to extract training text for each processing dataset. At this time, the processing system (100) inputs the data into any one of the text analysis algorithms of Hannanum, Kkma, Komoran, Mecab, and Okt to extract the text. The processing system (100) can then perform refinement on the extracted training text.

[0044] For example, the processing system (100) can remove words that are unnecessary for the learning of the language model from the training text, such as prepositions and articles. At this time, the processing system (100) can perform the removal based on the frequency of occurrence of the training text. The processing system (100) can also perform tagging on the parts of speech of the training text, and accordingly, can train the language model to distinguish homonyms. In addition, the processing system (100) can convert the training text into vector values. Based on the vectorized training text, the processing system (100) can derive the similarity between words in the training text by applying Euclidean distance or cosine similarity, etc.

[0045] In the step (S300) of training a language model, the processing system (100) trains a language model based on each training data. The processing system (100) trains the language model by inputting each training data into it. At this time, the processing system (100) continuously calculates accuracy during the training process and can adjust the parameters of the language model to increase the calculated accuracy. Accordingly, the processing system (100) trains the language model with the optimal language model for each training data.

[0046] In the step of performing language processing (S400), the processing system (100) performs language processing using a learned language model. At this time, the processing system (100) performs target source information acquisition (S410), language model search (S420), and language processing execution (S430).

[0047] In obtaining target nuclear power plant information (S410), the processing system (100) obtains target nuclear power plant facility information from the target nuclear power plant.

[0048] In the language model search (S420), the processing system (100) searches for a language model suitable for the target nuclear power plant from multiple language models based on the target nuclear power plant facility information. Here, the processing system (100) compares the target nuclear power plant facility information with the nuclear power plant facility information to search for similar facility information among the nuclear power plant facility information that is similar to the target nuclear power plant facility information. Then, the processing system (100) searches for classification data classified as similar facility information among the classification data. Then, the processing system (100) selects the aggregated data in which the searched classification data is aggregated from among the multiple aggregated data. Then, the processing system (100) selects language models trained with training text extracted from the processing data generated from the selected aggregated data among the multiple language models. Then, the processing system (100) finally selects the language model with the highest performance among the selected language models.

[0049] In the language processing execution (S430), the processing system (100) performs language processing by inputting the target source text information provided from the target source into the finally selected language model.

[0050] Thus, a language processing system using a language model according to the present invention can train a language model based on nuclear power plant text information provided from each nuclear power plant. Here, the present invention performs language model training using nuclear power plant text information merged with natural language text information, and can perform language processing specialized for the nuclear power plant domain using the trained language model.

[0051] Accordingly, the present invention includes the effect of improving accuracy so that the language model can be applied to the nuclear power plant site.

[0052] In addition, the present invention includes the effect of facilitating the training of a language model specialized for the source domain by merging source text information and natural language text information, even in the event of a lack of source text information.

[0053] An embodiment of the present invention described above and illustrated in the drawings should not be interpreted as limiting the technical scope of the present invention. The scope of protection of the present invention is limited only by the matters described in the claims, and a person skilled in the art may modify or change the technical scope of the present invention in various forms. Accordingly, such modifications and changes will fall within the scope of protection of the present invention insofar as they are obvious to a person skilled in the art.

Claims

Claim 1 A step in which a computational processing unit merges original text information and pre-constructed natural language text information to generate a processing dataset; a step in which the computational processing unit performs natural language processing on the processing dataset to extract training text for language model training; The method includes the step of the computational processing unit training a language model based on the training text and performing language processing specialized for the nuclear power plant domain using the trained language model, wherein the nuclear power plant text information includes history information and work information generated during an inspection of nuclear power plant facilities by a worker, and the natural language text information includes language used in everyday conversations and general documents, and the nuclear power plant text information is obtained from each of a plurality of nuclear power plants including different nuclear power plant facilities, and includes first and second nuclear power plant text information obtained from first and second nuclear power plants including nuclear power plant facility a, third nuclear power plant text information obtained from third nuclear power plants including nuclear power plant facility b, and fourth nuclear power plant text information obtained from fourth nuclear power plants including nuclear power plant facility c, and in the step of generating the processing dataset, the computational processing unit integrates the first to fourth nuclear power plant text information to generate integrated data, and classifies the data included in the integrated data based on the type of facility to generate first classification data for nuclear power plant facility a, second classification data for nuclear power plant facility b, and the A third classification data for nuclear power plant facility c is generated, the first to third classification data are arbitrarily combined to sum at least two classification data among the nuclear power plant facilities to generate multiple different summed data, and the natural language text information is individually merged into each summed data to generate multiple processing datasets, and in the step of extracting the training text, the computational processing unit extracts the training text individually from each processing dataset, and in the language processing for a fifth nuclear power plant including nuclear power plant facility d, in the step of performing the language processing, the computational processing unit,A language processing method using a language model trained with recombinant data, characterized by comparing the above nuclear power plant facility d with the above nuclear power plant facilities a to c to search for a nuclear power plant facility similar to the above nuclear power plant facility d among the above nuclear power plant facilities a to c, searching for similar classification data for the above similar nuclear power plant facility among the above classification data, searching for a processing dataset containing the above similar classification data among the above processing datasets, searching for a language model trained with the above training text extracted from the above searched processing dataset among the above trained language models, inputting the fifth nuclear power plant text information obtained from the fifth nuclear power plant into the model having the highest performance among the above searched language models to perform language processing, wherein in the step of extracting the above training text, the computational processing unit applies the above processing dataset to a text analysis algorithm to extract the above training text, wherein the text analysis algorithm includes any one of Hannannum, Kkma, Komoran, Mecab, and Okt, and the language model includes a Recurrent Neural Network-Language Model (RNN-LM). Claim 2 delete Claim 3 delete Claim 4 delete Claim 5 delete Claim 6 delete Claim 7 delete Claim 8 delete Claim 9 delete

Images