Recommendation method for automatic kana processing using artificial intelligence

Abstract

To provide an automatic pseudonymization technique recommendation method using artificial intelligence.SOLUTION: The method includes steps of: receiving an input of a training dataset including a plurality of pieces of data including a column name, a data type, a pseudonymization technique recommendation; adding, for each of the plurality of pieces of data, a data type vector obtained from the data type to a word vector obtained corresponding to the column name to obtain a numeric vector, and labeling the numeric vector with the pseudonymization technique recommendation to generate a plurality of pieces of training data; training a first learning model using the plurality of pieces of training data so that the first learning model outputs a pseudonymization technique recommendation in response to an input of the numeric vector; obtaining a numeric vector corresponding to each column of the dataset to be pseudonymized; and inputting, into the trained first learning model, the numeric vector obtained corresponding to each column of the dataset to be pseudonymized to obtain a pseudonymization technique recommendation for each column of the dataset to be pseudonymized.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to a method for recommending a kana processing technique, and more particularly to a method for recommending an automatic kana processing technique using artificial intelligence. [Background technology]

[0002] In the past, users selected and used pseudonymization methods based on their own common sense, learning, or preferences. For example, for a string called "name," one user might choose masking, another might choose two-way encryption, and still another might choose a heuristic pseudonymization method. Different pseudonymization methods have been selected and used based on the personal experience and preferences of the user who selected the pseudonymization method.

[0003] The problem with the conventional method of selecting pseudonymous methods is that the selection of methods is mainly biased towards the experience and preferences of individual users, which means that there is a need to take over the process whenever a new person in charge changes, resulting in continuous human and time consumption. Also, there is a need to search all past processing history to refer to past methods for a specific column, which is a time-consuming task. Summary of the Invention [Problem to be solved by the invention]

[0004] The technical problem that the present invention aims to solve is to provide a method for recommending an automatic kana processing technique using artificial intelligence. [Means for solving the problem]

[0005] In order to solve the above-mentioned technical problems, a method for recommending an automatic kana processing method using artificial intelligence, which is realized by a computing device according to the present invention, includes the steps of receiving as input a training dataset containing a plurality of data items, each including a column name, a data type, and a recommended kana processing method; for each of the plurality of data items, obtaining a numeric vector by adding a data type vector obtained from the data type to a word vector obtained corresponding to the column name, and labeling the numeric vector with the recommended kana processing method, thereby generating a plurality of training data items; training a first learning model using the plurality of training data items so as to output a recommended kana processing method when a numeric vector is input; obtaining a numeric vector corresponding to each column of a dataset to be processed by kana; and inputting the obtained numeric vector corresponding to each column of the dataset to be processed by kana into the trained first learning model, thereby obtaining a recommended kana processing method for each column of the dataset to be processed by kana.

[0006] The step of obtaining a numeric type vector corresponding to each column of the dataset to be processed by kana may include the steps of inputting column names corresponding to each column of the dataset to be processed by kana into a second learning model and obtaining a word vector for each column of the dataset to be processed by kana; obtaining a data type vector corresponding to the data type of each column of the dataset to be processed by kana; and adding a data type vector to the word vector obtained for each column of the dataset to be processed by kana, and obtaining a numeric type vector corresponding to each column of the dataset to be processed by kana.

[0007] The plurality of data included in the learning dataset may further include industry classification information.

[0008] The numeric vector obtained in the step of generating the training data may further include an industrial classification vector obtained from the industrial classification information.

[0009] The step of obtaining a numeric vector corresponding to each column of the dataset to be processed by kana may include the steps of inputting column names corresponding to each column of the dataset to be processed by kana into a second learning model and obtaining a word vector for each column of the dataset to be processed by kana; obtaining a data type vector corresponding to the data type of each column of the dataset to be processed by kana; obtaining an industrial classification vector corresponding to industrial classification information previously set for the dataset to be processed by kana; and obtaining a numeric vector corresponding to each column of the dataset to be processed by kana by adding a data type vector and an industrial classification vector to the word vector obtained for each column of the dataset to be processed by kana.

[0010] The method may further include training or retraining the second learning model using a plurality of column names obtained from the training dataset.

[0011] The first training model may be a decision tree model, and the second training model may be a word embedding model.

[0012] The word embedding model may be a FastText model, and the decision tree model may be a Gradient Boosting Trees model.

[0013] The method may further include performing pseudonym processing on the dataset to be pseudonym-processed in accordance with the recommended pseudonym processing technique obtained for each column of the dataset to be pseudonym-processed.

[0014] An embodiment of the present invention for solving the above technical problem may include a computer-readable recording medium having a program recorded thereon that causes a computer to execute the method.

[0015] To solve the above technical problem, a computing device according to an embodiment of the present invention includes a processor and a memory that stores instructions or programs executable by the processor, and can perform the method when the instructions or programs are executed by the processor. [Effects of the Invention]

[0016] The present invention has the advantage that it is possible to automatically recommend a kana processing method to a user who is not familiar with kana processing work, based on the history of past applications of kana processing methods. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a block diagram illustrating an outline of a system for recommending an automatic kana processing method using artificial intelligence according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram illustrating a schematic configuration of a first computing device according to an embodiment of the present invention. [Figure 3] FIG. 2 is a block diagram illustrating a schematic configuration of a second computing device according to an embodiment of the present invention. [Figure 4] 1 is a flowchart illustrating a learning process of an artificial intelligence-based learning model according to an embodiment of the present invention. [Figure 5] 1 conceptually illustrates data processed during the learning process of an artificial intelligence-based learning model according to one embodiment of the present invention. [Figure 6] 1 is a flowchart illustrating a process for recommending a kana processing method using artificial intelligence according to an embodiment of the present invention. [Figure 7] 1 conceptually illustrates data processed in the process of recommending a kana processing method using artificial intelligence according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0018] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily carry out the present invention.

[0019] The terms used in this specification are for the purpose of describing the embodiments and are not intended to limit the present invention. In this specification, the singular includes the plural unless otherwise specified. The terms "comprises" and / or "comprising" used in this specification do not exclude the presence or addition of one or more other elements other than the elements listed. The same reference numerals refer to the same elements throughout this specification, and "and / or" includes each and every combination of one or more of the listed elements. Although terms such as "first," "second," etc. are used to describe various elements, it is understood that these elements are not limited by these terms. These terms are used merely to distinguish one element from another. Therefore, it is understood that a first element referred to below may also be a second element within the technical spirit of the present invention.

[0020] As used herein, the term "computing device" includes a variety of devices capable of performing computations and providing results to a user. For example, computing devices include desktop PCs, notebook computers, and server computers, as well as smartphones, tablet PCs, cellular phones, PCS (Personal Communication Service) phones, synchronous / asynchronous IMT-2000 (International Mobile Telecommunication-2000) mobile terminals, Palm PCs, and personal digital assistants (PDAs).

[0021] FIG. 1 is a block diagram showing an outline of a system for recommending an automatic kana processing method using artificial intelligence according to an embodiment of the present invention.

[0022] Referring to FIG. 1, a recommendation system for an automatic kana processing technique using artificial intelligence according to the present invention may include a first computing device 100 and a second computing device 200.

[0023] The first computing device 100 and the second computing device 200 can exchange various types of information and data via a communication network.

[0024] The communication network may include a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), the Internet, 2G, 3G, 4G, LTE mobile communication networks, Bluetooth (registered trademark), Wi-Fi (registered trademark), Wibro, satellite communication networks, and LPWA (Low Power Wide Area) networks such as LoRa and Sigfox, and the communication method may be any method, whether wired or wireless.

[0025] When the first computing device 100 is requested by the second computing device 200 to recommend a kana processing method for a dataset to be processed by kana, the first computing device 100 can generate and provide information on the recommended kana processing method for each column of the dataset (hereinafter referred to as "recommended kana processing method information").

[0026] The second computing device 200 is provided with recommended pseudonym processing techniques for the dataset from the first computing device 100, and is then able to perform pseudonym processing operations on the dataset.

[0027] FIG. 2 is a block diagram illustrating a schematic configuration of a first computing device according to an embodiment of the present invention.

[0028] 2, the first computing device 100 may include at least one processor 110 and at least one memory 120. In addition, the first computing device 100 may further include a communication device, a user input / output device, etc.

[0029] The processor 110 may execute instructions and / or computer programs stored in the memory 120 to implement an artificial intelligence-based method for recommending automatic kana processing techniques on the first computing device 100.

[0030] The memory 120 is a recording medium readable by the processor 110 and may store an operating system and at least one instruction and / or program, etc. The memory 120 may also store various information and data related to the execution of a method for recommending an automatic pseudonym processing technique using artificial intelligence in the first computing device 100.

[0031] FIG. 3 is a block diagram illustrating a schematic configuration of a second computing device according to one embodiment of the present invention.

[0032] 3, the second computing device 200 may include at least one processor 210 and at least one memory 220. In addition, the second computing device 200 may further include a communication device, a user input / output device, etc.

[0033] The processor 210 executes instructions and / or computer programs stored in the memory 220, and after the second computing device 200 requests the first computing device 100 to recommend a pseudonym processing method for the dataset to be pseudonym processed, the processor 210 can perform pseudonym processing on the dataset to be pseudonym processed in accordance with the recommended pseudonym processing method information provided by the first computing device 100.

[0034] The memory 220 is a recording medium readable by the processor 210 and may store an operating system and at least one or more instructions and / or programs, etc. The memory 220 may also store various information and data related to performing pseudonymization processing on a dataset to be pseudonymized in the second computing device 200.

[0035] FIG. 4 is a flowchart illustrating the learning process of an artificial intelligence-based learning model according to one embodiment of the present invention, and FIG. 5 conceptually shows data processed during the learning process of an artificial intelligence-based learning model according to one embodiment of the present invention.

[0036] 4 and 5, first, the first computing device 100 may receive a training dataset 500 for training a first training model and / or a second training model (S410). Here, the first training model and the second training model may be artificial intelligence-based neural network models. Hereinafter, the first training model will be referred to as a kana processing method recommendation model, and the second training model will be referred to as a word embedding model.

[0037] The data set 500 input in step S410 may include, for example, a plurality of data items corresponding to column names, data types, and recommended kana processing methods, as shown in Table 1.

[0038] [Table 1]

[0039] Table 1 illustrates n pieces of data, each containing a column name, a data type, and a recommended kana processing method. These pieces of data may be created by a data kana processing expert, or may be created based on existing kana-processed data. For example, a method in which an expert matches a column name-data type pair with a recommended kana processing method may be used, or a kana-processed dataset may be analyzed for each column, including the column name, data type, and the kana processing method applied to that column, to create multiple pieces of data, such as those illustrated in Table 1. Of course, training datasets created by various methods other than those described here can be used to train a kana processing method recommendation model and / or a word embedding model.

[0040] The first computing device 100 then separates (S420) column names, data types, and recommended kana processing techniques 510, 520, 530 from the dataset 500 input in step S410, and trains (S430) the word embedding model 111 using the multiple column names 510 separated from the multiple data. In step S430, the word embedding model 111 can be trained to output a corresponding quantified vector (word vector) when a column name is input, and if the word embedding model 111 has already been trained, it can be retrained in step S430.

[0041] Here, word embedding is a method of representing words as quantified vectors, and can be realized using word embedding models such as Word2Vec, FastText, and Glove (Global Vectors for Word Representation). Among the word embedding models, the FastText model, which is strong in recognizing words themselves, can be used to convert column names consisting of Hangul character strings into numeric data.

[0042] The FastText model divides words into individual characters and creates information for each character based on information about surrounding characters. In this way, the FastText model creates vector data based on the relationships between characters, so if the characters used in two different words are similar, the vector data will also be similar.

[0043] After the training of the word embedding model 111 is completed, the first computing device 100 can input each of the multiple column names 510 previously separated from the multiple data as input data to the word embedding model 111, and obtain multiple word vectors 530 for each column name 510 (S440).

[0044] Then, in step S440, the first computing device 100 can create multiple numeric vectors 550 by adding a data type vector corresponding to the data type 520 corresponding to each column name to the word vector 540 obtained for each column name (S450). For example, the numeric vectors 550 can be created by adding a numeric data type "0" and a character data type "1" to each word vector 530. Specifically, since the data type of the word vector obtained for the column name "Name" in Table 1 is "character," a numeric vector can be created by adding a "1."

[0045] Next, the first computing device 100 can label each of the multiple numeric vectors 550 created in step S450 with a recommended kana processing method 530 to generate multiple training data 560 (S460). For example, if there are n recommended kana processing methods, the labeling data can be created using a one-hot encoding method, which is an n-dimensional vector in which the position value corresponding to the recommended kana processing method is set to "1" and the remaining position values ​​are set to "0".

[0046] Finally, the first computing device 100 can train a kana processing method recommendation model using the multiple training data 560 generated in step S460 (S470). In step S470, the kana processing method recommendation model can be trained so that when a numeric vector obtained corresponding to a column name and data type is input, a recommended kana processing method is output corresponding to the input. Here, the kana processing method recommendation model can be an artificial intelligence neural network model based on learning with supervised data. For example, a map learning model such as a decision tree, Random Forest, Gradient Boosting Trees, CART (Classification and Regression Trees), ID3 (Iterative Dichotomizer 3), or C5.0 (Successor of ID3, C4.5) can be used. In particular, in this embodiment, a gradient boosting tree model that applies gradient boosting technology, an ensemble technology, can be used to achieve a more accurate decision tree model.

[0047] FIG. 6 is a flowchart showing the process of recommending a kana processing method using artificial intelligence according to one embodiment of the present invention, and FIG. 7 conceptually shows the data processed in the process of recommending a kana processing method using artificial intelligence according to one embodiment of the present invention.

[0048] 6, first, the first computing device 100 may receive a request for a recommendation of a pseudonym processing method from the second computing device 200 (S610). In step (S610), the second computing device 200 may obtain a column name and a data type for each column of the dataset to be pseudonym processed, and transfer data 700 including the column name and data type corresponding to each column to the first computing device 100. Of course, depending on the embodiment, the second computing device 200 may immediately transfer the dataset to be pseudonym processed to the first computing device 100, and the first computing device 100 may directly obtain data 700 including the column name and data type corresponding to each column of the dataset to be pseudonym processed.

[0049] The following method can be used to obtain data 700 from a dataset to be kana processed. For example, in the case of an Excel dataset such as Table 2, the contents of each cell in the first row can be checked to determine the column name, and the data values ​​from the second row to a predetermined row for each column can be checked to determine the data type.

[0050] [Table 2]

[0051] Referring to Table 2, in the first row, we can find the column names "ID," "Name," "Age," "Gender," and "Email." Then, we can check the data values ​​for each column. We can see that the data types of the first and third columns are "Numeric," and the data types of the second, fourth, and fifth columns are "Character."

[0052] Next, the first computing device 100 can input the column names 710 obtained corresponding to each column of the dataset to be kana processed into the word embedding model 111 to obtain word vectors 730 corresponding to each column (S620).

[0053] In addition, the first computing device 100 may add a data type vector corresponding to the data type 720 to the word vector 730 obtained for each column of the dataset to be kana processed, thereby creating a numeric type vector 740 for each column of the dataset to be kana processed (S630).

[0054] Next, the first computing device 100 inputs the multiple numeric vectors 740 obtained corresponding to each column of the dataset to be processed into the previously trained recommended model 1130 for the kana processing method, and can obtain a recommended kana processing method 750 for each column of the dataset to be processed (S640).

[0055] The first computing device 100 may then provide the second computing device 200 with the recommended pseudonym processing techniques obtained for each column of the dataset to be pseudonym processed (S650).

[0056] Next, the second computing device 200 can perform pseudonym processing for each column of the dataset to be pseudonym processed (S660) according to the recommended pseudonym processing method provided by the first computing device 100. According to an embodiment, before step (S660), the second computing device 200 can also provide the user with recommended information on the pseudonym processing method, allowing the user to select whether or not to apply the recommended pseudonym processing method for each column.

[0057] Meanwhile, although an embodiment in which the first computing device 100 and the second computing device 200 are separated has been described above, it is also possible to integrate the first computing device 100 and the second computing device 200 into a single device, and perform the following on a single computing device: learning a word embedding model and a kana processing method recommendation model; using the learned model to generate kana processing method recommendation information for a dataset to be processed; and performing kana processing work on a dataset to be processed.

[0058] Meanwhile, the data items that require pseudonymization may differ depending on the industry in which the data is used, and different pseudonymization methods may need to be applied. In consideration of this, according to an embodiment, each of the multiple data included in the training dataset may further include industry classification information such as education, health / medical care, finance, and communications.

[0059] For each of the multiple data included in the training dataset, a word vector, a data type vector, and an industry classification vector can be obtained, and the word vector, data type vector, and industry classification vector can be concatenated to obtain a numeric vector. For example, if there are n pieces of industry classification information, the industry classification vector can be created using one-hot encoding technology as a one-hot vector, which is an n-dimensional vector in which the position value corresponding to that industry classification information is set to "1" and the remaining position values ​​are set to "0."

[0060] As described above, training data is generated by labeling a vector corresponding to the recommended kana processing method for a numeric vector including an industry classification vector, and then a recommended model for the kana processing method is trained using the training data thus generated.

[0061] Furthermore, when the first computing device 100 receives a request from the second computing device 200 to recommend a pseudonym processing method, it can also receive industrial classification information corresponding to the industrial field in which the dataset to be pseudonym processed will be utilized. In this case, the second computing device 200 can add an industrial classification vector when creating a numeric vector for each column of the dataset to be pseudonym processed. This allows the second computing device 200 to recommend a pseudonym processing method suited to the industrial field in which the dataset to be pseudonym processed will be utilized.

[0062] The embodiments described above may be implemented using hardware components, software components, and / or a combination of hardware and software components. For example, the devices, methods, and components described in the embodiments may be implemented using one or more general-purpose or special-purpose computing devices, such as a processor, controller, arithmetic logic unit (ALU), digital signal processor, microcomputer, field programmable gate array (FPGA), programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. A processing device may execute an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a processing device may be described as being a single device, but those skilled in the art will recognize that a processing device may include multiple processing elements and / or multiple types of processing elements. For example, a processing device may include multiple processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

[0063] Software may include computer programs, code, instructions, or a combination of one or more of these, which may configure or, independently or collectively, instruct a processing device to operate as desired. The software and / or data may be permanently or temporarily embodied in some type of machine, component, physical device, virtual device, computer storage medium, or device to be analyzed by the processing device or to provide instructions or data to the processing device. The software may be distributed across network-coupled computer systems and stored or executed in a distributed manner. The software and data may be stored on one or more computer-readable storage media.

[0064] Methods according to the embodiments may be embodied in the form of program instructions executable by various computer means and recorded on a computer-readable medium. The computer-readable medium may include, alone or in combination, program instructions, data files, data structures, and the like. The program instructions recorded on the medium may be those specially designed and constructed for the embodiments, or may be those well known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape; optical media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include not only machine code, such as that produced by a compiler, but also high-level language code executable by a computer using an interpreter, for example. The above hardware devices may be configured to operate as one or more software modules to perform the operations of the embodiments, or vice versa.

[0065] Although the embodiments have been described above with limited drawings, those skilled in the art can apply various technical modifications and variations based on the above. For example, the described techniques may be performed in an order different from that described, and / or the components of the described systems, structures, devices, circuits, etc. may be combined or combined in a manner different from that described, or may be replaced or substituted by other components or equivalents, and still achieve suitable results.

Claims

1. 1. A method for recommending automatic kana processing techniques using artificial intelligence implemented in a computing device including a processor and a memory storing instructions or programs executable by the processor, the method comprising: receiving, by the computing device, a training dataset including a plurality of data, the training dataset including column names, data types, and a recommended pseudonym processing technique; the computing device generates a plurality of training data by concatenating a data type vector obtained from the data type with a word vector obtained corresponding to the column name for each of the plurality of data, and labeling the numeric vector with the recommended kana processing method; training a first learning model using the plurality of training data sets, the first learning model being configured to output a recommended kana processing method when a numeric vector is input; determining, by the computing device, a numeric vector corresponding to each column of a dataset to be pseudonymized; The computing device inputs the obtained numeric vectors corresponding to each column of the dataset to be processed into the trained first learning model to obtain a recommended kana processing method for each column of the dataset to be processed; Including, The step of obtaining a numeric vector corresponding to each column of the dataset to be kana processed includes: inputting column names corresponding to each column of the dataset to be processed using kana characters into a second learning model, and obtaining word vectors for each column of the dataset to be processed using kana characters; determining a data type vector corresponding to the data type of each column of the dataset to be kana processed; and concatenating a data type vector with the word vector obtained for each column of the data set to be kana processed, to obtain a numeric type vector corresponding to each column of the data set to be kana processed; The data types include at least a numeric type (Numeric) and a character type (Character), Recommendations for automated kana processing techniques using artificial intelligence.

2. The plurality of data included in the learning dataset further includes industry classification information, an industrial classification vector obtained from the industrial classification information is further added to the numerical vector obtained in the step of generating the learning data; The step of obtaining a numeric vector corresponding to each column of the dataset to be kana processed includes: inputting column names corresponding to each column of the dataset to be processed using kana characters into a second learning model, and obtaining word vectors for each column of the dataset to be processed using kana characters; determining a data type vector corresponding to the data type of each column of the dataset to be kana processed; A step of obtaining an industrial classification vector corresponding to industrial classification information preset for the dataset to be subjected to the pseudonym processing; a step of adding a data type vector and an industry classification vector to the word vector obtained for each column of the dataset to be kana processed, and obtaining a numeric type vector corresponding to each column of the dataset to be kana processed; The method for recommending an automatic kana processing technique using artificial intelligence according to claim 1, comprising:

3. The method for recommending an automatic kana processing technique using artificial intelligence as described in claim 1 or 2, further comprising a step of training or retraining a second learning model using a plurality of column names obtained from the training dataset.

4. the first learning model is a decision tree model; The method for recommending an automatic kana processing technique using artificial intelligence according to claim 3 , wherein the second learning model is a word embedding model.

5. the word embedding model is a FastText model, The method for recommending an automatic kana processing technique using artificial intelligence according to claim 4, wherein the decision tree model is a Gradient Boosting Trees model.

6. The method for recommending an automatic kana processing method using artificial intelligence as described in claim 1, further comprising a step of performing kana processing on the dataset to be processed according to the recommended kana processing method obtained for each column of the dataset to be processed.

7. A computer-readable recording medium storing a program for causing a computer to execute the method for recommending an automatic kana processing technique using artificial intelligence according to claim 1 or 2.

8. 1. A computing device comprising: a processor; a memory for storing instructions or programs executable by said processor; A computing device in which, when the instructions or program are executed by the processor, the method for recommending automatic kana processing techniques using artificial intelligence according to claim 1 or 2 is performed.

Images