Programming language conversion device, method for converting programming language, and programming language conversion system

Abstract

To provide a programming language conversion device which can convert a source code written in a programming language before a transition when the programming code is used in a legacy system to a source code written in a programming language after transition.SOLUTION: A programming language conversion device 100 includes: a data masking unit 220 for masking a specific type of token of a first source code written in a first programming language with a place holder different according to tokens; and a pre-learning unit 250 and a main learning unit 260 for learning the first source code masked by the place holder and making a translation model for converting the first source code to a second source code written in a second programming language.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a programming language conversion device, a programming language conversion method, and a programming language conversion system. [Background technology]

[0002] In recent years, machine translation from a pre-migration programming language (e.g., COBOL language) with a small amount of source code to a new language (e.g., Java (registered trademark) language) with a large amount of source code has been considered. However, in conventional rule-based machine translation, it is very costly to implement and update the rules and tools. For this reason, programming language conversion processing based on deep learning using AI (Artificial Intelligence) has been considered. The programming language conversion processing is, for example, a process of converting existing source code created in a pre-migration programming language into source code created in a post-migration programming language.

[0003] Deep learning is mainly carried out in three phases: "dataset preprocessing," "pre-training," and "actual training." Dataset preprocessing is the process of converting source code written in two different programming languages ​​for training into a form that can be input to a deep learning model. Pre-training is the process of teaching a deep learning model the characteristics (e.g., syntax, grammar, etc.) of two programming languages. This training is a process in which a deep learning model learns to convert source code in an old programming language into source code in a new programming language, thereby mapping source code in the old programming language to source code in the new programming language.

[0004] Conventional data set pre-processing will now be described. In conventional dataset preprocessing, a conventional programming language conversion device extracts source code written in the programming language before migration from various software project folders. Based on the extracted source code, a selection process is performed to remove missing source code, and initial processing is performed, and the source code is then compiled into a single dataset file. For example, the dataset file contains one line per source code file.

[0005] Next, a conventional programming language conversion device converts the dataset file into a dataset suited to the respective purposes of pre-learning and main learning. The pre-learning dataset is a dataset in which each file of source code is compiled on one line, and has the same format as the dataset file created in the above-mentioned dataset pre-processing. On the other hand, the main learning dataset is a dataset in which each function (e.g., method or function) of the source code is compiled on one line. Therefore, one line of data in the pre-learning dataset becomes multiple lines of data in the main learning dataset. Although the compiled dataset file formats are different between the pre-learning dataset and the main learning dataset, the data content is the same.

[0006] Next, a conventional programming language conversion device converts the pre-training dataset and the training dataset into binary files that can be input into a deep learning model through standard natural language processing processes (such as word segmentation).

[0007] In the pre-learning, a conventional programming language conversion device creates a language model by associating a pre-learning data set created from source code written in a programming language before migration with a pre-learning data set created from source code written in a programming language after migration. The language model is basic knowledge such as grammar and syntax related to the programming language in which the source code is written, and is also referred to as a translation model in the following description.

[0008] In this training, a conventional programming language conversion device takes as input a training dataset created from source code written in the pre-migration programming language and a training dataset created from source code written in the post-migration programming language, adds a translation layer to the trained language model, and constructs and trains a translation model.

[0009] Non-Patent Document 1 discloses a technique for unsupervised translation of programming languages. [Prior art documents] [Non-patent literature]

[0010] [Non-Patent Document 1] Marie-Anne Lachaux, et al. [online] Unsupervised Machine Translation of Programming Languages, [Retrieved June 1, 2023] Internet (URL: https: / / arxiv.org / abs / 2006.03511) Summary of the Invention [Problem to be solved by the invention]

[0011] Deep learning-based machine translation basically requires a parallel data set, i.e. a data set with bilingual translations. In addition, in order to create a deep learning model, a large amount of data is required for each of the above-mentioned processes. However, source codes written in pre-migration programming languages ​​such as COBOL used in legacy systems are not publicly available, and there are fewer source codes compared to natural language samples. For this reason, unlike natural language translation, parallel data sets often do not exist in the first place for programming language conversion, so unsupervised learning is necessary. However, since there is little data available for learning, it has been difficult to improve the quality of deep learning models created by unsupervised learning.

[0012] Furthermore, when conventional programming language conversion processing is applied as is, the following problems arise. The first challenge is that strings and constants written in the source code overwhelm the dictionary, causing learning noise. A dictionary is a collection of all tokens contained in preprocessed source code. There is a limit to the size of the dictionary for learning. When a dictionary is built through preprocessing similar to natural language processing, the dictionary is overwhelmed by tokens that do not affect the logic of the program. For example, many strings and constants do not affect the logic of a program, and source code created in Japan is often written in Japanese. Words with the same meaning often appear different words to a deep learning model, so the dictionary becomes more bloated than necessary.

[0013] In addition, in a deep learning model, it is necessary to assign a unique vector to each token without exception and train the model, and in the training process, tensors are assigned to all tokens. Tokens that appear infrequently (e.g., comments written in source code) become noise in the training process, and the training quality of the vector itself becomes low. For example, source code written in a programming language before the migration may have annotations written in Japanese or special constants defined. If such annotations and constants are included in the dictionary used in the dataset preprocessing, the dictionary will be overwhelmed and will easily become noise in pre-training and main training. Furthermore, due to the nature of a deep learning model being a black box, noise may have a negative impact on the entire model.

[0014] The second challenge is that in the case of unsupervised learning, if the topics (functions) of the training data in both languages ​​are different, it significantly affects the translation quality of the translation model. For example, if the pre-migration program was created on the assumption that it would be batch processing executed locally, and the post-migration program was created on the assumption that it would be database access processing executed on the web, the topics of each program will be different. In this way, the quality of translation will decrease in a translation model trained by matching the pre-migration program and the post-migration program, which have different topics.

[0015] The present invention has been made in view of the above circumstances, and has an object to improve the quality of a translation model for converting source code written in a first programming language into a second programming language. [Means for solving the problem]

[0016] The programming language conversion device according to the present invention includes a masking unit that masks specific types of tokens in a first source code written in a first programming language with a different placeholder for each token, and a learning unit that learns the source code masked with the placeholders and creates a translation model for converting the first source code into a second source code written in a second programming language. Effect of the Invention

[0017] According to the present invention, the quality of the translation model is improved, so that the source code written in a first programming language can be correctly translated into the source code written in a second programming language. Problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0018] [Figure 1] 1 is a block diagram showing an example of a hardware configuration of a programming language conversion device according to a first embodiment of the present invention. [Diagram 2] 1 is a block diagram showing an example of a functional configuration of a programming language conversion device according to a first embodiment of the present invention. [Diagram 3] FIG. 2 is a diagram showing the contents of a source file group according to the first embodiment of the present invention. [Figure 4] FIG. 2 is a diagram showing the contents of a mask source file group according to the first embodiment of the present invention. [Diagram 5] FIG. 2 is a diagram showing the contents of a placeholder mapping group according to the first embodiment of the present invention. [Figure 6]FIG. 2 is a diagram showing the contents of a pre-learning data group according to the first embodiment of the present invention. [Figure 7] FIG. 2 is a diagram showing the contents of a main training data group according to the first embodiment of the present invention. [Figure 8] 4 is a flowchart showing processing in a learning process of the programming language conversion device according to the first embodiment of the present invention. [Figure 9] 5 is a flowchart showing an example of detailed processing of a data masking process according to the first embodiment of the present invention. [Figure 10] FIG. 2 is a diagram illustrating an example of an AST according to the first embodiment of the present invention. [Figure 11] 4 is a flowchart showing an example of data processing for pre-learning according to the first embodiment of the present invention. [Figure 12] FIG. 2 is a diagram showing an example of a dictionary according to the first embodiment of the present invention. [Figure 13] 4 is a flowchart showing an example of data processing for main learning according to the first embodiment of the present invention. [Figure 14] 4 is a flowchart showing an example of processing in a translation process of the programming language conversion device according to the first embodiment of the present invention. [Figure 15] FIG. 11 is a block diagram showing an example of the configuration of a programming language conversion system according to a second embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. In this specification and the drawings, components having substantially the same functions or configurations are designated by the same reference numerals, and redundant description will be omitted.

[0020] [First embodiment] Fig. 1 is a block diagram showing an example of a hardware configuration of a programming language conversion device 100 according to the first embodiment. The programming language conversion device 100 is an example of a computer capable of implementing the programming language conversion process according to the present embodiment. The programming language conversion device 100 according to the present embodiment implements a programming language conversion method in which each functional block shown in Fig. 2, which will be described later, cooperates with each other by executing the programming language conversion process according to the present embodiment.

[0021] The programming language conversion device 100 includes a processor 110 , a main memory device 120 , an auxiliary memory device 130 , an input device 140 , an output device 150 and a communication device 160 .

[0022] The processor 110 reads out program code of software for realizing each function according to the present embodiment from the main memory device 120 and executes it. For example, a read only memory (ROM) or a random access memory (RAM) is used as the main memory device 120. Variables, parameters, etc. generated during the arithmetic processing of the processor 110 are temporarily written to the main memory device 120, and these variables, parameters, etc. are read out by the processor 110 as appropriate. However, for the processor 110, a central processing unit (CPU), a micro processing unit (MPU), a graphics processing unit (GPU), etc. are used.

[0023] The input device 140 may be, for example, a keyboard, a mouse, etc., and allows the user to input predetermined operations and give instructions. The output device 150 may be, for example, a liquid crystal display monitor, and displays the results of the processing performed by the programming language conversion device 100 to the user.

[0024] The auxiliary storage device 130 may be, for example, a hard disk drive (HDD), a solid state drive (SSD), a magneto-optical disk, or a non-volatile memory. In addition to an operating system (OS) and various parameters, a program for operating the programming language conversion device 100 is recorded in the auxiliary storage device 130. The auxiliary storage device 130 records programs, data, and the like required for the processor 110 to operate, and is used as an example of a computer-readable non-transitory storage medium that stores a program executed by the programming language conversion device 100.

[0025] The communication device 160 may be, for example, a network interface card (NIC), and various data may be transmitted and received between devices via a local area network (LAN) or dedicated line connected to a terminal of the NIC.

[0026] Next, an example of the functional configuration of the programming language conversion device 100 and each database will be described with reference to FIGS. Fig. 2 is a block diagram showing an example of a functional configuration of the programming language conversion device 100. In the following explanation, each process will be explained with the numbers indicating the process order given in Fig. 2.

[0027] The programming language conversion device 100 includes a data filtering unit 210, a data masking unit 220, a pre-learning data processing unit 230, a main learning data processing unit 240, a pre-learning unit 250, a main learning unit 260, a translation unit 270, a post-processing unit 280, and an information storage unit 300. The information storage unit 300 includes a source file group 310, a learning candidate group 320, a mask source file group 330, a pre-learning data group 340, a main learning data group 350, a translation model group 360, and a placeholder mapping group 370.

[0028] (1) Reading source files First, the data filtering unit 210 filters the first source code based on the topic. For example, the data filtering unit 210 reads a source file written in the pre-migration program and a source file written in the post-migration program from the source file group 310 of the information storage unit 300, and filters the source code written in the source file. For this purpose, the data filtering unit 210 uses one or more types of filtering methods on the data set of the post-migration programming language, and classifies the source code by topic based on a part or all of the source code, making use of the characteristics of the programming language. Then, the data filtering unit 210 filters out source code with obviously different topics (uses).

[0029] Here, an example of a filtering method used in the data filtering unit 210 will be described. (a) Keyword-based filtering method The data filtering unit 210 filters, for example, source code written in Java based on the keyword of the package name of the library to be imported, and filters out source code with a different purpose. For example, if com.android is written in the source code, it is source code for a mobile application, and if HttpServlet is written, it is source code for a Web application, so both are subject to filtering out. In this way, when filtering source code, the data filtering unit 210 can arbitrarily combine information such as import, comments, annotations, etc. written in the source code and determine whether to exclude this information.

[0030] (b) Topic-based classification methods The data filtering unit 210 utilizes the characteristics of the programming language to split variable names, identify source code into individual words, and then perform filtering using a topic classification method. In filtering using a topic classification method, a static analysis of the source code is performed, and tokenization (splitting) is performed on all source code. For example, if a character string written in the source code is "CamelCase", it is split into two words, "camel" and "case". Also, if the character string is "snake_case", which is two words connected by an underscore, the underscore is removed and it is split into two words, "snake" and "case".

[0031] Since programming languages ​​have strict structures, the data filtering unit 210 can extract only a portion of information from source code using a topic-based classification method, and can significantly improve the efficiency of classification with little loss of accuracy. The data filtering unit 210 can also exclude specific types of tokens, such as character strings (e.g., comments and annotations), written in source code from being input to the topic classification method by performing AST (Abstract Syntax Tree) analysis. Furthermore, the data filtering unit 210 can also classify comments and annotations into topics by extracting only one of the comments and annotations and inputting it to the topic classification method.

[0032] As a topic classification method, a method may be used in which clustering is performed on all or part of the source code using a statistical method such as Linear Discriminant Analysis (LDA) and clearly different topics are detected based on the words with high weights in each cluster. When using the above-mentioned AST analysis and LDA together, the execution time of LDA can be significantly reduced by removing unnecessary data in advance using AST analysis and then performing LDA. Furthermore, a method based on deep learning (e.g., neural network) may be used as a topic classification method.

[0033] FIG. 3 is a diagram showing the contents of the source file group 310. As shown in FIG. The source file group 310 includes a source file 311 written in COBOL language (an example of a first programming language) as a pre-migration program, and a source file 312 written in Java language as a post-migration program. The processing function performed by the source file 311 is the same as the processing function performed by the source file 312. The source files 311 and 312 show tokens 311a, 311b, 312a, and 312b to be masked by a data masking process described later. The tokens are character strings, constants, and the like written in the source code, and are to be replaced with placeholders. The tokens are identified by word-level division using a division algorithm for natural language processing, or by sub-word-level division using a statistical method.

[0034] (2) Writing candidates for learning Returning to the explanation of Figure 2. The data filtering unit 210 performs a filtering process based on source files 311 written in the pre-migration program read from the source file group 310 and source files 312 written in the post-migration program. The filtering process removes source files that are unnecessary for both pre-learning and main learning. After that, the data filtering unit 210 writes data that can be used as learning data into a learning candidate group 320 in the information storage unit 300 as learning candidates. The learning candidate group 320 is a collection of multiple source files.

[0035] (3) Reading out the learning candidates Next, the data masking unit 220 masks the filtered first source code with a different placeholder for each token. For example, the data masking unit 220 reads out learning candidates from the information storage unit 300 and performs a data masking process to mask a predetermined portion in each source file.

[0036] (4) Masking process Next, the data masking unit 220 masks a specific type of token in the first source code of the source file 311 written in the COBOL language with a different placeholder for each token. For this purpose, the data masking unit 220 uses a parser that analyzes a program to replace constants and character strings in the source code of the pre-migration program and the post-migration program with placeholders to create a data set in which the source files subjected to data masking processing are used as mask source files. After that, the data masking unit 220 writes the mask source files to the mask source file group 330 in the information storage unit 300.

[0037] (4A) Map writing Additionally, the data masking unit 220 writes a map indicating the correspondence between the placeholders and the constants and character strings to the placeholder mapping group 370 of the information storage unit 300. In the placeholder mapping group 370, the original character strings masked in the source file and the placeholders that masked the original character strings are recorded in association with each other.

[0038] FIG. 4 is a diagram showing the contents of the mask source file group 330. As shown in FIG. The mask source file group 330 includes a mask source file 331 in which a part of a source file 311 written in the pre-migration program is masked, and a mask source file 332 in which a part of a source file 312 written in the post-migration program is masked. The tokens 311a, 311b, 312a, and 312b shown in Fig. 3 are masked by mask information 331a, 331b, 332a, and 332b, respectively.

[0039] FIG. 5 is a diagram showing the contents of the placeholder mapping group 370. As described above, the mask information is also written to the placeholder mapping group 370. A placeholder is a different character string that replaces a character string written in an original source file. The placeholder mapping 371 indicates that the original character string "MAINLOOP" of the source file 311 written in the COBOL language is masked by a placeholder called "$var1", and the original character string "CLOSE" is masked by a placeholder called "$var2". The placeholder mapping 372 indicates that the original character string "MAINLOOP" of the source file 312 written in the Java language is masked by a placeholder called "$var1", and the original character string "CLOSE" is masked by a placeholder called "$var2". In this way, the mask information 331a, 331b, 332a, and 332b shown in FIG. 4 are read as placeholders.

[0040] (5) and (6) Processing of data for pre-learning Returning to the explanation of Figure 2. The pre-learning data processing unit 230 performs a process of converting the first source code masked with a placeholder into pre-learning data. For example, the pre-learning data processing unit 230 reads the mask source files 331 and 332 from the mask source file group 330 and performs data processing for pre-learning. After that, the pre-learning data processing unit 230 writes the pre-learning data files 341 and 342 obtained by performing data processing for pre-learning on the mask source files 331 and 332 into the pre-learning data group 340 of the information storage unit 300.

[0041] FIG. 6 is a diagram showing the contents of the pre-learning data group 340. As shown in FIG. The pre-training data group 340 includes pre-training data files 341 and 342 that are the results of pre-training data processing performed on the mask source files 331 and 332. The file name of the pre-training data file 341 is given the name “COBOL_pretrain_dataset”, and the file name of the pre-training data 342 is given the name “Java_pretrain_dataset”.

[0042] The pre-learning data file 341 is a single line of source code written in the mask source file 331 shown in FIG. 4. Note that in the drawing, the right end of the source code is wrapped around. "NEW_LINE" is added to each line break in the pre-learning data file 341. A semicolon is added to each line break in the pre-learning data 342. The pre-learning data files 341 and 342 contain placeholders 341a, 341b, 342a, and 342b that correspond to the placeholders added to the mask source files 331 and 332.

[0043] Returning to the explanation of Figure 2. (7) and (8) Pre-learning process The pre-learning unit 250 learns the first source code masked with a placeholder and creates a translation model for converting the first source code into a second source code written in the Java language (an example of a second programming language). The translation model is an example of a deep learning model created in this embodiment.

[0044] The pre-learning unit 250 reads the pre-learning data files 341, 342 from the pre-learning data group 340 and performs pre-learning. As described above, the pre-learning is a process of learning the characteristics (syntax, grammar, etc.) of the pre-migration programming language and the post-migration programming language, and creates a translation model that has learned the characteristics of the COBOL language and the Java language based on the pre-learning data. After that, the pre-learning unit 250 writes the result of the pre-learning as a translation model in the translation model group 360 of the information storage unit 300. The translation model is, for example, a single binary file.

[0045] (9) and (10) Processing of data for this study Next, the main learning data processing unit 240 performs a process of converting the first source code masked with the placeholder into main learning data. For example, the main learning data processing unit 240 reads the mask source files 331 and 332 from the mask source file group 330 and performs main learning data processing. After that, the main learning data processing unit 240 writes main learning data files 351 and 352 obtained by performing main learning data processing on the mask source files 331 and 332 into the main learning data group 350 in the information storage unit 300.

[0046] In addition, the process of (9) reading a mask source file in the processing of data for this training and the process of (5) reading a mask source file in the processing of data for pre-training described above may be performed simultaneously or their order may be reversed.

[0047] FIG. 7 is a diagram showing the contents of the main training data group 350. As shown in FIG. The main training data group 350 includes main training data files 351 and 352 that are the results of main training data processing performed on the mask source files 331 and 332. The file name of the main training data file 351 is given the name "COBOL_train_dataset", and the file name of the main training data 352 is given the name "Java_train_dataset".

[0048] The main training data file 351 is obtained by deleting unnecessary information from the mask source file 331 for main training. The main training data files 351 and 352 include placeholders 351a, 351b, 352a, and 352b that correspond to the placeholders added to the mask source files 331 and 332.

[0049] Returning to the explanation of Figure 2. (11), (12), (13) Processing of this study Next, the main learning unit 260 creates a translation model based on the data for main learning. For example, the main learning unit 260 reads main learning data files 351 and 352 from the main learning data group 350, reads a pre-trained translation model from the translation model group 360, and performs main learning. The main learning is a process of creating a translation model for converting source code written in a programming language before the migration into source code written in a programming language after the migration. After that, the main learning unit 260 writes the result of the main learning into the translation model group 360 of the information storage unit 300 as a translation model.

[0050] (14) Translation processing Next, the translation unit 270 applies the translation model to the first source code to convert it into a second source code. For example, the translation unit 270 obtains a translation result by applying data input from the outside to the translation model read from the translation model group 360. The data input from the outside is, for example, source code written in the programming language before the migration. However, it is necessary to make the data input from the outside in a state in which the translation model can be applied. Therefore, the main learning data processing unit 240 performs main learning data processing on the data input from the outside. The translation model is applied to the data that has been subjected to the main learning data processing.

[0051] The translation result obtained from the translation unit 270 is in a state where the character string has been replaced with a placeholder. If the user does not instruct post-processing and there is a desire to obtain the translation result in this state, the translation unit 270 outputs a second source code in which a specific token is masked with a placeholder. In other words, the translation unit 270 outputs the translation result in a state where the character string has been replaced with a placeholder. The output translation result is displayed or printed by the output device 150 shown in FIG. 1.

[0052] (15) Post-processing When a user instructs post-processing, the post-processing unit 280 performs post-processing to return the placeholder to its original state, and outputs the post-processed second source code. For example, the post-processing unit 280 replaces the placeholder of the source code translated by the translation unit 270 with the original character string obtained by referring to the placeholder mapping group 370, and automatically restores the source code. The post-processed translation result is displayed or printed by the output device 150 shown in FIG. 1.

[0053] <Details of each process> Next, the details of the programming language conversion method performed in each functional unit of the programming language conversion device 100 will be described with reference to Fig. 8 to Fig. 13. In the following description, the process of writing various data to the information storage unit 300 may be omitted.

[0054] 8 is a flowchart showing the processing of the learning process of the programming language conversion device 100. Here, the processing up to the actual learning will be explained.

[0055] First, the data filtering unit 210 filters the source files 311, 312 read as learning data from the source file group 310 (S1). The filtered source files 311, 312 are stored in a candidate group 320 for learning.

[0056] Next, the data masking unit 220 performs a data masking process (S2). In the data masking process, the data masking unit 220 masks the source files 311 and 312 that have been read out from the learning candidate group 320 as learning data.

[0057] Next, the pre-learning data processing unit 230 performs pre-learning data processing (S3). In the pre-learning data processing, the pre-learning data processing unit 230 performs pre-learning preprocessing on both the source code written in the programming language before the migration and the source code written in the programming language after the migration. The results of the pre-learning data processing are stored in the pre-learning data group 340.

[0058] Next, the pre-learning unit 250 performs pre-learning based on the pre-learning data files 341 and 342 read from the pre-learning data group 340 (S4).

[0059] Next, the main learning data processing unit 240 performs main learning data processing (S5). In the main learning data processing, pre-processing for main learning is performed on both the source code written in the programming language before the main learning data processing unit 240 is migrated and the source code written in the programming language after the migration. The result of the main learning data processing is stored in the main learning data group 350.

[0060] Finally, main learning unit 260 performs main learning using main learning data files 351, 352 read from main learning data group 350 and the translation model read from translation model group 360 (S6), and ends the main process. After main learning is performed, the translation model is stored in translation model group 360.

[0061] Fig. 9 is a flowchart showing an example of detailed processing of the data masking process shown in step S2 of Fig. 8. In the following description, information included in the placeholder mapping group 370 and indicating the correspondence between the original character strings and the placeholders is called a map.

[0062] First, the data masking unit 220 creates a map (S11). Next, the data masking unit 220 analyzes the source code of the source file 311 read from the learning candidate group 320 in Fig. 2, and converts it into an AST (S12). The conversion process into an AST is performed by, for example, a parser included in the data masking unit 220. Here, the AST will be described with reference to Fig. 10.

[0063] FIG. 10 is a diagram illustrating an example of an AST. An AST is a tree structure that represents the grammatical structure of source code. In the AST, tokens (character strings, constants, etc.) of the parsed source code are converted into nodes nx (x is an integer equal to or greater than 0). Node n0 is assigned to the root of the AST. Furthermore, leaves are generated for node n0, with node numbers assigned to them, such as nodes n1, n2, .... By converting the source code into an AST in this way, the program for the language conversion processing according to this embodiment becomes easier to handle.

[0064] Returning to FIG. After step S12, the data masking unit 220 repeats the following process for each node of the AST (S13). First, the data masking unit 220 determines whether the node is a character string or a constant (S14). The node determined in step S14 is subject to the determination regardless of the characters contained in the character string. For example, a period added to a character string is also subject to the determination.

[0065] If the node is neither a character string nor a constant (NO in S14), the data masking unit 220 proceeds to step S20, counts up the node number by "1", and performs the determination in step S14 again.

[0066] On the other hand, if the node is either a character string or a constant (YES in S14), the data masking unit 220 determines whether the value of the node has already been recorded in the map (S15). For example, it is determined whether the original character string "MAINLOOP" shown in FIG. 5 has been recorded in the map.

[0067] If the value of the node has already been recorded in the map (YES in S15), the data masking unit 220 rewrites the value of the node to the placeholder recorded in the map (S16).

[0068] On the other hand, if the node value is not recorded in the map (NO in S15), the data masking unit 220 creates a new placeholder in the map (S17). Next, the data masking unit 220 adds the node value and the correspondence between the placeholder to the map (S18). For example, if the original character string "MAINLOOP" shown in FIG. 5 is not recorded in the map, "$var1" is created as a new placeholder, and the correspondence between "MAINLOOP" and "$var1" is added to the map.

[0069] Next, the data masking unit 220 rewrites the value of the node in the AST with the new placeholder recorded in the map (S19). After the NO determination in step S14, S16, or S19, the process proceeds to step S20, where the data masking unit 220 counts up the node number by "1" and performs the determination in step S14 again.

[0070] After the repeated processing for all the nodes of the AST is completed, the data masking unit 220 converts the processed AST including the nodes rewritten by the placeholders into source code and outputs it (S21). Then, the process returns to step S3 in FIG.

[0071] FIG. 11 is a flowchart showing an example of the pre-learning data processing shown in step S3 of FIG.

[0072] The pre-learning data processing unit 230 starts repeated processing for each of the mask source files 331, 332 read from the mask source file group 330 (S31). Next, the pre-learning data processing unit 230 performs static analysis of the read mask source files (S32). Here, it is assumed that the static analysis is performed on the mask source file 331 first.

[0073] Next, the pre-learning data processing unit 230 processes the mask source file 331 so that the source code is reduced to one line based on the result of the static analysis (S33). Next, the pre-learning data processing unit 230 writes the processing result to the pre-learning data file 341, which is a language file corresponding to the mask source file (S34).

[0074] Next, the pre-learning data processing unit 230 determines whether there is an unprocessed mask source file (S35). If there is an unprocessed mask source file, the process returns to step S32 and static analysis is performed. Here, static analysis is performed on the mask source file 332.

[0075] If it is determined in step S35 that there are no unprocessed mask source files, the repeated processing ends for the mask source files 331, 332. Next, the pre-learning data processing unit 230 combines each of the mask source files 331, 332 into a single file (S36).

[0076] Next, the pre-learning data processing unit 230 performs BPE (Byte Pair Encoding) subword division (S37). BPE is a process of further dividing low-frequency words in a document using statistical methods. Words obtained by further dividing low-frequency words are called subwords. Division information is generated by BPE subword division. For example, the character string "@@" written in the pre-learning data files 341 and 342 shown in FIG. 6 indicates that a character string has been divided by BPE subword division, and the divided character string including "@@" is called division information. For example, the character string 341c in FIG. 6 is "dc-@@ func-@@ code", which indicates that "DC-FUNC-CODE" in the character string 331c shown in FIG. 4 has been divided.

[0077] Finally, the pre-learning data processing unit 230 writes the dictionary to the pre-learning data group 340, and writes the division information to the translation model group 360 (S38). After that, the process returns to step S4 in FIG.

[0078] FIG. 12 is a diagram illustrating an example of the dictionary. As described above, the dictionary is a collection of all tokens written in the preprocessed source code. Fig. 12 shows an example of a dictionary created based on the source file 311 shown in Fig. 3. The dictionary is created with unnecessary tokens removed by the preprocessing. This dictionary is then written to the pre-training data set 340.

[0079] FIG. 13 is a flowchart showing an example of the main learning data processing shown in step S5 of FIG.

[0080] The main learning data processing unit 240 starts repeated processing for each of the mask source files 331, 332 read from the mask source file group 330 (S41). Next, the main learning data processing unit 240 performs static analysis of the read mask source files (S42). Here, it is assumed that the static analysis is performed on the mask source file 331 first.

[0081] Next, the main learning data processing unit 240 processes the mask source file 331 so that each method is written in one line based on the result of the static analysis (S43). Next, the main learning data processing unit 240 writes the processing result to the main learning data file 351, which is a language file corresponding to the mask source file (S44).

[0082] Next, the main learning data processing unit 240 determines whether there is an unprocessed mask source file (S45). If there is an unprocessed mask source file, the process returns to step S42 and static analysis is performed. Here, static analysis is performed on the mask source file 332.

[0083] If it is determined in step S45 that there are no unprocessed mask source files, the repeated processing ends for the mask source files 331, 332. Next, the main learning data processing unit 240 combines each of the mask source files 331, 332 into a single file (S46).

[0084] Next, the main learning data processing unit 240 performs BPE subword division (S47). Division information is generated by the BPE subword division.

[0085] Finally, the main training data processing unit 240 writes the dictionary and the division information to the main training data group 350 (S48). After that, the process returns to step S6 in FIG.

[0086] FIG. 14 is a flowchart showing an example of the processing in the translation process of the programming language conversion device 100.

[0087] First, the data masking unit 220 masks the pre-migration program, which is data input from the outside (S51).

[0088] Next, the translation unit 270 performs pre-processing on the masked pre-migration program (S52). This pre-processing is the main learning data processing by the main learning data processing unit 240 shown in Fig. 2. In this pre-processing, the division information is used.

[0089] Next, the translation unit 270 reads out a translation model from the translation model group 360, and translates the pre-processed pre-migration program into a post-migration program (S53). Next, the translation unit 270 determines whether or not there is a post-processing instruction from the user (S54).

[0090] The output of the deep learning model is uncertain. Even if the placeholders in the output source code are matched to the map, there is no guarantee that the source code written in the correct migrated programming language will be obtained. For this reason, it may be necessary for the user to visually check the placeholders and return them to the original strings.

[0091] If there is no instruction for post-processing (NO in S54), the translation unit 270 outputs the translated post-migrated program (S56). The post-migrated program that is output without post-processing is in a state where it is masked with a placeholder. For example, the mask source file 332 shown in FIG. 4 is displayed on the output device 150. Note that the map read from the placeholder mapping group 370 may also be displayed on the output device 150.

[0092] The masked character string may be written in Japanese, and when changing or correcting this character string to English, it is desirable for the post-migration program to be in a state where it is masked with placeholders. For this reason, the user manually operates the input device 140, checking the placeholders one by one, and restoring the placeholders to the original character string.

[0093] The maps stored in the placeholder mapping group 370 may have different correspondences between placeholders and character strings for each project that uses the source code. In this case, by converting the placeholders to character strings according to the project, source code with different descriptions for each project is created even if the processing content is the same.

[0094] If there is an instruction for post-processing (YES in S54), the post-processing unit 280 carries out the post-processing (S55). The post-processing replaces the placeholders in the migrated program with the original character strings. Thereafter, the translation unit 270 outputs the translated migrated program (S56). The migrated program that has been post-processed and output has the placeholders replaced with the original character strings. For example, the source file 312 shown in FIG. 3 is displayed on the output device 150. The user checks whether the placeholders in the displayed source file 312 have been correctly restored to the original character strings.

[0095] In the programming language conversion device 100 according to the first embodiment described above, the data filtering unit 210 classifies source code by topic and filters out source files with different topics, thereby improving the quality of the translation model.

[0096] In addition, the data masking unit 220 uses a parser to create a mask source file in which constants and character strings written in each source code are replaced with placeholders, and stores the correspondence between the constants and character strings and the placeholders in the placeholder mapping group 370. This prevents character strings and constants from overwhelming the dictionary as in the past, and reduces noise in pre-learning and main learning. As a result, learning efficiency is improved, and a certain level of quality can be maintained with a small legacy language dataset.

[0097] Furthermore, the pre-learning unit 250 creates a translation model that has been pre-learned based on the pre-learning source file. Furthermore, the main learning unit 260 performs main learning using the translation model based on the main learning source file. The translation model created in this way can be used by the translation unit 270 to obtain high-quality translation results. Furthermore, by providing the translation model to other programming language conversion devices 100 or incorporating it into a translation program (not shown), the translation quality in devices other than the programming language conversion device 100 that created the translation model can also be improved.

[0098] In addition, the translation unit 270 can translate an externally input source file written in a pre-migration programming language into a source file written in a post-migration programming language by applying the translation model. Therefore, the time required for translation can be shortened compared to conventional methods.

[0099] In addition, highly accurate translation is possible using the translation model updated through this learning.

[0100] [Second embodiment] Next, a programming language conversion system according to a second embodiment of the present invention will be described. FIG. 15 is a block diagram showing an example of the configuration of a programming language conversion system 1000.

[0101] The programming language conversion system 1000 includes a preprocessing device 1100, a translation model creation device 1200, and a DB device 1300, which are connected to one another via a network N and are capable of transmitting and receiving various types of data.

[0102] The preprocessing device 1100 includes the data filtering unit 210 and the data masking unit 220 shown in Fig. 2. Before the learning process according to this embodiment, the service provider performs data filtering and data masking.

[0103] The translation model creation device 1200 includes the pre-learning data processing unit 230, the main learning data processing unit 240, the pre-learning unit 250, the main learning unit 260, the translation unit 270, and the post-processing unit 280 shown in FIG.

[0104] 2, the source file group 310, the learning candidate group 320, the mask source file group 330, the pre-learning data group 340, the main learning data group 350, the translation model group 360, and the placeholder mapping group 370. The DB device 1300 may be a device managed by a service provider.

[0105] By configuring the programming language conversion system 1000 in this way, it is possible to perform data filtering and data masking in advance. Therefore, the translation model creation device 1200 can perform pre-learning and main learning based on the mask source file that has been subjected to data masking, and create a translation model.

[0106] [Variations] The programming language conversion device 100 according to the embodiment described above is an example in which the present invention is applied to unsupervised learning, but the present invention may also be applied to supervised learning. In supervised learning, the data filtering unit 210 may be removed from the programming language conversion device 100.

[0107] Also, the programming language before migration may be C, and the programming language after migration may be Python. Another possible combination is that the programming language before migration is JCL (Job Control Language), and the programming language after migration is Java. In this way, the combination of the programming language before migration and the programming language after migration may be arbitrary.

[0108] Even if the programming language is the same, the command format may differ depending on the version. For example, Python version 2.X and version 3.X have different formats and are not compatible. In this case, the pre-migration program can be created in Python version 2.X (first programming language), and the post-migration program can be created in Python version 3.X (second programming language).

[0109] The present invention is not limited to the above-described embodiment, and various other applications and modifications are possible without departing from the gist of the present invention as set forth in the claims. For example, the above-mentioned embodiment describes the system configuration in detail and specifically in order to explain the present invention in an easily understandable manner, and is not necessarily limited to a system having all of the described configurations. In addition, it is also possible to add, delete, or replace part of the configuration of the present embodiment with other configurations. In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. In reality, it can be considered that almost all components are connected to each other. [Explanation of symbols]

[0110] 100... programming language conversion device, 110... processor, 120... main memory device, 210... data filtering unit, 220... data masking unit, 230... data processing unit for pre-learning, 240... data processing unit for main learning, 250... pre-learning unit, 260... main learning unit, 270... translation unit, 280... post-processing unit, 300... information storage unit, 310... source file group, 320... learning candidate group, 330... mask source file group, 340... pre-learning data group, 350... main learning data group, 360... translation model group, 370... placeholder mapping group

Claims

1. a masking unit that masks specific types of tokens in a first source code written in a first programming language with placeholders that differ for each token; a learning unit that learns the first source code masked with the placeholders and creates a translation model for converting the first source code into second source code written in a second programming language. Programming language converter.

2. a filtering unit that filters the first source code based on a topic; The masking unit masks the filtered first source code with a different placeholder for each token.

2. The programming language conversion device according to claim 1.

3. a translation unit that applies the translation model to the first source code to convert it into the second source code; 3. The programming language conversion device according to claim 2.

4. The translation unit outputs the second source code masked with the placeholder when there is no instruction for post-processing.

4. The programming language conversion device according to claim 3.

5. a post-processing unit that performs post-processing to restore the placeholder to its original state when a post-processing instruction is given, and outputs the second source code that has undergone post-processing; 4. The programming language conversion device according to claim 3.

6. a pre-learning data processing unit that converts the first source code masked with the placeholder into pre-learning data; The learning unit includes a pre-learning unit that creates the translation model that has learned features of the first programming language and the second programming language based on the pre-learning data.

6. The programming language conversion device according to claim 5.

7. a data processing unit for actual training that converts the first source code masked with the placeholder into data for actual training; The learning unit includes a main learning unit that creates the translation model based on the main learning data.

7. The programming language conversion device according to claim 6.

8. a masking unit that masks specific types of tokens in a first source code written in a first programming language with placeholders that differ for each token; learning the first source code masked with the placeholders and creating a translation model for converting the first source code into second source code written in a second programming language; Programming language conversion methods.

9. A preprocessing device and a translation model creation device are provided, The pretreatment device comprises: masking specific types of tokens in a first source code written in a first programming language with placeholders that differ for each of the tokens; The translation model creation device a learning unit that learns the source code masked with the placeholders and creates a translation model for converting the first source code into a second source code written in a second programming language; Programming language conversion system.

Images