Code conversion device, method, and program

Abstract

To provide a code conversion device that can convert existing codes into codes capable of obtaining the same results to reduce data processing time.SOLUTION: Input means 81 receives an input of a target program code. Extraction means 82 extracts, from the target program code, a first program code indicating that aggregation processing of each column in a data frame is performed on the basis of a predetermined column value for a data frame representing data in a two-dimensional tabular format, and a second program code indicating predefined processing to be performed on each data frame for which the aggregation processing is performed and a function that calls the predefined processing. Conversion means 83 converts the extracted codes into alternative program code, which is program code indicating a process that does not make a function call that invokes the predefined processing by the function and that can obtain the same result as the processing by the first program code and the second program code.SELECTED DRAWING: Figure 18

Description

[Technical field]

[0001] The present disclosure relates to a code conversion device, a code conversion method, and a code conversion program that convert a code used by a computer for processing. [Background technology]

[0002] To efficiently use data stored in a database, program code is generally defined and processing is performed based on the defined code. In recent years, the demand and market for analyzing large amounts of data has expanded, and in order to analyze large amounts of data at high speed, it is necessary to create programs that are conscious of speed.

[0003] For example, Patent Document 1 describes a system that analyzes a data processing program created by a defined script and sequentially processes data. When analyzing the script, the system described in Patent Document 1 converts or complements arguments of application program interface (API) functions so that they match the arguments of the original API functions in the library. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2017-120611 A Summary of the Invention [Problem to be solved by the invention]

[0005] It is possible to generate a program capable of executing data processing from a script by using the system described in Patent Document 1. However, the system described in Patent Document 1 does not take into consideration the time required for data processing.

[0006] It is expected that the time required for processing will increase as the amount of data to be processed increases. Therefore, it is preferable to be able to convert existing code into code that can obtain similar results in order to reduce the time required for data processing.

[0007] Therefore, an object of the present disclosure is to provide a code conversion device, a code conversion method, and a code conversion program that can convert existing code into code that produces similar results so as to reduce the processing time for data. [Means for solving the problem]

[0008] The code conversion device according to the present disclosure is characterized by comprising an input means for accepting input of a target program code, which is the program code to be processed; an extraction means for extracting from the target program code a first program code indicating an aggregation process for each column in a data frame representing two-dimensional tabular data based on the values ​​of a specified column, and a second program code indicating a predefined process to be performed on each data frame on which the aggregation process has been performed and a function that invokes that process; a conversion means for converting the extracted first program code and second program code into an alternative program code, which is program code indicating a process that does not perform a function call to invoke the process using the function, and that obtains results similar to those obtained by the process by the first program code and the second program code; and an output means for outputting the alternative program code.

[0009] The code conversion method according to the present disclosure is characterized in that it accepts input of a target program code, which is the program code to be processed, extracts from the target program code a first program code that indicates an aggregation process for each column in a data frame representing two-dimensional tabular data based on the values ​​of a specified column, and a second program code that indicates a predefined process to be performed on each data frame on which the aggregation process has been performed and a function that invokes that process, converts the extracted first program code and second program code into an alternative program code, which is program code that indicates a process that does not perform a function call to invoke the process using the function, and that obtains results similar to those obtained by the process by the first program code and the second program code, and outputs the alternative program code.

[0010] The code conversion program according to the present disclosure is characterized in that it causes a computer to execute an input process for accepting input of a target program code, which is the program code to be processed; an extraction process for extracting from the target program code a first program code indicating an aggregation process for each column in a data frame representing two-dimensional tabular data based on the values ​​of a specified column, and a second program code indicating a predefined process to be performed on each data frame on which the aggregation process has been performed and a function that invokes that process; a conversion process for converting the extracted first program code and second program code into an alternative program code, which is program code indicating a process that does not perform a function call to invoke the process using the function, and that obtains results similar to those obtained by the process by the first program code and the second program code; and an output process for outputting the alternative program code. Effect of the Invention

[0011] According to the present disclosure, existing code can be converted to code that achieves similar results so that the processing time on the data can be reduced. [Brief description of the drawings]

[0012] [Figure 1] 1 is a block diagram showing an example of the configuration of an embodiment of a code conversion device; [Diagram 2] FIG. 11 is an explanatory diagram illustrating an example of an aggregation process. [Diagram 3] FIG. 11 is an explanatory diagram showing an example of a correspondence relationship between codes. [Figure 4] 13 is a flowchart showing an example of the operation of the code conversion device. [Diagram 5] 13 is a flowchart illustrating an example of an extraction process. [Figure 6] 13 is a flowchart illustrating an example of an extraction process. [Figure 7] 13 is a flowchart illustrating an example of a conversion process. [Figure 8] 13 is a flowchart illustrating an example of a conversion process. [Figure 9] FIG. 11 is an explanatory diagram showing an example of an extraction process and a conversion process. [Figure 10] FIG. 11 is an explanatory diagram showing a first specific example of the conversion process; [Figure 11] FIG. 11 is an explanatory diagram showing a first specific example of the conversion process; [Figure 12] FIG. 11 is an explanatory diagram showing a second specific example of the conversion process. [Figure 13] FIG. 11 is an explanatory diagram showing a second specific example of the conversion process. [Figure 14] FIG. 11 is an explanatory diagram showing a third specific example of the conversion process. [Figure 15] FIG. 11 is an explanatory diagram showing a third specific example of the conversion process. [Figure 16] FIG. 11 is an explanatory diagram showing a fourth specific example of the conversion process. [Figure 17] FIG. 11 is an explanatory diagram showing a fourth specific example of the conversion process. [Figure 18] 1 is a block diagram showing an overview of a code conversion device according to the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, an embodiment of the present disclosure will be described with reference to the drawings. In the following, a case where data represented in a two-dimensional tabular format is processed will be described. An example of data represented in such a format is a data frame. In the following description, data in a two-dimensional tabular format may also be referred to as a data frame.

[0014] 1 is an explanatory diagram showing an example of the configuration of an embodiment of a code conversion device according to the present disclosure. The code conversion device 100 according to this embodiment includes a storage unit 10, an input unit 20, a code extraction unit 30, a code conversion unit 40, and an output unit 50.

[0015] The storage unit 10 stores various information used when the code conversion device 100 performs processing. The storage unit 10 may also store information received by the input unit 20 (described later) and processing results by the code conversion unit 40. The storage unit 10 is realized by, for example, a magnetic disk.

[0016] The input unit 20 accepts input of a program code to be processed (hereinafter, referred to as a target program code). The input unit 20 may acquire the target program code from the storage unit 10, for example, or may acquire the target program code from an external storage server (not shown). The input unit 20 may also accept input of the target program code from a user via a user interface.

[0017] In the following description, Python is exemplified as a programming language, and Pandas is exemplified as a library used to write the target program code. However, the target program code does not need to be written based on Pandas, and any programming language other than Python can be used. The code extraction unit 30 and the code conversion unit 40 described later only need to perform processing according to the rules of the target program code used.

[0018] The code extraction unit 30 extracts, from the input target program code, program code that can be converted into code that can shorten the processing time for a data frame. Specifically, the code extraction unit 30 extracts program code (hereinafter referred to as a first program code) that indicates performing an aggregation process for each column in a two-dimensional tabular data (i.e., a data frame) based on the values ​​of a predetermined column. In the case of Pandas, an example of the aggregation process extracted as the first program code is the groupby() method.

[0019] Fig. 2 is an explanatory diagram showing an example of aggregation processing. Fig. 2 shows an example of processing for dividing the data in the data frame df into groups of survivors and deceased people (Survived) and calculating the average age.

[0020] Furthermore, the code extraction unit 30 extracts, from the input target program code, a program code (hereinafter, referred to as a second program code) indicating a predefined process to be performed on each data frame on which the above-mentioned aggregation process has been performed and a function that calls that process. In the case of Pandas, an example of a function extracted as the second program code is the apply() method, and an example of a predefined process is an external function (user-defined function).

[0021] The code conversion unit 40 converts the extracted code (more specifically, the first program code and the second program code) into code that can shorten the processing time for the data frame. Note that the conversion of the code here includes not only the conversion of the extracted code itself, but also the conversion of other code that becomes necessary as a result of the conversion (is affected by the conversion).

[0022] Specifically, the code conversion unit 40 converts the extracted code into program code (hereinafter, alternative program code) that indicates processing that does not make a function call to call processing by the extracted function and that obtains results similar to those of the processing by the extracted code. That is, the code conversion unit 40 converts the target program code so that processing on the aggregated data frame can be realized by processing that does not make a function call to an external function.

[0023] The code conversion unit 40 may generate an alternative program code by converting a program code included in a function indicated by the second program code into a predetermined corresponding program code. In this case, a correspondence relationship between a code that can reduce a processing time (converted code) and a code that can be converted into the code (pre-conversion code) may be stored in advance in the storage unit 10, and the code conversion unit 40 may read the correspondence relationship and perform the conversion process.

[0024] It is preferable that the code conversion unit 40 converts the first program code and the second program code into an alternative program code when the type of the argument of the function matches the type of the return value of the function. This ensures that the program code is converted into a process that produces the same result. An example of a case where the types match is when both arguments are data frames.

[0025] FIG. 3 is an explanatory diagram showing an example of the correspondence between codes. FIG. 3 shows an example of the correspondence used in the case of Pandas. For example, the second row of the correspondence table shown in FIG. 3 shows that the code (df[“C”]=df.groupby([“A”][“B”].transform(“first”)) that can reduce the processing time corresponds to the process (g[“C”]=g[“B”]+iloc[0]) that overwrites the values ​​of other elements with the value of the element in the first row of the aggregated data frame.

[0026] For example, in the case of Pandas shown above, the target program code is assumed to be code in which an external function func() is called by a function call using the apply() method, and that external function processes the data frame that has been aggregated using the groupby() method.

[0027] In this case, the code conversion unit 40 converts the process of calling the external function func() using the apply() method into program code for a process that can obtain the same result without calling the external function func().

[0028] In the processing by the alternative program code converted in this way, the processing of calling functions is suppressed, so that the processing time for data can be shortened even if the program code achieves the same results. Note that the specific processing using the correspondence relationship illustrated in FIG. 3 will be described later.

[0029] The output unit 50 outputs the alternative program code. The output unit 50 may display the alternative program code on a display device (not shown), or may store the alternative program code in the storage unit 10. The output unit 50 may also transmit the alternative program code to a device (not shown) that executes the alternative program code.

[0030] The input unit 20, the code extraction unit 30, the code conversion unit 40, and the output unit 50 are realized by a computer processor (e.g., a CPU (Central Processing Unit), a GPU (Graphics Processing Unit)) that operates according to a program (a code conversion program).

[0031] For example, the program may be stored in the storage unit 10 of the code conversion device 100, and the processor may read the program and operate as the input unit 20, the code extraction unit 30, the code conversion unit 40, and the output unit 50 in accordance with the program. Furthermore, the functions of the code conversion device 100 may be provided in a SaaS (Software as a Service) format.

[0032] Furthermore, the input unit 20, the code extraction unit 30, the code conversion unit 40, and the output unit 50 may each be realized by dedicated hardware. Furthermore, a part or all of the components of each device may be realized by a general-purpose or dedicated circuit, processor, etc., or a combination of these. These may be configured by a single chip, or may be configured by multiple chips connected via a bus. A part or all of the components of each device may be realized by a combination of the above-mentioned circuits, etc., and a program.

[0033] In addition, when some or all of the components of the code conversion device 100 are realized by a plurality of information processing devices, circuits, etc., the plurality of information processing devices, circuits, etc. may be centrally arranged or distributed. For example, the information processing devices, circuits, etc. may be realized as a client-server system, a cloud computing system, etc., in a form in which each is connected via a communication network.

[0034] Next, the operation of the code conversion device 100 of this embodiment will be described. Fig. 4 is a flowchart showing an example of the operation of the code conversion device 100 of this embodiment. The input unit 20 accepts an input of a target program code (step S11). The code extraction unit 30 extracts, from the target program code, a first program code indicating an aggregation process to be performed, and a second program code indicating a process to be performed on the data frame on which the aggregation process has been performed and a function that calls that process (step S12).

[0035] The code conversion unit 40 converts the extracted program code into an alternative program code that can obtain the same result without calling a function (step S13). Then, the output unit 50 outputs the alternative program code (step S14).

[0036] Next, a specific example of the process performed by the code extraction unit 30 will be described. In the following description, a specific example using the above-mentioned Pandas will be described. In addition, in the following description, in accordance with the correspondence table illustrated in FIG. 3, an operation of referencing and overwriting with an iloc property, an arithmetic operation, and an aggregation operation will be exemplified as codes to be extracted by the code extraction unit 30. However, the codes to be extracted are not limited to these codes. Any process can be extracted as long as it is a process that does not involve a function call and a conversion process that can obtain similar results can be defined in the correspondence table.

[0037] 5 and 6 are flowcharts showing an example of extraction processing by the code extraction unit 30. Here, a function in the target program code is denoted as Func, and a data frame that is an argument of the function call is denoted as g.

[0038] First, the code extraction unit 30 searches for the function Func to be called, and determines whether or not the data frame returned by the function is the same as the data frame input to the function (i.e., as an argument) (step S101 in FIG. 5). If they are the same (Yes in step S101), the code extraction unit 30 performs a process to determine whether the function can be accelerated. In doing so, the code extraction unit 30 performs a check process starting from the bottom line in the function (step S102). On the other hand, if they are not the same, the code extraction unit 30 determines that the function cannot be accelerated, and ends the process (step S120).

[0039] The code extraction unit 30 reads one line of processing in the function Func (step S103). The code extraction unit 30 determines whether the operations on both the right and left sides of the read line (hereinafter referred to as the target line) are operations on g (step S104). If both the right and left sides are operations on g (Yes in step S104), the code extraction unit 30 determines whether those operations may change the number of lines of g (step S105).

[0040] If there is a possibility that the number of lines of g can be changed (Yes in step S105), the code extraction unit 30 determines that the processing cannot be accelerated and ends the process (step S120). On the other hand, if there is no possibility that the number of lines of g can be changed (No in step S105), the code extraction unit 30 determines whether the operation on the target line is an operation of referencing and overwriting by the iloc property (hereinafter referred to as the first operation) (step S106). If the operation on the target line is the first operation (Yes in step S106), the code extraction unit 30 sets flag #1 to the target line (step S130).

[0041] Similarly, if the operation on the target row is not the first operation (No in step S106) but is an arithmetic operation (Yes in step S107), the code extraction unit 30 sets flag #1 to the target row (step S130). Furthermore, if the operation on the target row is not an arithmetic operation (No in step S107) but is an aggregation operation (Yes in step S108), the code extraction unit 30 sets flag #1 to the target row (step S130). Then, if the operation on the target row is not an aggregation operation (No in step S108), the code extraction unit 30 determines that the processing cannot be accelerated and ends the processing (step S120). Note that the order of the processing from step S106 to step S108 may be reversed.

[0042] In step S130 after setting flag #1 for the target line, code extraction unit 30 determines whether or not there is a line above the target line (i.e., a previous process) (step S109). If there is a line above the target line (Yes in step S109), code extraction unit 30 sets the line above as the target line (step S110) and repeats the process from step S103 onwards, which reads one line of the process in function Func.

[0043] On the other hand, if there is no line above the target line (No in step S109), the code extraction unit 30 determines whether or not flag #1 is set for all lines in the function Func (step S111). If flag #1 is not set for all lines (No in step S111), the code extraction unit 30 determines that speed-up is not possible and ends the process (step S120). On the other hand, if flag #1 is set for all lines (Yes in step S111), the code extraction unit 30 determines that speed-up is possible (step S140).

[0044] In step S104, if neither the right nor left side is an operation on g (No in step S104), that is, if processing is performed on a data frame df other than g, processing from step S201 onward illustrated in FIG. 6 is performed.

[0045] Specifically, the code extraction unit 30 determines whether a data frame df other than g has been generated by an equality filter using the key to be aggregated (groupkey) (step S201). If a data frame df other than g has been generated (Yes in step S201), the code extraction unit 30 sets flag #1 to the target line and replaces the flag of the line to which flag #2 is set with flag #1 (step S202). After that, the process from step S109 onwards in FIG. 5 is performed.

[0046] On the other hand, if a data frame df other than g has not been generated (No in step S201), the code extraction unit 30 determines whether the operation of the target row is an operation that refers to data (hereinafter, referred to as a second operation) (step S203). If the operation of the target row is the second operation (Yes in step S203), the code extraction unit 30 sets flag #2 to the target row (step S210).

[0047] Similarly, if the operation on the target row is not the second operation (No in step S203) but is an arithmetic operation (Yes in step S204), the code extraction unit 30 sets flag #2 on the target row (step S210). Furthermore, if the operation on the target row is not an arithmetic operation (No in step S204) but is an aggregation operation (Yes in step S205), the code extraction unit 30 sets flag #2 on the target row (step S210). Then, if the operation on the target row is not an aggregation operation (No in step S205), the code extraction unit 30 determines that the processing cannot be accelerated (step S120) and ends the processing. The order of the processing from step S203 to step S205 may be reversed.

[0048] In step S210, after flag #2 is set for the target line, The processes from step S109 onward in FIG. 5 are carried out.

[0049] Next, a specific example of the process performed by the code conversion unit 40 will be described. Figures 7 and 8 are flowcharts showing an example of the conversion process by the code conversion unit 40. Here, it is assumed that the extraction process illustrated in Figures 5 and 6 has been performed, and that the conversion process is performed on the target code that has been determined to be capable of being accelerated. Also, the code conversion unit 40 starts the process from the bottommost line in the target function (step S301).

[0050] The code conversion unit 40 reads one line of processing in the function Func (step S302). The code conversion unit 40 determines whether the operation of the read line (i.e., the target line) is an operation on g on both the right and left sides (step S303). If both the right and left sides are operations on g (Yes in step S303), the code conversion unit 40 determines whether the operation of the target line is an operation of referencing and overwriting by the iloc property (i.e., the first operation) (step S304). If the operation of the target line is the first operation (Yes in step S304), the code conversion unit 40 converts the code of the target line into code for speeding up (step S305).

[0051] Similarly, if the operation on the target row is not the first operation (No in step S304) but is an arithmetic operation (Yes in step S306), the code conversion unit 40 converts the code on the target row into a code for speeding up the processing (step S307). Furthermore, if the operation on the target row is not an arithmetic operation (No in step S306) but is an aggregation operation (Yes in step S308), the code conversion unit 40 converts the code on the target row into a code for speeding up the processing (step S309). Then, if the operation on the target row is not an aggregation operation (No in step S308), the code conversion unit 40 ends the processing without converting the code. Note that the order of the determination processing may be changed in the processing from step S304 to step S309.

[0052] In step S311 after the code is converted, the code conversion unit 40 determines whether or not there is a line above the target line (i.e., a previous process) (step S311). If there is a line above the target line (Yes in step S311), the code conversion unit 40 sets the line above as the target line (step S312) and repeats the processes from step S302 onwards, which read one line of the process in the function Func.

[0053] On the other hand, if there is no line above the target line (No in step S311), the code conversion unit 40 ends the process (conversion process into code for speeding up) (step S320).

[0054] In step S303, if neither the right nor left side is an operation on g (No in step S303), that is, if processing is performed on a data frame df other than g, processing from step S401 onward illustrated in FIG. 8 is performed.

[0055] Specifically, the code conversion unit 40 determines whether a data frame df other than g is generated by an equality filter using the key to be aggregated (groupkey) (step S401). If a data frame df other than g is generated (Yes in step S401), the code conversion unit 40 converts the code of the target row into a code for accelerating the code (step S402).

[0056] On the other hand, if a data frame df other than g has not been generated (No in step S401), the code conversion unit 40 determines whether the operation of the target row is an operation of referencing and overwriting by the iloc property (i.e., the first operation) (step S403). If the operation of the target row is the first operation (Yes in step S403), the code conversion unit 40 converts the code of the target row into a code for speeding up (step S404).

[0057] Similarly, if the operation on the target row is not the first operation (No in step S403) but is an arithmetic operation (Yes in step S405), the code conversion unit 40 converts the code on the target row into a code for speeding up the processing (step S406). Furthermore, if the operation on the target row is not an arithmetic operation (No in step S405) but is an aggregation operation (Yes in step S407), the code conversion unit 40 converts the code on the target row into a code for speeding up the processing (step S408). Then, if the operation on the target row is not an aggregation operation (No in step S407), the code conversion unit 40 ends the processing without converting the code. Note that the order of the determination processing may be changed in the processing from step S403 to step S408.

[0058] After the process of converting into a code for speeding up is performed (that is, after steps S402, S404, S406, and S408), the processes from step S311 onward in FIG. 7 are performed.

[0059] Here, the extraction process and conversion process shown in the flowcharts exemplified in Figures 5 to 8 will be described using specific program code. Figure 9 is an explanatory diagram showing an example of the extraction process and conversion process.

[0060] First, the code extraction unit 30 judges whether the returned data frame is the same as the input (i.e., argument) data frame. In the code shown in FIG. 9, both are the same data frame g.

[0061] Next, the code extraction unit 30 reads the bottom line (the third line of the process) and determines whether the operations on both the right and left sides of the target line are operations on g. Here, since neither the right nor the left side is an operation on g, the code extraction unit 30 determines whether a data frame df other than g has been generated by the equality filter using the key to be aggregated (groupkey).

[0062] Here, since no data frame df other than g has been generated, the code extraction unit 30 judges whether it is an operation to refer to data, an arithmetic operation, or an aggregation operation. Since it corresponds to an aggregation operation here, flag #2 is set to the target line. Then, since a line above the target line (i.e., the previous process) exists, that line (the second line of the process) is read.

[0063] The code extraction unit 30 determines whether the operation on the target row is an operation on g on both the right and left sides. Here, since neither the right nor left sides are operations on g, the code extraction unit 30 determines whether a data frame df other than g has been generated by an equality filter using the aggregation key (groupkey). Here, since a data frame df other than g has not been generated, the code extraction unit 30 determines whether the operation is an operation to reference data, an arithmetic operation, or an aggregation operation. Here, since it corresponds to an arithmetic operation, flag #2 is set for the target row. Then, since a row above the target row (i.e., the previous process) exists, that row (the first row of the process) is read.

[0064] The code extraction unit 30 determines whether the operation on the target row is an operation on g on both the right and left sides. Here, since neither the right and left sides are operations on g, the code extraction unit 30 determines whether a data frame df other than g has been generated by an equality filter using the aggregation key (groupkey). Here, since a data frame df other than g has been generated, the code extraction unit 30 sets flag #1 to the target row. Furthermore, the code extraction unit 30 replaces the flag of the row with flag #2 set to flag #1.

[0065] Then, since there is no line above the target line (i.e., a previous process), the code extraction unit 30 judges whether or not flag #1 is set for all lines. In this case, flag #1 is set for all lines, so the code extraction unit 30 judges that speedup is possible.

[0066] As described above, in this embodiment, the input unit 20 accepts input of a target program code, and the code extraction unit 30 extracts a first program code and a second program code from the target program code. Then, the code conversion unit 40 converts the extracted code into an alternative program code, i.e., a program code indicating a process that does not perform a function call and that obtains a result similar to that of the process by the extracted code, and the output unit 50 outputs the alternative program code.

[0067] Therefore, in order to reduce the processing time for data, the existing code can be converted into code that can obtain similar results. That is, the code conversion unit 40 changes the program code that performs a function call after an aggregation operation into a program code that does not perform a function call, thereby reducing the time required for processing.

[0068] The operation of the code conversion device of the present disclosure will be described below using a specific example. As described above, the code extraction unit 30 extracts the first program code and the second program code from the target program code. In the following description, it is assumed that func(g) is a program code indicating a predefined process that takes a data frame as an argument. In this case, apply(func) and func(g) are examples of the second program code, and are program codes indicating a predefined process func performed on a data frame and a function apply that calls func.

[0069] As described above, groupby is an example of the first program code, which is a program code indicating that aggregation processing is performed on each column in a data frame based on the value of a specific column. Therefore, in this specific example, the code extraction unit 30 extracts a program code indicating the content of groupby+apply(func).

[0070] In addition, apply in pandas is a method that emphasizes convenience and is a convenient method that allows programmers to intuitively write the functions they want to achieve. On the other hand, if code is written in the format of groupby+apply(func), it is necessary to call func as many times as the number of groups in the aggregated data frame, which may slow down the calculation. On the other hand, the code conversion device of the present disclosure makes it possible to automatically detect such written code and convert it into code for speedup.

[0071] First, a first specific example of the conversion process will be described. Figures 10 and 11 are explanatory diagrams showing a first specific example of the conversion process. Figures 10 and 11 illustrate a process in which, when grouping is performed by the value of column A in data frame df1, the value of column B in the group is overwritten with the first value of other column B.

[0072] When the code is written in the format of groupby+apply(func), this process can be expressed by the code CD1 shown in FIG. 10. First, when data frame df1 is grouped by column A, for example, a data frame g is obtained in which value A1 has been aggregated. The same applies for other values. Code CD1 shows the process of calling function func with data frame g as an argument, the number of times being the number of data frames that have been aggregated. As a result, data frame r1 is obtained.

[0073] On the other hand, as illustrated in Fig. 11, the code CD1 illustrated in Fig. 10 can be converted into a code CD2 for accelerating the processing speed. The code conversion unit 40 may generate the code CD2, which is an alternative program code, based on the correspondence illustrated in Fig. 3 (specifically, the second row of the table), for example. This results in a data frame r2, the contents of which are the same as those of the data frame r1 illustrated in Fig. 10.

[0074] Next, a second specific example of the conversion process will be described. Figures 12 and 13 are explanatory diagrams showing a second specific example of the conversion process. Figures 12 and 13 illustrate a process in which, when the data frame df2 is grouped by the values ​​of column A, the values ​​of other columns B and C are added.

[0075] When the code is written in the format of groupby+apply(func), this process can be expressed by the code CD3 shown in FIG. 12. First, when data frame df2 is grouped by column A, for example, a data frame g is obtained in which value A1 has been aggregated. The same applies to other values. Code CD3 shows the process of calling function func with data frame g as an argument, the number of times being the number of data frames that have been aggregated. As a result, data frame r3 is obtained.

[0076] On the other hand, as illustrated in Fig. 13, the code CD3 illustrated in Fig. 12 can be converted into a code CD4 for accelerating the processing speed. The code conversion unit 40 may generate the code CD4, which is an alternative program code, based on the correspondence illustrated in Fig. 3 (specifically, the third row of the table), for example. This results in a data frame r4, the contents of which are the same as those of the data frame r3 illustrated in Fig. 12.

[0077] In this process, the code conversion unit 40 may create a new column D and set the added value in the new column, instead of updating column B with the added value.

[0078] Next, a third specific example of the conversion process will be described. Figures 14 and 15 are explanatory diagrams showing a third specific example of the conversion process. Figures 14 and 15 illustrate a process of calculating the sum of the values ​​of other columns B in a group when grouping is performed by the values ​​of column A in data frame df3.

[0079] When the code is written in the format of groupby+apply(func), this process can be expressed by the code CD5 shown in FIG. 14. First, when data frame df3 is grouped by column A, for example, a data frame g is obtained in which value A1 has been aggregated. The same applies to other values. Code CD5 shows the process of calling function func with data frame g as an argument, the number of times equal to the number of data frames that have been aggregated. As a result, a data frame r5 is obtained.

[0080] On the other hand, as illustrated in Fig. 15, the code CD5 illustrated in Fig. 14 can be converted into a code CD6 for accelerating the processing speed. The code conversion unit 40 may generate the code CD6, which is a substitute program code, based on the correspondence illustrated in Fig. 3 (specifically, the fourth row of the table), for example. This results in a data frame r6, the contents of which are the same as those of the data frame r5 illustrated in Fig. 14.

[0081] Next, a fourth specific example of the conversion process will be described. FIG. 16 and FIG. 17 are explanatory diagrams showing a fourth specific example of the conversion process. FIG. 16 and FIG. 17 illustrate a process of extracting data that matches a value contained in an ID column of a data frame df4 from an ID column that is a key (groupkey) to be aggregated in another data frame df5. Note that a filter operation that extracts data equal to a groupkey from another data frame can be considered as a join operation. One example of such an operation is an operation of calculating statistical data from a data frame df5 that stores a customer's past product purchase records, using the customer ID of the data frame df4 as a key.

[0082] When the code is written in the format of groupby+apply(func), this process can be expressed as the code CD7 shown in Fig. 16. The code CD7 shows the process of calling the function func as many times as the number of data frames that have been aggregated, using the data frames grouped by ID as arguments.

[0083] First, data frame df4 is grouped by column ID, and for example, a row corresponding to value A1 is searched for in data frame df5 to obtain data frame df6. Then, the sum of the values ​​in the obtained data frame df6 is calculated. The same is done for other values. As a result, data frame r7 is obtained.

[0084] On the other hand, as illustrated in Fig. 17, the code CD7 illustrated in Fig. 16 can be converted into a code CD8 for speeding up. The code conversion unit 40 may generate the code CD8, which is an alternative program code, based on the correspondence illustrated in Fig. 3 (specifically, the first row of the table), for example. This results in a data frame r11, the contents of which are the same as those of the data frame r7 illustrated in Fig. 16.

[0085] The conversion process of the fourth specific example will be described in detail below. First, the code conversion unit 40 converts the target program code into code that generates a data frame df7 by copying the data frame df4 and assigning an index for later use. Next, the code conversion unit 40 converts the target program code into code that generates a data frame d8 by joining the data frames df5 and df7 by column IDs.

[0086] In this example, when we look at the column IDs of the data frame df7, the value A1 corresponds to the topmost Index=0. When we look at the entire data frame df5, there are multiple (2) rows where the value A1 corresponds to the column ID. In this case, the row with Index=0 in the data frame df7 is duplicated and joined.

[0087] Similarly for other indexes, the row with index=1 is duplicated twice, and the row with index=2 is duplicated once. The rows with index=3 and index=4 are duplicated once, and the row with index=5 is not duplicated. Here, matching column A and duplicating the corresponding row in data frame df5 multiple times corresponds to the operation of extracting data frame df5 with a filter for rows with matching keys in apply(func).

[0088] The code conversion unit 40 converts the target program code into code that performs an aggregation process using the index of the data frame d8 as a key, calculates a sum, and generates a data frame d9 that indicates the calculated result. The code conversion unit 40 then converts the target program code into code that combines the copied data frames df4 and df9 to generate a data frame df10. Note that, taking into consideration the presence of missing values ​​as shown in ID=C2, the code conversion unit 40 converts the target program code into code that generates a data frame df11 in which the missing values ​​are filled with 0.

[0089] Next, an overview of the present disclosure will be described. FIG. 18 is a block diagram showing an overview of a code conversion device according to the present disclosure. A code conversion device 80 (for example, a code conversion device 100) according to the present disclosure includes an input means 81 (for example, an input unit 20) for receiving an input of a target program code, which is a program code to be processed; an extraction means 82 (for example, a code extraction unit 30) for extracting from the target program code a first program code (for example, a groupby() method) indicating an aggregation process of each column in a data frame representing two-dimensional tabular data based on a value of a predetermined column, as well as a second program code indicating a predefined process (for example, a user-defined function) performed on each data frame on which the aggregation process has been performed and a function (for example, an apply() method) for calling the process; a conversion means 83 (for example, a code conversion unit 40) for converting the extracted first program code and second program code into an alternative program code, which is a program code indicating a process that does not call a function to call the process by the function and that obtains a result similar to that of the process by the first program code and the second program code; and an output means 84 (for example, an output unit 50) for outputting the alternative program code.

[0090] Such an arrangement allows existing code to be converted to code that achieves similar results, so that processing time on the data can be reduced.

[0091] Specifically, the conversion means 83 may generate the alternative program code by converting the program code included in the function indicated by the second program code into a predetermined corresponding program code (for example, a correspondence table as illustrated in FIG. 2).

[0092] Furthermore, the conversion means 83 may convert the first program code and the second program code into an alternative program code when the type of an argument used to call a process matches the type of a return value of the process.

[0093] In addition, the conversion means 83 may convert an operation within a predefined process expressed as an expression connected by an equal sign into code for speeding up the process when both the right and left sides of the expression are operations on a data frame that is used as an argument when invoking the process, and when there is no possibility that the number of rows of the data frame that is used as an argument will change due to the operation.

[0094] In addition, the conversion means 83 may convert the code indicated by an operation within a predefined process that is expressed by an equation connected by an equal sign into code for speeding up the operation if at least one of the right and left sides of the equation is not an operation on a data frame that is used as an argument when calling the process.

[0095] In addition, when a data frame other than the data frame that is the argument is generated by an equality filter using a key to be aggregated in the data frame, the conversion means 83 may convert the code indicated by the operation expressed in an equation connected by an equality sign into a code for speeding up the operation. [Explanation of symbols]

[0096] 10 Storage section 20 Input section 30 Code Extraction Section 40 Code conversion section 50 Output section 100 Code conversion device

Claims

1. an input means for receiving an input of a target program code which is a program code to be processed; an extraction means for extracting, from the target program code, a first program code indicating an aggregation process for each column in a data frame representing two-dimensional tabular data based on a value of a predetermined column, and a second program code indicating a predefined process to be performed on each data frame on which the aggregation process has been performed and a function for calling the process; a conversion means for converting the extracted first program code and the extracted second program code into an alternative program code which is a program code indicating a process that does not perform a function call to call the process by the function and that obtains a result similar to that of the process by the first program code and the second program code; and an output means for outputting the alternative program code. A code conversion device comprising:

2. The conversion means converts the program code included in the function indicated by the second program code into a predetermined corresponding program code to generate a substitute program code.

2. The code conversion device according to claim 1.

3. The conversion means converts the first program code and the second program code into an alternative program code when the type of an argument used to call the process matches the type of a return value of the process.

3. The code conversion device according to claim 1 or 2.

4. The conversion means converts a code indicated by an operation expressed by an equation connected by an equal sign among operations in a predefined process into a code for accelerating the operation when both the right side and the left side of the equation are operations on a data frame that is an argument when invoking the process, and when there is no possibility that the operation will change the number of rows of the data frame that is the argument.

3. The code conversion device according to claim 1 or 2.

5. The conversion means converts a code indicated by an operation expressed by an equation connected by an equal sign among operations in a predefined process into a code for accelerating the operation when at least one of the right side and the left side of the equation is not an operation on a data frame that is used as an argument when the process is called.

3. The code conversion device according to claim 1 or 2.

6. The conversion means converts a code indicated by an operation expressed by an expression linked by an equality sign into a code for accelerating the operation when a data frame other than the data frame is generated by a filter using an equality sign based on a key to be aggregated in the data frame as an argument.

6. The code conversion device according to claim 5.

7. Accepting input of a target program code which is a program code to be processed; extracting from the target program code a first program code indicating an aggregation process for each column in a data frame representing two-dimensional tabular data based on a value of a predetermined column, and a second program code indicating a predefined process to be performed on each data frame on which the aggregation process has been performed and a function for calling the process; converting the extracted first program code and the extracted second program code into an alternative program code which is a program code indicating a process that does not perform a function call to call the process by the function and that obtains a result similar to that of the process by the first program code and the second program code; Output the alternative program code A code conversion method comprising:

8. The conversion means converts the program code included in the function indicated by the second program code into a predetermined corresponding program code to generate a substitute program code. The code conversion method according to claim 7.

9. On the computer, an input process for accepting input of a target program code which is a program code to be processed; an extraction process for extracting, from the target program code, a first program code indicating an aggregation process for each column in a data frame representing two-dimensional tabular data based on values ​​of a predetermined column, and a second program code indicating a predefined process to be performed on each data frame on which the aggregation process has been performed and a function for calling the predefined process; a conversion process for converting the extracted first program code and the extracted second program code into an alternative program code which is a program code indicating a process that does not perform a function call to call the process by the function and that obtains a result similar to that of the process by the first program code and the second program code; and Output process for outputting the alternative program code A code conversion program to execute the above.

10. On the computer, In the conversion process, a program code included in a function indicated by the second program code is converted into a predetermined corresponding program code to generate an alternative program code. The code conversion program according to claim 9.

Images