Optimizing method for intermediate representations and optimizing program for intermediate representations

Abstract

An optimizing method for intermediate representation operators comprising: generating an intermediate representation from user program; detecting a known optimization pattern in the current intermediate representation and previous intermediate representations; checking whether the detected optimization pattern can be applied to a target intermediate representation operator without any performance issues; and replacing the target intermediate representation operator with the optimized intermediate representation operator as per the detected optimization pattern.

Description

OPTIMIZING METHOD FOR INTERMEDIATE REPRESENTATIONS AND OPTIMIZING PROGRAM FOR INTERMEDIATE REPRESENTATIONS

[0001] The present invention relates to an optimizing method for intermediate representations and an optimizing program for intermediate representations.

[0002] The collected data is rarely used directly to make application of the collected data. Collected data is generally pre-processed, called data maintenance, prior to application. Data maintenance is said to spend 80% of the time for data analysis, and as the data size increases, it becomes a bottleneck that needs to be improved to higher speeds.

[0003] For example, Patent Literature 1 describes a technique for reducing the time required for preprocessing to generate training data for application in machine learning.

[0004] [PTL 1] International Publication WO2024 / 023892A1

[0005] The disclosures of the above prior art document shall be incorporated by reference into this document. The following analysis has been made by the inventors.

[0006] There is a technique for improving efficiency for preprocessing in data analysis which generates an intermediate representation operator (IR-OP), but the optimizing IR-OP is performed when all the IR-OPs are generated. The more IR-OPs the system accumulates, the more time the system has to compile. It is also hard to check all the accumulated IR-OPs for all known patterns during the optimization phase.

[0007] In view of the above problems, it is an object of the present invention to provide an optimizing method for intermediate representations and an optimizing program for intermediate representations that contribute to reduce the total execution time of an intermediate representation-based program.

[0008] According to a first aspect of the present disclosure, there is provided an optimizing method for intermediate representation operators, comprising: generating an intermediate representation from user program; detecting a known optimization pattern in the current intermediate representation and previous intermediate representations; checking whether the detected optimization pattern can be applied to a target intermediate representation operator without any performance issues; and replacing the target intermediate representation operator with the optimized intermediate representation operator as per the detected optimization pattern.

[0009] According to a second aspect of the present disclosure, there is provided an optimizing program for intermediate representation operators causing a computer to perform: generating an intermediate representation from user program; detecting a known optimization pattern in the current intermediate representation and previous intermediate representations; checking whether the detected optimization pattern can be applied to a target intermediate representation operator without any performance issues; and replacing the target intermediate representation operator with the optimized intermediate representation operator as per the detected optimization pattern.

[0010] Further, this program can be stored in a computer-readable storage medium. The storage medium may be non-transitory one such as a semiconductor memory, a hard disk, a magnetic recording medium, an optical recording medium, and the like. The present invention can also be realized as a computer program product.

[0011] According to each aspect of the present invention, there can be provided an optimizing method for intermediate representations and an optimizing program for intermediate representations that contribute to reduce the total execution time of an intermediate representation-based program.

[0012] Fig. 1 shows a flow chart of optimizing method for intermediate representations.Fig. 2 shows the difference between a conventional optimization and the optimization in the present disclosure.Fig. 3 shows an example of a null bitmap.Fig. 4 shows an example of NullAggregateOp(“sum”) by using a null bitmap.Fig. 5 illustrates the input Data Frame and temporal data frame.Fig. 6 illustrates the input Data Frame and temporal data frame.Fig. 7 illustrates an example of optimization for multiplication of a vector with a scalar.Fig. 8 illustrates an example of operation using loc and iloc leading to the same result.Fig. 9 illustrates an example of operation using loc.Fig. 10 illustrates an example of operation using iloc.

[0013] Fig. 1 shows a flow chart of optimizing method for intermediate representations. As shown in Fig. 1, the optimizing method starts with the user API calls, which are contained in user program.

[0014] The first step (S1) generates the intermediate representations (IRs) for the respective API call. Here, the intermediate representations (IRs) for the respective API call are usually intermediate representation operators (IR-OPs) that perform the respective API call.

[0015] The second step (S2) detects a known optimization pattern in the current IR and previous IRs. Here, the second step (S2) is performed during the creation of current IR. In other words, the second step (S2) is performed during the first step (S1). If the second step (S2) is yes (i.e. a known optimization pattern is detected), the process goes to the next. If the second step (S2) is no (i.e. a known optimization pattern is not detected), the process goes finish.

[0016] The third step (S3) checks whether the detected optimization pattern can be applied to target IR-OPs without any performance issues. The detected optimization pattern is not always work well and some optimization may reduce the performance. The third step (S3) is performed to avoid this problem. If the third step (S3) is yes, the process goes to the next. If the third step (S3) is no, the process goes finish.

[0017] The fourth step (S4) replaces the generated IR with the optimized IR as per the detected optimization pattern. During the generation of IR, if an optimization pattern is detected and performance gain can be guaranteed from that optimization, we can create a better IR-Op by applying the optimization pattern instead of accumulating the IR-Ops and inspecting them later at the compilation stage. This will speed-up the total execution time of an IR-based application.

[0018] The above optimizing method for intermediate representation operators can be implemented in an optimizing program for intermediate representation operators to cause a computer to perform the above process.

[0019] Fig. 2 shows the difference between a conventional optimization and the optimization in the present disclosure. Both of the conventional optimization and the optimization in the present disclosure starts with same user program, which contains API calls. The conventional optimization and the optimization in the present disclosure generates the intermediate representations (IRs) from the user program. <user program> df = read_csv(“data.csv”) r1 = df.isnull() r2 = r1.sum() print(r2)

[0020] In the conventional optimization, IR optimizing parse is performed when the intermediate representations (IRs) are generated for the entire user program. In this example, the pair of %2 = is_null(%1) and %3 = aggregate(%2, “sum”) is replaced to %2 = null_aggregate(%1, “sum”). Finally, the optimized intermediate representations are executed in an appropriate kernel.

[0021] In the optimization in the present disclosure in contrast with the conventional optimization, IR optimizing parse is performed during the generation of the intermediate representations. In this example, the pair of %2 = is_null(%1) and %3 = aggregate(%2, “sum”) is replaced to %2 = null_aggregate(%1, “sum”) when the intermediate representation %3 = aggregate(%2, “sum”) is generated. In other words, the optimization in the present disclosure detects optimization pattern in the current IR and previous IR and optimizes the detected IR when the optimization pattern is detected. In the optimization in the present disclosure, any special IR optimizing parse is not necessary because optimized intermediate representations are accumulated. Finally, the optimized intermediate representations are executed in an appropriate kernel.

[0022] The operator null_aggregate() is performed by using null bitmap. Fig. 3 shows an example of a null bitmap. A null bitmap is a lightweight data structure (consumes 1 bit for each value) that represents each valid value (non-missing / non-null) as “1” and each invalid value (missing / null) as “0”. As shown in Fig. 3, only 2 bytes of data to represent the validity of 16 elements. Column-wise null count can easily be computed from this null bitmap. A null bitmap can be extracted only for columns having null elements.

[0023] Fig. 4 shows an example of NullAggregateOp(“sum”) by using a null bitmap. As shown in Fig. 4, sum of null elements in this table can be easily computed by using the null bitmap. NullAggregateOp(“sum”) by using a null bitmap does not need to create intermediate table of Boolean data as a result of IsNullOP. NullAggregateOp(“sum”) by using a null bitmap saves memory and execution time because a null bitmap is more lightweight than Boolean table.

[0024] Even though an optimization pattern is detected during the IR creation, we cannot always decide whether to perform a specific optimization for that pattern without inspecting the entire program.

[0025] For example, let’s consider there is a known pattern to replace “Op_A + OP_B” as “Op_AB” for the following intermediate representations. < intermediate representations > %1 = Op_A %2 = Op_B %3 = sum_op(%1, 2)

[0026] But in this program, even if we replace Op_A+Op_B with Op_AB, the result of Op_A is used at some other place to calculate some addition, hence we cannot ignore processing the Op_A. The optimized intermediate representations may be as follows. Hence even after replacing Op_A+Op_B with Op_AB, we may not achieve expected performance gain. < optimized intermediate representations > %1 = Op_A %2 = Op_AB %3 = sum_op(%1, 2)

[0027] Despite of the challenges mentioned in the above example, if we can ensure the conversion is full proof and will always bring performance in a user program by automatic conversion of IR-OP during the IR creation, it is better to implement those conversion to save a significant amount of compilation overhead.

[0028] The decision whether to apply the optimization during IR construction can be made in either of the following ways: Automatically: programmatically checking some conditions: parent IR has only single use (when used in chained expression) the output can be determined without generating the current IR. Manually: with some domain knowledge analysis: there is no increase in computational cost even in the worst-case situation. there exists an alternative way which is guaranteed to reduce the execution cost.

[0029] <Automatic Conversion: Case-1 (Chained Expression)> Chained Expression res = Op_A(…).Op_B(…) When the methods are called as a chain of operations as shown above (which is the most common way of calling a series of methods), it can be guaranteed that the results from the intermediate operations will only be used for a single time. For example, output of Op_A will be used only as the input of Op_B as it is used in a chained expression. When creating IR for Op_B, programmatically it is possible to detect that the input operation (Op_A) has a single usage (by checking its reference count etc.) in the entire program.

[0030] Hence it can safely be converted to the optimized OP (Op_AB) as follows: res = Op_A(…).Op_B(…) ->->-> res = Op_AB(…)

[0031] Rule: It is possible to create a better IR automatically at the time of its construction when the following conditions are met: A known pattern is detected while creating the current IR (e.g., while creating Op_B using Op_A, we can detect that a known optimized pattern Op_AB exists.) It can be ensured that the input IR has only single usage (by checking its reference count etc.).

[0032] <Automatic Conversion: Case-2 (Extraction of Index Values) (optimization possible)> < user program > df = read_csv(“data.csv”) random_indices = np.random.choice(len(df), 3) #e.g., [3,5,1] data_index = df.take(random_indices).index < generated intermediate representations > %1 = read_csv(“data.csv”) %2 = take_rows(%1, [3, 5, 1]) %3 = get_index(%2) Fig. 5 illustrates the input Data Frame and temporal data frame. In this case, the input Data Frame has numeric RangeIndex. Hence even after extracting the index for given rows, the result will be same as in input. So, its better to eliminate the need for actual get_index OP for such cases.

[0033] While creating %3 (get_index OP), we can detect the input (%2) is a take_rows OP on input data %1. Programmatically it is possible to detect that %1 has a numeric Range Index. Hence, it is guaranteed that even after executing the get_index OP, the result will be same as in input of take_rows OP ([3,5,1]). Since the output can be determined without generating the IR for get_index, we can safely eliminate the need for actual get_index OP with the target row-indices ([3,5,1]).

[0034] <Automatic Conversion: Case-2 (Extraction of Index Values) (optimization impossible)> < user program > df = read_csv(“data.csv”) random_indices = np.random.choice(len(df), 3) #e.g., [3,5,1] data_index = df.take(random_indices).index < generated intermediate representations > %1 = read_csv(“data.csv”) %2 = take_rows(%1, [3, 5, 1]) %3 = get_index(%2) Fig. 6 illustrates the input Data Frame and temporal data frame. In this case, The input Data Frame has non-numeric Index. Hence the index values for the target rows (3,5,1) are (D,F,B). So, elimination of get_index OP is not possible.

[0035] While creating %3 (get_index OP), we can detect the input (%2) is a take_rows OP on input data %1. Programmatically it is possible to detect that %1 has a non-numeric Index. Therefore, the optimization of programmatically eliminating the get_index OP is not possible for such cases.

[0036] <Manual Conversion> When a known pattern is detected, but it cannot be programmatically guaranteed that the input IR has only single usage, we can use domain specific knowledge to decide whether the optimization is safe to apply without impacting performance for the following cases: CASE-1: There will be no increase in calculation cost even if the optimization is applied. CASE-2: There exists an alternative way which is guaranteed to reduce the execution cost.

[0037] <Manual Conversion: Case-1: Example: Vector-Scalar Multiplication> < user program > df = read_csv(“days.csv”) res = df * 24 * 60 * 60 print(res) < rewritten program 1> df = read_csv(“days.csv”) res = binop_mul_vec_scal( binop_mul_vec_scal( binop_mul_vec_scal(df, 24), 60 ), 60 ) print(res) Chain Expression is detected, so this is easy to optimize as: res = binop_mul_vec_scal(df, 86400)

[0038] < rewritten program 2> df = read_csv(“days.csv”) t1 = df * 24 t2 = t1 * 60 t3 = t2 * 60 print(t3) Not used in chained expression. Hence while creating IR-op for t2, it can not be guaranteed whether input (t1) has a single usage in the entire program, so automatic optimization is difficult. But we can use domain knowledge to ensure that there will be no increase in computation cost even if we apply the optimization. Therefore, we can safely convert to a better-op while creating IR for t2, t3.

[0039] < generated intermediate representations > %1 = read_csv(“days.csv”) %2 = binop_mul_vec_scalar(%1, 24) %3 = binop_mul_vec_scalar(%2, 60) %4 = binop_mul_vec_scalar(%3, 60) print(%4) In this example, the normal operation of multiplication of a vector with a scalar will be performed 3 times as shown above to compute the result of t3. But the same can be optimized in one step without affecting the result. Fig. 7 illustrates an example of optimization for multiplication of a vector with a scalar.

[0040] < rewritten program 3 > df = read_csv(“days.csv”) t1 = df * 24 t2 = df * 24 * 60 t3 = df * 24 * 60 * 60 print(t3) The vector-scalar multiplication operations are performed on same data as a sequence. In order to compute “t3”, we need to compute “t2” and in order to compute “t2” we need to compute “t1”. With the proposed approach we can remove the dependency such that at any point of time there is only single vector-scalar multiplication. If we need only t2, no need to compute t1. If we need only t3, no need to compute t1 and r2. We can safely perform such optimization while constructing the new IR, i.e., if the input of current IR is a binop_mul_vec_scalar OP, then just remove the dependency of current OP with previous OP and create a single IR by multiplying the scalars. While constructing %3, it can be detected that the input (%2) is a binop_mul_vec_scalar OP on input data %1. So, the IR for %3 can be optimized at the time of its creation as follows: < optimized intermediate representations 1> %2 = binop_mul_vec_scalar(%1, 24) %3 = binop_mul_vec_scalar(%1, 24*60) And the same step can be done for %4 as well. Initially it is dependent on %3, but since %3 is also a binop_mul_vec_scalar OP on input data %1, it can be optimized at the time of its creation as follows: < optimized intermediate representations 2> %3 = binop_mul_vec_scalar(%1, 24*60) %4 = binop_mul_vec_scalar(%1, 24*60*60)

[0041] How can we say that there will be no increase in computation cost even when the optimization is applied? Domain Knowledge Analysis: If we need only one of the results from t1, t2, t3, there will be only one vector-scalar multiplication. So, we can save quite a significant computation cost. If we need any two of them, there will be only two vector-scalar multiplications. Even if we need all three (t1, t2, t3) results in later parts of the program, the maximum number of vector-scalar multiplication will be 3 (which is same as in original case). So, there will be no loss in terms of performance.

[0042] <Manual Conversion: Case-2: Example: loc / iloc-setter> There exists two ways to access some intended rows of a given column: using loc: we need to specify the actual index values for the respective rows (D, F, B). using iloc: we need to specify the zero-based numeric location of the respective rows (3, 5, 1). Fig. 8 illustrates an example of operation using loc and iloc leading to the same result.

[0043] Therefore, when there is some calculation related to index-location to index-value computation to use loc(), we can eliminate the computation by making it an iloc() operation at the time of IR creation.

[0044] < user program > df = read_csv(“data.csv”) random_indices = np.random.choice(len(df), 3) # e.g., [3,5,1] df.loc[df.take(random_indices).index, “col1”] = 100 < generated intermediate representations > %1 = read_csv(“data.csv”) %2 = take_rows(%1, [3, 5, 1]) %3 = get_index(%2) loc_setter(%1, %3, “col1”, 100) Fig. 9 illustrates an example of operation using loc. When creating the IR for loc_setter, it can be detected that the index-location to index-value computation steps are involved (as a combination of take_rows followed by get_index Ops).

[0045] Fig. 10 illustrates an example of operation using iloc. We can safely eliminate the need for index-location to index-value computation, by choosing the alternative way of assignment using iloc(). < rewritten user program > df = read_csv(“data.csv”) random_indices = np.random.choice(len(df), 3) # e.g., [3, 5, 1] df.iloc[random_indices, “col1”] = 100

[0046] How can we say that there will be no increase in computation cost even when the optimization is applied? Domain Knowledge Analysis: During the creation of loc_setter, programmatically we can detect that index-location to index-value is being computed (since the input of loc_setter, %3 is a get_index operator obtained from %2, take_rows operator). We know that there exists an alternative way, iloc_setter which can be performed using the index-location itself. So, the loc_setter can be replaced with iloc_setter as follows using the domain knowledge and the need for computing index-location to index-value (%2 and %3) can also be eliminated which can save significant performance cost. Now, in the worst-case situation, when %3 or %2 are used in some other part of the program, there will be no increase in actual computation cost. The combined cost for (take_rows, get_index, loc_setter) is almost same as the combined cost for (take_rows, get_index, iloc_setter). Therefore, such optimization at IR level can safely be applied when constructing the IR for loc_setter in this case. < optimized intermediate representations > %1 = read_csv(“data.csv”) iloc_setter(%1, [3,5,1], “col1”, 100)

[0047] In this example, the optimization pattern is replacing a process of extracting a specific row and then extracting the index with a process that directly specifies the index. Fig. 5 illustrates the process of IR1 and IR2. As illustrated in Fig. 5, the process of IR1 contains more process than the process of IR2. This is because IR2 is optimized from IR1. In this example, it is possible to convert IR to better IR-OP while generating IR.

[0048] It is possible to eliminate some unnecessary operators when optimizing the IR during its creation which will speed-up the compilation phase, which will further speed-up the execution of entire program. Implementation of such domain pattern specific IR optimization at python level is much easier than doing it later at C++ level (during compilation). Optimization patterns can be added quickly and debugged at python level, reducing the development time.

[0049] <Supplementary Notes> The present disclosure may be represented as form of Supplementary Notes, without limitation. (Note 1) As described in the first aspect of the present disclosure. (Note 2) The optimizing method for intermediate representation operators according to Note 1, wherein the optimization pattern is replacing a pair of IsNullOP and AggregatdOP to NullAggregateOP. (Note 3) The optimizing method for intermediate representation operators according to Note 2, wherein NullAggregateOP is performed by using a null bitmap. (Note 4) The optimizing method for intermediate representation operators according to Note 1, wherein the optimization pattern is replacing plural multiplications to a single multiplication. (Note 5) The optimizing method for intermediate representation operators according to Note 1, wherein the optimization pattern is replacing a process of extracting a specific row and then extracting an index with a process that directly specifies the index. (Note 6) As described in the second aspect of the present disclosure. (Note 7) The optimizing program for intermediate representation operators according to Note 6, wherein the optimization pattern is replacing a pair of IsNullOP and AggregatdOP to NullAggregateOP. (Note 8) The optimizing program for intermediate representation operators according to Note 7, wherein NullAggregateOP is performed by using a null bitmap. (Note 9) The optimizing program for intermediate representation operators according to Note 6, wherein the optimization pattern is replacing plural multiplications to a single multiplication. (Note 10) The optimizing program for intermediate representation operators according to Note 6, wherein the optimization pattern is replacing a process of extracting a specific row and then extracting the index with a process that directly specifies the index.

[0050] While each example embodiment of the present invention has been described, it is to be noted that it is possible to modify or adjust the example embodiments or examples within the whole disclosure of the present invention (including the Claims) and based on the basic technical concept thereof. Further, it is possible to variously combine or select (or at least partially remove) a wide variety of the disclosed elements (including the individual elements of the individual claims, the individual elements of the individual example embodiments or examples, and the individual elements of the individual figures) within the scope of the whole disclosure of the present invention. That is, it is self-explanatory that the present invention includes any types of variations and modifications to be done by a skilled person according to the whole disclosure including the Claims and the technical concept of the present invention. Particularly, any numerical ranges disclosed herein should be interpreted that any intermediate values or subranges falling within the disclosed ranges are also concretely disclosed even without specific recital thereof. Further, the disclosure of each Patent Literature cited above is incorporated herein in its entirety by reference thereto.

[0051] S1 first step S2 second step S3 third step S4 fourth step

Claims

1. An optimizing method for intermediate representation operators, comprising: generating an intermediate representation from user program; detecting a known optimization pattern in the current intermediate representation and previous intermediate representations; checking whether the detected optimization pattern can be applied to a target intermediate representation operator without any performance issues; and replacing the target intermediate representation operator with the optimized intermediate representation operator as per the detected optimization pattern.

2. The optimizing method for intermediate representation operators according to claim 1, wherein the optimization pattern is replacing a pair of IsNullOP and AggregatdOP to NullAggregateOP.

3. The optimizing method for intermediate representation operators according to claim 2, wherein NullAggregateOP is performed by using a null bitmap.

4. The optimizing method for intermediate representation operators according to claim 1, wherein the optimization pattern is replacing plural multiplications to a single multiplication.

5. The optimizing method for intermediate representation operators according to claim 1, wherein the optimization pattern is replacing a process of extracting a specific row and then extracting an index with a process that directly specifies the index.

6. An optimizing program for intermediate representation operators causing a computer to perform: generating an intermediate representation from user program; detecting a known optimization pattern in the current intermediate representation and previous intermediate representations; checking whether the detected optimization pattern can be applied to a target intermediate representation operator without any performance issues; and replacing the target intermediate representation operator with the optimized intermediate representation operator as per the detected optimization pattern.

7. The optimizing program for intermediate representation operators according to claim 6, wherein the optimization pattern is replacing a pair of IsNullOP and AggregatdOP to NullAggregateOP.

8. The optimizing program for intermediate representation operators according to claim 7, wherein NullAggregateOP is performed by using a null bitmap.

9. The optimizing program for intermediate representation operators according to claim 6, wherein the optimization pattern is replacing plural multiplications to a single multiplication.

10. The optimizing program for intermediate representation operators according to claim 6, wherein the optimization pattern is replacing a process of extracting a specific row and then extracting the index with a process that directly specifies the index.

Images