Optimizing method for intermediate representations, and optimizing program for intermediate representations

Abstract

An optimizing method for intermediate representation operators comprising: detecting a combination of IsNullOP(data) and AggregateOP(method) in the intermediate representation operators; and replacing the combination of IsNullOP(data) and AggregateOP(method) with NullAggregateOP(data, method) that is performed by using a null bitmap if a method in AggregateOP(method) is supported by NullAggregateOP(data, method) and a result of IsNullOP(data) is not used anywhere else in a user program.

Description

OPTIMIZING METHOD FOR INTERMEDIATE REPRESENTATIONS, AND OPTIMIZING PROGRAM FOR INTERMEDIATE REPRESENTATIONS

[0001] The present invention relates to an optimizing method for intermediate representations, and an optimizing program for intermediate representations.

[0002] The collected data is rarely used directly to make application of the collected data. Collected data is generally pre-processed, called data maintenance, prior to application. Data maintenance is said to spend 80% of the time for data analysis, and as the data size increases, it becomes a bottleneck that needs to be improved to higher speeds.

[0003] For example, Patent Literature 1 describes a technique for reducing the time required for preprocessing to generate training data for application in machine learning.

[0004] [PTL 1] International Publication WO2024 / 023892A1

[0005] The disclosures of the above prior art document shall be incorporated by reference into this document. The following analysis has been made by the inventors.

[0006] The performance of a user application is highly impacted by the choice of selected methods. Sometimes there exist several ways (APIs) to perform the same task (approach A, approach B, approach C, …). It might happen that approach B is the best and approach C is the worst in terms of computational performance. If the user selects the most inefficient method (e.g., C), the application will face significant issues.

[0007] Hence, it is necessary to create a solution where even if the user selects the most inefficient approach, the solution should auto-optimize, helping the user to experience speed up without much bothering about which approach to select. In this disclosure, we want to solve some similar issues when processing null data using compiler optimization (combination of IR Ops) technology.

[0008] In view of the above problems, it is an object of the present invention to highlight some common issues with Null data processing and how they can be addressed using compiler technology.

[0009] According to a first aspect of the present disclosure, there is provided an optimizing method for intermediate representation operators comprising: detecting a combination of IsNullOP(data) and AggregateOP(method) in the intermediate representation operators; and replacing the combination of IsNullOP(data) and AggregateOP(method) with NullAggregateOP(data, method) that is performed by using a null bitmap if a method in AggregateOP(method) is supported by NullAggregateOP(data, method) and a result of IsNullOP(data) is not used anywhere else in a user program.

[0010] According to a second aspect of the present disclosure, there is provided an optimizing method for intermediate representation operators comprising: detecting a filter that uses NotOP and IsNullOP for which target column is from the same table target for the filter; and replacing the filter that uses NotOP and IsNullOP with DropNAOP with subset as the target column, wherein DropNAOP is performed by using a null bitmap if the target table to be filtered is the same as the table on which NotOP+IsNullOP is performed and a result for NotOP+IsNullOP is not used anywhere in the user program.

[0011] According to a third aspect of the present disclosure, there is provided an optimizing program for intermediate representation operators causing a computer to perform: detecting a combination of IsNullOP(data) and AggregateOP(method) in the intermediate representation operators; and replacing the combination of IsNullOP(data) and AggregateOP(method) with NullAggregateOP(data, method) that is performed by using a null bitmap if a method in AggregateOP(method) is supported by NullAggregateOP(data, method) and a result of IsNullOP(data) is not used anywhere else in a user program.

[0012] According to a fourth aspect of the present disclosure, there is provided an optimizing program for intermediate representation operators causing a computer to perform: detecting a filter that uses NotOP and IsNullOP for which target column is from the same table target for the filter; and replacing the filter that uses NotOP and IsNullOP with DropNAOP with subset as the target column, wherein DropNAOP is performed by using a null bitmap if the target table to be filtered is the same as the table on which NotOP+IsNullOP is performed and a result for NotOP+IsNullOP is not used anywhere in the user program.

[0013] Further, this program can be stored in a computer-readable storage medium. The storage medium may be non-transitory one such as a semiconductor memory, a hard disk, a magnetic recording medium, an optical recording medium, and the like. The present invention can also be realized as a computer program product.

[0014] According to each aspect of the present invention, there can be provided an optimizing method for intermediate representations, and an optimizing program for intermediate representations that contribute to highlight some common issues with Null data processing and how they can be addressed using compiler technology.

[0015] Fig. 1 shows a sample house-pricing data with missing values marked as NA.Fig. 2 shows an example of process of conventional Null-Aggregate.Fig. 3 shows an example of the functionality of the data analyzing tool.Fig. 4 shows an example of process for the optimization of proposal 1Fig. 5 shows an example of a null bitmap.Fig. 6 shows an example of NullAggregateOp(“sum”) by using a null bitmap.Fig. 7 shows an example of operation of df[~df[“A”].isnull()].Fig. 8 shows an example of operation of df.isnull().sum() / df.isnull().count() * 100.Fig. 9 shows a flow chart of optimizing IR-OPs with NullAggregateOP.Fig. 10 shows a flow chart of optimizing IR-OPs with DropNAOP.

[0016] In real-world data, some values might be missing because of various reasons: incomplete data entry, lost files, system failure personal choice etc. Fig. 1 shows a sample house-pricing data with missing values marked as NA. Sometimes it is also termed as “null”.

[0017] In relational data analysis, there often a need to inspect the row-wise / column-wise distribution of missing (aka null) values. For example, percentage of null values in each row: average, number of null values in each row: sum, whether any value in a row is null: any, whether all values in a row are null: all.

[0018] In order to perform conventional Null-Aggregate, tabular data is (1) first converted into a table of Boolean data (where “True” indicates the respective value is a null data and “False” indicates the value is a valid / non-null data), and then (2) the intended aggregation method is performed on the Boolean table.

[0019] Here are some supported methods by a Data Analysis tool. 1. isnull(): It is used to inspect whether an element in an input tabular data is null or not. It is usually invoked as: df.isnull() where “df” is the input tabular data. It can also be invoked on some specific column of input table. For example, df[“A”].isnull() determines whether elements in input column A are null. It outputs True if the respective element is null and returns False if it is not null. 2. aggregate(func): It is used to perform some aggregate function on tabular data. For example, df.aggregate(“sum”) is used to perform row-wise sum of input data “df”, and df.aggregate(“mean”) is used to perform row-wise average of input data “df” 3. null_aggregate(func): It is used to perform some aggregate operation on the null status of the input tabular data. For example, df.null_aggreagte(“sum”) is used to count row-wise number of nulls in the input data df. ※Ideally, it is the same as df.isnull().aggregate(“sum”), but it is more efficient, as explained later. 4. filter(mask_vector): It is used to perform the filter operation on tabular data using a mask vector containing True / False, where True indicates the specific row to be kept and False indicates the specific row to be dropped. 5. not(mask-vector): It is used to invert a Boolean mask-vector. It is often used with the symbol “~”. For example, ~df[“A”].isnull() can be used to check whether elements of column A of input table “df” are not null. It returns True if not null and False if null. 6. dropna(subset): It is used to drop rows from the input table, “df” when any of the data in a given subset of columns for that specific row contains null. For example, df.dropna([“A”, “B”]) drops rows from table “df” when any element either in Column A or in Column B of a specific row is null.

[0020] It is required to improve a Data analysis tool with an in-built compiler that supports specific IR (Intermediate Representation) for each of these operations as follows (method -> IR): ● df.isnull() -> IsNullOP(df) ● df.aggregate(“sum”) -> AggregateOP(df, “sum”) ● df.null_aggregate(“sum”) -> NullAggregateOP(df, “sum”) ● df.filter(mask_vector) -> FilterOP(df, mask_vector) ● ~df[“A”].isnull() -> NotOP(IsNullOP(df[“A”])) ● df.filter(~df[“A”].isnull()) -> FilterOP(df, NotOP(IsNullOP(df[“A”]))) ● df.dropna([“A”]) -> DropNAOP(df, [“A”])

[0021] Fig. 2 shows an example of process of conventional Null-Aggregate. As shown in Fig. 2, tabular data T1 is first converted into a table of Boolean table T2, and then the intended aggregation method is performed on the Boolean table T2. For example, the result r1 indicates number of null values in each row (sum), the result r2 indicates mean of number of null values in each row (mean), the result r3 indicates whether any value in a row is null (any), and the result r4 indicates whether all values in a row are null (all).

[0022] There are two performance issues in the conventional Null-Aggregate: 1. isnull() results in a table of boolean columns of the same dimension as in the input data table, which means an increase in computational memory. We need to inspect each element in the input table to check if the value is missing. 2. Even if there are no missing values in a given column, it will be scanned, and a column of False values will be created. The column having False values will not impact the result of the count (x + 0 = x), which means waste of computation time.

[0023] Due to ASIS execution, there might be unnecessary redundant data processing when a user doesn’t take care of his application performance. For example, isnull() will be performed two times as written in the following user program. < user program > df = read_csv(“data.csv”) r1 = df.isnull().sum() r2 = df.isnull().mean()

[0024] For each method call, a data analyzing tool generates an intermediate representation (IR-OP) and optimizes the IR-OP using a JIT compiler during the execution. Hence, even if a user wrote his program as above, it can be optimized by the data analyzing tool by removing the duplicated OPs. However, the problem with the generation of an intermediate table of Boolean data still remains. < IR-OPs > %1 = read_csv(“data.csv”) %2 = is_null(%1) %3 = aggregate(%2, “sum”) %4 = is_null(%1) %5 = aggregate(%4, “mean”) <optimized IR-OPs> %1 = read_csv(“data.csv”) %2 = is_null(%1) %3 = aggregate(%2, “sum”) %4 = aggregate(%2, “mean”)

[0025] We can provide a new interface to the user for performing the null-aggregation, as follows: null_aggregate(data: table, method_name: str) However, a user will need to be aware of that new interface and an existing program cannot be optimized without manual code changes.

[0026] We propose an automated conversion of the above method calls using a data analyzing tool IR-optimization.

[0027] Fig. 3 shows an example of the functionality of the data analyzing tool. For each method in the user program, it first creates a specialized IR (as shown in Fig. 3). At the time of executing the methods, the internal compiler performs some compilation passes, where it tries to detect some specific pattern from the generated IRs. If a known pattern is detected and all the conditions for applying it are met, it replaces the pattern with an optimized solution. Then, the optimized code is executed by the execution unit.

[0028] (Proposal 1) Whenever the compiler detects the pattern (A) of calling “isnull() followed by an aggregation” (IsNullOP + AggregateOP), it shall replace it with the optimized method (B) null_aggregate (NullAggregateOP), when the following conditions are met: ● the aggregate method is supported by the null_aggregate() method ● and the result of isnull() is not used anywhere else in the user program.

[0029] Fig. 4 shows an example of process for the optimization of proposal 1.

[0030] (Example 1) Implement a new operator, named NullAggregateOP and during the compilation of the IR-OPs generated at the time of execution of a user program, if the combination of IsNullOP(data) and AggregateOP(method) is detected and the aggregate method is known, replace the combination of IR-OPs with NullAggregateOP(data, method). Auto-conversion does not need to expose new interface for user to learn and manually modify existing program. < user program > df = read_csv(“data.csv”) r1 = df.isnull().sum() r2 = df.isnull().mean() < generated IR-OPs > %1 = read_csv(“data.csv”) %2 = is_null(%1) # IsNullOP %3 = aggregate(%2, “sum”) # AggregateOp(“sum”) %4 = is_null(%1) # IsNullOP %5 = aggregate(%4, “mean”) # AggregateOp(“mean”) < replaced IR-OPs > %1 = read_csv(“data.csv”) %2 = null_aggregate(%1, “sum”) # NullAggregateOp(“sum”) %3 = null_aggregate(%1, “mean”) # NullAggregateOp(“mean”)

[0031] The aggregate method is performed by null bitmap. Fig. 5 shows an example of a null bitmap. A null bitmap is a lightweight data structure (consumes 1 bit for each value) that represents each valid value (non-missing / non-null) as “1” and each invalid value (missing / null) as “0”. As shown in Fig. 5, only 2 bytes of data to represent the validity of 16 elements. Column-wise null count can easily be computed from this null bitmap. A null bitmap can be extracted only for columns having null elements.

[0032] Fig. 6 shows an example of NullAggregateOp(“sum”) by using a null bitmap. As shown in Fig. 6, sum of null elements in this table can be easily computed by using the null bitmap. NullAggregateOp(“sum”) by using a null bitmap does not need to create intermediate table of Boolean data as a result of IsNullOP. NullAggregateOp(“sum”) by using a null bitmap save memory and execution time because a null bitmap is more lightweight than Boolean table.

[0033] Why null_aggregate() is better? A: There exists a conventional way of performing null-aggregation: performing isnull() to create a Boolean table and applying the intended aggregate operation (sum, mean etc.): For example, df.isnull().aggregate(“sum”) is used to calculate the row-wise number of nulls in input table “df.” B: There exists an optimized way of performing null-aggregation (proposed previously in a different patent specification): limiting the search area of the table containing nulls by using null-bitmap: extract only those columns having null from input table T and create an extracted table T1. slice the extracted table T1 in several chunks and further extract columns having null from each slice of T1 to limit the scope of checking nulls when performing sum, max, mean etc.

[0034] B is far better than A due to the limited usage of runtime memory and CPU. For example, when only a small number of null entries are in input data, it processes only that small part and finishes the computation. If there are no null entries at all, the execution finishes without any computation.

[0035] (Proposal 2) When the Filter operation is detected using Not(IsNull) (FilterOP + NotOP + IsNullOP), the same can be converted to DropNAOP when the following conditions are met: ● Target table to be filtered is the same as the table on which NotOP+IsNullOP is performed. ● The result for NotOP+IsNullOP is not used anywhere in the user program.

[0036] For example, df.filter(~df[“A”].isnull()) can be replaced with df.dropna([”A”]) but, df.filter(~df2[“A”].isnull()) should not be replaced with: df.dropna([”A”]).

[0037] (Example 2) Even if dropna() (operation for dropping the rows containing NA) is a known method to the programmer of dataframe library like pandas, it is often found that the same operation as dropna() is performed by combining a few method calls as follows: res = df[~df[“A”].isnull()]

[0038] Fig. 7 shows an example of operation of df[~df[“A”].isnull()]. Since null check is performed on “A” column of same table (df) which is to be filtered using the generated True-False mask, it can be optimized using dropna() as follows: res = df.dropna(subset=[“A”])

[0039] If target table for filter is %1, and is_null check is performed on column of table %1, hence, the following IR-ops can be optimized using dropna() as follows: <IR-ops for df[~df[“A”].isnull()]> %1 = read_csv(“data.csv”) %2 = project_table(%1, “A”) %3 = is_null(%2) %4 = not(%3) %4 = filter_table(%1, %4) <optimized IR-OPs> %1 = read_csv(“data.csv”) %2 = dropna(%1, “A”)

[0040] The operation drop() saves memory and execution time because the operation drop() can be performed by using a null bitmap.

[0041] Why dropna() is better? Dropping rows having nulls can be performed in either of the two ways: A: Using Filter Approach: Generate a true / false mask-vector indicating whether the respective element in a given column is null. Filter using the inverted mask-vector. mask = df[“A”].isnull() # checks whether data in A column is null result = df.filter(~mask) # keeps rows in input data (df) where mask is not True (not null) B: Optimized DropNA Approach (proposed previously in some different patent): Instead of checking nulls for each individual element of column A, slice the column into multiple chunks. For chunk that doesn’t contain any null (null-bitmap is full of 1s), there is nothing to be dropped, so copy it as it is. For chunk that contains null (some null bitmap is 0), perform the filter approach.

[0042] Again, B is far better than A due to the limited computational cost. For example, when only a small number of null entries are in input data, it processes only that small part and finishes the computation. If there are no null entries at all, the execution finishes without any computation.

[0043] (Proposal 3) Additionally, we can further combine the following null aggregator operations to create a better NullAggregateOP. For example, when an operation like the following is to be performed: df.null_aggregate(“sum”) / df.null_aggregate(“count”), since both aggregate operations are performed on the same data “df”, we can combine them to optimize it as follows: df.null_aggregate(“mean”), when the following conditions are met: ● the target table for both operators are same ● none of the operators are used anywhere in the user program.

[0044] In short, when the compiler detects a pattern of Division of NullAggregateOP(df, “sum”) by NullAggregateOP(df, “count”), it will replace it as NullAggregateOP(df, “mean”), since both operators are performed on the same input “df”. But when an operation like df.isnull().aggregate(“sum”) / df2.isnull().aggregate(“count”) is to be performed, the optimization cannot be performed since the target inputs are different (df and df2).

[0045] (Example 3) A user program may be written as follows to calculate the PERCENTAGE of missing values per row. Fig. 8 shows an example of operation of df.isnull().sum() / df.isnull().count() * 100. res = df.isnull().sum() / df.isnull().count() * 100

[0046] Since count() result (number of values in each row) will not change for any of the rows in a given table, we can simply optimize such poorly written program using IR optimization as follows: res = df.null_aggregate(“mean”) * 100

[0047] If a division uses NullAggregateOP(“sum”) and NullAggregateOP(“count”) and target tables are same, replace the IR-OPs for sum, count and division with NullAggregateOP(“mean”). < IR-OPs for df.isnull().sum() / df.isnull().count() * 100 > %1 = read_csv(“data.csv”) %2 = is_null(%1) %3 = aggregate(%2, “sum”) %4 = is_null(%1) %5 = aggregate(%4, “count”) %6 = binop_div(%3, %5) %7 = binop_mul(%6, 100) < optimized IR-OPs by NullAggregateOP > %1 = read_csv(“data.csv”) %2 = null_aggregate(%1, “sum”) %3 = null_aggregate(%1, “count”) %4 = binop_div(%2, %3) %5 = binop_mul(%4, 100) <Further optimized IR-OPs > %1 = read_csv(“data.csv”) %2 = null_aggregate(%1, “mean”) %3 = binop_mul(%2, 100)

[0048] (Optimized Flow: NullAggregateOP) Fig. 9 shows a flow chart of optimizing IR-OPs with NullAggregateOP. As shown in Fig. 9, the optimizing IR-OPs starts with the list of generated IR-OPs.

[0049] The first step (S1) determines whether or not IsNullOP+AggregateOP pattern is detected and Aggregate method is known. If the first step (S1) is yes, the second step (S2) replaces “IsNullOP+AggregateOP” to “NullAgggregateOP”. If the first step (S1) is no, the process goes finish.

[0050] The third step (S3) determines whether or not a division uses NullAggregateOP(“sum”) and NullAggregateOP(“count”) and target tables are same. If the third step (S3) is yes, the fourth step (S4) replaces the ops for sum, count and division with NullAggregateOP(“mean”). If the third step (S3) is no, the process goes finish.

[0051] The above optimizing method for intermediate representation operators can be implemented in an optimizing program for intermediate representation operators to cause a computer to perform the above process.

[0052] (Optimized Flow: DropNAOP) Fig. 10 shows a flow chart of optimizing IR-OPs with DropNAOP. As shown in Fig. 10, the optimizing IR-OPs starts with the list of generated IR-OPs.

[0053] The fifth step (S5) determines whether or not a filter is detected on NotOP+IsNullOP and target column for IsNullOP is from the same table target for the filter. If the fifth step (S5) is yes, the sixth step (S6) replace the IR-OPs with DropNAOP with subset as the target column. If the fifth step (S5) is no, the process goes finish.

[0054] The above optimizing method for intermediate representation operators can be implemented in an optimizing program for intermediate representation operators to cause a computer to perform the above process.

[0055] <Supplementary Notes> The present disclosure may be represented as form of Supplementary Notes, without limitation. <Note 1> As described in the first aspect of the present disclosure. <Note 2> The optimizing method for intermediate representation operators according to Note 1, comprising: detecting a division that uses NullAggregateOP(“sum”) and NullAggregateOP(“count”) and target tables are same; and replacing the division of NullAggregateOP(“sum”) by NullAggregateOP(“count”) with NullAggregateOP(“mean”) if the target table for both operators NullAggregateOP(“sum”) and NullAggregateOP(“count”) are same and none of the operators NullAggregateOP(“sum”) and NullAggregateOP(“count”) are used anywhere in a user program. <Note 3> As described in the second aspect of the present disclosure. <Note 4> The optimizing method for intermediate representation operators according to any one of Notes 1 to 3, wherein the intermediate representation operators are generated from a user program. <Note 5> The optimizing method for intermediate representation operators according to any one of Notes 1 to 3, wherein the null bitmap is extracted only for columns having null elements. <Note 6> As described in the third aspect of the present disclosure. <Note 7> The optimizing program for intermediate representation operators according to Note 6, causing a computer to perform: detecting a division that uses NullAggregateOP(“sum”) and NullAggregateOP(“count”) and target tables are same; and replacing the division of NullAggregateOP(“sum”) by NullAggregateOP(“count”) with NullAggregateOP(“mean”) if the target table for both operators NullAggregateOP(“sum”) and NullAggregateOP(“count”) are same and none of the operators NullAggregateOP(“sum”) and NullAggregateOP(“count”) are used anywhere in a user program. <Note 8> As described in the fourth aspect of the present disclosure. <Note 9> The optimizing program for intermediate representation operators according to any one of Notes 6 to 8, wherein the intermediate representation operators are generated from a user program. <Note 10> The optimizing program for intermediate representation operators according to any one of Notes 6 to 8, wherein the null bitmap is extracted only for columns having null elements.

[0056] While each example embodiment of the present invention has been described, it is to be noted that it is possible to modify or adjust the example embodiments or examples within the whole disclosure of the present invention (including the Claims) and based on the basic technical concept thereof. Further, it is possible to variously combine or select (or at least partially remove) a wide variety of the disclosed elements (including the individual elements of the individual claims, the individual elements of the individual example embodiments or examples, and the individual elements of the individual figures) within the scope of the whole disclosure of the present invention. That is, it is self-explanatory that the present invention includes any types of variations and modifications to be done by a skilled person according to the whole disclosure including the Claims and the technical concept of the present invention. Particularly, any numerical ranges disclosed herein should be interpreted that any intermediate values or subranges falling within the disclosed ranges are also concretely disclosed even without specific recital thereof. Further, the disclosure of each Patent Literature cited above is incorporated herein in its entirety by reference thereto.

[0057] T1 tabular data T2 Boolean table

Claims

1. An optimizing method for intermediate representation operators comprising: detecting a combination of IsNullOP(data) and AggregateOP(method) in the intermediate representation operators; and replacing the combination of IsNullOP(data) and AggregateOP(method) with NullAggregateOP(data, method) that is performed by using a null bitmap if a method in AggregateOP(method) is supported by NullAggregateOP(data, method) and a result of IsNullOP(data) is not used anywhere else in a user program.

2. The optimizing method for intermediate representation operators according to claim 1, comprising: detecting a division that uses NullAggregateOP(“sum”) and NullAggregateOP(“count”) and target tables are same; and replacing the division of NullAggregateOP(“sum”) by NullAggregateOP(“count”) with NullAggregateOP(“mean”) if the target table for both operators NullAggregateOP(“sum”) and NullAggregateOP(“count”) are same and none of the operators NullAggregateOP(“sum”) and NullAggregateOP(“count”) are used anywhere in a user program.

3. An optimizing method for intermediate representation operators comprising: detecting a filter that uses NotOP and IsNullOP for which target column is from the same table target for the filter; and replacing the filter that uses NotOP and IsNullOP with DropNAOP with subset as the target column, wherein DropNAOP is performed by using a null bitmap if the target table to be filtered is the same as the table on which NotOP+IsNullOP is performed and a result for NotOP+IsNullOP is not used anywhere in the user program.

4. The optimizing method for intermediate representation operators according to any one of claims 1 to 3, wherein the intermediate representation operators are generated from a user program.

5. The optimizing method for intermediate representation operators according to any one of claims 1 to 3, wherein the null bitmap is extracted only for columns having null elements.

6. An optimizing program for intermediate representation operators causing a computer to perform: detecting a combination of IsNullOP(data) and AggregateOP(method) in the intermediate representation operators; and replacing the combination of IsNullOP(data) and AggregateOP(method) with NullAggregateOP(data, method) that is performed by using a null bitmap if a method in AggregateOP(method) is supported by NullAggregateOP(data, method) and a result of IsNullOP(data) is not used anywhere else in a user program.

7. The optimizing program for intermediate representation operators according to claim 6, causing a computer to perform: detecting a division that uses NullAggregateOP(“sum”) and NullAggregateOP(“count”) and target tables are same; and replacing the division of NullAggregateOP(“sum”) by NullAggregateOP(“count”) with NullAggregateOP(“mean”) if the target table for both operators NullAggregateOP(“sum”) and NullAggregateOP(“count”) are same and none of the operators NullAggregateOP(“sum”) and NullAggregateOP(“count”) are used anywhere in a user program.

8. An optimizing program for intermediate representation operators causing a computer to perform: detecting a filter that uses NotOP and IsNullOP for which target column is from the same table target for the filter; and replacing the filter that uses NotOP and IsNullOP with DropNAOP with subset as the target column, wherein DropNAOP is performed by using a null bitmap if the target table to be filtered is the same as the table on which NotOP+IsNullOP is performed and a result for NotOP+IsNullOP is not used anywhere in the user program.

9. The optimizing program for intermediate representation operators according to any one of claims 6 to 8, wherein the intermediate representation operators are generated from a user program.

10. The optimizing program for intermediate representation operators according to any one of claims 6 to 8, wherein the null bitmap is extracted only for columns having null elements.

Images