Data model optimization system, data model optimization method, and data model optimization program

Abstract

In the present invention, a calculation unit (110) calculates, for each column in a plurality of tables and on the basis of database configuration information, a call count which is the number of times the column is called when data is acquired according to a data acquisition scenario. The calculation unit (110) calculates, for each column set in the plurality of tables, a call count which is the number of times the column set is called at the same timing when data is acquired from the database according to the data acquisition scenario. The calculation unit (110) calculates a degree of similarity between the names of the columns for each column set in the plurality of tables. A generation unit (130) generates a data model on the basis of the call count for each column, the call count for each column set, and the degree of similarity for each column set.

Description

Data model optimization system, data model optimization method, and data model optimization program

[0001] The present disclosure relates to techniques for generating data models for applications.

[0002] Patent Document 1 proposes a technique for generating new JSON format data using data written in JSON format and a conversion formula written in JSON format.

[0003] Special table 2017-535854 publication

[0004] Patent Literature 1 does not propose a technology for regenerating a transformation formula based on evaluation of the transformation formula described in the JSON format, and therefore is unable to generate a data model suitable for an application by taking into account the combination and division of database tables.

[0005] The present disclosure aims to enable the creation of a data model suited to an application.

[0006] The data model optimization system of the present disclosure comprises: a single call count calculation unit that calculates, for each column in the multiple tables, the number of calls that the column will be called when data is acquired from the database in accordance with the data acquisition scenario, based on database configuration information that indicates the configuration of multiple tables in a database and a data acquisition scenario that specifies data to be acquired from the database; a pair call count calculation unit that calculates, for each column set in the multiple tables, the number of calls that the column set will be called at the same time when data is acquired from the database in accordance with the data acquisition scenario, based on the database configuration information and the data acquisition scenario; a similarity calculation unit that calculates, for each column set in the multiple tables, a similarity between column names, based on the database configuration information; and a data model generation unit that generates a data model for expressing data acquired in accordance with the data acquisition scenario in a structure suitable for processing by an application that uses the data, based on the number of calls for each column, the number of calls for each column set, and the similarity for each column set.

[0007] According to the present disclosure, it is possible to generate a data model suitable for an application.

[0008] 1 is a diagram illustrating a configuration of a data model optimization system 100 according to a first embodiment. 2 is a diagram illustrating a configuration of a data processing system 200 according to the first embodiment. 3 is a diagram illustrating a functional configuration of the data processing system 200 according to the first embodiment. 4 is a flowchart illustrating a data model optimization method according to the first embodiment. 5 is a flowchart illustrating a data model optimization method according to the first embodiment. 6 is a diagram illustrating an example of a configuration of a database 212 according to the first embodiment. 7 is a diagram illustrating an example of a data acquisition scenario D02 according to the first embodiment. 8 is a diagram illustrating an example of a first edition model D20 according to the first embodiment. 9 is a diagram illustrating an example of a data conversion image according to the first embodiment. 10 is a diagram illustrating an example of a data model D31 according to the first embodiment. 11 is a diagram illustrating an example of a data model D31 according to the first embodiment. 12 is a diagram illustrating an example of a data model D31 according to the first embodiment. 13 is a diagram illustrating an example of a functional configuration of a data processing system 200 according to the first embodiment. 14 is a diagram illustrating a functional configuration of a data processing system 200 according to the second embodiment. 15 is a flowchart illustrating a data model optimization method according to the second embodiment. 16 is a flowchart illustrating a data model optimization method according to the second embodiment. 17 is a diagram illustrating a hardware configuration of a data model optimization system 100 according to an embodiment.

[0009] In the embodiments and drawings, the same or corresponding elements are denoted by the same reference numerals. The description of elements denoted by the same reference numerals as those already described will be omitted or simplified as appropriate. Arrows in the drawings primarily indicate the flow of data or the flow of processing.

[0010] First Embodiment A data model optimization system 100 will be described with reference to FIGS.

[0011] ***Description of Configuration*** The configuration of the data model optimization system 100 will be described with reference to Figure 1. The data model optimization system 100 is a computer equipped with hardware such as a processor 101, a memory 102, an auxiliary storage device 103, a communication device 104, and an input / output interface 105. These pieces of hardware are connected to each other via signal lines. The data model optimization system 100 may be configured with multiple computers instead of being configured with one computer (device).

[0012] The processor 101 is an IC that performs arithmetic processing and controls other hardware. For example, the processor 101 is a CPU. IC is an abbreviation for Integrated Circuit. CPU is an abbreviation for Central Processing Unit.

[0013] The memory 102 is a volatile or non-volatile storage device. The memory 102 is also called a primary storage device or a main memory. For example, the memory 102 is a RAM. Data stored in the memory 102 is saved in the secondary storage device 103 as needed. RAM is an abbreviation for Random Access Memory.

[0014] The auxiliary storage device 103 is a non-volatile storage device. For example, the auxiliary storage device 103 is a ROM, a HDD, a flash memory, or a combination of these. Data stored in the auxiliary storage device 103 is loaded into the memory 102 as needed. ROM is an abbreviation for Read Only Memory. HDD is an abbreviation for Hard Disk Drive.

[0015] The communication device 104 is a receiver and a transmitter. For example, the communication device 104 is a communication chip or NIC. Communication of the data model optimization system 100 is performed using the communication device 104. NIC is an abbreviation for Network Interface Card.

[0016] The input / output interface 105 is a port to which an input device and an output device are connected. For example, the input / output interface 105 is a USB terminal, the input devices are a keyboard and a mouse, and the output device is a display. Input and output to and from the data model optimization system 100 are performed using the input / output interface 105. USB is an abbreviation for Universal Serial Bus.

[0017] The data model optimization system 100 includes elements such as a calculation unit 110, an evaluation unit 120, and a generation unit 130. These elements are realized by software. The calculation unit 110 includes elements such as a single call count calculation unit 111, a pair call count calculation unit 112, and a similarity calculation unit 113. The evaluation unit 120 includes elements such as a data model evaluation unit 121. The generation unit 130 includes elements such as a data model comparison unit 131 and a data model generation unit 132.

[0018] The auxiliary storage device 103 stores a data model optimization program for causing the computer to function as the calculation unit 110, the evaluation unit 120, and the generation unit 130. The data model optimization program is loaded into the memory 102 and executed by the processor 101. The auxiliary storage device 103 also stores an OS. At least a portion of the OS is loaded into the memory 102 and executed by the processor 101. The processor 101 executes the data model optimization program while running the OS. OS is an abbreviation for Operating System.

[0019] Input and output data of the data model optimization program are stored in the storage unit 190. The memory 102 functions as the storage unit 190. However, a storage device such as the auxiliary storage device 103, a register in the processor 101, or a cache memory in the processor 101 may function as the storage unit 190 instead of or together with the memory 102.

[0020] The data model optimization program can be recorded (stored) in a computer-readable manner on a non-volatile recording medium such as an optical disk or flash memory.

[0021] The configuration of a data processing system 200 will be described with reference to FIG. 2 . The data processing system 200 is a computer system that uses the data model optimization system 100. The data processing system 200 includes a data platform 210 and an application unit 221. The data platform 210 is a computer system that includes the data model optimization system 100, a data conversion unit 211, and a database 212. The data conversion unit 211 is an element that executes data conversion software and is realized by a processing circuit (e.g., a processor) of a computer. The data conversion software causes the computer to function as the data conversion unit 211. The application unit 221 is an element that executes an application program for data processing and is realized by a processing circuit of the computer. The application program causes the computer to function as the application unit 221.

[0022] The data model optimization system 100 is introduced into a communication system between the application unit 221 and the data platform 210. The data model optimization system 100 generates a data model D1 and sends the data model D1 to the data conversion unit 211. The data conversion unit 211 receives the data model D1. The application unit 221 sends a data request D2 to the data platform 210. The data conversion unit 211 receives the data request D2. The data conversion unit 211 obtains data D4 requested in the data request D2 from the database 212 by querying the database 212 D3. The data conversion unit 211 converts the data D4 into data D5 based on the data model D1 and sends the data D5 to the application unit 221. The application unit 221 receives the data D5 and performs data processing using the data D5. The data model D1 represents only the data necessary for the application from a huge amount of source data (DB) in a structure (format and grouping) suitable for application processing. For example, data model D1 converts data D4 into data D5 as follows: Data model D1 represents a "person" whose internal structure includes name, age, and gender. Then, separate data D4 consisting of "Suzuki," "26," and "Female" is obtained from database 212. In this case, data model D1 stores "Suzuki" in the "Name" field of the internal structure, stores "26" in the "Age" field of the internal structure, and stores "Female" in the "Gender" field of the internal structure. As a result, data D4 is converted into data D5 representing a "person named Suzuki." The application uses this group of data D5.

[0023] 3 shows the functional configuration of the data model optimization system 100. The functions of each element of the data model optimization system 100 and the data input and output between the elements will be described later.

[0024] ***Description of Operation*** The operation procedure of the data model optimization system 100 corresponds to a data model optimization method. Also, the operation procedure of the data model optimization system 100 corresponds to a processing procedure by a data model optimization program.

[0025] The data model optimization method will be described with reference to Figures 4 and 5. In step S110, the calculation unit 110 calculates the number of calls for each column, the number of calls for each column set, and the similarity for each column set based on the database configuration information D01 and the data acquisition scenario D02.

[0026] The database configuration information D01 is data that indicates the configuration of a plurality of tables in the database 212. The database configuration information D01 is acquired from the database 212, for example.

[0027] 6 shows an example of the configuration of multiple tables in the database 212. The database 212 has a first table, a second table, and a third table. Time series data is registered in each table. The first table has columns such as "ID," "person's name," "person's age," "person's location," and "observation time." The second table has columns such as "ID," "person's name," "person's gender," "person's heart rate," and "observation time." The third table has columns such as "ID," "robot model number," "robot location," "remaining battery level of the robot," and "observation time." Database configuration information D01 shows this configuration of the database 212.

[0028] 4, the description of step S110 will be continued. The data acquisition scenario D02 is data that specifies data to be acquired from the database 212. The data acquisition scenario D02 is acquired from the application unit 221, for example.

[0029] FIG. 7 shows an example of a data acquisition scenario D02. The first scenario indicates that data for each column specified in the field to be used is acquired in the order specified in the timing field. Data for two or more columns specified in the field to be used at the same timing is acquired at the same time. "Same timing" may be read as "simultaneous." The second scenario indicates that data for multiple columns specified in the field to be used is acquired all at once.

[0030] 4, the details of step S110 will be described. The single call count calculation unit 111 calculates the number of calls for each column in the multiple tables indicated in the database configuration information D01 based on the data acquisition scenario D02. The calculated number of calls is the number of times a column is called when data is acquired from the database 212 according to the data acquisition scenario D02.

[0031] An example of calculating the number of calls for each column will be described based on the first scenario in Figure 7. In the first scenario, the column "Person's Age" is specified in the used fields column for each of the first and eleventh timings. Therefore, if the column "Person's Age" is not specified in the used fields column for the fifteenth timing and thereafter, the number of calls for the column "Person's Age" is 2. In the second scenario, the number of calls for each column specified in the used fields column is 1, and the number of calls for the other columns is 0.

[0032] Returning to FIG. 4, the description of step S110 continues. The pair call count calculation unit 112 calculates the call count for each column pair in the multiple tables indicated in the database configuration information D01 based on the data acquisition scenario D02. The calculated call count is the number of times a column pair is called at the same time when data is acquired from the database 212 according to the data acquisition scenario D02. A column pair consists of two or more columns. For example, each of all combinations of all columns in all tables in the database 212 is a column pair.

[0033] An example of calculating the number of calls for each column group will be described based on the first scenario in Figure 7. In the first scenario, the group of columns "Person's Age" and "Person's Heart Rate" is specified in the used fields column for each of the first and eleventh timings. Therefore, if the group of columns "Person's Age" and "Person's Heart Rate" is not specified in the used fields column for the fifteenth timing and later, the number of calls for the group of columns "Person's Age" and "Person's Heart Rate" is 2. In the second scenario, the number of calls for each column group specified in the used fields column is 1, and the number of calls for the other column groups is 0.

[0034] 4, the description of step S110 will be continued. The similarity calculation unit 113 calculates the similarity for each column pair in the multiple tables indicated in the database configuration information D01. The calculated similarity is the similarity between the names of the columns included in the column pair.

[0035] An example of calculating the similarity for each column pair will be described with reference to FIG. 6 . When the name strings of the columns completely match, the similarity of the column pair is calculated by multiplying the reference value by the number of columns. The number of columns refers to the number of columns included in the column pair. The name strings "Person's Name" in the first table and the column "Person's Name" in the second table completely match. Therefore, when the reference value is 10, the value "20" calculated by multiplying the reference value "10" by the number of columns "2" is the similarity of the pair of the column "Person's Name" in the first table and the column "Person's Name" in the second table. When the name strings of the columns do not completely match, the similarity of the column pair is calculated by multiplying the number of common words by the number of columns. The number of common words refers to the number of words common in the name strings of the columns. The column "Person's Position" in the first table and the column "Robot's Position" in the third table share the word "Position" in their name strings. Therefore, the value "2" calculated by multiplying the number of common words "1" by the number of columns "2" is the similarity between the pair of column "person position" in the first table and column "robot position" in the third table.

[0036] Returning to FIG. 4 , the description of step S110 will continue. The single call count calculation unit 111 stores the number of calls for each column in the storage unit 190. The pair call count calculation unit 112 stores the number of calls for each column pair in the storage unit 190. The similarity calculation unit 113 stores the similarity for each column pair in the storage unit 190. Data indicating the number of calls for each column, the number of calls for each column pair, and the similarity for each column pair will be referred to as calculation information D11.

[0037] In step S120, the data model evaluation unit 121 evaluates the data model D21 based on the data acquisition scenario D02 and calculates an evaluation value of the data model D21. The data model D21 is data that indicates rules for expressing data acquired according to the data acquisition scenario D02 in a structure suitable for processing by an application that uses the data.

[0038] In the first step S120, the first version model D20 is evaluated. The first version model D20 is a data model D21 that is generated in advance. For example, the first version model D20 is input to the data model optimization system 100, and the data model evaluation unit 121 receives the input first version model D20.

[0039] 8 shows an example of a first edition model D20. The first edition model D20 indicates rules for expressing data acquired from each table in the database 212 in a structure suitable for processing by an application that uses the data. Data model x is the first edition model D20 for the first table. Data model y is the first edition model D20 for the second table.

[0040] 9 shows an example of data whose structure has been converted in accordance with the first edition model D20. Data conversion image x shows data whose structure has been converted in accordance with data model x. Data conversion image y shows data whose structure has been converted in accordance with data model y.

[0041] 4, the description of step S120 will be continued. In step S120 from the second time onwards, the data model D31 generated in step S140 is evaluated as the data model D21. The data model D31 is stored in the storage unit 190 and is read out from the storage unit 190.

[0042] The evaluation value (score) of the data model D21 is a value obtained by evaluating the data model D21 based on the evaluation axis. The evaluation axis means a standard, rule, or condition for evaluation. An example of an evaluation axis is the number of queries to the database 212. The number of queries corresponds to the number of accesses to a table. The evaluation axis may be the amount of data communication or the number of data models. The evaluation axis may be a combination of factors such as the number of queries, the amount of data communication, and the number of data models. The evaluation axis may be related to communication performance. Other evaluation axes may also be used.

[0043] In the first embodiment, the smaller the evaluation value, the higher the evaluation of the data model D21, and the larger the evaluation value, the lower the evaluation of the data model D21.

[0044] The evaluation value of a data model is calculated as follows: The data model evaluation unit 121 simulates the behavior of the data processing system 200 (particularly, at least one of the data conversion unit 211 and the application unit 221) when the data model to be evaluated is used. Then, the data model evaluation unit 121 calculates the evaluation value of the data model based on the results of the simulation.

[0045] An example of calculation of an evaluation value will be described. In this example, the evaluation axis is the number of queries to the database 212. The database 212 has the table shown in FIG. 6. The data acquisition scenario D02 is the first scenario in FIG. 7, and the data model D21 to be evaluated is the first edition model D20 in FIG. 8. First, the data model evaluation unit 121 calculates an evaluation value for each timing indicated in the first scenario. At the first timing, data on "person's age" and "person's heart rate" are acquired. "Person's age" is a column in the first table, and "person's heart rate" is a column in the second table. Therefore, when the first edition model D20 is used, a query to the first table of the database 212 and a query to the second table of the database 212 are made. In other words, the number of queries to the database 212 is 2. Therefore, the evaluation value for the first timing is 2. The data model evaluation unit 121 then sums the evaluation values ​​for each timing. The calculated sum becomes the evaluation value for the first edition model D20.

[0046] Continuing the explanation of step S120, the evaluation value calculated for the data model D21 is referred to as the evaluation value D23. The data model evaluation unit 121 stores evaluation information D22 in the storage unit 190. The evaluation information D22 indicates the evaluation value D23 of the data model D21 in association with the identifier of the data model D21. The data model evaluation unit 121 stores the data model D21 in the storage unit 190.

[0047] In step S131, the data model comparison unit 131 compares the latest evaluation value D23 with the reference value D24, and determines whether the evaluation represented by the latest evaluation value D23 is higher than the evaluation represented by the reference value D24.

[0048] Specifically, the data model comparison unit 131 determines whether the latest evaluation value D23 is smaller than the reference value D24. If the latest evaluation value D23 is smaller than the reference value D24, the evaluation represented by the latest evaluation value D23 is higher than the evaluation represented by the reference value D24. If the latest evaluation value D23 is larger than the reference value D24, the evaluation represented by the latest evaluation value D23 is lower than the evaluation represented by the reference value D24.

[0049] The latest evaluation value D23 is the evaluation value D23 of the latest data model D21 among the generated data models D21. In other words, the latest evaluation value D23 is the evaluation value D23 calculated in the immediately preceding step S120. The latest evaluation value D23 is read from the storage unit 190.

[0050] The reference value D24 is an evaluation value that represents the highest evaluation among one or more evaluation values ​​D23 for one or more data models D21 generated before the latest data model D21. In other words, the reference value D24 is the smallest evaluation value among one or more evaluation values ​​D23 for one or more data models D21 generated before the latest data model D21. The reference value D24 used in the first step S131 is an initial value (e.g., the maximum value). The reference value D24 is stored in the storage unit 190 and read out from the storage unit 190.

[0051] If the latest evaluation value D23 is smaller than the reference value D24, that is, if the evaluation represented by the latest evaluation value D23 is higher than the evaluation represented by the reference value D24, the process proceeds to step S132. If the current evaluation value D23 is equal to or greater than the reference value D24, that is, if the evaluation represented by the latest evaluation value D23 is lower than the evaluation represented by the reference value D24, the process proceeds to step S151.

[0052] In step S132, the data model comparison unit 131 updates the reference value D24 to the latest evaluation value D23.

[0053] In step S133, the data model comparison unit 131 resets the non-update count to zero. The non-update count is the number of times the reference value D24 has not been updated. The non-update count is stored in the storage unit 190.

[0054] In step S140, the data model generation unit 132 generates a data model D31 based on the calculation information D11. The data model D31 is generated so that the evaluation value D23 of the data model D31 is small. In other words, the data model D31 is generated so that the evaluation represented by the evaluation value D23 of the data model D31 is high.

[0055] The data model D31 is generated by at least one of the following methods (1) to (3).

[0056] (1) The data model D31 is generated as follows. First, the data model generation unit 132 selects a column pair with a high call count based on the call count for each column pair. A pair count threshold is used for the selection. The pair count threshold is stored in the storage unit 190. The selected column pair is referred to as a target column pair. Specifically, the data model generation unit 132 compares the call count for each column pair with the pair count threshold, and selects a column pair whose call count is equal to or greater than the pair count threshold as the target column pair. Then, the data model generation unit 132 generates a data model D31 for expressing the data of each column of the target column pair only among the column pairs in the multiple tables indicated in the database configuration information D01, in a structure suitable for processing by an application that uses the data.

[0057] FIG. 10 shows an example of a data model D31 generated by method (1). The data model D31 in FIG. 10 is described below. When the first scenario in FIG. 7 is used, the pair of columns "Person's Age" and "Person's Heart Rate" is called at the first and eleventh timings. That is, the pair of columns "Person's Age" and "Person's Heart Rate" is called twice. When the pair count threshold is equal to or less than two, the pair of columns "Person's Age" and "Person's Heart Rate" is a target column pair. In the database 212 in FIG. 6, "Person's Age" is a column in the first table, and "Person's Heart Rate" is a column in the second table. When the first-edition model D20 in FIG. 8 is used, conversion is performed for each table, resulting in a high evaluation value D23 based on the number of queries or the amount of data communication. Therefore, the data model D31 in FIG. 10 is generated to lower the evaluation value D23. 10 is a data model D31 for expressing the data in the columns "person's age" and "person's heart rate" in a structure suitable for processing by an application that uses the data. Note that the columns "ID" and "observation time" are required items when reading data, so the data model D31 indicates the names of the columns "ID" and "observation time."

[0058] (2) The data model D31 is generated as follows. First, the data model generation unit 132 selects a column pair with high similarity from among combinations of columns with high call counts based on the call count for each column and the similarity for each column pair. A single call count threshold and a similarity threshold are used for this selection. The single call count threshold and the similarity threshold are stored in the storage unit 190. The selected column pair is referred to as a target column pair. Specifically, the data model generation unit 132 compares the call count for each column with the single call count threshold and selects, as target columns, each column whose call count is equal to or greater than the single call count threshold. Then, for each column pair that is a target column combination, the data model generation unit 132 compares the similarity of the column pair with a similarity threshold and selects, as target column pairs, a column pair whose similarity is equal to or greater than the similarity threshold. Then, the data model generation unit 132 generates a data model D31 for only the target column pair among the column pairs in the multiple tables indicated in the database configuration information D01, to represent the data of each column of the target column pair in a structure suitable for processing by an application that uses the data.

[0059] FIG. 11 shows an example of a data model D31 generated by method (2). The data model D31 in FIG. 11 is described below. When the first scenario in FIG. 7 is used, the column "Person Location" is called at the second and twelfth timings. The column "Robot Location" is called at the third and thirteenth timings. That is, the number of times each of the columns "Person Location" and "Robot Location" is called is two. When the single-count threshold is two or less, the columns "Person Location" and "Robot Location" are target columns. In the database 212 in FIG. 6, "Person Location" is a column in the first table, and "Robot Location" is a column in the third table. When the first-edition model D20 is used, conversion is performed for each table, so the evaluation value D23 based on the number of queries or the amount of data communication, etc., is high. The column pair "Person Location" and the column "Robot Location" share the word "location" in common, so the similarity between the column pair is high. Therefore, the similarity between the pair of columns "person location" and "robot location" is equal to or greater than the similarity threshold. Therefore, in order to lower the evaluation value D23, data model D31 in FIG. 11 is generated. Data model D31 in FIG. 11 is a data model D31 for expressing the data in the columns "person location" and "robot location" in a "location" structure suitable for processing by applications that use the data. When the character strings in the names of the columns share multiple common words, data model D31 indicates the conversion of the multiple common words.

[0060] (3) The data model D31 is generated as follows. First, the data model generation unit 132 selects a column pair with low similarity from among combinations of columns with high call counts based on the call count for each column and the similarity for each column pair. A single call count threshold and a similarity threshold are used for this selection. The single call count threshold and the similarity threshold are stored in the storage unit 190. The selected column pair is referred to as a target column pair. Specifically, the data model generation unit 132 compares the call count for each column with the single call count threshold and selects, as target columns, each column whose call count is equal to or greater than the single call count threshold. Then, for each column pair that is a target column combination, the data model generation unit 132 compares the similarity of the column pair with a similarity threshold and selects, as target column pairs, column pairs whose similarity is less than the similarity threshold. Then, for only the target column pair among the column pairs in the multiple tables indicated in the database configuration information D01, the data model generation unit 132 generates a data model D31 for representing the data of each column of the target column pair in separate structures suitable for processing by an application that uses the data.

[0061] FIG. 12 shows an example of a data model D31 generated by method (3). The data model D31 in FIG. 12 is described below. When the first scenario in FIG. 7 is used, the column "Person's Name" is called at the fourth and fourteenth timings. The column "Robot Model Number" is called at the fifth and fifteenth timings. That is, the number of times each of the columns "Person's Name" and "Robot Model Number" is called is two. When the single-count threshold is two or less, the columns "Person's Name" and "Robot Model Number" are target columns. In the database 212 in FIG. 6, "Person's Name" is a column in the first and second tables, and "Robot Model Number" is a column in the third table. When the first-edition model D20 is used, conversion is performed for each table, so the evaluation value D23 based on the number of inquiries or data communication volume, etc., is high. Because the pair of the columns "Person's Name" and "Robot Model Number" does not share any common words, the similarity between the pair of columns is low. Therefore, the similarity between the pair of columns "person's name" and "robot's model number" is less than the similarity threshold. Therefore, in order to lower the evaluation value D23, data model D31 in FIG. 12 is generated. Data model D31 in FIG. 12 is a data model D31 for expressing the data in the columns "person's name" and "robot's model number" in separate (independent) structures suitable for processing by the application that uses the data.

[0062] The explanation of step S140 continues. The data model generation unit 132 changes each of the pair count threshold, the single count threshold, and the similarity threshold. For example, thresholds such as the pair count threshold, the single count threshold, and the similarity threshold are changed using machine learning as follows: The data model generation unit 132 changes the thresholds to appropriate values ​​using a trained model. The trained model is generated in advance and stored in the storage unit 190. The trained model is generated by a learning device. The learning device is, for example, a device separate from the data model optimization system 100. The learning device generates the trained model by learning training data using, for example, a convolutional neural network (CNN). The training data indicates the relationship between the thresholds and the evaluation values ​​of the data model. For example, the training data indicates the relationship between the thresholds used in another data model optimization system and the evaluation values ​​of a data model generated in another data model optimization system.

[0063] 4, the description of step S140 continues. The data model generation unit 132 passes the data model D31 to the data model evaluation unit 121. The data model generation unit 132 also stores the data model D31 in the storage unit 190. After step S140, the process proceeds to step S120.

[0064] 5, the description will continue from step S151. In step S151, the data model comparison unit 131 adds 1 to the number of times that the data model has not been updated, thereby updating the number of times that the data model has not been updated.

[0065] In step S152, the data model comparison unit 131 compares the number of times that the data has not been updated with a non-update threshold value to determine whether the number of times that the data has not been updated has reached the non-update threshold value. The non-update threshold value is a threshold value for the number of times that the data has not been updated, and is stored in advance in the storage unit 190.

[0066] If the number of unupdated times has reached the unupdated threshold, the process proceeds to step S153. If the number of unupdated times has not reached the unupdated threshold, the process proceeds to step S140.

[0067] In step S153, the data model comparison unit 131 outputs the data model D1, which is the data model D21 corresponding to the reference value D24.

[0068] The data model D1 is output as follows: First, the data model comparison unit 131 selects evaluation information D22 that indicates the same evaluation value D23 as the reference value D24, and acquires a data model identifier from the selected evaluation information D22. Next, the data model comparison unit 131 acquires the data model D21 identified by the acquired data model identifier from the storage unit 190. Then, the data model comparison unit 131 outputs the acquired data model D21 as the data model D1. The output data model D1 is input to the data conversion unit 211. After step S153, the processing ends.

[0069] ***Effects of First Embodiment*** The purpose of the first embodiment is to generate a data model suitable for each application by simulating the behavior of the application's communication system, taking into consideration the merging and splitting of database tables, and to optimize communication efficiency. The data model optimization system 100 repeatedly generates a data model based on the number of column calls and column similarity, and evaluates the data model based on the evaluation axis.

[0070] The data model optimization system 100 includes a calculation unit 110, an evaluation unit 120, and a generation unit 130. The calculation unit 110 calculates the number of calls for a single column, the number of calls for a combination of columns, and column similarity based on the database configuration and the application's data acquisition scenario. The evaluation unit 120 aggregates evaluation values ​​based on the evaluation axis for cases where an application acquires data using a data model based on the data model and the application's data acquisition scenario. The generation unit 130 compares the evaluation value based on the evaluation result with the minimum value of the data model evaluation result after system startup. If the evaluation value exceeds the minimum value of the data model evaluation result after system startup, the generation unit 130 generates a data model that reduces the evaluation value based on the evaluated data model, the number of column calls, and the similarity. This makes it possible to generate a data model suitable for each application, taking into account the merging and splitting of database tables, and optimize communication efficiency.

[0071] The generation unit 130 generates a data model as follows based on the column call count, similarity, and the evaluated data model. The generation unit 130 generates a data model that converts only columns with a high number of simultaneous column call counts. The generation unit 130 generates a data model that converts only columns with a high number of individual column call counts and high column similarity. The generation unit 130 generates a data model that converts only columns with a high number of individual column call counts and low column similarity. This makes it possible to generate a data model suitable for each application, taking into account the merging and splitting of database tables.

[0072] The calculation unit 110 calculates column similarity from the character strings of the names of each column in a database table based on the database configuration and the data acquisition scenario of the application. This makes it possible to generate a data model in which columns with high column similarity can be converted into the same data model when generating a data model.

[0073] The generation unit 130 compares the data model evaluation value with the minimum value of the data model evaluation results after system startup. Then, in any of the following cases, the generation unit 130 generates a new data model based on the column call count, similarity, and the evaluated data model so that the data model evaluation value becomes smaller. The generation unit 130 generates a new data model when the data model evaluation value is lower than the minimum value of the data model evaluation results after system startup. The generation unit 130 generates a new data model when the data model evaluation value exceeds the minimum value of the data model evaluation results after system startup but the number of times the minimum value of the data model evaluation results has not been updated since system startup has not reached a threshold. This makes it possible to generate a data model with higher communication efficiency and suitable for the application.

[0074] The generation unit 130 compares the data model evaluation value with the minimum value of the data model evaluation results after system startup. If the data model evaluation value exceeds the minimum value of the data model evaluation results after system startup and the number of times the minimum value of the data model evaluation results has not been updated since system startup reaches a threshold, the generation unit 130 outputs a data model. The output data model is the data model whose data model evaluation value is the minimum value of the data model evaluation results since system startup. This makes it possible to output an optimal data model, thereby optimizing communication efficiency.

[0075] ***Example of First Embodiment*** Fig. 13 shows an example of the functional configuration of a data processing system 200. Input and output data of the data model optimization system 100 may be stored in a network storage 230 instead of or together with the storage unit 190. The network storage 230 is a storage unit provided outside the data model optimization system 100, and is composed of one or more storage devices. The data model optimization system 100 stores data in the network storage 230 and retrieves data from the network storage 230 by communicating with the network storage 230.

[0076] *** Supplementary Note to First Embodiment *** When data from various fields is handled, as in smart cities, a data platform is constructed to collect and manage the data. Data integration across fields uses a software platform that allows applications to handle data using a common model. Such a software platform converts data according to a data model defined in the interface of the data platform and provides the converted data to the application. Therefore, the efficiency of communication between the application and the data platform depends on the data model. The first embodiment is a technology for functions implemented within a data platform that handles various data, as in smart cities. Application developers may not be aware of the database configuration of the data platform. Therefore, the data acquisition scenario of an application only indicates the fields of data used and the order in which the data is used.

[0077] Second Embodiment A mode for outputting a data model D1 for which a target evaluation has been obtained will be described below, focusing mainly on the differences from the first embodiment, with reference to FIGS.

[0078] ***Description of Configuration*** The configuration of the data processing system 200 will be described with reference to FIG. 14. The configuration of the data processing system 200 is the same as that in the first embodiment. However, in the data processing method, a target value D03 is used. The target value D03 is a value that represents a target evaluation, and is set in the setting file 191.

[0079] ***Description of Operation*** The data model optimization method will be described with reference to Figures 15 to 17. In step S210, the calculation unit 110 calculates the number of calls for each column, the number of calls for each column set, and the similarity for each column set, based on the database configuration information D01 and the data acquisition scenario D02. Step S210 is the same as step S110 in the first embodiment.

[0080] In step S220, the data model evaluation unit 121 evaluates the data model D21 based on the data acquisition scenario D02 to calculate an evaluation value of the data model D21. Step S220 is the same as step S120 in the first embodiment.

[0081] In step S231, the data model comparison unit 131 compares the latest evaluation value D23 with the target value D03, and determines whether the evaluation represented by the latest evaluation value D23 is higher than the evaluation represented by the target value D03.

[0082] Specifically, the data model comparison unit 131 determines whether the latest evaluation value D23 is smaller than the target value D03. If the latest evaluation value D23 is smaller than the target value D03, the evaluation represented by the latest evaluation value D23 is higher than the evaluation represented by the target value D03. If the latest evaluation value D23 is larger than the target value D03, the evaluation represented by the latest evaluation value D23 is lower than the evaluation represented by the target value D03.

[0083] The target value D03 is acquired from the setting file 191. The setting file 191 is stored in advance in the storage unit 190, for example.

[0084] If the latest evaluation value D23 is smaller than the target value D03, that is, if the evaluation represented by the latest evaluation value D23 is higher than the evaluation represented by the target value D03, the process proceeds to step S232. If the current evaluation value D23 is equal to or greater than the target value D03, that is, if the evaluation represented by the latest evaluation value D23 is lower than the evaluation represented by the target value D03, the process proceeds to step S241.

[0085] In step S232, the data model comparison unit 131 outputs the data model D1. The data model D1 is the data model D21 corresponding to the latest evaluation value D23. In other words, the data model D1 is the latest data model D21.

[0086] The data model D1 is output as follows: First, the data model comparison unit 131 selects the evaluation information D22 indicating the latest evaluation value D23, and acquires a data model identifier from the selected evaluation information D22. Next, the data model comparison unit 131 acquires the data model D21 identified by the acquired data model identifier from the storage unit 190. Then, the data model comparison unit 131 outputs the acquired data model D21 as the data model D1. The output data model D1 is input to the data conversion unit 211. After step S232, the processing ends.

[0087] The process when the latest evaluation value D23 is equal to or greater than the target value D03 in step S231 is the same as the process in the steps following step S131 in embodiment 1. That is, steps S241 to S243 are the same as steps S131 to S133 in embodiment 1. Also, step S250 is the same as step S140 in embodiment 1. Also, steps S261 to S263 are the same as steps S151 to S153 in embodiment 1.

[0088] ***Effects of Second Embodiment*** According to the second embodiment, it is possible to generate a data model that satisfies the target values ​​by using the target value setting file for evaluation.

[0089] The generation unit 130 compares the data model evaluation value, the target value in the target value setting file, and the minimum value of the data model evaluation results after system startup. Then, in any of the following cases, the generation unit 130 generates a new data model so that the data model evaluation value is reduced based on the column call count, similarity, and the evaluated data model. The generation unit 130 generates a new data model when the data model evaluation value exceeds the target value based on the target value setting file but falls below the minimum value of the data model evaluation results after system startup. The generation unit 130 generates a new data model when the data model evaluation value exceeds the target value in the target value setting file, the data model evaluation value exceeds the minimum value of the data model evaluation results after system startup, and the number of times the minimum value of the data model evaluation results has not been updated after system startup has not reached a threshold. This makes it possible to generate a data model with higher communication efficiency and that satisfies the target value.

[0090] The generation unit 130 compares the data model evaluation value, the target value in the target value setting file, and the minimum value of the data model evaluation result after system startup. Then, in any of the following cases, the generation unit 130 outputs a data model whose data model evaluation value is lower than the target value in the target value setting file, or a data model whose data model evaluation value is the minimum value of the data model evaluation result after system startup. The generation unit 130 outputs a data model when the data model evaluation value is lower than the target value in the target value setting file. The generation unit 130 outputs a data model when the data model evaluation value exceeds the target value in the target value setting file, the data model evaluation value exceeds the minimum value of the data model evaluation result after system startup, and the number of times the minimum value of the data model evaluation result has not been updated since system startup has reached a threshold. This makes it possible to output a data model that satisfies the target value, or a data model that does not satisfy the target value but has high communication efficiency.

[0091] *** Supplementary Information about the Embodiment *** The hardware configuration of the data model optimization system 100 will be described with reference to Fig. 18. The data model optimization system 100 includes a processing circuit 109. The processing circuit 109 is hardware that realizes a calculation unit 110, an evaluation unit 120, and a generation unit 130. The processing circuit 109 may be dedicated hardware, or may be a processor 101 that executes a program stored in a memory 102.

[0092] When the processing circuit 109 is dedicated hardware, the processing circuit 109 may be, for example, a single circuit, a multiple circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof. ASIC is an abbreviation for Application Specific Integrated Circuit. FPGA is an abbreviation for Field Programmable Gate Array.

[0093] The data model optimization system 100 may include multiple processing circuits replacing the processing circuit 109 .

[0094] In the processing circuit 109, some functions may be realized by dedicated hardware, and the remaining functions may be realized by software or firmware.

[0095] Thus, the functionality of the data model optimization system 100 may be implemented in hardware, software, firmware, or a combination thereof.

[0096] Each embodiment is an example of a preferred embodiment and is not intended to limit the technical scope of the present disclosure. Each embodiment may be implemented in part or in combination with other embodiments. Procedures described using flowcharts, etc. may be modified as appropriate.

[0097] The "part" of each element of the data model optimization system 100 may be read as a "process," a "step," a "circuit," or a "circuitry."

[0098] 100 Data model optimization system, 101 Processor, 102 Memory, 103 Auxiliary storage device, 104 Communication device, 105 Input / output interface, 109 Processing circuit, 110 Calculation unit, 111 Single call count calculation unit, 112 Pair call count calculation unit, 113 Similarity calculation unit, 120 Evaluation unit, 121 Data model evaluation unit, 130 Generation unit, 131 Data model comparison unit, 132 Data model generation unit, 190 Storage unit, 191 Configuration file, 200 Data processing system, 210 Data platform, 211 Data conversion unit, 212 Database, 221 Application unit, 230 Network storage, D1 Data model, D2 Data request, D3 Inquiry, D01 Database configuration information, D02 Data acquisition scenario, D03 Target value, D11 Calculation information, D20 First edition model, D21 Data model, D22 Evaluation information, D23 Evaluation value, D24 Reference value, D31 Data model.

Claims

1. A data model optimization system comprising: a single call count calculation unit that calculates, for each column in the multiple tables, the number of calls that the column will be called when data is acquired from the database in accordance with the data acquisition scenario, based on database configuration information indicating the configuration of a plurality of tables in a database and a data acquisition scenario that specifies data to be acquired from the database; a set call count calculation unit that calculates, for each column set in the multiple tables, the number of calls that the column set will be called at the same time when data is acquired from the database in accordance with the data acquisition scenario, based on the database configuration information and the data acquisition scenario; a similarity calculation unit that calculates, for each column set in the multiple tables, a similarity between column names, based on the database configuration information; and a data model generation unit that generates a data model for expressing data acquired in accordance with the data acquisition scenario in a structure suitable for processing by an application that uses the data, based on the number of calls for each column, the number of calls for each column set, and the similarity for each column set.

2. The data model optimization system according to claim 1, further comprising a data model evaluation unit which, each time a data model is generated, evaluates the generated data model based on the data acquisition scenario to calculate an evaluation value of the generated data model, and the data model generation unit generates the new data model so that the evaluation represented by the evaluation value of the new data model is high.

3. The data model optimization system according to claim 2, further comprising a data model comparison unit, wherein the data model comparison unit compares a latest evaluation value, which is an evaluation value of the latest data model among the generated data models, with a reference value, which is an evaluation value representing the highest evaluation among one or more evaluation values ​​for one or more data models generated before the latest data model, to determine whether the evaluation represented by the latest evaluation value is higher than the evaluation represented by the reference value, and the data model generation unit generates the new data model if the evaluation represented by the latest evaluation value is higher than the evaluation represented by the reference value.

4. The data model optimization system of claim 3, wherein the data model comparison unit updates the reference value to the latest evaluation value when the evaluation represented by the latest evaluation value is higher than the evaluation represented by the reference value, and does not update the reference value to the latest evaluation value when the evaluation represented by the latest evaluation value is lower than the evaluation represented by the reference value; and the data model generation unit generates the new data model when the evaluation represented by the latest evaluation value is lower than the evaluation represented by the reference value but a non-update count, which is the number of times the reference value has not been updated, does not reach a non-update threshold.

5. The data model optimization system according to claim 4, wherein the data model comparison unit outputs a data model corresponding to the reference value when the number of times that the data has not been updated reaches the non-update threshold value.

6. The data model optimization system according to claim 2, further comprising a data model comparison unit, which compares a latest evaluation value, which is an evaluation value of the latest data model among the generated data models, with a target value, determines whether the evaluation represented by the latest evaluation value is higher than the evaluation represented by the target value, and outputs the latest data model when the evaluation represented by the latest evaluation value is higher than the evaluation represented by the target value.

7. The data model optimization system according to claim 6, wherein the data model generation unit generates the new data model when the evaluation represented by the latest evaluation value is lower than the evaluation represented by the target value.

8. The data model optimization system of claim 7, wherein the data model comparison unit compares the latest evaluation value with a reference value which is an evaluation value representing the highest evaluation among one or more evaluation values ​​for one or more data models generated before the latest data model, and determines whether the evaluation represented by the latest evaluation value is higher than the evaluation represented by the reference value; and the data model generation unit generates the new data model when the evaluation represented by the latest evaluation value is lower than the evaluation represented by the target value and the evaluation represented by the latest evaluation value is higher than the evaluation represented by the reference value.

9. The data model optimization system of claim 8, wherein the data model comparison unit updates the reference value to the latest evaluation value when the evaluation represented by the latest evaluation value is higher than the evaluation represented by the reference value, and does not update the reference value to the latest evaluation value when the evaluation represented by the latest evaluation value is lower than the evaluation represented by the reference value; and the data model generation unit generates the new data model when the evaluation represented by the latest evaluation value is lower than the evaluation represented by the target value, and the evaluation represented by the latest evaluation value is lower than the evaluation represented by the reference value but the number of times the reference value has not been updated does not reach an unupdate threshold.

10. The data model optimization system according to claim 9, wherein the data model comparison unit outputs a data model corresponding to the reference value when the evaluation represented by the latest evaluation value is lower than the evaluation represented by the target value but the number of times that the data has not been updated reaches the non-update threshold value.

11. A data model optimization system as claimed in any one of claims 1 to 10, wherein the data model generation unit selects a column set that is called frequently based on the number of calls for each column set as a target column set, and generates a data model for expressing data of each column of the target column set only among the column sets in the plurality of tables in a structure suitable for processing of an application that uses the data.

12. A data model optimization system as claimed in any one of claims 1 to 11, wherein the data model generation unit selects a column set with high similarity from among combinations of columns with high call frequencies based on the number of calls for each column and the similarity for each column set as a target column set, and generates a data model for expressing data of each column of the target column set only among the column sets in the plurality of tables in a structure suitable for processing of an application that uses the data.

13. A data model optimization system as claimed in any one of claims 1 to 12, wherein the data model generation unit selects a column set with a low similarity from among combinations of columns with a high number of calls based on the number of calls for each column and the similarity for each column set as a target column set, and generates a data model for expressing data of each column of the target column set only among the column sets in the multiple tables in separate structures suitable for processing of an application that uses the data.

14. A data model optimization method comprising the steps of: calculating, for each column in the multiple tables, the number of calls that the column will be called when data is acquired from the database in accordance with the data acquisition scenario based on database configuration information indicating the configuration of multiple tables in a database and a data acquisition scenario specifying data to be acquired from the database; calculating, for each column group in the multiple tables, the number of calls that the column group will be called at the same time when data is acquired from the database in accordance with the data acquisition scenario based on the database configuration information and the data acquisition scenario; calculating, for each column group in the multiple tables, a similarity between column names based on the database configuration information; and generating a data model for expressing data acquired in accordance with the data acquisition scenario in a structure suitable for processing by an application that uses the data, based on the number of calls for each column, the number of calls for each column group, and the similarity for each column group.

15. A data model optimization program for causing a computer to execute the following steps: a single call count calculation process for calculating, for each column in the multiple tables, the number of calls that the column will be called when data is acquired from the database in accordance with the data acquisition scenario, based on database configuration information indicating the configuration of multiple tables in a database and a data acquisition scenario specifying data to be acquired from the database; a pair call count calculation process for calculating, for each column group in the multiple tables, the number of calls that the column group will be called at the same time when data is acquired from the database in accordance with the data acquisition scenario, based on the database configuration information and the data acquisition scenario; a similarity calculation process for calculating, for each column group in the multiple tables, the similarity between column names, based on the database configuration information; and a data model generation process for generating a data model for expressing data acquired in accordance with the data acquisition scenario in a structure suitable for processing by an application that uses the data, based on the number of calls for each column, the number of calls for each column group, and the similarity for each column group.

Images