Lineage Management System and Lineage Management Method

Abstract

To provide an information processing system which can manage a lineage more properly.SOLUTION: A lineage unit management system 3 determines a lineage unit on the basis of a content of processing of data processing of generating output data including at least one element from input data including at least one element. A lineage management system 4 generates lineage information showing a correspondence between the at least one element of the input data and the at least one element of the output data according to the lineage unit. Since the lineage information is generated according to the lineage unit according to the content of the data processing, a more proper lineage management becomes possible.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a lineage management system and a lineage management method.

Background Art

[0002] In recent years, machine learning models have attracted attention, and in particular, in fields such as medicine and nursing care, highly reliable machine learning models are required. In order to ensure the reliability of a machine learning model, it is necessary to build the machine learning model using appropriate learning data. Since learning data is generated by processing data acquired at a site or the like, in order to determine whether the learning data is appropriate, lineage management for managing lineage information that can track the data transition up to the learning data is required.

[0003] Patent Documents 1 and 2 disclose techniques for realizing lineage management. In the techniques described in Patent Documents 1 and 2, by analyzing a query that requests data processing, the correspondence relationship between the input data and the output data of the data processing corresponding to the query is specified, and lineage information is generated based on the correspondence relationship.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the technologies described in Patent Documents 1 and 2, since the correspondence relationship between the elements of the input data and the output data is specified in units of tables or columns, it is impossible to obtain detailed lineage information, and sufficient lineage management may not be possible. For example, in data processing, when input data having a vertical structure is converted into output data having a horizontal structure, the correspondence relationship between the columns of the input data and the columns of the output data is one-to-many. Therefore, it is difficult to track the elements of the input data from the elements of the output data using lineage information based on columns.

[0006] An object of the present disclosure is to provide a lineage management system and a lineage management method capable of more appropriate lineage management.

Means for Solving the Problems

[0007] A lineage management system according to an aspect of the present disclosure is a lineage management system that generates lineage information indicating the correspondence relationship between elements of input data including one or more elements and output data including one or more elements generated from the input data. A rule management unit that determines a lineage unit, which is a unit for defining the correspondence relationship, based on the processing content of data processing for generating the output data from the input data, and a lineage management unit that generates the lineage information according to the lineage unit.

Effects of the Invention

[0008] According to the present invention, more appropriate lineage management becomes possible.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Modes for Carrying Out the Invention

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0011] (First Embodiment) FIG. 1 is a diagram showing the configuration of an information processing system according to a first embodiment of the present disclosure. The information processing system shown in FIG. 1 includes a data management system 1, a data analysis system 2, a lineage unit management system 3, and a lineage management system 4. Further, the data management system 1, the data analysis system 2, the lineage unit management system 3, and the lineage management system 4 are communicably connected to each other via a network 5. Further, at least one of the data management system 1, the data analysis system 2, the lineage unit management system 3, and the lineage management system 4 may be communicably connected to a terminal (not shown) used by a user who uses the information processing system via the network 5.

[0012] FIG. 2 is a diagram showing an example of the hardware configuration of the data management system 1. As shown in FIG. 2, the data management system 1 includes a storage device 51, a CPU 52, an input device 53, an output device 54, and a network interface (NW I / F) 55, and they are connected to each other via a bus line 56.

[0013] The storage device 51 includes a main storage device (not shown) such as a memory and an auxiliary storage device (not shown) such as an HDD (Hard Disk Drive) and an SSD (Solid State Drive). The storage device 51 stores a program that defines the operation of the CPU 52 and various information used and generated by the CPU 52. The CPU 52 is a processor that reads a program stored in the storage device 51 and executes various processes by executing the read program.

[0014] The input device 53 is a device for inputting various information from the user, and the output device 54 is a device for outputting (for example, displaying) various information to the user. The network interface 55 is a device that is communicably connected to external devices such as the data management system 1, the data analysis system 2, the lineage management system 4, and the terminal via the network 5.

[0015] Note that since the hardware configurations of the data management system 1, the data analysis system 2, and the lineage management system 4 are the same as the hardware configuration of the lineage unit management system 3 shown in FIG. 2, the description thereof is omitted.

[0016] FIG. 3 is a diagram showing an example of the functional configuration of the data management system 1. The data management system 1 shown in FIG. 3 is a processing unit that executes data processing, and includes a database 11 and a database management unit 12.

[0017] The database 11 is a storage unit that stores data used and generated in data processing. The data is data including one or more elements, and in this embodiment, is table data having a table structure. In this case, each element of the data is stored in each cell of the table.

[0018] The database management unit 12 manages the data stored in the database 11. For example, the database management unit 12 performs data processing according to a query that is a data processing request from the user. Specifically, the database management unit 12 reads data from the database 11 according to the query, performs data processing on the input data that is the read data, and stores the output data that is the data generated by the data processing in the database 11. The query is described in SQL statements in this embodiment.

[0019] FIG. 4 is a diagram showing an example of the functional configuration of the data analysis system 2. The data analysis system 2 shown in FIG. 4 is an analysis unit that analyzes data processing, and includes a data processing acquisition unit 21, a data processing analysis unit 22, and a data processing storage unit 23.

[0020] The data processing acquisition unit 21 acquires the execution logs and queries of the data processing performed by the database management unit 12 of the data management system 1.

[0021] The data processing analysis unit 22 analyzes the execution log, which is the log information of the data processing acquired by the data processing acquisition unit 21, and generates data processing information indicating the content of the data processing.

[0022] The data processing storage unit 23 stores the data processing information generated by the data processing analysis unit 22.

[0023] FIG. 5 is a diagram showing an example of the functional configuration of the lineage unit management system 3. The lineage unit management system 3 shown in FIG. 5 is a rule management unit that determines a lineage unit, which is a lineage rule defining the correspondence relationship between the elements of the input data and the elements of the output data of the data processing, and includes a lineage unit determination condition storage unit 31, a threshold value storage unit 32, a lineage unit management unit 33, a lineage unit estimated value calculation unit 34, and a lineage unit determination unit 35.

[0024] The lineage unit determination condition storage unit 31 stores a lineage unit determination condition table indicating the lineage unit determination conditions, which are the determination conditions for determining the lineage unit. In this embodiment, there are a plurality of lineage unit determination conditions. The threshold value storage unit 32 stores a lineage unit determination table, which is a threshold value table indicating the determination threshold values, which are the threshold values for determining the lineage unit. There may be a plurality of determination threshold values.

[0025] The lineage unit management unit 33 sets the lineage unit determination condition table and the lineage unit determination table in the lineage unit determination condition storage unit 31 and the threshold value storage unit 32 based on an instruction from the user.

[0026] The lineage unit estimation value calculation unit 34 calculates a lineage unit estimation value, which is an estimated value for determining the lineage unit of target data (input data and output data) in data processing, based on the data processing information stored in the data processing storage unit 23 of the data analysis system 2 and the lineage unit determination condition table stored in the lineage unit determination condition storage unit 31. The lineage unit estimation value is, for example, a value corresponding to the correspondence relationship between the elements of the input data and the output data of the data processing. Specifically, the lineage unit estimation value calculation unit 34 determines whether the target data meets the lineage unit determination conditions shown in the lineage unit determination condition table based on the data processing information, and calculates the lineage unit estimation value based on the determination result.

[0027] The lineage unit determination unit 35 compares the lineage unit estimation value calculated by the lineage unit estimation value calculation unit 34 with the determination threshold value shown in the lineage unit determination table stored in the threshold value storage unit 32, and determines the lineage unit of the target data based on the comparison result.

[0028] FIG. 6 is a diagram showing an example of the functional configuration of the lineage management system 4. The lineage management system 4 shown in FIG. 6 is a lineage management unit that generates lineage information indicating the correspondence relationship between the elements of the target data, and includes a lineage management unit 41, a lineage recording unit 42, a lineage display unit 43, a column unit lineage storage unit 44, a conditional expression unit lineage storage unit 45, and a cell unit lineage storage unit 46.

[0029] The lineage management unit 41 generates the lineage information of the target data based on the lineage unit determined by the lineage unit determination unit 35.

[0030] The lineage recording unit 42 records the lineage information generated by the lineage management unit 41 in a storage unit corresponding to each lineage unit. In this embodiment, the lineage units include a "column unit", which is a rule that defines the correspondence of each element of the target data in column units; a "conditional expression unit", which is a rule that defines the correspondence of each element of the target data in units of conditional expressions related to cells; and a "cell unit", which is a rule that defines the correspondence of each element of the target data in cell units. The lineage recording unit 42 stores the lineage information in column units in the column unit lineage storage unit 44, stores the lineage information in conditional expression units in the conditional expression unit lineage storage unit 45, and stores the lineage information in cell units in the cell unit lineage storage unit 46.

[0031] The lineage display unit 43 displays various information. For example, the lineage display unit 43 displays the lineage information stored in the column unit lineage storage unit 44, the conditional expression unit lineage storage unit 45, and the cell unit lineage storage unit 46. The display destination of the information is not particularly limited, and it may be an output device such as the lineage management system 4, or a display screen of a terminal used by the user.

[0032] Each functional unit shown in FIGS. 3 to 6 is realized, for example, by the CPU 52 shown in FIG. 2 reading a program stored in the storage device 51 and executing the read program.

[0033] FIGS. 7 and 8 are diagrams showing an example of data recorded in the database 11 of the data management system 1. In FIGS. 7 and 8, as data, data related to health checkups (hereinafter abbreviated as health checkups), particularly data related to BMI (Body Mass Index) values, are exemplified, but the type of data is not particularly limited.

[0034] In the examples of FIGS. 7 and 8, the database 11 has, as data, a basic disease-by-patient number table 100, a first health check table 110, and a second health check table 120 shown in FIG. 7, and a basic disease cumulative table 200, a health check date table 210, and a BMI value abnormality table 220 shown in FIG. 8.

[0035] The number-of-patients-by-underlying-disease table 100 has a column 101 for storing the district number that identifies the district where the health check was conducted, a column 102 for storing the health check date and time which is the date and time when the health check was conducted, a column 103 for storing the number of patients determined to have hypertension, which is the number of hypertension patients, and a column 104 for storing the number of patients determined to have diabetes, which is the number of diabetes patients.

[0036] The first health check table 110 has a column 111 for storing the district number, a column 112 for storing the health check date and time, and a column 113 for storing the number of patients with a BMI value of 30 or more, which is the number of patients with a BMI value of 30 or more.

[0037] The second health check table 120 has a column 121 for storing the district number, a column 122 for storing the health check date and time, and a column 123 for storing the number of patients with an abnormal BMI value, which is the number of patients with an abnormal BMI value.

[0038] The underlying-disease cumulative table 200 has a column 201 for storing the district number, a column 202 for storing the health check date and time, and a column 203 for storing the number of patients with underlying diseases, which is the number of patients with underlying diseases.

[0039] The health check date table 210 has a column 211 for storing the district number, a column 212 for storing the health check date and time, and a column 212 for storing the number of patients with a BMI value of 30 or more.

[0040] The abnormal-BMI-value table 220 has a column 221 for storing the health check date and time, a column 222 for storing the number of patients with an abnormal BMI value in district 3 (the district with district number "3"), and a column 223 for storing the number of patients with an abnormal BMI value in district 4 (the district with district number "4").

[0041] FIG. 9 is a diagram showing an example of an execution log of data processing. The execution log 300 shown in FIG. 9 has columns 301 to 305. Column 301 stores an execution ID that identifies the executed data processing. Column 302 stores an input table name that identifies an input table which is input data used in the data processing. Column 303 stores an output table name that identifies an output table which is output data generated in the data processing. Column 304 stores execution SQL information indicating a query that requested the executed data processing. Column 305 stores an execution time which is the date and time when the data processing was executed.

[0042] FIG. 10 is a diagram showing an example of a lineage unit determination condition table. The lineage unit determination condition table 400 shown in FIG. 10 has columns 401 to 404.

[0043] Column 401 stores a condition ID that identifies the lineage unit determination condition. Column 402 stores a determination criterion which is the lineage unit determination condition. Column 403 stores status information indicating whether the determination criterion is used for the determination of the lineage unit. Column 404 stores a weight value which is a numerical value assigned to the determination criterion.

[0044] In this embodiment, the determination criteria include "the output data is data extracted from the input data according to specific conditions", "the number of input / output records (the number of records of the input data and the output data) does not match", "the output data cannot be expressed by an aggregate function of the input data (including a combination of multiple aggregate functions)", "the elements of the input data correspond to different output destination columns according to conditions", and "a lineage unit is set for the input data". The aggregate function is a function provided in SQL (such as SUM and MAX). Note that the output data of a certain data processing may be the input data of another data processing. In that case, a lineage unit is already set for the input data of the other data processing.

[0045] When the status information is used for determination on a lineage-by-lineage basis, it indicates "Active", and when it is not used for determination on a lineage-by-lineage basis, it indicates "Non-Active". The weights are all the same value in the example of Figure 10, but they may also be different values.

[0046] Figure 11 is a diagram showing an example of a lineage unit determination table. The lineage unit determination table shown in Figure 11 has columns 501 to 503.

[0047] Column 501 stores the threshold ID that identifies the determination threshold. Column 502 stores the determination threshold. Column 502 stores the lineage unit corresponding to the determination threshold.

[0048] Figures 12 to 14 are diagrams showing an example of lineage information.

[0049] Figure 12 is a diagram showing an example of a column unit lineage table that is lineage information in column units. The column unit lineage table 600 shown in Figure 12 has columns 601 to 608.

[0050] Column 601 stores the lineage ID that identifies the lineage information. Column 602 stores the lineage unit. In Figures 12 to 14, the column unit is set to "1", the conditional expression unit is set to "2", and the cell unit is set to "3" as the lineage unit. Column 603 stores the input table name that identifies the input data. Column 604 stores the input column name that identifies the column in the input data that has a correspondence with the output data. Column 605 stores the processing content of the data processing. Column 606 stores the output table name that identifies the output data. Column 607 stores the output column name that identifies the output column in the output data that has a correspondence with the column of the input column name. Column 608 stores the registration time, which is the date and time when the lineage information was registered.

[0051] FIG. 13 is a diagram showing an example of a conditional expression unit lineage table which is lineage information in units of conditional expressions. The conditional expression unit lineage table 700 shown in FIG. 13 has columns 701 to 709.

[0052] Column 701 stores a lineage ID for identifying lineage information. Column 702 stores the lineage unit. Column 703 stores the input table name. Column 704 stores the input column name. Column 705 stores the conditional expression. Column 706 stores the processing content for data processing. Column 707 stores the output table name. Column 708 stores the output column name for identifying the output column. Column 709 stores the registration time.

[0053] The conditional expression stored in column 705 is a condition regarding the cells included in the column of the input column name. For example, in the example of FIG. 13, it is a condition for associating the cells whose health check date and time value is "2021 / 07 / 01".

[0054] FIG. 14 is a diagram showing an example of a cell unit lineage table which is lineage information in units of cells. The cell unit lineage table 800 shown in FIG. 14 has columns 801 to 812.

[0055] Column 801 stores an ID for identifying the lineage. Column 802 stores the lineage unit. Column 803 stores the input table name. Column 804 stores the input column name. Column 805 stores an input identification key for identifying the cells in the input data that have a correspondence with the cells of the output data, and column 806 stores the input identification value which is the value of the input identification key.

[0056] Column 807 stores the processing content of the data processing. Column 808 stores the output table name. Column 809 stores the output column name. Column 810 stores an output identification key for identifying the cells in the output data that have a correspondence with the cells of the input data, and column 811 stores the output identification value which is the value of the output identification key. Column 812 stores the registration time of this entry.

[0057] FIG. 15 is a flowchart for explaining an example of the operation of the information system according to the present embodiment.

[0058] First, the lineage management system 4 sets the lineage unit determination condition and the determination threshold value in the lineage unit determination condition storage unit 31 and the threshold value storage unit 32 of the lineage unit management system 3, respectively (step S101).

[0059] After that, when the database management unit 12 of the data management system 1 receives a query from a user terminal or the like, it reads data from the database 11 according to the query, executes data processing on the input data which is the read data, and stores the output data which is the data generated by the data processing in the database 11. At this time, the database management unit 12 generates an execution log of the data processing and stores it in the database 11 (step S102).

[0060] The data processing acquisition unit 21 of the data analysis system 2 detects the execution of the data processing by the data management system 1 and acquires the execution log corresponding to the data processing (step S103).

[0061] The data processing analysis unit 22 analyzes the execution log acquired by the data processing acquisition unit 21, generates data processing information indicating the content of the data processing, and stores it in the data processing storage unit 23 (step S104).

[0062] After that, the lineage unit estimated value calculation unit 34 of the lineage unit management system 3 executes an estimated value calculation process (see FIG. 16) for calculating a lineage unit estimated value based on the data processing information stored in the data processing storage unit 23 and the lineage unit determination condition table stored in the lineage unit determination condition storage unit 31 (step S105).

[0063] The lineage unit determination unit 35 determines the lineage unit of the target data based on the lineage unit estimated value calculated by the lineage unit estimation value calculation unit 34 and the lineage unit determination table stored in the threshold value storage unit 32 (step S106). Specifically, the lineage unit determination unit 35 compares the lineage unit estimated value with the determination threshold value in the lineage unit determination table, and determines the lineage unit of the target data based on the comparison result.

[0064] Then, the lineage management unit 41 of the lineage management system 4 generates lineage information of the target data based on the lineage unit determined by the lineage unit determination unit 35 (step S107).

[0065] The lineage recording unit 42 stores the lineage information generated by the lineage management unit 41 in any one of the column unit lineage storage unit 44, the conditional expression unit lineage storage unit 45, and the cell unit lineage storage unit 46 according to the lineage unit (step S108).

[0066] Thereafter, the lineage display unit 43 displays various information. For example, the lineage display unit 43 displays the lineage information stored in the column unit lineage storage unit 44, the conditional expression unit lineage storage unit 45, and the cell unit lineage storage unit 46 (step S109), and ends the process. Note that the lineage display unit 43 may process and display the lineage information.

[0067] FIG. 16 is a flowchart for explaining an example of the lineage unit estimated value calculation process in step S105 of FIG. 15.

[0068] In the lineage unit estimated value calculation process, first, the lineage unit estimated value calculation unit 34 determines whether the target data conforms to the determination criterion 1 "the output data is data extracted from the input data according to specific conditions", which is the determination criterion with the ID of "1" in FIG. 10 (step S201).

[0069] When it meets the determination criterion 1, the lineage unit estimated value calculation unit 34 sets the determination value "A" corresponding to the determination criterion 1 to 1 (step S202). On the other hand, when it does not meet the determination criterion 1, the lineage unit estimated value calculation unit 34 sets the determination value "A" to 0 (step S203).

[0070] Subsequently, the lineage unit estimated value calculation unit 34 determines whether the target data meets the determination criterion 2 "the number of output records does not match" which is the determination criterion with the ID of "2" in FIG. 10 (step S204).

[0071] When it meets the determination criterion 2, the lineage unit estimated value calculation unit 34 sets the determination value "B" corresponding to the determination criterion 2 to 1 (step S205). On the other hand, when it does not meet the determination criterion 2, the lineage unit estimated value calculation unit 34 sets the determination value "B" to 0 (step S206).

[0072] Subsequently, the lineage unit estimated value calculation unit 34 determines whether the target data meets the determination criterion 3 "the output data cannot be expressed as an aggregate function of the input data" which is the determination criterion with the ID of "3" in FIG. 10 (step S207).

[0073] When it meets the determination criterion 3, the lineage unit estimated value calculation unit 34 sets the determination value "C" corresponding to the determination criterion 3 to 1 (step S208). On the other hand, when it does not meet the determination criterion 3, the lineage unit estimated value calculation unit 34 sets the determination value "C" to 0 (step S209).

[0074] Subsequently, the lineage unit estimated value calculation unit 34 determines whether the target data meets the determination criterion 4 "the elements of the input data correspond to different output destination columns according to the conditions" which is the determination criterion with the ID of "4" in FIG. 10 (step S210).

[0075] If it meets the determination criterion 4, the lineage unit estimated value calculation unit 34 sets the determination value "D" corresponding to the determination criterion 4 to 1 (step S211). On the other hand, if it does not meet the determination criterion 4, the lineage unit estimated value calculation unit 34 sets the determination value "D" to 0 (step S212).

[0076] Subsequently, the lineage unit estimated value calculation unit 34 determines whether the target data meets the determination criterion 5 "the lineage unit is set in the input data", which is the determination criterion with the ID of "5" in FIG. 10 (step S213).

[0077] If it meets the determination criterion 5, the lineage unit estimated value calculation unit 34 sets the determination value "E" corresponding to the determination criterion 5 to 1 (step S214). On the other hand, if it does not meet the determination criterion 5, the lineage unit estimated value calculation unit 34 sets the determination value "E" corresponding to the determination criterion 5 to 0 (step S215).

[0078] After that, the lineage unit estimated value calculation unit 34 calculates the weighted sum of the determination values A to E of each of the determination criteria 1 to 5 using the weight values of each of the determination criteria 1 to 5 shown in FIG. 10 (step S216). Let the weight values of each of the determination criteria 1 to 5 be x 1 ~x 5 Then, the weighted sum Y is Y = Ax 1 + bx 2 + Cx 3+ Dx 4 + Ex 5 and so on.

[0079] The lineage unit estimated value calculation unit 34 calculates the weighted sum Y as the lineage unit estimated value (step S217) and ends the lineage unit estimated value calculation process.

[0080] For example, in the case of data processing that adds the value in column 103 and the value in column 104 of the patient number table 100 by underlying disease shown in FIG. 7 to generate the cumulative underlying disease table 200 shown in FIG. 8, the target data (the patient number table 100 by underlying disease and the cumulative underlying disease table 200) only corresponds to criterion 3. Therefore, the determination value C is 1, the other determination values are 0, and the lineage unit estimated value becomes 1. In this case, when using the lineage unit determination table 500, the lineage unit becomes the column unit.

[0081] Also, in the case of data processing that extracts the value in column 112 of the first health check table 110 shown in FIG. 7 where the value is "2021-07-01" to generate the health check date table 210 shown in FIG. 8, the target data (the first health check table 110 and the health check date table 210) only corresponds to criteria 1 and 3. Therefore, the determination values A and C are 1, the other determination values are 0, and the lineage unit estimated value becomes 2. In this case, when using the lineage unit determination table 500, the lineage unit becomes the conditional expression unit.

[0082] Also, in the case of data processing that calculates the total of the number of patients with a BMI value of 30 or more and the number of patients with an abnormal BMI value in regions 3 and 4 in the first health check table 110 and the second health check table 120 shown in FIG. 7 to generate the abnormal BMI value table 220 shown in FIG. 8, the target data (the first health check table 110, the second health check table 120, and the abnormal BMI value table 220) corresponds to criteria 1 to 4. Therefore, the determination values A to D are 1, the determination value E is 0, and the lineage unit estimated value becomes 4. In this case, when using the lineage unit determination table 500, the lineage unit becomes the cell unit.

[0083] Note that it is assumed that no lineage unit is set in the patient number table 100 by underlying disease, the first health check table 110, and the second health check table 120 shown in FIG. 7.

[0084] FIGS. 17 to 20 are diagrams showing an example of the display screen displayed by the lineage display unit 43.

[0085] FIG. 17 is a diagram showing an example of a main screen. The main screen 1000 shown in FIG. 17 is a screen displayed by processes such as steps S101 and S109 in FIG. 15, and includes a setting button 1001 and a display button 1002. The setting button 1001 is a button for setting the lineage unit determination condition and the determination threshold. The display button 1002 is a button for displaying lineage information.

[0086] FIG. 18 is a diagram showing an example of a lineage unit determination condition setting screen. The lineage unit determination condition setting screen 1100 shown in FIG. 18 is a screen for setting the lineage unit determination condition and the determination threshold, and is displayed, for example, when the setting button 1001 in FIG. 17 is pressed.

[0087] The lineage unit determination condition setting screen 1100 has a lineage unit determination condition table 1101, an add button 1102, a modify button 1103, a delete button 1104, a lineage unit determination table 1105, a modify button 1106, and a return button 1107.

[0088] The lineage unit determination condition table 1101 shows the content of the currently set lineage unit determination condition table. The add button 1102 is a button for adding a determination criterion to the lineage unit determination condition table. The modify button 1103 is a button for modifying the content of the lineage unit determination condition table. The delete button 1104 is a button for deleting a determination criterion from the lineage unit determination condition table.

[0089] The lineage unit determination table 1105 shows the content of the currently set lineage unit determination table. The modify button 1106 is a button for modifying the content of the lineage unit determination table.

[0090] The return button 1108 is a button for ending the setting of the lineage unit determination condition and the determination threshold and returning to the main screen 1000.

[0091] FIG. 19 is a diagram showing an example of a lineage display content input screen. The lineage display content input screen 1200 shown in FIG. 19 is a screen for setting the content of the lineage information to be displayed, and is displayed, for example, when the display button 1002 in FIG. 17 is pressed.

[0092] The lineage display content input screen 1200 has an item input field 1201, a target unit input field 1203, a target data name input field 1204, a display lineage unit input field 1205, an execute button 1206, and a return button 1207.

[0093] The item input field 1201 is a field for inputting the items of the lineage information to be displayed. The target unit input field 1203 is a field for inputting the unit of the lineage information to be displayed. The target data name input field 1204 is a field for inputting the name of the data (output data) of the lineage information to be displayed. The display lineage unit input field 1205 is a field for inputting the lineage unit of the data of the lineage information to be displayed.

[0094] The execute button 1206 is a button for finalizing the content input in the input fields 1201 to 1205 and displaying the lineage information. The return button 1207 is a button for canceling the display of the lineage information and returning to the main screen 1000.

[0095] FIG. 20 is a diagram showing an example of a data lineage display screen. The data lineage display screen 1300 shown in FIG. 20 has input data 1301, output data 1302, and link information 1303.

[0096] The input data 1301 and the output data 1302 are data having a correspondence relationship with each other. The link information 1303 is information indicating the correspondence relationship between the input data 1301 and the output data 1302, and in the example of FIG. 20, indicates the relationship between the cells having a correspondence relationship with each other in the input data 1301 and the output data 1302.

[0097] As described above, according to this embodiment, the lineage unit management system 3 determines a lineage unit based on the content of data processing for generating output data including one or more elements from input data including one or more elements. The lineage management system 4 generates lineage information indicating the correspondence between the elements of the input data and the elements of the output data according to the lineage unit. Therefore, since the lineage information is generated according to the lineage unit corresponding to the content of the data processing, more appropriate lineage management becomes possible.

[0098] Also, in this embodiment, the lineage unit is determined based on the lineage unit estimated value and the lineage unit determination table. Specifically, the lineage unit estimated value is calculated based on the determination result of whether the target data including the input data and the output data meets the lineage unit determination condition. Therefore, since the lineage unit is determined based on an appropriate determination condition according to the data processing, more appropriate lineage management becomes possible.

[0099] Also, in this embodiment, since there are a plurality of lineage unit determination conditions, the lineage unit can be determined more appropriately.

[0100] Also, in this embodiment, the lineage unit is determined according to the lineage unit estimated value, which is the sum of the weight values assigned to each of the lineage unit determination conditions that the target data meets. Therefore, since it is possible to determine the lineage unit in consideration of the importance of the lineage unit determination conditions, etc., the lineage unit can be determined more appropriately.

[0101] Also, in this embodiment, the lineage unit includes a column unit, a cell unit, and a conditional expression unit. Therefore, it becomes possible to determine a lineage unit suitable for table data.

[0102] (Second Embodiment) Next, the second embodiment will be described.

[0103] In this embodiment, compared with the first embodiment, the calculation process of the lineage unit estimated value in step S105 of FIG. 15 is different.

[0104] FIG. 21 is a flowchart for explaining an example of the calculation process of the lineage unit estimated value in this embodiment.

[0105] In the calculation process of the lineage unit estimated value in this embodiment, first, the lineage unit estimated value calculation unit 34 acquires the lineage unit determination table from the threshold storage unit 32 (step S301), and acquires the lineage unit determination condition table from the lineage unit determination condition storage unit 31 (step S302).

[0106] Based on the data processing information stored in the data processing storage unit 23 of the data analysis system 2, the lineage unit estimated value calculation unit 34 determines whether the target data in the data processing corresponds to any of the determination criteria (lineage unit determination conditions) shown in the lineage unit determination condition table (step S303). This determination can be made, for example, by performing the processing from steps S201 to S215 in FIG. 16.

[0107] If it corresponds to any of the determination criteria, the lineage unit estimated value calculation unit 34 calculates the sum of the respective weight values of the corresponding determination criteria as the lineage unit estimated value based on the lineage unit determination condition table (step S304). Then, the lineage unit determination unit 35 compares the lineage unit estimated value with the determination threshold value in the lineage unit determination table, determines the lineage unit of the target data based on the comparison result (step S305), and ends the processing.

[0108] On the other hand, if it does not correspond to any of the determination criteria, the lineage unit determination unit 35 determines the lineage unit of the target data based on the lineage unit determination table (step S306) and ends the processing.

[0109] As described above, according to this embodiment, even when the target data does not correspond to any of the determination criteria, it is possible to determine an appropriate lineage rule.

[0110] The above-described embodiments of the present disclosure are examples for explaining the present disclosure, and are not intended to limit the scope of the present disclosure only to those embodiments. Those skilled in the art can implement the present disclosure in various other modes without departing from the scope of the present disclosure.

Explanation of Signs

[0111] 1: Data management system 2: Data analysis system 3: Lineage unit management system 4: Lineage management system 11: Database 12: Database management unit 21: Data processing acquisition unit 22: Data processing analysis unit 23: Data processing storage unit 31: Lineage unit determination condition storage unit 32: Threshold value storage unit 33: Lineage unit management unit 34: Lineage unit estimated value calculation unit 35: Lineage unit determination unit 41: Lineage management unit 42: Lineage recording unit 43: Lineage display unit 44: Column unit lineage storage unit 45: Conditional expression unit lineage storage unit 46: Cell unit lineage storage unit

Claims

1. In a lineage management system that generates lineage information indicating the correspondence between elements of input data including one or more elements and output data including one or more elements generated from the input data, a rule management unit that determines a lineage unit, which is a unit for defining the correspondence, based on the content of data processing for generating the output data from the input data; a lineage management system having a lineage management unit that generates the lineage information according to the lineage unit.

2. The rule management unit calculates an estimated value of the lineage unit corresponding to the correspondence, and determines the lineage unit based on the estimated value of the lineage unit and a threshold table indicating the relationship between the lineage unit and a threshold value. The lineage management system according to claim 1.

3. The rule management unit determines whether target data including the input data and the output data meets a determination condition regarding the correspondence, and calculates the estimated value of the lineage unit based on the determination result. The lineage management system according to claim 2.

4. For each of a plurality of the determination conditions, the rule management unit determines whether the target data meets the determination condition, and calculates the estimated value of the lineage unit based on the determination condition that the target data meets. The lineage management system according to claim 3.

5. The rule management unit calculates, as an estimated value of the lineage unit, the sum of numerical values pre-assigned to each of the determination conditions that the target data meets. The lineage management system according to claim 4.

6. The input data and the output data are table data having a table structure, and the elements are stored in each cell of the table data. The lineage management system according to claim 1.

7. The lineage unit is either a column unit of the table data or a cell unit of the table data. The lineage management system according to claim 6.

8. The lineage unit is either a column unit of the table data, a cell unit of the table data, or a conditional expression unit related to the cell of the table data. The lineage management system according to claim 6.

9. A lineage management method by a lineage management system that has a processor and generates lineage information indicating a correspondence relationship between each element of input data including one or more elements and output data including one or more elements generated from the input data, wherein the processor determines a lineage unit that is a unit defining the correspondence relationship based on the content of data processing for generating the output data from the input data, and the processor generates the lineage information according to the lineage unit. A lineage management method.

Images