Data table maintenance method, device, storage medium and electronic device

Abstract

The present application provides a data table maintenance method, device, storage medium and electronic device, the method comprising: obtaining a first attribute value and a second attribute value of a data table in a directed acyclic graph; wherein the first attribute value is determined according to the computational cost and total storage cost of the data table; the second attribute value is determined according to the user query value and system query value of the data table; calculating the ratio of the second attribute value to the first attribute value; comparing the calculated ratio with a preset ratio; in response to the ratio being less than the preset ratio, outputting a notification carrying the ratio and setting a timer for the data table; in response to receiving an instruction not to delete the data table during the timer timing process, deleting the timer set for the data table; in response to the timer timing expiring and not receiving an instruction not to delete the data table, deleting the data table. The method can accurately perform data table maintenance at a low cost.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to a data table maintenance method, device, storage medium, and electronic device. Background Art

[0002] In the data ETL process, as demands increase and data scale grows, computing and storage costs become increasingly higher.

[0003] Some data tables have not been used for a long time or are rarely used. If these data tables are still stored and calculated, a lot of computing and storage costs will be consumed.

[0004] In related technologies, data tables are generally optimized through manual analysis and sorting, thereby saving the calculation and storage costs of the data tables; however, manual maintenance of data tables is relatively costly and prone to errors. Summary of the Invention

[0005] In view of this, the present application provides a data table maintenance method, device, storage medium and electronic device, which can accurately perform data table maintenance at a low cost.

[0006] To solve the above technical problems, the technical solution of this application is implemented as follows:

[0007] In one embodiment, a data table maintenance method is provided, the method comprising:

[0008] A data table maintenance method, characterized in that the method comprises:

[0009] Obtaining a first attribute value and a second attribute value of a data table in a directed acyclic graph; wherein the first attribute value is determined based on a computational cost and a total storage cost of the data table; and the second attribute value is determined based on a user query value and a system query value of the data table;

[0010] calculating a ratio of the second attribute value to the first attribute value;

[0011] comparing the calculated ratio with a preset ratio;

[0012] In response to the ratio being less than the preset ratio, outputting a notification carrying the ratio and setting a timer for the data table;

[0013] In response to receiving an instruction not to delete the data table during the timer timing process, deleting the timer set for the data table;

[0014] In response to the timer timing out and no instruction not to delete the data table is received, the data table is deleted.

[0015] The step of obtaining the first attribute value of the data table in the directed acyclic graph includes:

[0016] Obtaining a total storage cost of the data table; wherein the total storage cost is determined based on the data table's own storage cost and the total storage cost of upstream data tables;

[0017] Obtaining a computational cost for the data table; wherein the computational cost is determined based on the computational resources consumed for computing the data table; the computational resources are obtained by weighting the accumulated values ​​of the CPU and memory time consumed for computing the data table;

[0018] A first attribute value of the data table is obtained by performing a weighted summation of the total storage cost and the computation cost of the data table.

[0019] Wherein, the method further comprises:

[0020] When obtaining the total storage cost of the data table, if there are N downstream data tables of the upstream data table of the data table, the total storage cost of the data table is calculated using 1 / N of the total storage cost of the upstream data table; if there is one downstream data table of the upstream data table of the data table, the storage cost of the data table is calculated using the total storage cost of the upstream data table; wherein N is an integer greater than 1.

[0021] Wherein, obtaining the second attribute value of the data table in the directed acyclic graph includes:

[0022] Obtaining the number of times a user queries the data table within different preset time periods, and performing a weighted summation of the number of times within different preset time periods to obtain a user query value of the data table;

[0023] Obtaining the systems and businesses that have historically queried the data table, and determining the system query value of the data table based on the weights corresponding to the systems and businesses that have queried the data table;

[0024] The second attribute value of the data table is obtained by weighted summing the user query value and the system query value.

[0025] Wherein, the method further comprises:

[0026] Obtaining a second attribute value of a downstream data table of the data table and a second attribute value of the data table;

[0027] Obtaining and recording a third attribute value of the data table; wherein the third attribute value is determined based on a second attribute value of a downstream data table of the data table and the second attribute value of the data table;

[0028] In response to insufficient computing resources, the computing resources of the data table having a high third attribute value are preferentially used.

[0029] Wherein, the method further comprises:

[0030] Obtaining the generation time of the data table in the current update cycle and the earliest query time of the data table in the current update cycle;

[0031] Calculating the difference between the query time and the generation time;

[0032] Comparing the calculated difference with the preset difference;

[0033] In response to the calculated difference being greater than a preset difference, in the next update cycle, the time for updating the data table is delayed, and the delayed time for updating the data table is no later than the query time in the current cycle.

[0034] Wherein, the method further comprises:

[0035] In response to the generation of the data table, recording a generation time of the data table;

[0036] In response to the data table being queried, the query time and the subject of the query are recorded, where the subject is a user or a system.

[0037] In another embodiment, a data table maintenance device is provided, the device comprising:

[0038] an acquiring unit configured to acquire a first attribute value and a second attribute value of a data table in a directed acyclic graph; wherein the first attribute value is determined based on a computation cost and a storage cost of the data table; and the second attribute value is determined based on a user query value and a system query value of the data table;

[0039] a calculating unit, configured to calculate a ratio of the second attribute value to the first attribute value;

[0040] a comparing unit configured to compare the calculated ratio with a preset ratio;

[0041] a timing unit configured to set a timer for the data table in response to the ratio being less than the preset ratio;

[0042] an output unit configured to output a notification carrying the ratio in response to the ratio being less than the preset ratio;

[0043] The deleting unit is configured to execute, in response to receiving an instruction not to delete the data table during the timer timing process, deleting the timer set for the data table; in response to the timer timing expiring and no instruction not to delete the data table is received, deleting the data table.

[0044] In another embodiment, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the steps of the data table maintenance method are implemented when the processor executes the program.

[0045] In another embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the steps of the data table maintenance method are implemented.

[0046] As can be seen from the above technical solution, in the above embodiment, by obtaining the first attribute value and the second attribute value of each data table, when it is determined that the ratio of the second attribute value to the first attribute value of the data table is less than a preset ratio, a notification carrying the ratio is output, and a timer is set for the data table; if during the timer timing process, an instruction is received not to delete the data table, the timer is deleted; if the timer times out and no instruction is received not to delete the data table, the data table is deleted; wherein, the first attribute value is determined based on the calculation cost and total storage cost of the data table; the second attribute value is determined based on the user query value and system query value of the data table. In this way, determining whether to delete a data table with a relatively small cost ratio (the ratio of the second attribute value to the first attribute value) can greatly save storage costs and calculation costs. This solution can accurately maintain data tables at a low cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0048] Figure 1 This is a schematic diagram of a data table maintenance process in an embodiment of the present application;

[0049] Figure 2 This is another data table maintenance process diagram in an embodiment of the present application;

[0050] Figure 3 A diagram of a directed acyclic graph;

[0051] Figure 4 This is a schematic diagram of the resource allocation process in the embodiment of this application;

[0052] Figure 5 A schematic diagram of the time flow for adjusting and updating the data table according to an embodiment of the present application;

[0053] Figure 6This is a structural diagram of a data table maintenance device in an embodiment of the present application;

[0054] Figure 7 A schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0056] The terms "first", "second", "third", "fourth", etc. (if any) in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe the order or precedence of the objects. It should be understood that the numbers used in this way are interchangeable where appropriate, so that the embodiments of the invention described herein can, for example, be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or that are inherent to these processes, methods, products or apparatus.

[0057] The following specific embodiments are used to describe the technical solution of the present invention in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0058] In the ETL process, data tables are stored by generating a directed acyclic graph based on their lineage relationships. As the number of tables stored in a database increases, computing and storage costs increase. Related technologies rely on manual maintenance of database tables, which is costly and error-prone.

[0059] Based on this, a data table maintenance method is provided in an embodiment of the present application, which maintains the data table based on the blood relationship of the data and the directed acyclic graph. Specifically, by obtaining the first attribute value and the second attribute value of each data table, when it is determined that the ratio of the second attribute value to the first attribute value of the data table is less than the preset ratio, a notification carrying the ratio is output, and a timer is set for the data table; if during the timer timing process, an instruction not to delete the data table is received, the timer is deleted; if the timer is timed out and no instruction not to delete the data table is received, the data table is deleted; wherein, the first attribute value is determined according to the calculation cost and the total storage cost of the data table; the second attribute value is determined according to the user query value and the system query value of the data table. In this way, determining whether to delete a data table with a relatively small cost ratio (the ratio of the second attribute value to the first attribute value) can greatly save storage costs and calculation costs. This solution can accurately maintain data tables at a low cost.

[0060] In an embodiment of the present application, multiple data tables in the database are stored in the form of a directed acyclic graph. In specific implementation, the SQL statements used in the data processing process or the scripts used in the data processing process are obtained, and data lineage relationships are generated through the SQL statements or the SQL statements contained in the processing scripts, and a directed acyclic graph is formed based on the data lineage relationships.

[0061] In response to the generation of the data table, recording the generation time of the generated data table;

[0062] In response to a data table being queried, the query time and the query subject are recorded, which can be a user or a system. In specific implementations, when a system corresponds to multiple businesses, the business corresponding to the system query must also be recorded.

[0063] In the embodiment of the present application, a data table is maintained in a directed acyclic graph. The following is a detailed description of the data table maintenance process in the embodiment of the present application in conjunction with the accompanying drawings.

[0064] See also Figure 1 , Figure 1 This is a schematic diagram of a data table maintenance process in an embodiment of the present application. The specific steps are:

[0065] Step 101, obtain a first attribute value and a second attribute value of a data table in a directed acyclic graph; wherein the first attribute value is determined based on the computational cost and the total storage cost of the data table; and the second attribute value is determined based on the user query value and the system query value of the data table.

[0066] Step 102: Calculate the ratio of the second attribute value to the first attribute value.

[0067] Step 103: compare the calculated ratio with a preset ratio.

[0068] Step 104 : In response to the ratio being less than a preset ratio, outputting a notification carrying the ratio and setting a timer for the pair of data tables.

[0069] Step 105: In response to receiving an instruction not to delete the data table during the timer timing process, the timer set for the data table is deleted, and this process ends.

[0070] Step 106: In response to the timer timing out and no instruction not to delete the data table is received, the data table is deleted.

[0071] In this embodiment, the first attribute value and the second attribute value of each data table are obtained. When it is determined that the ratio of the second attribute value to the first attribute value of the data table is less than a preset ratio, a notification containing the ratio is output, and a timer is set for the data table. If an instruction is received not to delete the data table during the timer timing process, the timer is deleted. If the timer expires and no instruction is received not to delete the data table, the data table is deleted. The first attribute value is determined based on the computational cost and total storage cost of the data table; the second attribute value is determined based on the user query value and system query value of the data table. In this way, determining whether to delete a data table with a relatively small cost ratio (the ratio of the second attribute value to the first attribute value) can significantly save storage and computation costs. This solution can accurately maintain data tables at a low cost.

[0072] The maintenance of data tables specifically involves optimizing the data tables. According to actual needs, data tables that are rarely used or basically unused are deleted to save storage and computing costs.

[0073] See also Figure 2 , Figure 2 This is another data table maintenance process diagram in the embodiment of this application. The specific steps are:

[0074] Step 201 : Obtain the total storage cost of a data table in a directed acyclic graph; wherein the total storage cost is determined based on the storage cost of the data table itself and the total storage cost of upstream data tables.

[0075] Each data table occupies storage resources during storage. The cost of the occupied storage resources is called the data table's own storage cost. For example, if the occupied storage resources are MGB and the unit storage cost is K, then M×K is the data table's own storage cost.

[0076] When obtaining the total storage cost of a data table, if the upstream data table of the data table has N downstream data tables, the total storage cost of the data table is calculated using 1 / N of the total storage cost of the upstream data tables; that is, the sum of the storage cost of the data table itself and the total storage cost of 1 / N of the upstream data tables is used as the total storage cost of the data table; where N is an integer greater than 1

[0077] If the upstream data table of the data table has a downstream data table, the storage cost of the data table is calculated using the total storage cost of the upstream data table; that is, the total storage cost of the data table is the sum of its own storage cost and the total storage cost of the upstream data table.

[0078] See also Figure 3 , Figure 3 A diagram of a directed acyclic graph. Figure 3 In the table, data tables A and B point to data table C, that is, data table C is the upstream data table of data tables A and B, data table C points to data table F, and data table F is the upstream data table of data table C; data tables E and F point to data table G, and data table G is the upstream data table of data tables E and F.

[0079] Take data table A to calculate the storage cost RSCa as an example:

[0080] RSCa=SCa+1 / 2RSCc;

[0081] Among them, SCa is the storage cost of data table A itself, RSCc is the total storage cost of data table C upstream of data table A. Since data table C has two downstream data tables, the storage cost allocated to data table A is half of its total storage cost.

[0082] Step 202, obtaining the computational cost of the data table; wherein the computational cost is determined based on the computational resources consumed by computing the data table; the computational resources are obtained by weighting the accumulated values ​​of the CPU and memory time consumed by the computation.

[0083] like Figure 3 In the example, data table C generates data table A and data table B through a calculation process. Data table C consumes CPU and memory when calculating to generate data table A, and this process lasts for a period of time. Therefore, the calculation cost of data table A is the cumulative value of the CPU and memory consumed during the calculation process. During the specific calculation, the weighted sum of the cumulative value of CPU and memory is taken to obtain the calculation cost of data table A.

[0084] Here, there is no specific restriction on the weights set for the cumulative value of the CPU and the cumulative value of the memory, and they can be set according to actual needs, such as 1 / 5 and 4 / 5.

[0085] For the most downstream data table in the directed acyclic graph, since it is not used during calculation, the calculation cost is determined to be 0.

[0086] Step 203: Perform a weighted summation of the storage cost and the computation cost to obtain the first attribute value of the data table. Execute step 207.

[0087] In a specific implementation, the storage cost and the computing cost may be normalized first, and then weighted summed to obtain the first attribute value of the data table.

[0088] Steps 201 to 203 provide a specific process of obtaining the first attribute value of a data table in a directed acyclic graph.

[0089] Step 204 : obtaining the number of times the user queries the data table within different preset time periods, and performing a weighted summation of the number of times within different preset time periods to obtain the user query value of the data table.

[0090] Each time a user queries, a data query and the query time will be recorded. In this way, the number of times the user queries the data table within multiple different preset time periods can be obtained based on the current time. As for the number of different preset time periods, the number can be set according to the actual application scenario and is not limited in the embodiment of the present application.

[0091] For example, you can obtain the number of user queries within the last 7 days, the last 30 days, and the last 60 days.

[0092] The weights here are set according to the actual application scenario and are not restricted.

[0093] Step 205 : obtaining the systems that have historically queried the data table and the businesses corresponding to the systems, and determining the system query value of the data table according to the weights of the businesses corresponding to the systems that have queried the data table.

[0094] When the system queries a data table for a certain business, the system and business that have queried the data table are recorded. Repeated queries do not need to be recorded repeatedly.

[0095] For a data table, assuming that the system query records are: system 1, business 1; system 2, business 2, and business 3, then the sum of the corresponding weights of system 1 business 1, system 2 business 2, and system 3 business 3 is taken as the system query value of the data table.

[0096] Step 206: The user query value and the system query value are weighted and summed to obtain the second attribute value of the data table.

[0097] In specific implementation, there is no restriction on the weighted value here, and it is set according to the actual application scenario.

[0098] Steps 204 to 206 provide a specific process of obtaining the second attribute value of the data table in the directed acyclic graph.

[0099] When the data table is not queried by the user or the system, the second attribute value is determined to be 0.

[0100] The execution process of step 201 to step 203 and the execution process of step 204 to step 206 can be executed successively or simultaneously, and this is not limited in the embodiment of the present application.

[0101] Step 207: Calculate the ratio of the second attribute value to the first attribute value.

[0102] The first attribute value can be regarded as the actual total cost of the data table, the second attribute value can be regarded as the total value of the data table, and the ratio of the second attribute value to the first attribute value can be regarded as the cost ratio of the data table.

[0103] Step 208: Compare the calculated ratio with the preset ratio.

[0104] When the ratio of the second attribute value to the first attribute value is less than the preset ratio, it indicates that the cost ratio of the data table is relatively low, and the data table is rarely queried or has not been queried for a long time.

[0105] Step 209: In response to the ratio being less than a preset ratio, output a notification carrying the ratio, and set a timer for the data table.

[0106] In response to the calculated ratio being not less than the preset ratio, no notification is output for the data table, and no timer is set, and maintenance is performed in the previous maintenance manner.

[0107] Here, the notification carrying the ratio for the data table output is output on a certain device, and relevant personnel can receive the notification through text or voice on the device, determine whether to delete the data table based on the ratio carried in the notification, and send corresponding instructions.

[0108] Step 210: In response to receiving an instruction not to delete the data table during the timer timing process, the timer set for the data table is deleted, and this process ends.

[0109] Here, the timer set for the data table is deleted, that is, the data table returns to the previous maintenance method for maintenance without additional processing.

[0110] Step 211 : In response to the timer timing out and no instruction not to delete the data table is received, the data table is deleted.

[0111] When the timer expires, no instruction not to delete the data table is received, indicating that the data table does not need to be maintained any further and can be directly deleted.

[0112] In specific implementation, this embodiment may perform maintenance according to a maintenance cycle.

[0113] In this embodiment, the first attribute value and the second attribute value of each data table are obtained. When it is determined that the ratio of the second attribute value to the first attribute value of the data table is less than a preset ratio, a notification containing the ratio is output, and a timer is set for the data table. If an instruction is received not to delete the data table during the timer timing process, the timer is deleted. If the timer expires and no instruction is received not to delete the data table, the data table is deleted. The first attribute value is determined based on the computational cost and total storage cost of the data table; the second attribute value is determined based on the user query value and system query value of the data table. In this way, determining whether to delete a data table with a relatively small cost ratio (the ratio of the second attribute value to the first attribute value) can significantly save storage and computation costs. This solution can accurately maintain data tables at a low cost.

[0114] See also Figure 4 , Figure 4 This is a schematic diagram of the resource allocation process in the embodiment of this application. The specific steps are:

[0115] Step 401: Acquire a second attribute value of a downstream data table of a data table in a directed acyclic graph, and the second attribute value of the data table.

[0116] The method for determining the second attribute value of a data table is as follows:

[0117] The first step is to obtain the number of times the user queries the data table within different preset time periods, and perform a weighted sum of the number of times within different preset time periods to obtain the user query value of the data table.

[0118] Each time a user queries, a data query and the query time will be recorded. In this way, the number of times the user queries the data table within multiple different preset time periods can be obtained based on the current time. As for the number of different preset time periods, the number can be set according to the actual application scenario and is not limited in the embodiment of the present application.

[0119] For example, you can obtain the number of user queries within the last 7 days, the last 30 days, and the last 60 days.

[0120] The weights here are set according to the actual application scenario and are not restricted.

[0121] The second step is to obtain the systems that have historically queried the data table and the businesses corresponding to the systems, and determine the system query value of the data table based on the weights of the businesses corresponding to the systems that have queried the data table.

[0122] When the system queries a data table for a certain business, the system and business that have queried the data table are recorded. Repeated queries do not need to be recorded repeatedly.

[0123] For a data table, assuming that the system query records are: system 1, business 1; system 2, business 2, and business 3, then the sum of the corresponding weights of system 1 business 1, system 2 business 2, and system 3 business 3 is taken as the system query value of the data table.

[0124] Step 3: Take the weighted sum of the user query value and the system query value to obtain the second attribute value of the data table.

[0125] In specific implementation, there is no restriction on the weighted value here, and it is set according to the actual application scenario.

[0126] Step 402 : Obtain and record a third attribute value of the data table; wherein the third attribute value is determined based on the second attribute value of the downstream data table of the data table and the second attribute value of the data table.

[0127] If a data has multiple downstream data tables, the second attribute values ​​of the multiple downstream data tables are accumulated.

[0128] Step 403: In response to insufficient computing resources, the computing resources of the data table with a high third attribute value are preferentially used.

[0129] In response to the sufficient computing resources, usage of the computing resources of all data tables may be satisfied.

[0130] In the embodiment of the present application, the third attribute value can be regarded as the score value of the data table, which is used to evaluate the importance of the data table. When computing resources are insufficient, the use of computing resources of the data table with a high third attribute value is prioritized to ensure the stability of the entire computing chain.

[0131] See also Figure 5 , Figure 5 This is a flowchart of adjusting and updating the data table time in this embodiment of the application. The specific steps are:

[0132] Step 501: Obtain the generation time of the data table in the current update cycle and the earliest query time of the data table in the current update cycle.

[0133] Set the update cycle of the data table. At the end of the current cycle, obtain the generation time of the data table in the current update cycle and the earliest queried time.

[0134] Step 502: Calculate the difference between the query time and the generation time.

[0135] Step 503: compare the calculated difference with a preset difference.

[0136] In a specific implementation, the difference is converted into a time value in the same unit as the preset difference. For example, if the preset difference is seconds, the difference is converted into a time value in seconds.

[0137] Step 504 : In response to the calculated difference being greater than the preset difference, in the next update cycle, the time for updating the data table is delayed, and the delayed time for updating the data table is no later than the query time of the current cycle.

[0138] In response to the calculated difference being not greater than the preset difference, the time for updating the data table is not adjusted in the next update cycle.

[0139] In the embodiment of the present application, based on the usage of the data table in the current update cycle, the update cycle of the data table in the next update cycle is determined, and some data tables in the directed acyclic graph can be scheduled not to be updated at the same time, thereby avoiding the occurrence of resource usage scheduling peaks.

[0140] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present disclosure, and will not be described in detail here.

[0141] Based on the same inventive concept, the present application also provides a data table maintenance device. Figure 6 , Figure 6 This is a schematic diagram of the structure of the data table maintenance device in the embodiment of the present application. The data table maintenance device includes:

[0142] The acquisition unit 601 is configured to acquire a first attribute value and a second attribute value of a data table in a directed acyclic graph; wherein the first attribute value is determined based on a computation cost and a total storage cost of the data table; and the second attribute value is determined based on a user query value and a system query value of the data table.

[0143] A calculation unit 602 is configured to calculate a ratio of the second attribute value to the first attribute value;

[0144] A comparison unit 603 is configured to compare the calculated ratio with a preset ratio;

[0145] The timing unit 604 is configured to set a timer for the data table in response to the ratio being less than the preset ratio;

[0146] The output unit 605 is configured to output a notification carrying the ratio in response to the ratio being less than the preset ratio;

[0147] The deletion unit 606 is configured to execute, in response to receiving an instruction not to delete the data table during the timer timing process, deleting the timer set for the data table; in response to the timer timing expiring and no instruction not to delete the data table is received, deleting the data table.

[0148] In another embodiment,

[0149] The acquisition unit 601 is configured to execute, when acquiring the total storage cost of the data table, if there are N downstream data tables of the upstream data table of the data table, using 1 / N of the total storage cost of the upstream data table to calculate the total storage cost of the data table; if there is one downstream data table of the upstream data table of the data table, using the total storage cost of the upstream data table to calculate the storage cost of the data table; wherein N is an integer greater than 1.

[0150] In another embodiment,

[0151] The acquisition unit 601 is configured to execute the acquisition of the second attribute value of the data table in the directed acyclic graph, obtain the number of times the user queries the data table in different preset time periods, and perform a weighted sum of the number of times in different preset time periods to obtain the user query value of the data table; obtain the systems and businesses that have historically queried the data table, and determine the system query value of the data table based on the weights corresponding to the systems and businesses that have queried the data table; and perform a weighted sum of the user query value and the system query value to obtain the second attribute value of the data table.

[0152] In another embodiment, the apparatus further comprises: a processing unit 607;

[0153] The acquiring unit 601 is further configured to acquire a second attribute value of a downstream data table of the data table and the second attribute value of the data table; acquire and record a third attribute value of the data table; wherein the third attribute value is determined based on the second attribute value of the downstream data table of the data table and the second attribute value of the data table;

[0154] The processing unit 607 is configured to execute, in response to insufficient computing resources, to give priority to using computing resources of the data table with a high third attribute value.

[0155] In another embodiment,

[0156] The acquiring unit 601 is further configured to acquire the generation time of the data table in the current update cycle and the earliest query time of the data table in the current update cycle;

[0157] The calculation unit 602 is further configured to calculate the difference between the query time and the generation time;

[0158] The comparison unit 603 is further configured to compare the calculated difference with a preset difference;

[0159] The processing unit 607 is further configured to, in response to the calculated difference being greater than a preset difference, delay updating the data table in the next update cycle, and the delayed updating time of the data table is no later than the query time of the current cycle.

[0160] In another embodiment,

[0161] The acquisition unit 601 is configured to record the generation time of the data table in response to the generation of the data table; and record the query time and the query subject in response to the data table being queried, where the subject is a user or a system.

[0162] The units in the above embodiments may be integrated into one body or deployed separately; they may be combined into one unit or further divided into multiple sub-units.

[0163] In another embodiment, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the steps of the data table maintenance method are implemented when the processor executes the program.

[0164] In another embodiment, a computer-readable storage medium is provided, on which computer instructions are stored. When the instructions are executed by a processor, the steps in the data table maintenance method can be implemented.

[0165] Figure 7 Schematic diagram of the physical structure of the electronic device provided by the embodiment of the present invention. Figure 7 As shown, the electronic device may include: a processor (Processor) 710, a communication interface (Communications Interface) 720, a memory (Memory) 730 and a communication bus 740, wherein the processor 710, the communication interface 720, and the memory 730 communicate with each other via the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the following method:

[0166] Obtaining a first attribute value and a second attribute value of a data table in a directed acyclic graph; wherein the first attribute value is determined based on a computational cost and a total storage cost of the data table; and the second attribute value is determined based on a user query value and a system query value of the data table;

[0167] Calculating a ratio of the second attribute value to the first attribute value;

[0168] Comparing the calculated ratio with the preset ratio;

[0169] In response to the ratio being less than a preset ratio, outputting a notification carrying the ratio and setting a timer for the data table;

[0170] In response to receiving an instruction not to delete the data table during the timer timing process, deleting the timer set for the data table;

[0171] In response to the timer timing out and no instruction not to delete the data table is received, the data table is deleted.

[0172] In addition, the logic instructions in the above-mentioned memory 730 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0173] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0174] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0175] The flowcharts and block diagrams in the accompanying drawings of the present application show the possible implementation architecture, functions and operations of the systems, methods and computer program products according to the various embodiments disclosed in the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in the order of the standards in different figures. For example, the boxes represented by two connections can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of the boxes in the block diagram or flowchart, can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0176] Those skilled in the art will appreciate that the features described in the various embodiments and / or claims disclosed in this application may be combined and / or coupled in various ways, even if such combinations and / or couplings are not explicitly disclosed in this application. In particular, without departing from the spirit and teachings of this application, the features described in the various embodiments and / or claims of this application may be combined and / or coupled in various ways, and all such combinations and / or couplings fall within the scope disclosed in this application.

[0177] The principles and implementation methods of the present invention are described herein using specific embodiments. The description of the above embodiments is only used to help understand the method of the present invention and its core ideas, and is not intended to limit this application. For those skilled in the art, changes can be made in the specific implementation methods and application scope based on the ideas, spirit and principles of the present invention. Any modifications, equivalent replacements, improvements, etc. made therein should be included within the scope of protection of this application.

Claims

1. A data table maintenance method, characterized in that: The method comprises: Obtaining a first attribute value and a second attribute value of a data table in a directed acyclic graph; wherein the first attribute value is determined based on a computational cost and a total storage cost of the data table; and the second attribute value is determined based on a user query value and a system query value of the data table; calculating a ratio of the second attribute value to the first attribute value; comparing the calculated ratio with a preset ratio; In response to the ratio being less than the preset ratio, outputting a notification carrying the ratio and setting a timer for the data table; In response to receiving an instruction not to delete the data table during the timer timing process, deleting the timer set for the data table; In response to the timer timing out and no instruction not to delete the data table is received, deleting the data table; Wherein, obtaining the first attribute value of the data table in the directed acyclic graph includes: Obtaining a total storage cost of the data table; wherein the total storage cost is determined based on the data table's own storage cost and the total storage cost of upstream data tables; Obtaining a computational cost for the data table; wherein the computational cost is determined based on the computational resources consumed for computing the data table; the computational resources are obtained by weighting the accumulated values ​​of the CPU and memory time consumed for computing the data table; Obtaining a first attribute value of the data table by performing a weighted summation of the total storage cost and the computation cost of the data table; Obtaining a second attribute value of a data table in the directed acyclic graph includes: Obtaining the number of times a user queries the data table within different preset time periods, and performing a weighted summation of the number of times within different preset time periods to obtain a user query value of the data table; Obtaining the systems and businesses that have historically queried the data table, and determining the system query value of the data table based on the weights corresponding to the systems and businesses that have queried the data table; The second attribute value of the data table is obtained by weighted summing the user query value and the system query value.

2. The method according to claim 1, characterized in that The method further comprises: When obtaining the total storage cost of the data table, if there are N downstream data tables of the upstream data table of the data table, the total storage cost of the data table is calculated using 1 / N of the total storage cost of the upstream data table; if there is one downstream data table of the upstream data table of the data table, the storage cost of the data table is calculated using the total storage cost of the upstream data table; wherein N is an integer greater than 1.

3. The method according to claim 1, characterized in that The method further comprises: Obtaining a second attribute value of a downstream data table of the data table and a second attribute value of the data table; Obtaining and recording a third attribute value of the data table; wherein the third attribute value is determined based on a second attribute value of a downstream data table of the data table and the second attribute value of the data table; In response to insufficient computing resources, the computing resources of the data table having a high third attribute value are preferentially used.

4. The method according to claim 1, wherein The method further comprises: Obtaining the generation time of the data table in the current update cycle and the earliest query time of the data table in the current update cycle; Calculating the difference between the query time and the generation time; Comparing the calculated difference with the preset difference; In response to the calculated difference being greater than a preset difference, in the next update cycle, the time for updating the data table is delayed, and the delayed time for updating the data table is no later than the query time in the current cycle.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: In response to the generation of the data table, recording a generation time of the data table; In response to the data table being queried, the query time and the subject of the query are recorded, where the subject is a user or a system.

6. A data table maintenance device, characterized in that: The device comprises: an acquiring unit configured to acquire a first attribute value and a second attribute value of a data table in a directed acyclic graph; wherein the first attribute value is determined based on a computation cost and a storage cost of the data table; and the second attribute value is determined based on a user query value and a system query value of the data table; a calculating unit, configured to calculate a ratio of the second attribute value to the first attribute value; a comparing unit configured to compare the calculated ratio with a preset ratio; a timing unit configured to set a timer for the data table in response to the ratio being less than the preset ratio; an output unit configured to output a notification carrying the ratio in response to the ratio being less than the preset ratio; a deleting unit configured to, in response to receiving an instruction not to delete the data table during the timer timing process, delete the timer set for the data table; and in response to expiration of the timer and no instruction not to delete the data table is received, delete the data table; Among them, the acquisition unit is specifically used to obtain the first attribute value of the data table in the directed acyclic graph, including: obtaining the total storage cost of the data table; wherein the total storage cost is determined based on the data table's own storage cost and the total storage cost of the upstream data table; obtaining the computational cost of the data table; wherein the computational cost is determined based on the computing resources consumed by computing the data table; the computing resources are obtained by weighting the accumulated time value of the CPU and memory consumed by the computation; the first attribute value of the data table is obtained by weighted summing the total storage cost and the computational cost of the data table; when obtaining the second attribute value of the data table in the directed acyclic graph, including: obtaining the number of times the user queries the data table within different preset time periods, and weighted summing the number of times within different preset time periods to obtain the user query value of the data table; obtaining the systems and businesses that have historically queried the data table, and determining the system query value of the data table based on the weights corresponding to the systems and businesses that have queried the data table; and obtaining the second attribute value of the data table by weighted summing the user query value and the system query value.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images