Data table life cycle management method and related device

Abstract

The invention discloses a data table life cycle management method and a related device, which can be used in the field of big data, in the method, firstly, a storage medium type and calling information of a data table are acquired; then, based on the storage medium type and the calling information of the data table, determining a life cycle recommendation strategy of the data table; then, based on a life cycle recommendation strategy and a target user identifier corresponding to the data table, outputting recommendation information; and finally, in response to a life cycle strategy modification operation executed by the target user based on the recommendation information, updating the current life cycle. In this way, the life cycle recommendation strategy of the data table is determined according to the information of multiple aspects such as the data use condition and the storage cost, and the life cycle better meeting the requirement can be set for the data table.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and in particular to a data table life cycle management method and related devices. Background Art

[0002] As more and more data is stored in the database, in order to reduce storage costs, the method of setting a life cycle for the data table is usually adopted to manage the data so as to realize automatic data cleanup.

[0003] Currently, developers usually manually set the life cycle of a data table when defining the data table structure. However, it is difficult for developers to make accurate judgments on the actual value and usage frequency of the data in advance, and it is impossible to accurately set the life cycle of the data table. If the life cycle is set too long, useless data will be retained for a long time, occupying storage resources; if the life cycle is set too short, valuable data will be cleared prematurely, affecting daily use.

[0004] Therefore, how to set a life cycle for the data table that better meets the needs becomes a problem that needs to be solved. Summary of the invention

[0005] Based on the above problems, the present application provides a data table life cycle management method and related devices, which can set a life cycle for the data table that better meets the needs.

[0006] The embodiments of the present application disclose the following technical solutions:

[0007] In a first aspect, an embodiment of the present application provides a method for managing a data table life cycle, the method comprising:

[0008] Acquire the storage medium type and call information of the data table; the call information includes the last access date and the last update date of each partition of the data table;

[0009] Determine a recommended life cycle strategy for the data table based on the storage medium type and call information of the data table;

[0010] Based on the life cycle recommendation strategy and the target user identifier corresponding to the data table, output recommendation information; the recommendation information at least includes: storage space change information and data table partition life cycle change information generated by adopting the life cycle recommendation strategy compared with adopting the current life cycle;

[0011] In response to a lifecycle policy modification operation performed by a target user based on the recommendation information, the current lifecycle is updated.

[0012] Optionally, the determining a recommended lifecycle strategy for the data table based on the storage medium type and call information of the data table includes:

[0013] Based on the storage medium type and call information of the data table, a recommended life cycle strategy for the data table is periodically determined at preset time intervals.

[0014] Optionally, the determining a recommended lifecycle strategy for the data table based on the storage medium type and call information of the data table includes:

[0015] Based on the call information of the data table and the first weight, obtaining a first data item;

[0016] Obtaining a second data item based on a storage cost coefficient corresponding to the storage medium type of the data table and a second weight;

[0017] Determine a recommended life cycle value of the data table based on the first data item, the second data item, and a mapping relationship between the call information and the time margin;

[0018] Based on the recommended life cycle values ​​of multiple data tables, a life cycle recommendation strategy is obtained.

[0019] Optionally, obtaining the first data item based on the call information of the data table and the first weight includes:

[0020] Based on the date difference between the most recently accessed date and the most recently updated date of the data table partition, the maximum date difference among the corresponding date differences of the multiple data table partitions is obtained;

[0021] A first data item is obtained based on the maximum date difference and the first weight.

[0022] Optionally, obtain call information of the data table, including:

[0023] Scan the historical full task execution log to obtain the input table partition name, output table partition name, and task execution date of each task;

[0024] Based on the input table partition name, output table partition name and task execution date of each task, obtain the access date and update date corresponding to each task;

[0025] The maximum value of the access dates corresponding to the tasks is determined as the most recent access date of the data table partition, and the maximum value of the update dates corresponding to the tasks is determined as the most recent update date of the data table partition to obtain the call information of the data table.

[0026] Optionally, after updating the current life cycle in response to the target user performing a life cycle policy modification operation based on the recommendation information, the method further includes:

[0027] Scanning a data table based on a preset scanning cycle;

[0028] If there is a target data table or target data table partition whose remaining life cycle duration is less than a preset duration threshold, data information to be cleaned is sent based on the target user identifier corresponding to the target data table or target data table partition; the data information to be cleaned includes the name, storage capacity, last access date and estimated cleanup time of the target data table or target data table partition.

[0029] Optionally, after updating the current life cycle in response to the target user performing a life cycle policy modification operation based on the recommendation information, the method further includes:

[0030] In response to a whitelisting operation performed by a target user on a target data table or a target data table partition, the life cycle of the target data table or the target data table partition is determined to be long-term preservation.

[0031] Optionally, after determining the recommended lifecycle strategy for the data table based on the storage medium type and call information of the data table, the method further includes:

[0032] If the deviation between the recommended lifecycle strategy corresponding to the target data table or the target data table partition and the current lifecycle is greater than a preset deviation threshold, a first storage cost corresponding to the recommended lifecycle value and a second storage cost corresponding to the current lifecycle are calculated;

[0033] Based on the target user identifier corresponding to the target data table or the target data table partition, the first storage cost, the second storage cost, and the data table partition lifecycle change information generated by adopting the lifecycle recommendation strategy compared to adopting the current lifecycle are sent.

[0034] In a second aspect, an embodiment of the present application provides a data table life cycle management device, the device comprising: an acquisition module, a determination module, an output module and an update module;

[0035] The acquisition module is used to acquire the storage medium type and call information of the data table; the call information includes the last access date and the last update date of each partition of the data table;

[0036] The determination module is used to determine a recommended life cycle strategy for the data table based on the storage medium type and call information of the data table;

[0037] The output module is used to output recommendation information based on the life cycle recommendation strategy and the target user identifier corresponding to the data table; the recommendation information at least includes: storage space change information and data table partition life cycle change information generated by adopting the life cycle recommendation strategy compared with adopting the current life cycle;

[0038] The updating module is used to update the current life cycle in response to a life cycle policy modification operation performed by a target user based on the recommendation information.

[0039] In a third aspect, an embodiment of the present application provides a data table life cycle management device, the device comprising: a memory and a processor;

[0040] The memory is used to store program code and transmit the program code to the processor;

[0041] The processor is used to execute the steps of the data table life cycle management method described in any implementation of the first aspect according to the program code.

[0042] Compared with the prior art, this application has the following beneficial effects:

[0043] The embodiment of the present application provides a method for managing the life cycle of a data table, in which, first, the storage medium type and call information of the data table are obtained; then, based on the storage medium type and call information of the data table, the recommended life cycle strategy of the data table is determined; then, based on the recommended life cycle strategy and the target user identifier corresponding to the data table, the recommended information is output; finally, in response to the life cycle strategy modification operation performed by the target user based on the recommended information, the current life cycle is updated. Thus, the recommended life cycle strategy of the data table is determined based on information such as data usage and storage cost, so that a life cycle that better meets the needs can be set for the data table. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 A flow chart of a method for managing a data table life cycle provided in an embodiment of the present application;

[0046] Figure 2 A schematic diagram of a data table life cycle management device provided in an embodiment of the present application;

[0047] Figure 3 A structural diagram of a data table life cycle management device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The data table life cycle management method and related devices provided in the present application can be used in the big data field. The above is only an example and does not limit the application field of the data table life cycle management method and related devices provided in the present application.

[0049] The terms "first", "second", "third" and "fourth" etc. in the specification, claims and drawings of this application are used to distinguish different objects rather than to limit a specific order.

[0050] In the embodiments of the present application, the words "as an example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "as an example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the words "as an example" or "for example" is intended to present the relevant concepts in a specific way.

[0051] The terms used in the implementation section of this application are only used to explain the specific embodiments of this application and are not intended to limit this application.

[0052] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0053] See also Figure 1 , which is a flow chart of a data table life cycle management method provided in an embodiment of the present application, the method comprising:

[0054] S101: Obtain storage medium type and call information of a data table.

[0055] The call information includes the most recent access date and the most recent update date of each partition of the data table. As an example, by scanning the historical full task execution log, each data task can be parsed by the table lineage method to obtain the input table partition name, output table partition name and task execution date of each task; then, based on the input table partition name, output table partition name and task execution date of each task, the access date and update date corresponding to each task are obtained; finally, the maximum value of the access dates corresponding to each task is determined as the most recent access date of the data table partition, and the maximum value of the update dates corresponding to each task is determined as the most recent update date of the data table partition, to obtain the call information of the data table.

[0056] For example, for a data query task, if the task execution date is August 28, 2024, the task SQL code is:

[0057] select xxxx from table1

[0058] where ds = 'Partition 1'

[0059] The table lineage analysis result is: the input table partition name is "Table 1. Partition 1". Combined with the task execution date, it can be obtained that the access date of Table 1. Partition 1 in this task is August 28, 2024.

[0060] For another example, for a data insertion task, if the task execution date is August 29, 2024, the task SQL code is:

[0061] Insert into table table2partition(ds='partition 2')

[0062] select xxxx from table1

[0063] where ds = 'Partition 1'

[0064] The table lineage analysis result is: the input table partition is named "Table 1. Partition 1", and the output table partition is named "Table 2. Partition 2". Combined with the task execution date, it can be obtained that the access date of Table 1. Partition 1 in this task is August 29, 2024, and the update date of Table 2. Partition 2 is August 29, 2024.

[0065] The obtained access date and update date are grouped according to the data table partition to obtain the access date and update date corresponding to each task; the maximum value in each group of access date and update date is calculated, and the maximum value in the access date corresponding to each task is determined as the most recent access date of the data table partition, and the maximum value in the update date corresponding to each task is determined as the most recent update date of the data table partition to obtain the call information of the data table.

[0066] Exemplarily, the storage medium type of the data table can be obtained from the metadata management system, which can include three main types: hot storage, cold storage, and archive storage, where the storage cost of hot storage is higher than that of cold storage, and the storage cost of cold storage is higher than that of archive storage.

[0067] S102: Determine a recommended lifecycle strategy for the data table based on the storage medium type and call information of the data table.

[0068] As an example, the life cycle recommendation strategy of a data table can be periodically determined according to a preset time interval. For example, if the preset time interval is 7 days, and the last life cycle recommendation strategy of a data table is determined on August 22, 2024, the life cycle recommendation strategy of the data table is re-determined on August 29, 2024. In this way, the life cycle recommendation strategy of a data table can be regularly updated to better adapt to changes in business needs and data access patterns, and improve the timeliness of the life cycle recommendation strategy.

[0069] For each data table, a first data item can be obtained based on the calling information of the data table and the first weight; a second data item can be obtained based on the storage cost coefficient corresponding to the storage medium type of the data table and the second weight; and a recommended life cycle value of the data table can be determined based on the mapping relationship between the first data item, the second data item, and the calling information and the time margin.

[0070] Specifically, in the embodiment of the present application, the life cycle of the data table is considered from the two aspects of data update and usage and storage cost. The recommended life cycle value p(s) of the data table s can be calculated by the following formula:

[0071] Among them, α is the first weight, β is the second weight, and γ is the third weight. The values ​​of α, β and γ can be flexibly adjusted according to actual needs. The larger the value, the greater the influence of the corresponding variable on the p(s) value; s(i).lv is the last access date of partition s(i); s(i).lm is the last update date of partition s(i); cost(s) is the storage cost coefficient; buffer is the time margin.

[0072] As an example, given a data table s containing partitions s(i) (s1, s2, ..., sn), for each partition s(i), calculate the date difference between the most recently accessed date s(i).lv and the most recently updated date s(i).lm, then select the largest date difference among all partitions to obtain the maximum date difference among the corresponding date differences of multiple data table partitions; finally, calculate the product of the variable of the maximum date difference and the first weight α to obtain the first data item.

[0073] For example, data table 1 contains partition 1 and partition 2, where the last update date of partition 1 is January 1, 2024, and the last access date of partition 1 is January 2, 2024, and the date difference between the two is 1 day; the last update date of partition 2 is January 2, 2024, and the last access date of partition 1 is January 4, 2024, and the date difference between the two is 2 days. Therefore, the maximum date difference among the corresponding date differences of multiple data table partitions is 2 days.

[0074] As an example, the storage cost coefficient cost(s) is positively correlated with the storage cost corresponding to the storage medium type of the data table.

[0075] As an example, the mapping relationship between the call information and the time margin can be specifically a mapping relationship between the predefined maximum date difference interval and the time margin buffer. The larger the maximum date difference, the larger the corresponding time margin. Therefore, even if the data access mode changes, the lifecycle recommendation strategy can still maintain a certain degree of flexibility and can better adapt to changes in business needs and data access modes.

[0076] By integrating the recommended life cycle values ​​from multiple data tables, we can get the recommended life cycle strategy.

[0077] S103: Outputting recommendation information based on the lifecycle recommendation strategy and the target user identifier corresponding to the data table.

[0078] As an example, the recommendation information can be output to the terminal device of the data owner (i.e., the target user) via email or other instant messaging methods, where the target user identifier can be the email address or phone number of the data owner. The output frequency of the recommendation information can be set according to actual needs, for example, once a week or once a month. In this way, the target user can receive the lifecycle recommendation strategy regularly, which enhances the transparency of data management.

[0079] The content of the recommendation information may include but is not limited to: the current life cycle, the recommended life cycle strategy, the storage space change information and data table partition life cycle change information generated by the recommended life cycle strategy compared to the current life cycle, and operation suggestions, etc.; wherein, the storage space change information may include the estimated amount of storage space that can be saved after cleaning up the data table partitions by adopting the recommended life cycle strategy; the data table partition life cycle change information may include a list of data table partition names and the amount of life cycle change after the recommended life cycle strategy is adopted; the operation suggestions may include, for example, whether to recommend adjusting the current life cycle.

[0080] Optionally, in the recommendation information output to the target user, a button or link can be provided for redirecting the page for the target user to the lifecycle modification page. For example, a "Modify Now" button can be provided, and the target user can click the button to enter the corresponding management interface to modify the lifecycle policy.

[0081] Optionally, it is possible to check whether the current life cycle of the data table is consistent with the life cycle recommendation strategy at preset time intervals. If the deviation between the life cycle recommendation strategy corresponding to the target data table or the target data table partition and the current life cycle is greater than the preset deviation threshold, it means that there is a significant difference between the life cycle recommendation strategy and the current life cycle. At this time, the first storage cost corresponding to the life cycle recommendation value and the second storage cost corresponding to the current life cycle can be calculated, and based on the target user identifier corresponding to the target data table or the target data table partition, the first storage cost, the second storage cost, and the data table partition life cycle change information generated by the life cycle recommendation strategy compared to the current life cycle are sent. In this way, when there is a large difference between the life cycle recommendation strategy and the current life cycle, the target user can be notified in time and prompted to optimize.

[0082] For example, the first storage cost, the second storage cost, and the data table partition life cycle change information generated by the life cycle recommendation strategy compared to the current life cycle can be sent to the target user corresponding to the target data table or the target data table partition based on the target user identifier corresponding to the target data table or the target data table partition by email or other instant messaging methods. The data table partition life cycle change information generated by the life cycle recommendation strategy compared to the current life cycle may include: the name, storage capacity, last access date, and expected cleaning time change of the data table partition corresponding to the data table partition to be cleared in advance or delayed; the target user is the data responsible person of the target data table or the target data table partition.

[0083] S104: In response to the target user performing a lifecycle policy modification operation based on the recommendation information, the current lifecycle is updated.

[0084] Optionally, after the target user modifies the lifecycle policy, a review page including all modification details may be displayed. After the target user reviews and confirms the modification, the current lifecycle is updated in response to the lifecycle policy modification operation performed by the target user based on the recommended information.

[0085] Exemplarily, the lifecycle policy modification operation performed by the target user based on the recommendation information may be to use the recommended lifecycle policy as the new current lifecycle, or to perform a customized modification on the lifecycle policy.

[0086] In the embodiment of the present application, first, the storage medium type and call information of the data table are obtained; then, based on the storage medium type and call information of the data table, the life cycle recommendation strategy of the data table is determined; then, based on the life cycle recommendation strategy and the target user identifier corresponding to the data table, the recommendation information is output; finally, in response to the life cycle policy modification operation performed by the target user based on the recommendation information, the current life cycle is updated. Thus, the life cycle recommendation strategy of the data table is determined based on multiple aspects of information such as data usage and storage cost, and a life cycle that better meets the needs can be set for the data table.

[0087] In addition, in another embodiment provided by the present application, the data table can also be scanned based on a preset scanning cycle; if there is a target data table or a target data table partition whose remaining life cycle is less than a preset time threshold, the data information to be cleaned is sent based on the target user identifier corresponding to the target data table or the target data table partition. The data information to be cleaned may include but is not limited to the name, storage capacity, last access date, and estimated cleaning time of the target data table or the target data table partition; the scanning cycle can be set according to actual needs, for example, it can be one week or one month, etc.

[0088] After the target user receives the information that the data is about to be cleaned, the target user can perform the operation of adding to the whitelist or modifying the life cycle for a specific data table or data table partition based on actual needs. In response to the target user performing the operation of adding to the whitelist on the target data table or the target data table partition, the life cycle of the target data table or the target data table partition is determined to be long-term preservation.

[0089] In this way, the target user can regularly receive information about the data that is about to be cleaned up. When a data table or data table partition is close to its life cycle, the target user can be reminded to understand and handle it in time, so that the target user can grasp the data status in time and avoid important data being automatically deleted without the target user's knowledge.

[0090] In another embodiment provided by the present application, for a data table that needs to comply with industry standards, its life cycle can be set based on the industry standards. For example, a law requires listed companies to retain all financial records for at least seven years. For a data table that records financial records, its recommended life cycle value is equal to the prescribed retention period, that is, seven years.

[0091] In another embodiment provided by the present application, historical information sent to a target user and operations performed by the target user can be saved for subsequent tracking and auditing. The historical information includes sent recommendation information and data information to be cleaned up. The target user can view the historical information and operations performed to confirm the usage of the data table and historical change information.

[0092] See also Figure 2 , this figure is a schematic diagram of a data table life cycle management device provided in an embodiment of the present application, and the device includes: an acquisition module 201, a determination module 202, an output module 203 and an update module 204.

[0093] The acquisition module 201 is used to acquire the storage medium type and call information of the data table; the call information includes the last access date and the last update date of each partition of the data table;

[0094] A determination module 202, configured to determine a recommended life cycle strategy for a data table based on a storage medium type and call information of the data table;

[0095] The output module 203 is used to output recommendation information based on the life cycle recommendation strategy and the target user identifier corresponding to the data table; the recommendation information at least includes: storage space change information and data table partition life cycle change information generated by adopting the life cycle recommendation strategy compared with adopting the current life cycle;

[0096] The updating module 204 is configured to update the current life cycle in response to a life cycle policy modification operation performed by the target user based on the recommendation information.

[0097] Therefore, the recommended life cycle strategy for the data table can be determined based on multiple information such as data usage and storage cost, so that a life cycle that better meets the needs can be set for the data table.

[0098] Optionally, the determination module 202 is specifically configured to: periodically determine a recommended life cycle strategy for the data table based on a storage medium type and call information of the data table at a preset time interval.

[0099] Optionally, the determination module 202 includes a first calculation unit, a second calculation unit, a third calculation unit and a determination unit; wherein the first calculation unit is used to obtain a first data item based on the calling information of the data table and a first weight; the second calculation unit is used to obtain a second data item based on a storage cost coefficient corresponding to the storage medium type of the data table and a second weight; the third calculation unit is used to determine the recommended life cycle value of the data table based on the first data item, the second data item and a mapping relationship between the calling information and the time margin; the determination unit is used to obtain a life cycle recommendation strategy based on the life cycle recommendation values ​​of multiple data tables.

[0100] Optionally, the first calculation unit is specifically used to: obtain the maximum date difference among the corresponding date differences of multiple data table partitions based on the date difference between the last access date and the last update date of the data table partition; and obtain the first data item based on the maximum date difference and the first weight.

[0101] Optionally, an acquisition unit is used to scan the historical full task execution log to obtain the input table partition name, output table partition name and task execution date of each task; based on the input table partition name, output table partition name and task execution date of each task, obtain the access date and update date corresponding to each task; determine the maximum value of the access date corresponding to each task as the most recent access date of the data table partition, and determine the maximum value of the update date corresponding to each task as the most recent update date of the data table partition to obtain the call information of the data table.

[0102] Optionally, another data table life cycle management device provided in an embodiment of the present application also includes a scanning module, which is used to: scan the data table based on a preset scanning cycle; if there is a target data table or a target data table partition whose remaining life cycle duration is less than a preset duration threshold, then send data information to be cleaned based on the target user identifier corresponding to the target data table or the target data table partition; the data information to be cleaned includes the name, storage capacity, last access date and estimated cleanup time of the target data table or the target data table partition.

[0103] Optionally, another data table lifecycle management device provided in an embodiment of the present application also includes an execution module for determining the lifecycle of the target data table or target data table partition as long-term preservation in response to a whitelisting operation performed by a target user on the target data table or target data table partition.

[0104] Optionally, another data table lifecycle management device provided in an embodiment of the present application also includes a deviation calculation module, which is used to calculate a first storage cost corresponding to the recommended lifecycle value and a second storage cost corresponding to the current lifecycle when the deviation between the recommended lifecycle strategy corresponding to the target data table or the target data table partition and the current lifecycle is greater than a preset deviation threshold; based on the target user identifier corresponding to the target data table or the target data table partition, send the first storage cost, the second storage cost, and the data table partition lifecycle change information generated by adopting the recommended lifecycle strategy compared to adopting the current lifecycle.

[0105] See also Figure 3 , this figure is a structural diagram of a data table life cycle management device provided in an embodiment of the present application, and the device includes: a memory 301 and a processor 302.

[0106] Memory 301: used to store program codes and transmit the program codes to the processor.

[0107] Processor 302: used to execute the steps of the above-mentioned data table life cycle management method according to the instructions in the program code.

[0108] In addition, the present application also provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on a data table life cycle management device, the data table life cycle management device executes the steps of the above-mentioned data table life cycle management method.

[0109] It should be noted that each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments. The device and storage medium embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components indicated as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. A person of ordinary skill in the art can understand and implement it without creative work.

[0110] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for managing a data table life cycle, characterized in that: The method comprises: Acquire the storage medium type and call information of the data table; the call information includes the last access date and the last update date of each partition of the data table; Determine a recommended life cycle strategy for the data table based on the storage medium type and call information of the data table; Based on the life cycle recommendation strategy and the target user identifier corresponding to the data table, output recommendation information; the recommendation information at least includes: storage space change information and data table partition life cycle change information generated by adopting the life cycle recommendation strategy compared with adopting the current life cycle; In response to a lifecycle policy modification operation performed by a target user based on the recommendation information, the current lifecycle is updated.

2. The method according to claim 1, characterized in that The determining of a recommended lifecycle strategy for the data table based on the storage medium type and call information of the data table includes: Based on the storage medium type and call information of the data table, a recommended life cycle strategy for the data table is periodically determined at preset time intervals.

3. The method according to claim 1, characterized in that The determining of a recommended lifecycle strategy for the data table based on the storage medium type and call information of the data table includes: Based on the call information of the data table and the first weight, obtaining a first data item; Obtaining a second data item based on a storage cost coefficient corresponding to the storage medium type of the data table and a second weight; Determine a recommended life cycle value of the data table based on the first data item, the second data item, and a mapping relationship between the call information and the time margin; Based on the recommended life cycle values ​​of multiple data tables, a life cycle recommendation strategy is obtained.

4. The method according to claim 3, characterized in that The obtaining of the first data item based on the calling information of the data table and the first weight includes: Based on the date difference between the most recently accessed date and the most recently updated date of the data table partition, the maximum date difference among the corresponding date differences of the multiple data table partitions is obtained; A first data item is obtained based on the maximum date difference and the first weight.

5. The method according to claim 1, characterized in that Get the call information of the data table, including: Scan the historical full task execution log to obtain the input table partition name, output table partition name, and task execution date of each task; Based on the input table partition name, output table partition name and task execution date of each task, obtain the access date and update date corresponding to each task; The maximum value of the access dates corresponding to the tasks is determined as the most recent access date of the data table partition, and the maximum value of the update dates corresponding to the tasks is determined as the most recent update date of the data table partition to obtain the call information of the data table.

6. The method according to claim 1, characterized in that After updating the current life cycle in response to the life cycle policy modification operation performed by the target user based on the recommendation information, the method further includes: Scanning a data table based on a preset scanning cycle; If there is a target data table or target data table partition whose remaining life cycle duration is less than a preset duration threshold, data information to be cleaned is sent based on the target user identifier corresponding to the target data table or target data table partition; the data information to be cleaned includes the name, storage capacity, last access date and estimated cleanup time of the target data table or target data table partition.

7. The method according to claim 1, characterized in that After updating the current life cycle in response to the life cycle policy modification operation performed by the target user based on the recommendation information, the method further includes: In response to a whitelisting operation performed by a target user on a target data table or a target data table partition, the life cycle of the target data table or the target data table partition is determined to be long-term preservation.

8. The method according to claim 1, characterized in that After determining the recommended life cycle strategy of the data table based on the storage medium type and call information of the data table, the method further includes: If the deviation between the recommended lifecycle strategy corresponding to the target data table or the target data table partition and the current lifecycle is greater than a preset deviation threshold, a first storage cost corresponding to the recommended lifecycle value and a second storage cost corresponding to the current lifecycle are calculated; Based on the target user identifier corresponding to the target data table or the target data table partition, the first storage cost, the second storage cost, and the data table partition lifecycle change information generated by adopting the lifecycle recommendation strategy compared to adopting the current lifecycle are sent.

9. A data table life cycle management device, characterized in that: The device comprises: an acquisition module, a determination module, an output module and an update module; The acquisition module is used to acquire the storage medium type and call information of the data table; the call information includes the last access date and the last update date of each partition of the data table; The determination module is used to determine a recommended life cycle strategy for the data table based on the storage medium type and call information of the data table; The output module is used to output recommendation information based on the life cycle recommendation strategy and the target user identifier corresponding to the data table; the recommendation information at least includes: storage space change information and data table partition life cycle change information generated by adopting the life cycle recommendation strategy compared with adopting the current life cycle; The updating module is used to update the current life cycle in response to a life cycle policy modification operation performed by a target user based on the recommendation information.

10. A data table life cycle management device, characterized in that: The device comprises: a memory and a processor; The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the steps of the data table life cycle management method according to any one of claims 1 to 8 according to the program code.

Images