Methods, devices, storage media and electronic equipment for identifying overdue data

Abstract

This invention provides a method, apparatus, storage medium, and electronic device for identifying expired data. The method includes: determining, based on the metadata of a database, a set of corresponding table names, partition table information, operation record information, and table name information corresponding to each preset data dimension; determining a database table as a first database table or a second database table based on the table name set, partition table information, and preset time partition identification conditions; determining the time information of each first database table based on the partition table information; determining the time information of each second database table based on the operation record information; determining whether a database table is an expired data table based on its time information; and identifying a target expired data table corresponding to each preset data dimension among the expired data tables, thereby obtaining the expired data identification result corresponding to each preset data dimension. The method of this invention can identify expired data for both partitioned and non-partitioned tables, and has a wide range of applications.

Description

Technical Field

[0001] This invention relates to the field of database technology, and in particular to a method and apparatus for identifying overdue data, a storage medium, and an electronic device. Background Technology

[0002] With the advent of the big data cloud era, data storage architecture has been evolving from data silo architecture to co-built and shared data platform architecture, and Massively Parallel Processing (MPP) databases have become one of the commonly used data storage architectures.

[0003] In the data lifecycle management process of information systems, it is necessary to identify expired data in the database and delete it promptly. Currently, the identification method for expired data in MPP databases is usually based on the structure of partitioned tables. A partitioned table is a database table formed by dividing the data of a large table into multiple smaller subsets (i.e., partitions). Data is divided based on a partition key, which consists of one or more columns on the large table and is used to determine which partition a specific row of data belongs to. Data is assigned to the corresponding partition according to the partition key value. The process of identifying expired data generally involves performing a full table scan, checking the time partition field of each data table one by one, and determining the lifecycle stage of the data based on the time partition field, thereby identifying expired data.

[0004] In real-world data storage scenarios, MPP databases contain not only partitioned tables but also non-partitioned tables, and some partitioned tables are not partitioned by time. For non-partitioned tables or partitioned tables not partitioned by time, the time partition field of the data cannot be obtained. Therefore, existing methods cannot identify expired data in these tables, resulting in insufficient data identification coverage, limited applicability, and difficulty in achieving full-lifecycle automatic management of all data. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method for identifying expired data to solve the problems that existing identification methods cannot identify expired data in non-partitioned tables, have a small data coverage, and are difficult to achieve automated lifecycle management of all data.

[0006] This invention also provides an overdue data identification device to ensure the practical implementation and application of the above method.

[0007] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0008] A method for identifying overdue data includes:

[0009] When it is necessary to identify expired data in the database, determine the metadata information and table name set corresponding to the database, wherein the table name set includes the table name of each database table in the database;

[0010] Based on the metadata information, determine the partition table information, operation record information, and table name information corresponding to each preset data dimension;

[0011] Based on the set of table names and the partition table information, it is determined whether each database table meets the preset time partition identification conditions, and the database table that meets the time partition identification conditions is determined as the first database table, and the database table that does not meet the time partition identification conditions is determined as the second database table.

[0012] Based on the partition table information, determine the time information corresponding to each of the first database tables;

[0013] Based on the operation record information, determine the time information corresponding to each of the second database tables;

[0014] For each database table, based on the time information corresponding to the database table, it is determined whether the database table meets the preset data expiration conditions. If the database table meets the data expiration conditions, the database table is determined to be an expired data table.

[0015] For each preset data dimension, based on the table name information corresponding to the preset data dimension, the target overdue data table corresponding to the preset data dimension is determined in each of the overdue data tables;

[0016] For each preset data dimension, the data of all target overdue data tables corresponding to the preset data dimension are summarized to obtain the overdue data identification result corresponding to the preset data dimension.

[0017] Optionally, in the above method, determining whether each database table meets the preset time partition identification conditions based on the table name set and the partition table information includes:

[0018] Obtain all partition table names from the partition table information;

[0019] For each of the database tables, it is determined whether a target table name exists in each of the partition table names. If the target table name exists in each of the partition table names, the database table is identified as the third database table. If the target table name does not exist in each of the partition table names, the database table is determined to not meet the time partition identification conditions. The target table name is the partition table name that matches the table name of the database table.

[0020] In the partition table information, the partition type corresponding to each of the third database tables is determined;

[0021] For each of the third database tables, if the partition type corresponding to the third database table is characterized by time partitioning, then the third database table is determined to meet the time partitioning identification condition.

[0022] Optionally, in the above method, determining the time information corresponding to each of the first database tables based on the partition table information includes:

[0023] For each of the first database tables, the time partition field corresponding to the first database table is determined in the partition table information, and the time partition field is used as the time information corresponding to the first database table.

[0024] Optionally, in the above method, determining the time information corresponding to each of the second database tables based on the operation record information includes:

[0025] For each of the second database tables, the operation record set corresponding to the second database table is determined in the operation record information. The operation record set includes the operation record of each table operation corresponding to the second database table.

[0026] For each of the second database tables, determine whether there is an operation record for data insertion in the operation record set corresponding to the second database table;

[0027] For each of the second database tables, if there is an operation record for a data insertion operation in the operation record set corresponding to the second database table, then the last data insertion operation record in the operation record set corresponding to the second database table shall be taken as the target operation record corresponding to the second database table.

[0028] For each of the second database tables, if there is no data insertion operation record in the operation record set corresponding to the second database table, then the creation operation record in the operation record set corresponding to the second database table shall be used as the target operation record corresponding to the second database table.

[0029] The operation time in the target operation record corresponding to each of the second database tables is used as the time information corresponding to each of the second database tables.

[0030] Optionally, in the above method, determining whether the database table meets the preset data expiration conditions based on the time information corresponding to the database table includes:

[0031] Determine the preset storage duration for this database table;

[0032] Based on the time information corresponding to the database table and the current time, determine the current storage duration corresponding to the database table;

[0033] Determine whether the current storage duration is greater than or equal to the preset storage duration;

[0034] If the current storage duration is greater than or equal to the preset storage duration, then the database table is determined to meet the data expiration condition.

[0035] Optionally, in the above method, determining the target overdue data table corresponding to the preset data dimension in each of the overdue data tables based on the table name information corresponding to the preset data dimension includes:

[0036] In the table name information corresponding to the preset data dimension, determine multiple dimension data corresponding to the preset data dimension and multiple table names corresponding to each dimension data;

[0037] For each table name corresponding to each dimension data, determine whether each overdue data table matches the table name, and take the overdue data table that matches the table name as the overdue data table corresponding to the dimension data;

[0038] The overdue data table corresponding to each of the aforementioned dimensions is used as the target overdue data table corresponding to that preset data dimension.

[0039] Optionally, in the above method, the step of summarizing all target overdue data tables corresponding to the preset data dimension to obtain the overdue data identification result corresponding to the preset data dimension includes:

[0040] Retrieve data from each target overdue data table corresponding to the preset data dimension;

[0041] The data of each dimension corresponding to the preset data dimension, the table name of the overdue data table corresponding to each dimension of the preset data dimension, and the data of each target overdue data table corresponding to the preset data dimension are used as the overdue identification result corresponding to the preset data dimension.

[0042] An overdue data identification device, comprising:

[0043] The first determining unit is used to determine the metadata information and table name set corresponding to the database when it is necessary to identify overdue data in the database. The table name set includes the table name of each database table in the database.

[0044] The second determining unit is used to determine the partition table information, operation record information, and table name information corresponding to each preset data dimension based on the metadata information.

[0045] The first judgment unit is used to determine whether each database table meets the preset time partition identification conditions based on the set of table names and the partition table information, and to determine the database table that meets the time partition identification conditions as the first database table, and the database table that does not meet the time partition identification conditions as the second database table.

[0046] The third determining unit is used to determine the time information corresponding to each of the first database tables based on the partition table information;

[0047] The fourth determining unit is used to determine the time information corresponding to each of the second database tables based on the operation record information;

[0048] The second judgment unit is used to determine, for each database table, whether the database table meets the preset data expiration condition based on the time information corresponding to the database table. If the database table meets the data expiration condition, the database table is determined to be an expired data table.

[0049] The fifth determining unit is used to determine the target overdue data table corresponding to each preset data dimension in each of the overdue data tables based on the table name information corresponding to the preset data dimension.

[0050] The aggregation unit is used to aggregate all target overdue data tables corresponding to each preset data dimension to obtain the overdue data identification result corresponding to the preset data dimension.

[0051] A storage medium comprising stored instructions, wherein, when the instructions are executed, the device in which the storage medium resides executes the overdue data identification method as described above.

[0052] An electronic device includes a memory and one or more instructions, wherein one or more instructions are stored in the memory and configured to be executed by one or more processors as described above for overdue data identification.

[0053] An overdue data identification method based on the above embodiments of the present invention includes: determining the metadata information and table name set corresponding to the database; determining partition table information, operation record information, and table name information corresponding to each preset data dimension based on the metadata information; determining whether each database table meets preset time partition identification conditions based on the table name set and partition table information, and identifying database tables that meet the conditions as first database tables and database tables that do not meet the conditions as second database tables; determining the time information corresponding to each first database table based on the partition table information; determining the time information corresponding to each second database table based on the operation record information; determining whether each database table meets preset data overdue conditions based on the time information corresponding to each database table, and identifying database tables that meet the conditions as overdue data tables; for each preset data dimension, determining the target overdue data table corresponding to the preset data dimension in the overdue data table based on the table name information corresponding to the preset data dimension; and summarizing the data of all target overdue data tables corresponding to each preset data dimension to obtain the overdue data identification result corresponding to each preset data dimension. Applying the method provided in this embodiment of the invention, time information can be obtained from partition table information for time-partitioned database tables, while time information can be obtained from operation record information for other database tables. The lifecycle of this type of database table is measured by operation time. Overdue data can be identified for all database tables, offering broad coverage and wide applicability, which is beneficial for achieving automated lifecycle management of all data and reducing manual maintenance time. Furthermore, the overdue data identification results are summarized according to different preset data dimensions, which helps meet the data analysis needs of different business areas. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0055] Figure 1 A flowchart of a method for identifying overdue data provided in an embodiment of the present invention;

[0056] Figure 2 This is an architecture diagram of an overdue data identification system provided in an embodiment of the present invention;

[0057] Figure 3 An example diagram illustrating an overdue data identification process provided in an embodiment of the present invention;

[0058] Figure 4This is another example diagram illustrating an overdue data identification process provided in an embodiment of the present invention;

[0059] Figure 5 Another example diagram illustrating an overdue data identification process provided in an embodiment of the present invention;

[0060] Figure 6 This is a schematic diagram of the structure of an overdue data identification device provided in an embodiment of the present invention;

[0061] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0062] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] In this application, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0064] As the background technology indicates, MPP databases are widely used in enterprise-level data lakes, data warehouse systems, and other data processing fields. MPP databases possess high performance, high availability, and high scalability. However, current analysis of expired data in MPP databases cannot quickly and accurately parse the lifecycle information of massive amounts of data, making it difficult to meet the needs of data storage management. Secondly, existing management strategies for expired data identification results analyze and summarize expired results at the table level, which can be used to adjust the storage lifecycle strategy of a single table. However, in actual application scenarios, different business domains involve a large number of corresponding tables. When managing the data lifecycle, the operations and maintenance personnel of each business domain still need to find the results they need from the identification results, meaning the identification results cannot directly meet the needs of each business domain.

[0065] Therefore, this invention provides a method for identifying overdue data. By combining partition table information and operation record information, the lifecycle of a database table can be identified. This method can identify overdue data for all database tables, has a wide data coverage, and provides statistical identification results from multiple dimensions, which is beneficial for meeting the needs of different business areas.

[0066] This invention provides a method for identifying overdue data. The method can be applied to an overdue data identification system, and its execution entity can be the system's server. The method flowchart is shown below. Figure 1 As shown, it includes:

[0067] S101: When it is necessary to identify overdue data in the database, determine the metadata information and table name set corresponding to the database, wherein the table name set includes the table name of each database table in the database;

[0068] The database mentioned in the method provided in this invention is an MPP database, namely a massively parallel processing database. In a shared-database cluster, each node has an independent disk storage system and memory system. Business data is divided among the nodes according to the database model and application characteristics. Each data node is interconnected through a dedicated network or a commercial general network, and they cooperate to perform computations, providing database services as a whole.

[0069] The method provided in this invention can periodically identify overdue data in the database, such as identifying it at a certain time on the 15th of each month or at a certain time every Monday. When overdue data identification is required, the metadata information of the database can be obtained. The metadata information includes various types of metadata of the database, specifically the required metadata tables, such as metadata tables recording the summary of each table in the database, metadata tables recording the summary of each partition table, metadata tables recording the inheritance hierarchy of tables, metadata tables recording the operation records of tables in the database, etc. Metadata refers to data that describes database data and can be used to describe the structured data attribute information. For example, technical metadata is used to describe the technical details and processing rules of data entities and data processing processes, such as table structure, ETL mapping relationships, etc. A partition table refers to a database table created and stored in the form of partitions.

[0070] In the method provided by this invention, the table name of each database table in the database can be obtained based on the metadata recording the table name in the metadata information, resulting in a set of table names. The database table refers to a table whose lifecycle needs to be identified, i.e., whether it has expired.

[0071] S102: Based on the metadata information, determine the partition table information, operation record information, and table name information corresponding to each preset data dimension;

[0072] In the method provided by this invention, partition table information can be determined based on the metadata describing the partition tables in the database. The partition table information includes a summary of each partition table in the database, such as table name, partition type, etc. Operation record information can be determined based on the metadata describing database operation records. The operation record information includes all operation records for each database table in the database, such as operation type, operation time, etc. During the process of determining various information based on metadata information, standard formatting and other operations can be performed on the data extracted from the metadata information as needed.

[0073] In the method provided by this invention, multiple data dimensions can be pre-set according to business needs. These pre-set data dimensions can be user dimensions, tablespace dimensions, or other custom dimensions. The term "user" refers to the role managing MPP database access permissions. A role can own database objects (including tables, indexes, etc.) and can also assign permissions on these objects to other roles. A user is a role with login permissions. A tablespace refers to a directory in the file system that describes the physical file storage location of MPP database objects (including tables, indexes, etc.).

[0074] In the method provided by this invention, the table name information corresponding to each preset data dimension can be determined based on the metadata describing the summary of the database tables in the metadata information. That is, the database tables are classified according to the preset data dimensions, and the table name information corresponding to each preset data dimension includes the table names of each database table corresponding to all objects under that preset data dimension. For example, under the user dimension, the table name information includes the table names of all tables created by each user.

[0075] S103: Based on the set of table names and the partition table information, determine whether each database table meets the preset time partition identification conditions, and determine the database table that meets the time partition identification conditions as the first database table, and the database table that does not meet the time partition identification conditions as the second database table.

[0076] In the method provided by this invention, time partition identification conditions can be preset to identify partitioned tables that are partitioned by time. Database tables that meet these conditions indicate that they contain corresponding time partition fields, representing the time period from which the data in the database table belongs. Based on the table name set and partitioned table information, it can be determined whether each database table meets the preset time partition identification conditions, that is, whether each database table is a time-partitioned table. For each database table, if it meets the preset time partition identification conditions, it is designated as the first database table; otherwise, it is designated as the second database table.

[0077] S104: Based on the partition table information, determine the time information corresponding to each of the first database tables;

[0078] In the method provided by this embodiment of the invention, for each first database table, the time partition field corresponding to the database table can be obtained from the partition table information. The time partition field is a field that characterizes the time interval or time point to which the data belongs. The life cycle of the database table is measured by this time data, and the time data is used as the time information corresponding to the database table. The first database table is a database table that meets the time partition identification conditions, that is, the first database table is a partition table partitioned by time, and the partition table information records its corresponding partition information.

[0079] S105: Based on the operation record information, determine the time information corresponding to each of the second database tables;

[0080] In the method provided by this embodiment of the invention, for each second database table, the operation time field corresponding to the database table is obtained from the operation record information. The operation time field is a field that represents the time when a certain operation is performed on the database table. Specifically, it can refer to the operation time of the operation that created the table, or the operation time of the last data insertion operation on the table, etc. The life cycle of the database table is measured by the operation time data, which is used as its corresponding time information.

[0081] S106: For each database table, based on the time information corresponding to the database table, determine whether the database table meets the preset data expiration conditions. If the database table meets the data expiration conditions, then the database table is determined to be an expired data table.

[0082] In the method provided by this invention, data expiration conditions can be set according to actual needs. This involves assessing whether the lifecycle of a database table has reached a stage requiring destruction. Database tables meeting certain conditions are considered expired data. Specific conditions may include data storage duration exceeding a predetermined duration. Based on the time data in the time information corresponding to each database table (i.e., each first database table and each second database table) and the preset data expiration conditions, it is possible to assess whether the lifecycle of each database table is in a stage requiring data destruction. If a database table's lifecycle is in a stage requiring data destruction, then that database table is determined to be an expired data table, i.e., considered expired data.

[0083] S107: For each preset data dimension, based on the table name information corresponding to the preset data dimension, determine the target overdue data table corresponding to the preset data dimension in each of the overdue data tables;

[0084] In the method provided by the embodiments of the present invention, for each preset data dimension, the overdue data table associated with it is matched in each overdue data table through the table name contained in its corresponding table name information, and used as the target overdue data table corresponding to the preset data dimension.

[0085] It should be noted that in the specific implementation process, there may be multiple target overdue data tables corresponding to the preset data dimensions, there may be only one, or there may be no corresponding target overdue data table, depending on the specific identification and matching situation. This does not affect the implementation function of the method provided by the embodiments of the present invention. If there is no corresponding target overdue data table under a certain preset data dimension, then its overdue data identification result is no overdue data.

[0086] S108: For each preset data dimension, summarize the data of all target overdue data tables corresponding to the preset data dimension to obtain the overdue data identification result corresponding to the preset data dimension.

[0087] In the method provided by this invention, the data in the target overdue data table under each preset data dimension is summarized, and the summary result under each preset data dimension is used as the overdue data identification result for that preset data dimension, thus ending the identification process. The overdue data identification result for each preset data dimension is summarized by dimension data. For example, in the overdue data identification result for the user dimension, the usernames of all users can be output, along with the corresponding overdue data (i.e., the data in the target overdue data table associated with that user). The overdue data identification results corresponding to each preset data dimension can be stored in the system for querying by maintenance personnel. Alternatively, the result recipients for each preset data dimension can be pre-set, and the overdue data identification results for each preset data dimension can be sent to their corresponding result recipients via email or other means.

[0088] Based on the method provided in this embodiment of the invention, when it is necessary to identify overdue data in a database, the following steps are taken: First, determine the metadata information and table name set corresponding to the database. The table name set includes the table name of each database table in the database. Based on the metadata information, determine the partition table information, operation record information, and table name information corresponding to each preset data dimension. Based on the table name set and partition table information, determine whether each database table meets the preset time partition identification conditions, and identify the database tables that meet the conditions as first database tables and the database tables that do not meet the conditions as second database tables. Based on the partition table information, determine the time information corresponding to each first database table. Based on the operation record information, determine the time information corresponding to each second database table. Based on the time information corresponding to each database table, determine whether each database table meets the preset data overdue conditions, and identify the database tables that meet the conditions as overdue data tables. Based on the table name information corresponding to each preset data dimension, determine the target overdue data table corresponding to each preset data dimension among the overdue data tables. Finally, summarize the data of all target overdue data tables corresponding to each preset data dimension to obtain the overdue data identification result corresponding to each preset data dimension. Applying the method provided in this invention, for time-partitioned database tables, time information for measuring data lifecycle can be obtained through partition table information. For other database tables, time information can be obtained through operation record information. The lifecycle of this type of database table is measured by operation time. Overdue data can be identified for all database tables, offering broad coverage and wide applicability, which is beneficial for achieving automated lifecycle management of all data and reducing manual maintenance time. Secondly, the overdue data identification results are summarized according to different preset data dimensions, which helps meet the data analysis needs of different business areas. Furthermore, during the identification process, there is no need to perform a full table scan for each database table, saving resources and time, improving efficiency, and ensuring the normal use of the database.

[0089] exist Figure 1 Based on the method shown, the method provided in this embodiment of the invention, in step S103, the process of determining whether each database table meets the preset time partition identification conditions based on the set of table names and the partition table information, includes:

[0090] Obtain all partition table names from the partition table information;

[0091] In the method provided by this embodiment of the invention, the table names of all partition tables in the database are extracted from the partition table information to obtain the table names of all partition tables.

[0092] For each of the database tables, it is determined whether a target table name exists in each of the partition table names. If the target table name exists in each of the partition table names, the database table is identified as the third database table. If the target table name does not exist in each of the partition table names, the database table is determined to not meet the time partition identification conditions. The target table name is the partition table name that matches the table name of the database table.

[0093] In the method provided by this embodiment of the invention, the name of each database table is compared with the name of each partition table. For each database table, if there is a partition table with the same name as the database table, then the database table is identified as the third database table, that is, the database table is a partition table. If the name of each partition table is different from the name of the database table being compared, then the current database table is not a partition table, and it is determined that the current database table does not meet the preset time partition identification conditions.

[0094] In the partition table information, the partition type corresponding to each of the third database tables is determined;

[0095] In the method provided by this embodiment of the invention, the partition table information records the partition types of all partition tables. The partition type refers to the basis for dividing the data in the partition table, such as partitioning by time, partitioning by business, etc. For each third database table, the partition type corresponding to the table name of that database table can be found in the partition table information and used as the partition type corresponding to that database table.

[0096] For each of the third database tables, if the partition type corresponding to the third database table is characterized by time partitioning, then the third database table is determined to meet the time partitioning identification condition.

[0097] In the method provided by this invention, if the partition type of the partition table is characterized by time partitioning, it indicates that the partition table is a partition table created by partitioning by time, and the partition table records data generated within a certain time period, which is recorded in the partition table information. Therefore, if the partition type corresponding to the third database table is characterized by time partitioning, it is determined that the database table meets the preset time partitioning identification conditions. If the partition type corresponding to the third database table is not characterized by time partitioning, it can be determined that it does not meet the time partitioning identification conditions.

[0098] exist Figure 1 Based on the method shown, the method provided in this embodiment of the invention, in step S104, the process of determining the time information corresponding to each of the first database tables based on the partition table information, includes:

[0099] For each of the first database tables, the time partition field corresponding to the first database table is determined in the partition table information, and the time partition field is used as the time information corresponding to the first database table.

[0100] In the method provided by this embodiment of the invention, the partition table information records the time partition fields of all partition tables that are partitioned by time, that is, the fields indicating the time period to which the data of the corresponding partition table belongs. For each first database table, the time partition field corresponding to the table name can be found in the partition table information through the table name of the database table, and this time partition field is the time information corresponding to the database table.

[0101] exist Figure 1 Based on the method shown, the process of determining the time information corresponding to each of the second database tables according to the operation record information mentioned in step S105 of the method provided in this embodiment of the invention includes:

[0102] For each of the second database tables, the operation record set corresponding to the second database table is determined in the operation record information. The operation record set includes the operation record of each table operation corresponding to the second database table.

[0103] In the method provided by this embodiment of the invention, the operation record information records the operation records for each table of each database table. For each second database table, the operation record information can be searched to find the operation record that matches the table name, and all operation records that match the table name are combined into an operation record set corresponding to that database table. The operation record set includes operation records such as those corresponding to table creation operations, data insertion operations on the table, etc.

[0104] For each of the second database tables, determine whether there is an operation record for data insertion in the operation record set corresponding to the second database table;

[0105] For each of the second database tables, if there is an operation record for a data insertion operation in the operation record set corresponding to the second database table, then the last data insertion operation record in the operation record set corresponding to the second database table shall be taken as the target operation record corresponding to the second database table.

[0106] For each of the second database tables, if there is no data insertion operation record in the operation record set corresponding to the second database table, then the creation operation record in the operation record set corresponding to the second database table shall be used as the target operation record corresponding to the second database table.

[0107] The operation time in the target operation record corresponding to each of the second database tables is used as the time information corresponding to each of the second database tables.

[0108] In the method provided by this invention, the existence of a data insertion operation record in each operation record set can be determined based on the operation type of the operation records in each operation record set. That is, whether a data insertion operation exists among the various table operations performed on the database table. If a data insertion operation has been performed on the second database table, the operation record of the last data insertion operation is taken as the target operation record corresponding to that database table. The last data insertion operation refers to the data insertion operation with the latest operation time. If only one data insertion operation has been performed, then that data insertion operation is the last data insertion operation.

[0109] In the method provided by this embodiment of the invention, for a second database table, if no data insertion operation has been performed on it, the operation record of the creation operation corresponding to that database table is used as its corresponding target operation record. The creation operation refers to the operation of creating a table, and the operation time of this operation is the table creation time. For each second database table, the operation time in its corresponding target operation record is its corresponding time information.

[0110] exist Figure 1 Based on the method shown, in the method provided by this embodiment of the invention, the process of determining whether the database table meets the preset data expiration conditions based on the time information corresponding to the database table mentioned in step S106 includes:

[0111] Determine the preset storage duration for this database table;

[0112] In the method provided by this invention, the storage duration for each type of data can be preset according to actual needs. This can be done by setting a single, uniform duration or by setting different durations for each type of data. During the process of determining whether each database table meets the preset data expiration conditions, the preset storage duration corresponding to the current database table can be determined from the pre-configured storage time information.

[0113] Based on the time information corresponding to the database table and the current time, determine the current storage duration corresponding to the database table;

[0114] In the method provided by this embodiment of the invention, the time data in the time information corresponding to the current database table is used as the starting time point, and the current time is used as the ending time point. The time length between the ending time point and the starting time point is calculated, and this time length is used as the current storage duration corresponding to the current database table. If the time data contained in the time information is a time period, a certain time point in the time period can be selected as the starting time point. The selection strategy can be preset, for example, it can be set to select the beginning time point of the time period as the starting time point, or it can be set to select the end time point as the starting time point, etc.

[0115] Determine whether the current storage duration is greater than or equal to the preset storage duration;

[0116] If the current storage duration is greater than or equal to the preset storage duration, then the database table is determined to meet the data expiration condition.

[0117] In the method provided by this embodiment of the invention, the current storage duration corresponding to the current database table is compared with its corresponding preset storage duration. If the current storage duration is greater than or equal to the preset storage duration, the current database table is determined to meet the preset data expiration condition. If the current storage duration is less than the preset storage duration, the current database table is determined to not meet the preset data expiration condition.

[0118] exist Figure 1 Based on the method shown, in the method provided by the embodiments of the present invention, the process of determining the target overdue data table corresponding to the preset data dimension in each of the overdue data tables according to the table name information corresponding to the preset data dimension mentioned in step S107 includes:

[0119] In the table name information corresponding to the preset data dimension, determine multiple dimension data corresponding to the preset data dimension and multiple table names corresponding to each dimension data;

[0120] In the method provided by this invention, during the process of determining the target overdue data table corresponding to each preset data dimension, all dimension data corresponding to the current preset data dimension and all table names corresponding to each dimension data can be obtained from the table name information corresponding to the current preset data dimension. Dimension data refers to the objects under the preset data dimension. For example, under the user dimension, the corresponding dimension data are each user (specifically represented by username), and the table name corresponding to the dimension data is the table name of the table created by the user. Similarly, under the tablespace dimension, the corresponding dimension data are each tablespace, and the table name corresponding to the dimension data is the table name of the table belonging to the corresponding tablespace.

[0121] It should be noted that in the specific implementation process, the preset data dimension can be the dimension of a table. In this scenario, it can be understood that it corresponds to only one dimension data object, namely the database table, and the table names corresponding to this dimension data are the table names of all database tables.

[0122] For each table name corresponding to each dimension data, determine whether each overdue data table matches the table name, and take the overdue data table that matches the table name as the overdue data table corresponding to the dimension data;

[0123] The overdue data table corresponding to each of the aforementioned dimensions is used as the target overdue data table corresponding to that preset data dimension.

[0124] In the method provided by this embodiment of the invention, each table name corresponding to each dimension data can be compared with the table name of each overdue data table. If the table name of the overdue data table compared with the current table name is the same as the current table name, then the overdue data table is considered to match the current table name, and the overdue data table is taken as the overdue data table corresponding to the dimension data corresponding to the current table name, and is also the target overdue data table corresponding to the preset data dimension corresponding to the dimension data.

[0125] Based on the method provided in the above embodiments, the method provided in this embodiment of the invention includes the process of summarizing all target overdue data tables corresponding to the preset data dimension in step S108 to obtain the overdue data identification result corresponding to the preset data dimension, comprising:

[0126] Retrieve data from each target overdue data table corresponding to the preset data dimension;

[0127] The data of each dimension corresponding to the preset data dimension, the table name of the overdue data table corresponding to each dimension of the preset data dimension, and the data of each target overdue data table corresponding to the preset data dimension are used as the overdue identification result corresponding to the preset data dimension.

[0128] In the method provided by this invention, during the process of determining the overdue data identification result corresponding to each preset data dimension, data of each target overdue data table corresponding to the current preset data dimension can be read from the database. The data of the target overdue data tables are summarized in a form where dimension data is associated with the preset data dimension, and each dimension data is associated with its corresponding overdue data table. The overdue identification result corresponding to the current preset data dimension is then output, and these table data represent the overdue data under that dimension. For example, the overdue identification result for the table dimension outputs the table name of the overdue data table in the database and the data under that table. The overdue identification result for the user dimension outputs all users in the database, the table names of the overdue data tables created under each user, and the data under each table. The overdue identification result for the tablespace dimension outputs all tablespaces in the database, the table names of the overdue data tables created under each tablespace, and the data under each table.

[0129] To better illustrate the method provided in this embodiment of the invention, another method for identifying overdue data is presented below, in conjunction with a practical application scenario. The method provided in this embodiment can be applied to a bank's data center scenario, with the identification object being an MPP database. Python is chosen as the programming language for overdue data identification, and a series of automated tools are coded to complete data initialization, overdue identification, and periodic inspection.

[0130] The method provided in this embodiment of the invention can be implemented through an overdue data identification system, which is for... Figure 1 An instantiation of the method shown, such as Figure 2 The system architecture shown primarily includes an overdue data identification initialization module, an overdue data identification analysis module, and an overdue data identification aggregation module. The overdue data identification process provided in this embodiment mainly involves the following six metadata tables of the MPP database:

[0131] pg_tables: Provides access to useful information about each table in the database, including table name, schema name, and tablespace.

[0132] pg_partitions: Provides access to useful information about each partition table in the database, including partition type, partition range, etc.

[0133] pg_inherits: Provides access to useful information about the table inheritance hierarchy in the database, including child table names, parent table names, etc.

[0134] pg_class: Provides access to useful information about tables, sequences, and views in the database, including table name, data type, access permissions, etc.

[0135] pg_namespace: Provides access to useful information about database tablespaces, including the tablespace name, the user to which the tablespace belongs, etc.

[0136] pg_stat_last_operation: Provides access to useful information about database table operation records, including operation type, operation time, etc.

[0137] Here, a schema refers to a collection of tables. A schema can contain views, indexes, data types, functions, and operators.

[0138] The overdue data identification process provided in this embodiment of the invention identifies and summarizes overdue data at the user, tablespace, and custom dimensions. The identification process is mainly implemented through the overdue data identification initialization module, the overdue data identification analysis module, and the overdue data identification summarization module. The workflow of each module will be described below.

[0139] Overdue data identification initialization module;

[0140] The overdue data identification and initialization module is primarily responsible for preprocessing six MPP metadata tables, including: obtaining table lists under different dimensions, obtaining table partition information, obtaining table operation records, and data formatting. Based on the metadata information, it can automatically obtain table lists under dimensions such as tablespace and user, and can customize the mapping relationship between different tables and business domains according to user needs to achieve multi-dimensional overdue data result aggregation. Since the table data comes from different business scenarios, and their data types and formats vary, this module will perform standard formatting processing on the data so that subsequent modules can process it uniformly.

[0141] like Figure 3 The flowchart shown illustrates the initialization process for identifying and handling overdue data, which mainly includes:

[0142] Retrieve metadata;

[0143] Preprocessing is performed on the metadata to extract information, resulting in a list of tables under different dimensions, detailed information on partitioned tables, and detailed information on operation records;

[0144] The extracted information data is formatted.

[0145] In the method provided by this invention, by combining MPP metadata, table names can be automatically mapped to objects such as the user who created the table, the tablespace to which it belongs, and the project to which it belongs. The table list can be multi-dimensional data, such as a user-dimensional table list, where the first level structure is the user, and the second level is all the table names owned by a certain user.

[0146] In the method provided by this invention, formatting mainly refers to converting data extracted from different metadata tables, which are expressed in different forms, into the same form. For example, when parsing the pg_partitions metadata table, the processing performed on certain fields extracted from it includes data type and data format, such as truncating or converting the case of a string in a certain field, and storing the date uniformly in "year-month-day".

[0147] The main functionalities involved in this module are described in the table below:

[0148] Table 1

[0149]

[0150]

[0151] The table lists under different dimensions can all be parsed from the metadata table `pg_tables` (which stores basic information about all tables in the database, including table names, users, tablespaces, etc.). The table list under the tablespace (user) dimension refers to all available tablespace (user) names, as well as all table names under a specific tablespace (user). Custom dimensions are dimensions set by operations personnel according to business needs, such as personalized input of combinations of tablespaces and users. For example, in a human resources system, if 10 users and 2 tablespaces are created in the database, these 10 users and 2 tablespaces can be specified as a single entity. The output will then be the table list under the "Human Resources System" dimension, i.e., all table names created by this system. Similarly, users and tablespaces of a provincial branch can be treated as a single entity to obtain the overdue data status of that provincial branch. Users and tablespaces in a specific business domain, such as the "foreign exchange" scenario, can also be treated as a single entity to obtain the overdue data status of the "foreign exchange" domain, facilitating adjustments and optimizations of big data product data lifecycle strategies by management.

[0152] In the method provided by this invention, the table partition information is obtained by parsing `pg_partitions` and `pg_inherits`. The detailed operation record information of the table is obtained by parsing three metadata tables: `pg_stat_last_operation`, `pg_class`, and `pg_namespace`. The table partition information includes partition information for all tables, including whether the table is a partitioned table and detailed information about the partitioned table, such as the partition sub-table name, partition fields, and partition type. The detailed operation record information of the table includes operation records for all tables.

[0153] Overdue data identification and analysis module;

[0154] The overdue data identification and analysis module is primarily responsible for building an identification model and analyzing the initialized data. This module automatically identifies whether a table is a time-partitioned table. For time-partitioned tables, it automatically extracts the time information by parsing the partition information in the metadata. For non-partitioned tables and non-time-partitioned tables, it automatically extracts the time information by parsing the table operation record information in the metadata.

[0155] like Figure 4 The flowchart shown illustrates the main steps in identifying and analyzing overdue data:

[0156] Determine the table type: Based on the formatted data processed in the previous module, determine whether each table is a time-partitioned table;

[0157] Identify partitioned tables (partitioned by time): If the table is a partitioned table partitioned by time, obtain the partition time information of the table based on the detailed information of all partitioned tables;

[0158] Identify non-partitioned tables and partitioned tables not partitioned by time: If the table is a non-partitioned table / a non-partitioned table not partitioned by time, then obtain the operation record time information of the table based on the table's operation record information;

[0159] Retrieve Overdue Data from Tables (Overdue Result Analysis): Determine whether a data point has expired based on the table's partition / operation record time information, and obtain the overdue analysis results.

[0160] The main functionalities involved in this module are described in the table below:

[0161] Table 2

[0162]

[0163] Overdue data identification and aggregation module;

[0164] The overdue data identification and aggregation module is primarily responsible for outputting the identification results and summarizing them from multiple dimensions, including: table dimension results aggregation, user dimension results aggregation, tablespace dimension results aggregation, and custom dimension results aggregation. For example... Figure 5 The flowchart shown illustrates the process of identifying, summarizing, and processing overdue data, which mainly includes:

[0165] The results of the overdue analysis are summarized in multiple dimensions to obtain summary results for table dimensions, user dimensions, tablespace dimensions, and custom dimensions.

[0166] The main functionalities involved in this module are described in the table below:

[0167] Table 3

[0168]

[0169] The method provided in this invention supports multi-dimensional aggregation of overdue data identification results. Based on the mapping relationship, different tables in the same business domain can be processed and analyzed as a set to meet enterprise-level data lifecycle management needs.

[0170] The method provided in this invention is based on offline processing of backup metadata. The identification process does not require scanning the actual business data tables. Production metadata can be backed up and exported, and the overdue identification algorithm can be run offline on the batch processing server to parse the backup metadata. Running the identification algorithm offline isolates the identification process from the production database, which not only improves identification efficiency but also does not increase the load on the production operation of the MPP database, thus achieving rapid and accurate identification of overdue data in the MPP database.

[0171] The method provided in this invention obtains time information through table operation records. Since non-partitioned tables themselves do not contain time information, this solution obtains time information by parsing table operation record information (including operation user, operation type, operation time, etc.) in metadata such as `pg_stat_last_operation`, ultimately achieving the identification of expired data in non-partitioned tables. By parsing metadata, partition fields can be automatically identified for partitioned tables, and time information can be obtained based on the partition information. For non-partitioned tables, time information can be obtained based on operation records. Newly added business tables to be identified can be seamlessly extended for identification.

[0172] The method provided in this invention can summarize and analyze results from multiple dimensions. By utilizing the mapping between tablespaces, users, and tables, it can automatically summarize the identification results of overdue data from different dimensions. It can also customize the correspondence between business domains and tables according to actual needs to obtain summary results of custom dimensions, thus meeting the needs of enterprise-level data lifecycle management.

[0173] and Figure 1 Corresponding to the overdue data identification method shown, this embodiment of the invention also provides an overdue data identification device for identifying overdue data. Figure 1 The specific implementation of the method shown is illustrated in the following diagram. Figure 6 As shown, it includes:

[0174] The first determining unit 201 is used to determine the metadata information and table name set corresponding to the database when it is necessary to identify overdue data in the database. The table name set includes the table name of each database table in the database.

[0175] The second determining unit 202 is used to determine the partition table information, operation record information, and table name information corresponding to each preset data dimension based on the metadata information.

[0176] The first judgment unit 203 is used to determine whether each database table meets the preset time partition identification conditions based on the set of table names and the partition table information, and to determine the database table that meets the time partition identification conditions as the first database table, and the database table that does not meet the time partition identification conditions as the second database table.

[0177] The third determining unit 204 is used to determine the time information corresponding to each of the first database tables based on the partition table information;

[0178] The fourth determining unit 205 is used to determine the time information corresponding to each of the second database tables based on the operation record information;

[0179] The second judgment unit 206 is used to determine, for each database table, whether the database table meets the preset data expiration condition based on the time information corresponding to the database table. If the database table meets the data expiration condition, the database table is determined to be an expired data table.

[0180] The fifth determining unit 207 is used to determine the target overdue data table corresponding to each preset data dimension in each of the overdue data tables based on the table name information corresponding to the preset data dimension.

[0181] The aggregation unit 208 is used to aggregate all target overdue data tables corresponding to each preset data dimension to obtain the overdue data identification result corresponding to the preset data dimension.

[0182] Using the apparatus provided in this invention, time-partitioned database tables can obtain time information for measuring data lifecycle through partition table information, while other database tables can obtain time information through operation record information. The lifecycle of this type of database table is measured by operation time. Overdue data can be identified for all database tables, offering broad coverage and wide applicability, which is beneficial for automating the lifecycle management of all data and reducing manual maintenance time. Secondly, the overdue data identification results are summarized according to different preset data dimensions, which helps meet the data analysis needs of different business areas. Furthermore, during the identification process, there is no need to perform a full table scan on each database table, saving resources and time, improving efficiency, and ensuring the normal use of the database.

[0183] exist Figure 6 Based on the device shown, the device provided in this embodiment of the invention can be further extended to include multiple units. The functions of each unit can be found in the descriptions of the various embodiments of the overdue data identification method provided above, and will not be further illustrated here.

[0184] This invention also provides a storage medium, which includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to perform the overdue data identification method as described above.

[0185] This invention also provides an electronic device, the structural schematic of which is shown below. Figure 7 As shown, it specifically includes a memory 301 and one or more instructions 302, wherein one or more instructions 302 are stored in the memory 301 and configured to be executed by one or more processors 303 to perform the following operations:

[0186] When it is necessary to identify expired data in the database, determine the metadata information and table name set corresponding to the database, wherein the table name set includes the table name of each database table in the database;

[0187] Based on the metadata information, determine the partition table information, operation record information, and table name information corresponding to each preset data dimension;

[0188] Based on the set of table names and the partition table information, it is determined whether each database table meets the preset time partition identification conditions, and the database table that meets the time partition identification conditions is determined as the first database table, and the database table that does not meet the time partition identification conditions is determined as the second database table.

[0189] Based on the partition table information, determine the time information corresponding to each of the first database tables;

[0190] Based on the operation record information, determine the time information corresponding to each of the second database tables;

[0191] For each database table, based on the time information corresponding to the database table, it is determined whether the database table meets the preset data expiration conditions. If the database table meets the data expiration conditions, the database table is determined to be an expired data table.

[0192] For each preset data dimension, based on the table name information corresponding to the preset data dimension, the target overdue data table corresponding to the preset data dimension is determined in each of the overdue data tables;

[0193] For each preset data dimension, the data of all target overdue data tables corresponding to the preset data dimension are summarized to obtain the overdue data identification result corresponding to the preset data dimension.

[0194] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0195] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0196] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for identifying overdue data, characterized in that, include: When it is necessary to identify expired data in the database, determine the metadata information and table name set corresponding to the database, wherein the table name set includes the table name of each database table in the database; Based on the metadata information, determine the partition table information, operation record information, and table name information corresponding to each preset data dimension; Based on the set of table names and the partition table information, it is determined whether each database table meets the preset time partition identification conditions, and the database table that meets the time partition identification conditions is determined as the first database table, and the database table that does not meet the time partition identification conditions is determined as the second database table. Based on the partition table information, determine the time information corresponding to each of the first database tables; Based on the operation record information, determine the time information corresponding to each of the second database tables; For each database table, based on the time information corresponding to the database table, it is determined whether the database table meets the preset data expiration conditions. If the database table meets the data expiration conditions, the database table is determined to be an expired data table. For each preset data dimension, based on the table name information corresponding to the preset data dimension, the target overdue data table corresponding to the preset data dimension is determined in each of the overdue data tables; For each preset data dimension, the data of all target overdue data tables corresponding to the preset data dimension are summarized to obtain the overdue data identification result corresponding to the preset data dimension. The step of determining the time information corresponding to each of the second database tables based on the operation record information includes: For each of the second database tables, the operation record set corresponding to the second database table is determined in the operation record information. The operation record set includes the operation record of each table operation corresponding to the second database table. For each of the second database tables, determine whether there is an operation record for data insertion in the operation record set corresponding to the second database table; For each of the second database tables, if there is an operation record for a data insertion operation in the operation record set corresponding to the second database table, then the last data insertion operation record in the operation record set corresponding to the second database table shall be taken as the target operation record corresponding to the second database table. For each of the second database tables, if there is no data insertion operation record in the operation record set corresponding to the second database table, then the creation operation record in the operation record set corresponding to the second database table shall be used as the target operation record corresponding to the second database table. The operation time in the target operation record corresponding to each of the second database tables is used as the time information corresponding to each of the second database tables.

2. The method according to claim 1, characterized in that, The step of determining whether each database table meets the preset time partition identification conditions based on the set of table names and the partition table information includes: Obtain all partition table names from the partition table information; For each of the database tables, it is determined whether a target table name exists in each of the partition table names. If the target table name exists in each of the partition table names, the database table is identified as the third database table. If the target table name does not exist in each of the partition table names, the database table is determined to not meet the time partition identification conditions. The target table name is the partition table name that matches the table name of the database table. In the partition table information, the partition type corresponding to each of the third database tables is determined; For each of the third database tables, if the partition type corresponding to the third database table is characterized by time partitioning, then the third database table is determined to meet the time partitioning identification condition.

3. The method according to claim 1, characterized in that, The step of determining the time information corresponding to each of the first database tables based on the partition table information includes: For each of the first database tables, the time partition field corresponding to the first database table is determined in the partition table information, and the time partition field is used as the time information corresponding to the first database table.

4. The method according to claim 1, characterized in that, The step of determining whether a database table meets preset data expiration conditions based on the time information corresponding to the database table includes: Determine the preset storage duration for this database table; Based on the time information corresponding to the database table and the current time, determine the current storage duration corresponding to the database table; Determine whether the current storage duration is greater than or equal to the preset storage duration; If the current storage duration is greater than or equal to the preset storage duration, then the database table is determined to meet the data expiration condition.

5. The method according to claim 1, characterized in that, The step of determining the target overdue data table corresponding to the preset data dimension in each of the overdue data tables based on the table name information corresponding to the preset data dimension includes: In the table name information corresponding to the preset data dimension, determine multiple dimension data corresponding to the preset data dimension and multiple table names corresponding to each dimension data; For each table name corresponding to each dimension data, determine whether each overdue data table matches the table name, and take the overdue data table that matches the table name as the overdue data table corresponding to the dimension data; The overdue data table corresponding to each of the aforementioned dimensions is used as the target overdue data table corresponding to that preset data dimension.

6. The method according to claim 5, characterized in that, The step of summarizing all target overdue data tables corresponding to the preset data dimension to obtain the overdue data identification result corresponding to the preset data dimension includes: Retrieve data from each target overdue data table corresponding to the preset data dimension; The data of each dimension corresponding to the preset data dimension, the table name of the overdue data table corresponding to each dimension of the preset data dimension, and the data of each target overdue data table corresponding to the preset data dimension are used as the overdue identification result corresponding to the preset data dimension.

7. An overdue data identification device, characterized in that, include: The first determining unit is used to determine the metadata information and table name set corresponding to the database when it is necessary to identify overdue data in the database. The table name set includes the table name of each database table in the database. The second determining unit is used to determine the partition table information, operation record information, and table name information corresponding to each preset data dimension based on the metadata information. The first judgment unit is used to determine whether each database table meets the preset time partition identification conditions based on the set of table names and the partition table information, and to determine the database table that meets the time partition identification conditions as the first database table, and the database table that does not meet the time partition identification conditions as the second database table. The third determining unit is used to determine the time information corresponding to each of the first database tables based on the partition table information; The fourth determining unit is used to determine the time information corresponding to each of the second database tables based on the operation record information; The second judgment unit is used to determine, for each database table, whether the database table meets the preset data expiration condition based on the time information corresponding to the database table. If the database table meets the data expiration condition, the database table is determined to be an expired data table. The fifth determining unit is used to determine the target overdue data table corresponding to each preset data dimension in each of the overdue data tables based on the table name information corresponding to the preset data dimension. The aggregation unit is used to aggregate all target overdue data tables corresponding to each preset data dimension to obtain the overdue data identification result corresponding to the preset data dimension. The fourth determining unit is specifically configured as follows: For each of the second database tables, the operation record set corresponding to the second database table is determined in the operation record information. This operation record set includes the operation records for each table operation corresponding to the second database table. For each of the second database tables, it is determined whether there is an operation record for a data insertion operation in the operation record set corresponding to the second database table. For each of the second database tables, if there is an operation record for a data insertion operation in the operation record set corresponding to the second database table, the last data insertion operation record in the operation record set corresponding to the second database table is taken as the target operation record for the second database table. For each of the second database tables, if there is no operation record for a data insertion operation in the operation record set corresponding to the second database table, the creation operation record in the operation record set corresponding to the second database table is taken as the target operation record for the second database table. The operation time in the target operation record corresponding to each of the second database tables is used as the time information corresponding to each of the second database tables.

8. A storage medium, characterized in that, The storage medium includes stored instructions, wherein, when the instructions are executed, the device containing the storage medium is controlled to perform the overdue data identification method as described in any one of claims 1 to 6.

9. An electronic device, characterized in that, It includes a memory, and one or more instructions, wherein one or more instructions are stored in the memory and configured to be executed by one or more processors as described in any one of claims 1 to 6.

Images