Data retention method and computing device

Abstract

The embodiment of the invention provides a data retention method and computing equipment, and the method comprises the steps: obtaining a timestamp of first historical version data and a timestamp of second historical version data in historical version data through a server, and according to the size relationship among the timestamp of the first historical version data, the timestamp of the second historical version data and the target retention timestamp, determining historical version data to be cleaned from the historical version data. And after the server deletes the to-be-cleaned historical version data, the retained historical version data can be obtained. According to the method provided by the embodiment of the invention, the to-be-cleaned historical version data can be determined based on the timestamp of the first historical version data, the timestamp of the second historical version data and the target retention timestamp, so that the historical version data is retained from the time dimension, and the method is suitable for a scene in which the historical version data in a certain time period needs to be retained.

Description

Technical Field

[0001] The present application relates to the field of server technology, and in particular to a data retention method and a computing device. Background Art

[0002] With the development of Internet technology, the speed of product version updates is getting faster and faster. The historical version data generated during the version update process is usually retained in the database as an important reference data for improving products and discovering product problems. However, how to retain the historical version data of products has attracted much attention.

[0003] Currently, the server can clean up the historical version data according to the number of historical version data to be deleted. For example, the number of historical version data to be deleted is set as the first threshold, and the server cleans up the first threshold number of historical version data in the database starting from the earliest generated historical version data according to the order in which the historical version data is generated.

[0004] However, this method of retaining historical version data from the dimension of the number of historical version data cannot retain the historical version data generated in a certain time period. Therefore, a solution is urgently needed to retain the historical version data generated in a certain time period. Summary of the invention

[0005] Embodiments of the present application provide a data retention method and a computing device.

[0006] In order to achieve the above purpose, the embodiment of the present application adopts the following technical solution:

[0007] First aspect: The embodiment of the present application provides a data retention method, which obtains the timestamp of the first historical version data and the timestamp of the second historical version data in the historical version data; determines the historical version data to be cleaned up from the historical version data according to the size relationship between the timestamp of the first historical version data, the timestamp of the second historical version data and the target retention timestamp, and the target retention timestamp is used to indicate the historical version data to be cleaned up; deletes the historical version data to be cleaned up. Among them, the first historical version data is the historical version data with the longest retention time in the historical version data, and the second historical version data is the historical version data with the shortest retention time in the historical version data. The timestamp of the first historical version data and the timestamp of the second historical version data are used to clean up the historical version data.

[0008] In an embodiment of the present application, the historical version data to be cleaned up is determined based on the timestamp of the first historical version data, the timestamp of the second historical version data and the target retention timestamp, and the historical version data generated in a certain time period can be retained, thereby retaining the historical version data from the time dimension.

[0009] In a possible implementation, historical version data to be cleaned up is determined from the historical version data based on the size relationship among the timestamp of the first historical version data, the timestamp of the second historical version data, and the target retention timestamp, including: determining the timestamp of the third historical version data based on the size relationship among the timestamp of the first historical version data, the timestamp of the second historical version data, and the target retention timestamp, the third historical version data being the historical version data with a timestamp closest to the target retention timestamp in the historical version data; and determining the historical version data with a timestamp smaller than the timestamp of the third historical version data in the historical version data as the historical version data to be cleaned up. In the embodiment of the present application, the third historical version data with a timestamp closest to the target retention timestamp can be determined, so that the historical version data with a timestamp smaller than the timestamp of the third historical version data can be deleted later.

[0010] In a possible implementation, the timestamp of the third historical version data is determined based on the size relationship among the timestamp of the first historical version data, the timestamp of the second historical version data, and the target retention timestamp, including: when the target retention timestamp is smaller than the timestamp of the first historical version data, the timestamp of the first historical version data is determined as the timestamp of the third historical version data. In the embodiment of the present application, when the target retention timestamp is smaller than the timestamp of the first historical version data, it indicates that all current historical version data are historical version data that need to be retained, and by determining the timestamp of the first historical version data as the timestamp of the third historical version data, it is convenient to retain all current historical version data later.

[0011] In a possible implementation, the timestamp of the third historical version data is determined based on the size relationship among the timestamp of the first historical version data, the timestamp of the second historical version data, and the target retention timestamp, including: when the target retention timestamp is greater than the timestamp of the second historical version data, the timestamp of the second historical version data is determined as the timestamp of the third historical version data. In the embodiment of the present application, when the target retention timestamp is greater than the timestamp of the second historical version data, it indicates that all current historical version data are historical version data that need to be deleted, and by determining the timestamp of the second historical version data as the timestamp of the third historical version data, it is convenient to delete all current historical version data later.

[0012] In a possible implementation, the timestamp of the third historical version data is determined based on the size relationship among the timestamp of the first historical version data, the timestamp of the second historical version data, and the target retention timestamp, including: when the target retention timestamp is greater than or equal to the timestamp of the first historical version data, and less than or equal to the timestamp of the second historical version data, the timestamp of the intermediate historical version data is determined based on the number of historical version data; the timestamp of the third historical version data is determined based on the timestamp of the intermediate historical version data and the target retention timestamp. In the embodiment of the present application, by determining the timestamp of the intermediate historical version data between the timestamp of the first historical version data and the timestamp of the second historical version data, the third historical version data whose timestamp is closest to the target retention timestamp can be determined.

[0013] In one possible implementation, the timestamp of the third historical version data is determined based on the timestamp of the intermediate historical version data and the target retention timestamp, including: if in the historical version data, the timestamp of the intermediate historical version data is closest to the target retention timestamp, the timestamp of the intermediate historical version data is determined as the timestamp of the third historical version data; if in the historical version data, the timestamp of the intermediate historical version data is not closest to the target retention timestamp, the first historical version data or the second historical version data is updated to obtain the updated first historical version data or the updated second historical version data; based on the timestamp of the updated first historical version data or the timestamp of the updated second historical version data, the timestamp of the third historical version data is determined.

[0014] In one possible implementation, the timestamp of the intermediate historical version data is not closest to the target retention timestamp, and the first historical version data or the second historical version data is updated to obtain the updated first historical version data or the updated second historical version data, including: the timestamp of the intermediate historical version data is not closest to the target retention timestamp, and the size relationship between the timestamp of the intermediate historical version data and the target retention timestamp is determined; based on the size relationship, the first historical version data or the second historical version data is updated to obtain the updated first historical version data or the updated second historical version data. In an embodiment of the present application, the first historical version data or the second historical version data is updated to update the intermediate historical version data, and the third historical version data whose timestamp is closest to the target retention timestamp is determined, so as to achieve the retention of historical version data from the time dimension.

[0015] In one possible implementation, based on the size relationship, the first historical version data or the second historical version data is updated to obtain the updated first historical version data or the updated second historical version data, including: when the timestamp of the intermediate historical version data is less than the target retention timestamp, the first historical version data is updated to the subsequent historical version data of the intermediate historical version data to obtain the updated first historical version data, the timestamp of the subsequent historical version data is adjacent to the timestamp of the intermediate historical version data and is greater than the timestamp of the intermediate historical version data; when the timestamp of the intermediate historical version data is greater than the target retention timestamp, the second historical version data is updated to the previous historical version data of the intermediate historical version data version data, obtain the updated second historical version data, the timestamp of the previous historical version data is adjacent to the timestamp of the intermediate historical version data and is smaller than the timestamp of the intermediate historical version data; based on the timestamp of the updated first historical version data or the timestamp of the updated second historical version data, determine the timestamp of the third historical version data, including: if the timestamp of the updated first historical version data is closest to the target retention timestamp, determine the timestamp of the updated first historical version data as the timestamp of the third historical version data; or if the timestamp of the updated second historical version data is closest to the target retention timestamp, determine the timestamp of the updated second historical version data as the timestamp of the third historical version data.

[0016] In one possible implementation, obtaining the timestamp of first historical version data and the timestamp of second historical version data in historical version data includes: obtaining the information identifier of the first historical version data and the information identifier of the second historical version data in the historical version data; obtaining the allocation time of the information identifier of the first historical version data and the allocation time of the information identifier of the second historical version data, determining the allocation time of the information identifier of the first historical version data as the timestamp of the first historical version data, and determining the allocation time of the information identifier of the second historical version data as the timestamp of the second historical version data.

[0017] In a possible implementation, the step of determining the target retention timestamp includes: obtaining a retention time parameter; and obtaining the target retention timestamp based on the difference between the timestamp of the current time and the retention time parameter. In the embodiment of the present application, the historical version data to be cleaned up indicated by the target retention timestamp can be deleted, wherein the target retention timestamp can be determined based on the retention time parameter, and the retention time parameter can be set as needed, thereby achieving the retention of historical version data generated in a certain time period according to demand, and retaining the historical version data from the time dimension.

[0018] In a possible implementation, historical version data to be cleaned up is determined from the historical version data based on the size relationship among the timestamp of the first historical version data, the timestamp of the second historical version data, and the target retention timestamp, including: determining the information identifier of the third historical version data from the historical version data based on the size relationship among the timestamp of the first historical version data, the timestamp of the second historical version data, and the target retention timestamp, wherein in the historical version data, the timestamp corresponding to the information identifier of the third historical version data is closest to the target retention timestamp; and determining the historical version data in the historical version data whose information identifier is smaller than the information identifier of the third historical version data as the historical version data to be cleaned up. In the embodiment of the present application, the information identifier can indicate the generation data of the corresponding historical version data, and then when determining the information identifier of the third historical version data, the historical version data to be cleaned up can be determined based on the size of the information identifier of the historical version data, thereby improving the efficiency of determining the historical version data to be cleaned up.

[0019] In a possible implementation, by using the information identifier of the third historical version data as a global variable of the database, the historical version data corresponding to the information identifier smaller than the global variable can be determined as the historical version data to be cleaned up. On this basis, based on the automatic cleanup mechanism of the database, the historical version data to be cleaned up can be deleted.

[0020] In the second aspect, an embodiment of the present application provides a computing device, the computing device comprising: a processor and a memory;

[0021] The memory is used to store program code and transmit the program code to the processor;

[0022] The processor is used to execute the steps of the data retention method described above according to the instructions in the program code. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A schematic diagram of an application scenario of a data retention method provided in an embodiment of the present application;

[0024] Figure 2 A flowchart of a data retention method provided in an embodiment of the present application;

[0025] Figure 3 A flowchart of a method for determining historical version data to be cleaned up based on preset algorithm rules provided in an embodiment of the present application;

[0026] Figure 4 A flowchart of a method for determining a timestamp of third historical version data provided by an embodiment of the present application;

[0027] Figure 5A flowchart of a method for deleting historical version data to be cleaned up provided in an embodiment of the present application;

[0028] Figure 6 A structural diagram of cleaning historical version data based on the time dimension provided in an embodiment of the present application;

[0029] Figure 7 A schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The terms "first", "second", "third", etc. in the specification, claims and drawings of this application are used to distinguish different objects rather than to limit a specific order.

[0031] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0032] With the development of Internet technology, the speed of product version updates is getting faster and faster. The historical version data generated during the version update process is usually retained in the database as an important reference data for improving products and discovering product problems. However, how to retain the historical version data of products has attracted much attention.

[0033] An embodiment of the present application provides a data retention method, in which a server can obtain the timestamp of a first historical version data and the timestamp of a second historical version data in the historical version data, wherein the first historical version data is the historical version data with the longest retention time in the historical version data, and the second historical version data is the historical version data with the shortest retention time in the historical version data. According to the size relationship between the timestamp of the first historical version data, the timestamp of the second historical version data, and the target retention timestamp, the historical version data to be cleaned up can be determined. After the server deletes the historical version data to be cleaned up, the retained historical version data can be obtained. The method provided in the embodiment of the present application can determine the historical version data to be cleaned up based on the timestamp of the first historical version data, the timestamp of the second historical version data, and the target retention timestamp, so as to retain the historical version data from the time dimension, and is suitable for scenarios where historical version data needs to be retained for a certain period of time.

[0034] Combine the following Figure 1 The present application provides an application scenario of a data retention method.

[0035] Figure 1This is a schematic diagram of an application scenario of a data retention method provided in an embodiment of the present application, in which a server 1101 interacts with a client 1102 .

[0036] It should be noted that the client 1102 can be deployed in the server 1101 or in other computing devices. The embodiment of the present application does not specifically limit the deployment location of the client 1102.

[0037] The user writes the version data of the application program into the database (DB) running in the server 1101 through the client 1102. The database may include a relational database, such as a Mysql database, a SqlServer database, etc., and a non-relational database, such as a NoSQL database, etc.

[0038] The user can update or delete the version data in the database through the client 1102, where the update operation includes modifying the version data or adding content to the version data. The server 1101 responds to at least one of the delete instruction and the update instruction sent by the client 1102, and adds the deleted or updated version data to the dead tuple. The delete instruction is used to delete the version data in the database; the update instruction is used to update the version data in the database.

[0039] A dead tuple is a data row generated when an operation such as updating or deleting data in a database is performed. In an embodiment of the present application, the server can add a deletion mark to the version data in the database to add it to the dead tuple. The version data in the dead tuple can be called historical version data. The historical version data is not physically deleted from the database and still actually occupies the storage space of the database so that it can be restored later as needed.

[0040] Since the storage space of the database is limited, in order to ensure the performance of the database, the server can physically delete historical version data as needed to reclaim the storage space it occupies.

[0041] In the embodiment of the present application, the database's built-in automatic cleaning mechanism (Autovacuum) can be triggered automatically or manually to clean up some or all of the historical version data. Among them, Autovacuum is a background process in the database, which is used to clean up the historical version data in the dead tuple.

[0042] In a possible implementation, the user can set the first number threshold through the client 1102. When the historical version data reaches the first number threshold, the server 1101 can automatically trigger Autovacuum to clean up the historical version data. The first number threshold can be set according to demand. For example, the total number of historical version data that can be stored in the database can be used as the first number threshold, or 20% of the total number can be used as the first number threshold. It is understandable that the numerical value of the first number threshold is not specifically limited in the embodiments of the present application.

[0043] In a possible implementation, the user can set the second threshold value through the client 1102. When the version data in the database reaches the second threshold value, the server 1101 automatically triggers Autovacuum to clean up the historical version data and release storage space. The released storage space can be used to continue to store version data. It is understood that the value of the second threshold value is not specifically limited in the embodiments of the present application.

[0044] In one possible implementation, the user can set the activity threshold and update frequency threshold through the client 1102. When the activity of the historical version data in the database reaches the activity threshold, or the update frequency of the dead tuple reaches the update frequency threshold, the server 1101 automatically triggers Autovacuum to clean up the historical version data.

[0045] Among them, the activity level may indicate the frequency at which version data is written into the database; the update frequency may indicate the update frequency of dead tuples, that is, the frequency at which version data in the database is deleted or updated. It is understandable that the activity level threshold and the update frequency threshold may be set as required, and the values ​​of the activity level threshold and the update frequency threshold are not specifically limited in the embodiments of the present application.

[0046] In another possible implementation, the user can manually trigger the automatic cleanup mechanism of the server 1101 to enable the server 1101 to clean up the historical version data, thereby manually cleaning up the historical version data. For example, the user can send a historical version data cleanup instruction to the server 1101 through the client 1102 to enable the server 1101 to clean up the historical version data.

[0047] After the automatic cleanup mechanism is triggered, server 1101 can obtain the timestamp of the first historical version data and the timestamp of the second historical version data in the historical version data, and determine the historical version data whose timestamp is less than the target retention timestamp from the historical version data based on the size relationship among the timestamp of the first historical version data, the timestamp of the second historical version data and the target retention timestamp, and use it as the historical version data to be cleaned; delete the historical version data to be cleaned, thereby obtaining the retained historical version data.

[0048] The first historical version data is the historical version data with the longest retention time among the historical version data, and the second historical version data is the historical version data with the shortest retention time among the historical version data.

[0049] Combine the following Figure 2 A data retention method provided in an embodiment of the present application is introduced. The figure is a flow chart of a data retention method provided in an embodiment of the present application, including S101-S103.

[0050] S101. The server obtains a timestamp of first historical version data and a timestamp of second historical version data from historical version data.

[0051] In an embodiment of the present application, when the version data in the database is updated or deleted, the server adds the version data to the dead tuple to make it historical version data.

[0052] In the embodiment of the present application, the existence form of the historical version data in the database is not specifically limited. For example, multiple historical version data can exist in the form of a historical version data list. For ease of description, the following will only introduce the existence of multiple historical version data in the form of a historical version data list as an example.

[0053] The first historical version data is the historical version data that is first added to the death tuple in the current historical version data, and is the historical version data with the longest retention time in the historical version data list; the second historical version data is the historical version data that is last added to the death tuple in the current historical version data, and is the historical version data with the shortest retention time in the historical version data list.

[0054] The server may determine the historical version data first added to the dead tuple as the first historical version data and the historical version data last added to the dead tuple as the second historical version data based on the order in which the historical version data are added to the dead tuple.

[0055] In a possible implementation, the historical version data may be arranged in the historical version data list in the order in which they are added to the dead tuple. For example, in the historical version data list, the historical version data with a longer retention time is placed on the top and the historical version data with a shorter retention time is placed on the bottom. The server may determine the historical version data in the first row of the historical version data list as the first historical version data, and determine the historical version data in the last row as the second historical version data.

[0056] Each historical version data corresponds to a file directory that stores its generation time. After determining the first historical version data and the second historical version data, the server can read the timestamp of the first historical version data and the timestamp of the second historical version data respectively in the file directory corresponding to the first historical version data and the file directory corresponding to the second historical version data.

[0057] The timestamp may be a sequence of characters or coded information, and when each historical version of data is generated, the server will assign a timestamp corresponding to the historical version of data. For example, a larger timestamp indicates a later generation time of the historical version of data.

[0058] Therefore, the timestamp of the first historical version data can indicate the generation time of the first historical version data; and the timestamp of the second historical version data can indicate the generation time of the second historical version data.

[0059] In a possible implementation, each historical version data may correspond to a unique information identifier (Transaction ID, XID). The XID may be a monotonically increasing 32-bit integer, which may be used to indicate the generation order of the historical version data. It should be noted that the value and value range of the XID are not specifically limited in the embodiments of the present application, and the above is only an example.

[0060] Whenever the server deletes or updates version data in the database and adds the version data to the dead tuple to generate historical version data, it will assign a unique XID to the operation and associate the XID with the generated historical version data.

[0061] In one embodiment, the server may use the XID corresponding to the operation performed to generate the historical version data as the XID of the historical version data to associate the XID with the generated historical version data. For example, when the server performs a deletion operation on the version data in the database and generates the historical version data corresponding to the deletion operation, the XID assigned to the deletion operation is 1. In this case, the server may use 1 as the XID of the historical version data generated based on the deletion operation; thereafter, when the server performs an update operation on the version data in the database and generates the historical version data corresponding to the update operation, the XID assigned to the update operation is 2. In this case, the server may use 2 as the XID of the historical version data generated based on the update operation to achieve the association between the XID and the historical version data.

[0062] Since XID is monotonically increasing, the server can determine the order in which the historical version data is added to the dead tuple based on the size of XID, that is, determine the order in which the historical version data is generated. For example, when the server adds the first historical version data to the dead tuple, it will assign the first XID to the corresponding operation and associate the first XID with the first historical version data; when the second historical version data is then added to the dead tuple, it will assign the second XID to the corresponding operation and associate the second XID with the second historical version data. Among them, the second XID is larger than the first XID. Based on the size relationship between the second XID and the first XID, it can be determined that the time when the first historical version data was added to the dead tuple is earlier than the time when the second historical version data was added to the dead tuple.

[0063] The server reads the smallest information identifier and the largest information identifier from the historical version data list of the database, wherein the smallest information identifier corresponds to the first historical version data, and the largest information identifier corresponds to the second historical version data.

[0064] When the server assigns an information identifier to the historical version data, it will save the time of assigning the information identifier in the corresponding file directory, for example, in pg_commit_ts. The time of assigning the information identifier to the historical version data can be used as the generation time of the historical version data.

[0065] After determining the information identifier of the first historical version data and the information identifier of the second historical version data, the server can read the time when the information identifier is assigned to the first historical version data and the time when the information identifier is assigned to the second historical version data in the file directory corresponding to the first historical version data and the file directory corresponding to the second historical version data, respectively. The time when the information identifier is assigned to the first historical version data is the timestamp of the first historical version data, corresponding to the generation time of the first historical version data; the time when the information identifier is assigned to the second historical version data is the timestamp of the second historical version data, corresponding to the generation time of the second historical version data.

[0066] S102: The server determines the historical version data to be cleared from the historical version data according to a preset algorithm rule based on the target retention timestamp, the timestamp of the first historical version data, and the timestamp of the second historical version data.

[0067] The target retention timestamp is used to indicate the historical version data that the user expects to retain.

[0068] In a possible implementation, the target retention timestamp may be determined based on a retention time parameter in the database, wherein the retention time parameter is a configurable parameter used to indicate the time that the historical version data is allowed to be retained in the database.

[0069] Exemplarily, after the server obtains the retention time parameter in response to the user's configuration of the retention time parameter, it can obtain the target retention time stamp based on the difference between the timestamp of the current time and the retention time parameter, as shown in formula (1). The unit of the retention time parameter is a time unit, such as hours, minutes, seconds, etc.

[0070] Target retention timestamp = current timestamp - retention time parameter (1)

[0071] Based on the target retention timestamp, the historical version data in the historical version data list can be divided into historical version data with a timestamp equal to the target retention timestamp, historical version data with a timestamp less than the target retention timestamp, and historical version data with a timestamp greater than the target retention timestamp.

[0072] The historical version data that the user expects to retain can be at least one of the historical version data whose timestamp is equal to the target retention timestamp, the historical version data whose timestamp is less than the target retention timestamp, and the historical version data whose timestamp is greater than the target retention time. The following will only take the historical version data that the user expects to retain as the historical version data whose timestamp is greater than the target retention timestamp as an example for introduction.

[0073] S103: The server deletes the historical version data to be cleaned up.

[0074] In one possible implementation, the server can determine the timestamp of the third historical version data according to preset algorithm rules based on the target retention timestamp, the timestamp of the first historical version data, and the timestamp of the second historical version data, and determine the historical version data to be cleared in the historical version data list based on the timestamp of the third historical version data.

[0075] The third historical version data is the historical version data whose timestamp is closest to the target retention timestamp among all the historical version data in the current historical version data list.

[0076] Combine the following Figure 3 Introduce the preset algorithm rules provided in the embodiment of the present application, such as Figure 3 As shown, the figure is a flow chart of a method for determining historical version data to be cleaned up based on preset algorithm rules provided in an embodiment of the present application, including S201-S207.

[0077] S201. The server obtains a timestamp of the first historical version data, a timestamp of the second historical version data, and a target retention timestamp.

[0078] The method for obtaining the timestamp of the first historical version data and the timestamp of the second historical version data is the same as the above S101, which will not be repeated here.

[0079] It is understandable that in the embodiment of the present application, the order in which the server obtains the timestamp of the first historical version data, the timestamp of the second historical version data, and the target retention timestamp is not specifically limited. Exemplarily, the server can obtain the timestamp of the first historical version data, the timestamp of the second historical version data, and the target retention timestamp at the same time, or can obtain the timestamp of the first historical version data, the timestamp of the second historical version data, and the target retention timestamp one by one.

[0080] S202: The server determines whether the target retention timestamp is smaller than the timestamp of the first historical version data.

[0081] If the target retention timestamp is smaller than the timestamp of the first historical version data, it means that all historical version data in the current historical version data list are generated after the target retention timestamp, and all historical version data in the current historical version data list are historical version data that need to be retained. Therefore, S203 can be executed, and the server determines the timestamp of the first historical version data as the timestamp of the third historical version data.

[0082] If the target retention timestamp is greater than or equal to the timestamp of the first historical version data, execute S204.

[0083] S204: The server determines whether the target retention timestamp is greater than the timestamp of the second historical version data.

[0084] If the target retention timestamp is greater than the timestamp of the second historical version data, it means that all historical version data in the current historical version data list are generated before the target retention timestamp, and all historical version data in the current historical version data list are historical version data that need to be cleaned up. Therefore, S205 can be executed, and the server determines the timestamp of the second historical version data as the timestamp of the third historical version data.

[0085] If the target retention timestamp is less than or equal to the timestamp of the second historical version data, it means that the target retention timestamp is between the timestamp of the first historical version data and the timestamp of the second historical version data, that is, it is within the allocation time period of the historical version data. The allocation time period of the historical version data can be represented by the difference between the timestamp of the second historical version data and the timestamp of the first historical version data. In this case, S206 can be executed.

[0086] S206. The server determines a timestamp of the third historical version data within the allocated time period of the historical version data.

[0087] In a possible implementation, when the server determines the timestamp of the third historical version data within the allocated time period of the historical version data, the timestamp of the third historical version data may be determined based on the following steps: Figure 4 As shown, the figure is a flowchart of a method for determining the timestamp of the third historical version data provided by an embodiment of the present application, including S301-S306.

[0088] S301. The server determines the intermediate historical version data based on the number of all historical version data in the historical version data list, and obtains the timestamp of the intermediate historical version data.

[0089] Exemplarily, when the number of all historical version data in the current historical version data list is 9, the first historical version data is the historical version data with the earliest generation time in the current historical version data list, the second historical version data is the historical version data with the latest generation time in the current historical version data list, and the intermediate historical version data is the fifth historical version data generated in the current historical version data list. When the number of all historical version data in the current historical version data list is 10, the server can determine the fifth historical version data or the sixth historical version data generated in the current historical version data list as the intermediate historical version data.

[0090] After the server determines the intermediate historical version data, it can obtain the timestamp of the intermediate historical version data. The method for obtaining the timestamp is the same as the method for obtaining the timestamps of the first historical version data and the second historical version data, and will not be repeated here.

[0091] S302: The server determines whether the timestamp of the intermediate historical version data is closest to the target retention timestamp in the historical version data list.

[0092] If the timestamp of the intermediate historical version data is closest to the target retention timestamp, for example, the timestamp of the intermediate historical version data is equal to the target retention timestamp, or although the timestamp of the intermediate historical version data is not equal to the target retention timestamp, but among all historical version data, the timestamp of the intermediate historical version data is closest to the target retention timestamp, then execute S303 and the server determines the timestamp of the intermediate historical version data as the timestamp of the third historical version data.

[0093] If the timestamp of the intermediate historical version data is not the timestamp closest to the target retention timestamp, execute S304.

[0094] S304: The server determines whether the timestamp of the intermediate historical version data is less than the target retention timestamp.

[0095] If the timestamp of the intermediate historical version data is less than the target retention timestamp, it means that the historical version data generated before the timestamp of the intermediate historical version data are all historical version data that need to be cleaned up, but the historical version data generated between the timestamp of the intermediate historical version data and the target retention timestamp are also historical version data that need to be cleaned up. In order to determine the historical version data between the timestamp of the intermediate historical version data and the target retention timestamp, S305 can be executed.

[0096] S305: The server updates the first historical version data to the subsequent historical version data of the intermediate historical version data.

[0097] The subsequent historical version data of the intermediate historical version data is adjacent to the intermediate historical version data in the historical version data list, and is generated later than the generation time of the intermediate historical version data. When the subsequent historical version data of the intermediate historical version data is adjacent to the intermediate historical version data in the historical version data list, the timestamps of the two are adjacent, that is, no historical version data is generated in the time period between the timestamp of the subsequent historical version data and the timestamp of the intermediate historical version data.

[0098] After the server updates the first historical version data to the subsequent historical version data of the intermediate historical version data, it can be considered that the historical version data with the longest retention time in the current historical version data list is the subsequent historical version data of the intermediate historical version data.

[0099] On this basis, S301 can be re-executed. The difference from the last execution of S301 is that the historical version data with the longest retention time in the current historical version data list is the historical version data after the intermediate historical version data. Therefore, the timestamp of an intermediate historical version data can be re-determined within the time period between the timestamp of the historical version data after the intermediate historical version data and the timestamp of the historical version data with the shortest retention time, until the timestamp closest to the target retention timestamp is determined.

[0100] If the timestamp of the intermediate historical version data is greater than the target retention timestamp, execute S306.

[0101] S306: The server updates the second historical version data to the previous historical version data of the intermediate historical version data.

[0102] The previous historical version data of the intermediate historical version data is adjacent to the intermediate historical version data in the historical version data list and is generated earlier than the intermediate historical version data. When the previous historical version data of the intermediate historical version data is adjacent to the intermediate historical version data in the historical version data list, the timestamps of the two are adjacent, that is, no historical version data is generated in the time period between the timestamp of the previous historical version data and the timestamp of the intermediate historical version data.

[0103] After the server updates the second historical version data to the previous historical version data of the intermediate historical version data, it can be considered that the historical version data with the shortest retention time in the current historical version data list is the previous historical version data of the intermediate historical version data.

[0104] On this basis, S301 can be re-executed. The difference from the last execution of S301 is that the historical version data with the shortest retention time in the current historical version data list is the previous historical version data of the intermediate historical version data. Therefore, the timestamp of an intermediate historical version data can be re-determined within the time period between the timestamp of the previous historical version data of the intermediate historical version data and the timestamp of the historical version data with the longest retention time, until the timestamp closest to the target retention timestamp is determined and used as the timestamp of the third version data.

[0105] In the embodiment of the present application, the server can determine the timestamp of the third historical version data in the three cases that the target retention timestamp is less than the timestamp of the first historical version data, greater than the timestamp of the second historical version data, or within the allocated time period of the historical version data. After determining the timestamp of the third historical version data, S207 can be executed, and the server determines the historical version data in the historical version data list whose timestamp is less than the timestamp of the third historical version data as the historical version data to be cleaned up.

[0106] After determining the historical version data to be cleared, the server may execute the above S103 to delete the historical version data to be cleared and obtain the retained historical version data.

[0107] To sum up, according to the method provided in the embodiment of the present application, the server can determine the timestamp of the third historical version data based on the timestamp of the first historical version data, the timestamp of the second historical version data and the target retention timestamp, and determine the historical version data generated before the timestamp of the third historical version data as the historical version data to be cleaned up, and obtain the retained historical version data by deleting the historical version data to be cleaned up, thereby realizing the retention of historical version data from the time dimension. This is suitable for scenarios where historical version data needs to be retained for a certain period of time.

[0108] In another possible implementation, in the case where each historical version data corresponds to a unique information identifier, the server can determine the information identifier of the third historical version data in the historical version data list according to the target retention timestamp, the timestamp of the first historical version data, and the timestamp of the second historical version data in accordance with a preset algorithm rule. On this basis, the historical version data to be cleaned up is deleted by setting the global variable of the database to the information identifier of the third historical version data, thereby obtaining the retained historical version data.

[0109] Combine the following Figure 5 , a method for deleting the historical version data to be cleaned up by setting the global variable of the database to the information identifier of the third historical version data is introduced.

[0110] like Figure 5 As shown, this figure is a flow chart of a method for deleting historical version data to be cleaned up provided in an embodiment of the present application.

[0111] S401. The server obtains an information identifier of a first historical version of data, a timestamp of the first historical version of data, an information identifier of a second historical version of data, a timestamp of the second historical version of data, and a target retention timestamp.

[0112] It is understandable that in the embodiment of the present application, the order in which the server obtains the information identifier of the first historical version data, the timestamp of the first historical version data, the information identifier of the second historical version data, the timestamp of the second historical version data, and the target retention timestamp is not specifically limited. Exemplarily, the server can simultaneously obtain the information identifier of the first historical version data, the timestamp of the first historical version data, the information identifier of the second historical version data, the timestamp of the second historical version data, and the target retention timestamp, or can successively obtain the information identifier of the first historical version data, the timestamp of the first historical version data, the information identifier of the second historical version data, the timestamp of the second historical version data, and the target retention timestamp.

[0113] Among them, the information identifier of the first historical version data can be expressed as oldestXid; the information identifier of the second historical version data can be expressed as newestXid; and the target retention timestamp can be expressed as retention_ts.

[0114] S402: The server determines whether the target retention timestamp is smaller than the timestamp of the first historical version data.

[0115] If the target retention timestamp is smaller than the timestamp of the first historical version data, it indicates that all historical version data in the current historical version data list are historical version data that need to be retained, and thus S403 can be executed, where the server sets the global variable of the database to the information identifier of the first historical version data.

[0116] The global variable of the database can be represented as OldestXmin, which can be used to control the cleaning of historical version data, that is, the server can delete the historical version data corresponding to the XID less than OldestXmin. The server sets OldestXmin to oldestXid, which is equivalent to determining the first historical version data as the third historical version data.

[0117] If the target retention timestamp is greater than the timestamp of the first historical version data, execute S404.

[0118] S404: The server determines whether the target retention timestamp is greater than the timestamp of the second historical version data.

[0119] If the target retention timestamp is greater than the timestamp of the second historical version data, it means that all historical version data in the current historical version data list are historical version data that need to be cleared, so S405 can be executed, and the server sets the global variable of the database to the information identifier of the second historical version data.

[0120] The server sets OldestXmin to newestXid, which is equivalent to determining the second historical version data as the third historical version data.

[0121] If the target retention timestamp is less than or equal to the timestamp of the second historical version data, S406 may be executed.

[0122] S406: The server determines the timestamp of the third historical version data within the allocated time period of the historical version data.

[0123] In the case where each historical version data corresponds to a unique information identifier, the server may determine the timestamp of the third historical version data based on the following steps.

[0124] S407: The server determines the information identifier of the intermediate historical version data based on the information identifier of the first historical version data and the information identifier of the second historical version data, and obtains the timestamp of the intermediate historical version data.

[0125] Exemplarily, the information identifier of the intermediate historical version data can be calculated based on formula (2) as follows:

[0126] middleXid=oldestXid+(newestXid-oldestXid) / 2 (2)

[0127] Wherein, middleXid represents the information identifier of the intermediate historical version data. After the information identifier of the intermediate historical version data is determined, the timestamp (middleXid_ts) of the intermediate historical version data can be obtained from the file directory where the timestamp is stored based on the information identifier of the intermediate historical version data.

[0128] S408: The server determines whether the timestamp of the intermediate historical version data is closest to the target retention timestamp.

[0129] If the timestamp of the intermediate historical version data is closest to the target retention timestamp, for example, the timestamp of the intermediate historical version data is equal to the target retention timestamp, S409 is executed and the server sets the global variable of the database to the information identifier of the intermediate historical version data.

[0130] By setting OldestXmin to middleXid, the server is equivalent to determining the intermediate historical version data as the third historical version data.

[0131] If the timestamp of the intermediate historical version data is not the timestamp closest to the target retention timestamp, execute S410.

[0132] S410: The server determines whether the timestamp of the intermediate historical version data is less than the target retention timestamp.

[0133] If the timestamp of the intermediate historical version data is less than the target retention timestamp, S411 may be executed.

[0134] S411. The server updates the information identifier of the first historical version data to the value of the information identifier of the middle historical version data plus 1. That is, the server updates oldestXid to middleXid+1.

[0135] oldestXid corresponds to the historical version data with the longest retention time in the current historical version data list. After the server updates oldestXid to middleXid+1, it can be considered that the historical version data with information identifier middleXid+1 is the historical version data with the longest retention time in the current historical version data list.

[0136] On this basis, S407 can be re-executed, and S407 can be executed last time to re-determine the information identifier of the intermediate historical version data and obtain the timestamp of the intermediate historical version data until a timestamp closest to the target retention timestamp is determined.

[0137] If the timestamp of the intermediate historical version data is greater than the target retention timestamp, execute S412.

[0138] S412: The server updates the information identifier of the second historical version data to the value of the information identifier of the middle historical version data minus 1. That is, the server updates newestXid to middleXid-1.

[0139] newestXid corresponds to the historical version data with the shortest retention time in the current historical version data list. After the server updates newestXid to middleXid-1, it can be considered that the historical version data with information identifier middleXid-1 is the historical version data with the shortest retention time in the current historical version data list.

[0140] On this basis, S407 can be re-executed, compared with the last execution of S407, to re-determine the information identifier of the intermediate historical version data and obtain the timestamp of the intermediate historical version data until the timestamp closest to the target retention timestamp is determined, and the information identifier corresponding to the timestamp is the information identifier of the third historical version data.

[0141] In the embodiment of the present application, the server can determine the information identifier of the third historical version data in the three cases that the target retention timestamp is less than the timestamp of the first historical version data, greater than the timestamp of the second historical version data, or is within the allocated time period of the historical version data. After determining the information identifier of the third historical version data, S413 can be executed, and the server sets the global variable of the database to the information identifier of the third historical version data.

[0142] S414. The server deletes the historical version data to be cleaned up based on the global variables of the database, and obtains the retained historical version data.

[0143] In the process of deleting the historical version data to be cleaned up based on the global variables of the database, if the information identifier of the historical version data is less than oldestXmin, it means that the generation time of the historical version data is less than oldestXmin, and the historical version data will be deleted, otherwise it will be retained.

[0144] To sum up, the method provided in the embodiment of the present application can obtain the retained historical version data by setting the global variable of the database to the information identifier of the third historical version data, so as to treat the historical version data with an information identifier smaller than the global variable as the historical version data to be cleaned up and delete it, thereby realizing the retention of historical version data from the time dimension, and is suitable for scenarios where historical version data needs to be retained for a certain period of time.

[0145] For ease of understanding, the following Figure 6 The data retention method provided in the embodiment of the present application is generally introduced as follows: Figure 6 As shown, this figure is a structural diagram of cleaning historical version data based on the time dimension provided by an embodiment of the present application.

[0146] After the application writes the version data to the database running on the server, if the version data in the database is deleted or updated, the server adds the deleted or updated version data to the dead tuple based on the Redundant Arrays of Independent Disks (MVCC) mechanism. The version data added to the dead tuple is called historical version data.

[0147] The server can automatically or manually trigger the database's built-in automatic cleanup mechanism to clean up dead tuples corresponding to XIDs that are less than the global variable.

[0148] In the embodiment of the present application, the server determines the third historical version data whose timestamp is closest to the target retention timestamp based on the target retention timestamp, the timestamp of the first historical version data, and the timestamp of the second historical version data according to the preset algorithm rules based on the third historical version data determination module. The global variable is updated to the information identifier of the third historical version data, so that the historical version data whose information identifier is less than the information identifier of the third historical version data is deleted to obtain the retained historical version data.

[0149] To sum up, in the embodiment of the present application, the server can calculate the third historical version data according to preset algorithm rules based on the timestamp of the first historical version data, the timestamp of the second historical version data, and the target retention timestamp configured by the user, and retain the historical version data generated after the target retention timestamp as much as possible.

[0150] At the same time, the method provided in the embodiment of the present application can instruct the server to retain historical version data within the corresponding time period through the target retention timestamp configured by the user, thereby facilitating the identification of whether historical version data that needs to be restored exists and reducing the complexity of restoring historical version data.

[0151] like Figure 7 As shown, this figure is a structural schematic diagram of a computing device provided in an embodiment of the present application.

[0152] In an embodiment of the present application, the computing device may be a server, and the types of servers include but are not limited to rack servers, cabinet servers, and AI servers.

[0153] The computing device includes a memory 1200, a processor 1201 and a communication interface 1202; wherein the memory 1200 stores computer instructions, and the processor 1201 is used to execute the computer instructions, so that the computing device executes a data retention method shown above.

[0154] In some embodiments, the processor 1201 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. A general-purpose processor may be a microprocessor or any conventional processor.

[0155] In some embodiments, the memory 1200 may be a volatile memory or a non-volatile memory, such as a register. Specifically, a volatile memory refers to a memory in which the data stored inside will be lost when the power supply is interrupted. Among them, the volatile memory is mainly a random access memory (RAM), including a static random access memory (SRAM) and a dynamic random access memory (DRAM). A non-volatile memory refers to a memory in which the data stored inside will not be lost even if the power supply is interrupted. Common non-volatile memories include read-only memory (ROM), optical disks, magnetic disks, solid-state hard disks, and various memory cards based on flash memory technology.

[0156] In some embodiments, the memory 1200 has executable codes, and the processor 1201 executes the codes. The communication interface 1202 may be used to implement communication between a client and a server.

[0157] The bus can be a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of understanding, Figure 7 It is represented by only one thick line, but it does not mean that there is only one bus or one type of bus.

[0158] Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. The above is only a specific implementation method of the present application, but the protection scope of the present application is not limited thereto, and any changes or replacements within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A data retention method, It is characterized in that include: Obtaining a timestamp of first historical version data and a timestamp of second historical version data in the historical version data, wherein the first historical version data is the historical version data with the longest retention time among the historical version data, and the second historical version data is the historical version data with the shortest retention time among the historical version data, and the timestamp of the first historical version data and the timestamp of the second historical version data are used to clean up the historical version data; Determine the historical version data to be cleaned up from the historical version data according to the size relationship among the timestamp of the first historical version data, the timestamp of the second historical version data, and the target retention timestamp, wherein the target retention timestamp is used to indicate the historical version data to be cleaned up; Delete the historical version data to be cleaned up.

2. The method according to claim 1, It is characterized in that The determining the historical version data to be cleaned up from the historical version data according to the size relationship among the timestamp of the first historical version data, the timestamp of the second historical version data, and the target retention timestamp includes: Determine the timestamp of the third historical version data according to the size relationship among the timestamp of the first historical version data, the timestamp of the second historical version data and the target retention timestamp, wherein the third historical version data is the historical version data whose timestamp is closest to the target retention timestamp among the historical version data; The historical version data whose timestamp is smaller than the timestamp of the third historical version data is determined as the historical version data to be cleared.

3. The method according to claim 2, It is characterized in that The determining the timestamp of the third historical version data according to the size relationship among the timestamp of the first historical version data, the timestamp of the second historical version data, and the target retention timestamp includes: When the target retention timestamp is smaller than the timestamp of the first historical version data, the timestamp of the first historical version data is determined as the timestamp of the third historical version data.

4. The method according to claim 2, It is characterized in that The determining the timestamp of the third historical version data according to the size relationship among the timestamp of the first historical version data, the timestamp of the second historical version data, and the target retention timestamp includes: When the target retention timestamp is greater than the timestamp of the second historical version data, the timestamp of the second historical version data is determined as the timestamp of the third historical version data.

5. The method according to claim 2, It is characterized in that The determining the timestamp of the third historical version data according to the size relationship among the timestamp of the first historical version data, the timestamp of the second historical version data, and the target retention timestamp includes: When the target retention timestamp is greater than or equal to the timestamp of the first historical version data and less than or equal to the timestamp of the second historical version data, determining the timestamp of the intermediate historical version data according to the number of historical version data; The timestamp of the third historical version data is determined according to the timestamp of the intermediate historical version data and the target retention timestamp.

6. The method according to claim 5, It is characterized in that The determining the timestamp of the third historical version data according to the timestamp of the intermediate historical version data and the target retention timestamp includes: If, among the historical version data, the timestamp of the intermediate historical version data is closest to the target retention timestamp, the timestamp of the intermediate historical version data is determined as the timestamp of the third historical version data; If, among the historical version data, the timestamp of the intermediate historical version data is not closest to the target retention timestamp, updating the first historical version data or the second historical version data to obtain the updated first historical version data or the updated second historical version data; Based on the timestamp of the updated first historical version data or the timestamp of the updated second historical version data, a timestamp of the third historical version data is determined.

7. The method according to claim 6, It is characterized in that The timestamp of the intermediate historical version data is not closest to the target retention timestamp, updating the first historical version data or the second historical version data to obtain the updated first historical version data or the updated second historical version data includes: The timestamp of the intermediate historical version data is not closest to the target retention timestamp, and the size relationship between the timestamp of the intermediate historical version data and the target retention timestamp is determined; Based on the size relationship, the first historical version data or the second historical version data is updated to obtain the updated first historical version data or the updated second historical version data.

8. The method according to claim 7, It is characterized in that The updating of the first historical version data or the second historical version data based on the size relationship to obtain the updated first historical version data or the updated second historical version data includes: In the case where the timestamp of the intermediate historical version data is less than the target retention timestamp, the first historical version data is updated to the historical version data after the intermediate historical version data to obtain the updated first historical version data, and the timestamp of the latter historical version data is adjacent to the timestamp of the intermediate historical version data and is greater than the timestamp of the intermediate historical version data; in the case where the timestamp of the intermediate historical version data is greater than the target retention timestamp, the second historical version data is updated to the historical version data before the intermediate historical version data to obtain the updated second historical version data, and the timestamp of the previous historical version data is adjacent to the timestamp of the intermediate historical version data and is less than the timestamp of the intermediate historical version data; The determining the timestamp of the third historical version data based on the timestamp of the updated first historical version data or the timestamp of the updated second historical version data includes: If the timestamp of the updated first historical version data is closest to the target retention timestamp, the timestamp of the updated first historical version data is determined as the timestamp of the third historical version data; or if the timestamp of the updated second historical version data is closest to the target retention timestamp, the timestamp of the updated second historical version data is determined as the timestamp of the third historical version data.

9. The method according to claim 1, It is characterized in that The step of obtaining the timestamp of the first historical version data and the timestamp of the second historical version data in the historical version data includes: Obtaining information identifiers of first historical version data and second historical version data in the historical version data; Obtain the allocation time of the information identifier of the first historical version data and the allocation time of the information identifier of the second historical version data, determine the allocation time of the information identifier of the first historical version data as the timestamp of the first historical version data, and determine the allocation time of the information identifier of the second historical version data as the timestamp of the second historical version data.

10. The method according to claim 1, It is characterized in that The step of determining the target retention timestamp includes: Get retention time parameters; The target retention timestamp is obtained based on the difference between the current timestamp and the retention time parameter.

11. A computing device, It is characterized in that The computing device includes: a processor and a memory; The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the steps of a data retention method according to any one of claims 1 to 10 according to the instructions in the program code.

Images