A method, system, device and readable storage medium for cleaning operational records
Patent Information
- Application Number
- CN202611166755.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-31
- Publication Date
- 2026-10-27
AI Technical Summary
[0016]可见，本发明通过边缘端在裸金属实例运行过程中生成运行记录，并将运行记录写入运行记录表，通过异步数据同步通道将运行记录推送至中心端；其中，运行记录包括添加时间、结束时间和数据变更时间；中心端接收并持久化边缘端推送的运行记录，并在完成持久化后，更新与运行记录表对应的消费确认水位；其中，消费确认水位用于表征中心端对运行记录表的同步消费确认进度；边缘端响应于针对运行记录表中目标数据的清理任务，针对每个目标数据，获取与目标数据所属运行记录表对应的目标消费确认水位；边缘端基于目标数据的结束时间、添加时间、数据变更时间和目标消费确认水位确定候选清理记录；其中，候选清理记录的添加时间早于基于当前时间和预设保留周期确定的保留时间，并且候选清理记录的数据变更时间早于目标消费确认水位；边缘端基于候选清理记录对运行记录进行清理。
Smart Images
Figure CN122985771A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data synchronization technology, and in particular to a method, system, device, and readable storage medium for cleaning up operation records. Background Technology
[0002] In the center-edge architecture of a cloud computing platform, the edge data center (Region) is responsible for the runtime management of bare metal instances, continuously generating runtime record data (bmi_run_record table). These runtime records are pushed to the central side via an asynchronous data synchronization channel (DataSyncRoom) for aggregation, querying, auditing, and billing calculation. As instances continue to run and stop, the runtime record table on the Region's local machine will continuously expand, requiring periodic cleanup to reclaim storage space. General resource cleanup solutions rely on the local time attributes of the objects being cleaned up (such as whether the file creation time is within a user-preset time period). If cleanup is performed solely based on these local attributes, data that has not yet been synchronized to the central side may be mistakenly deleted.
[0003] It is evident that improving the accuracy of operation record cleanup is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0004] In view of this, the purpose of the present invention is to provide a method, system, electronic device and storage medium for cleaning up running records, which solves the technical problem of low accuracy in cleaning up running records in the prior art.
[0005] To solve the above technical problems, the present invention provides a method for cleaning up operation records, comprising: The edge device generates a running record during the operation of the bare metal instance and writes the running record into a running record table. The running record is then pushed to the central device through an asynchronous data synchronization channel. The running record includes the addition time, end time, and data change time. The central terminal receives and persists the operation records pushed by the edge terminal, and after persistence is completed, updates the consumption confirmation level corresponding to the operation record table; wherein, the consumption confirmation level is used to characterize the synchronous consumption confirmation progress of the central terminal on the operation record table. The edge terminal responds to the cleanup task for the target data in the operation record table, and for each target data, obtains the target consumption confirmation level corresponding to the operation record table to which the target data belongs; The edge terminal determines candidate cleanup records based on the end time, addition time, data change time, and target consumption confirmation level of the target data; wherein, the addition time of the candidate cleanup record is earlier than the retention time determined based on the current time and a preset retention period, and the data change time of the candidate cleanup record is earlier than the target consumption confirmation level; The edge terminal cleans the running records based on the candidate cleanup records.
[0006] Optionally, in response to a cleanup task for target data in the operation record table, the edge terminal obtains the target consumption confirmation level corresponding to the operation record table to which the target data belongs for each target data, including: The edge terminal determines the business table name field corresponding to the operation record table to which the target data belongs; The edge terminal queries the synchronization event record in the synchronization event table based on the business table name field; The edge device reads the consumption confirmation level from the queried synchronization event records and determines the read consumption confirmation level as the target consumption confirmation level corresponding to the target data.
[0007] Optionally, in response to a cleanup task for target data in the operation record table, the edge terminal obtains the target consumption confirmation level corresponding to the operation record table to which the target data belongs for each target data, including: The edge terminal determines whether there is a consumption confirmation level corresponding to the target data; If the edge terminal does not find a synchronization event record corresponding to the running record table to which the target data belongs, or if the consumption confirmation level in the found synchronization event record is empty, then the edge terminal determines that it has not obtained the target consumption confirmation level. After determining that a valid target consumption confirmation level has not been obtained, the edge terminal decides not to perform the deletion action on the target data. If the edge terminal obtains the target consumption confirmation level, then the edge terminal executes the step of determining candidate cleanup records based on the end time, addition time, data change time, and target consumption confirmation level of the target data.
[0008] Optionally, the edge endpoint determines candidate cleanup records based on the end time, addition time, data change time, and target consumption confirmation level of the target data, including: The edge terminal filters target data with an end time from the target data in the operation record table, and filters target data with an addition time earlier than the retention time from the target data with an end time; The edge end filters target data whose data change time is earlier than the target consumption confirmation level from the target data that meets the retention time, and determines the filtered target data as the candidate cleanup record.
[0009] Optionally, before filtering target data with data change times earlier than the target consumption confirmation level from the target data that meets the retention time at the edge end, and determining the filtered target data as the candidate cleanup record, the method further includes: The edge terminal determines from the target data with an end time whether the addition time is less than the retention time; When the retention time is less than the specified retention time, it is determined that target data with a data change time earlier than the target consumption confirmation level will be selected from the data that meets the retention time, and the selected target data will be determined as the candidate cleanup record.
[0010] Optionally, before obtaining the target consumption confirmation level corresponding to the operation record table to which the target data belongs for each target data in response to a cleanup task for target data in the operation record table at the edge end, the method further includes: When the data push is interrupted at the edge due to a master node abnormality or network partition, the central end stops advancing the consumption confirmation level. The edge end uses the consumption confirmation water level at which the advance stops as the target consumption confirmation water level during the period when the consumption confirmation water level stops advancing.
[0011] Optionally, the central terminal receives and persists the operation records pushed by the edge terminal, and after persistence is completed, updates the consumption confirmation level corresponding to the operation record table, including: After receiving and persisting the operation records pushed by the edge terminal, the central terminal updates the consumption confirmation level in the synchronization event table corresponding to the operation record table. The central terminal records the synchronous consumption confirmation progress of the operation record table at the central terminal through the consumption confirmation water level.
[0012] The present invention also provides a system for cleaning up operation records, comprising: The central terminal is used to receive and persist the operation records pushed by the edge terminal, and after the persistence is completed, update the consumption confirmation level corresponding to the operation record table; wherein, the consumption confirmation level is used to characterize the synchronous consumption confirmation progress of the central terminal on the operation record table. At the edge, a process is used to generate operation records during the operation of a bare metal instance, write the operation records into an operation record table, and push the operation records to the central end via an asynchronous data synchronization channel. The operation records include an addition time, an end time, and a data change time. In response to a cleanup task for target data in the operation record table, for each target data, a target consumption confirmation level corresponding to the operation record table to which the target data belongs is obtained. Candidate cleanup records are determined based on the end time, addition time, data change time, and target consumption confirmation level of the target data. The addition time of the candidate cleanup records is earlier than the retention time determined based on the current time and a preset retention period, and the data change time of the candidate cleanup records is earlier than the target consumption confirmation level. The operation records are then cleaned based on the candidate cleanup records.
[0013] The present invention also provides an electronic device, comprising: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the runtime record cleaning method described above.
[0014] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described running record cleaning method.
[0015] The present invention also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the above-described method.
[0016] As can be seen, this invention generates operation records at the edge during the operation of a bare metal instance, writes these records into an operation record table, and pushes them to the central end via an asynchronous data synchronization channel. The operation records include the addition time, end time, and data change time. The central end receives and persists the operation records pushed by the edge end, and after persistence, updates the consumption confirmation level corresponding to the operation record table. The consumption confirmation level represents the central end's progress in synchronizing consumption confirmation of the operation record table. In response to a cleanup task for target data in the operation record table, the edge end obtains the target consumption confirmation level corresponding to the operation record table to which the target data belongs for each target data. The edge end determines candidate cleanup records based on the target data's end time, addition time, data change time, and target consumption confirmation level. The addition time of the candidate cleanup records is earlier than the retention time determined based on the current time and a preset retention period, and the data change time of the candidate cleanup records is earlier than the target consumption confirmation level. The edge end cleans the operation records based on the candidate cleanup records.
[0017] The beneficial effects of this invention are as follows: Compared with the current method of cleaning up running records based solely on local time attributes, this invention can utilize the central end's synchronous consumption confirmation level (i.e., time), addition time, and end time to constrain the cleaning range at the edge end. This ensures that data not confirmed for consumption by the central end is not cleaned up, and that only data meeting the set end time and addition time conditions is cleaned up, thereby improving the accuracy of running record cleaning.
[0018] In addition, the present invention also provides a system, device and readable storage medium for cleaning up operation records, which also have the above-mentioned beneficial effects. Attached Figure Description
[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0020] Figure 1 A flowchart of a method for cleaning up runtime records provided in an embodiment of the present invention; Figure 2 A flowchart illustrating a method for cleaning up runtime records provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a runtime record cleaning system provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0022] In this embodiment, Region corresponds to the edge, DataSyncRoom corresponds to the asynchronous data synchronization channel, bmi_run_record table corresponds to the running record table, sync_event table corresponds to the synchronization event table, table_name corresponds to the data field in the synchronization event table used to identify the target business table, lastPushTime, last_push_time or sync_event.lastPushTime corresponds to the consumption confirmation level, optTime corresponds to the data change time, Leader corresponds to the master node, and ClusterVote corresponds to the master node election mechanism.
[0023] Please refer to Figure 1 , Figure 1 A flowchart illustrating a method for cleaning up runtime records, provided in an embodiment of the present invention. The method may include: S101, the edge device generates a running record during the operation of the bare metal instance and writes the running record into the running record table. The running record is then pushed to the central device through an asynchronous data synchronization channel. The running record includes the addition time, end time, and data change time.
[0024] In this embodiment, the edge terminal can be an edge data center or edge node in a cloud computing platform. The run record table in this embodiment can be a run record data table generated during the operation of a bare metal instance, such as the `bmi_run_record` table. This embodiment does not limit the specific name of the run record table, as long as it is used to record the running status data generated by the bare metal instance during its operation at the edge. In this embodiment, the addition time is the time when the run record is written to the run record table, the end time is the time recorded when the corresponding running process of the bare metal instance ends, and the data change time is the time corresponding to when the run record is added, updated, or otherwise changed. This embodiment does not limit the specific field name of the data change time; it can be `optTime`, or it can be any other field used to indicate the time of the most recent data change in the run record. This embodiment does not limit the specific implementation method of the asynchronous data synchronization channel, as long as the asynchronous data synchronization channel can push the edge run record to the central terminal. The asynchronous data synchronization channel can be `DataSyncRoom`, or it can also be an existing asynchronous message push channel between the edge terminal and the central terminal.
[0025] It should be noted that when multiple nodes exist at the edge and a node needs to be selected to perform data push, a master node election can be used to determine the master node responsible for data push. This embodiment does not limit the specific method of master node election. For example, a ClusterVote mechanism can be used for master node election; alternatively, a Redis (distributed lock) lock can be used to determine the current master node responsible for data push. In one possible implementation, the lock corresponding to the master node can be renewed according to a preset period and expire after a preset expiration time. For example, the renewal period can be 30 seconds, and the expiration time can be 120 seconds. This embodiment uses the master node to perform data push, avoiding duplicate pushes or synchronization progress chaos caused by multiple edge nodes pushing simultaneously.
[0026] S102, the central end receives and persists the operation records pushed by the edge end, and after the persistence is completed, updates the consumption confirmation level corresponding to the operation record table; wherein, the consumption confirmation level is used to characterize the central end's synchronous consumption confirmation progress of the operation record table.
[0027] In this embodiment, the central endpoint acts as the data consumer for the runtime records. After receiving the runtime records pushed by the edge endpoint, the central endpoint persists the runtime records to the corresponding data storage structure at the central endpoint and updates the consumption confirmation level after persistence is complete. The consumption confirmation level is neither a time estimated by the edge endpoint based on its local time, nor a fixed cleanup time preset by the user, but rather progress information used by the central endpoint to record the synchronization progress of consumption confirmation in the runtime record table.
[0028] It should be noted that, based on any of the above embodiments, the central end receiving and persisting the operation records pushed by the edge end, and updating the consumption confirmation level corresponding to the operation record table after persistence, may include: after the central end completes receiving and persisting the operation records pushed by the edge end, updating the consumption confirmation level corresponding to the operation record table in the synchronization event table; the central end records the synchronization consumption confirmation progress of the operation record table in the central end through the consumption confirmation level. In this embodiment, the synchronization event table can be a `sync_event` table. This embodiment does not limit the synchronization event table to exist only in the central end or the edge end. The synchronization event table can be maintained in both the central end and the edge end; or the synchronization event table can be maintained by the central end and the consumption confirmation level can be synchronized to the data records readable by the edge end. This embodiment, by recording the consumption confirmation level in the synchronization event table, enables the edge end to obtain the central end's synchronization consumption confirmation progress corresponding to the operation record table when performing cleanup tasks.
[0029] S103, the edge responds to the cleanup task for the target data in the operation record table, and for each target data, obtains the target consumption confirmation level corresponding to the operation record table to which the target data belongs.
[0030] In this embodiment, the target data is the data in the operation log table that needs to be determined whether it can be cleaned up. The target data can be all operation records in the operation log table; or it can be a portion of the operation records that have been pre-selected from the operation log table. This embodiment does not limit the specific triggering method of the cleanup task. The cleanup task can be a scheduled task; or it can be a task triggered by the edge cleanup task framework according to a preset period.
[0031] It should be noted that, in response to the cleanup task for target data in the operation record table, the edge terminal obtains the target consumption confirmation level corresponding to the operation record table to which the target data belongs for each target data. This may include: the edge terminal determining the business table name field corresponding to the operation record table to which the target data belongs; the edge terminal querying the synchronization event record in the synchronization event table based on the business table name field; the edge terminal reading the consumption confirmation level from the queried synchronization event record and determining the read consumption confirmation level as the target consumption confirmation level corresponding to the target data. In this embodiment, the business table name field is used to identify the business table corresponding to the synchronization event record. For example, when the business table name field in the synchronization event table is "operation record table," it indicates that the synchronization event record corresponds to the operation record table. By querying the synchronization event record through the business table name field, the edge terminal can obtain the target consumption confirmation level that matches the operation record table to which the target data belongs, avoiding the mixing of consumption confirmation levels between different business tables.
[0032] In one specific example, the run log table is a data table that records the running status of bare metal instances, and the consumption confirmation level is read from this synchronization event record. In this embodiment, the business table name field is not the table name of the synchronization event table itself, but rather a field in the synchronization event table used to identify the corresponding business table. Querying the synchronization event record through the business table name field allows the edge device to obtain the consumption confirmation level that matches the run log table to which the target data belongs, preventing the misuse of synchronization consumption confirmation progress from other business tables for run log table cleanup.
[0033] It should be further explained that, in response to the cleanup task for target data in the operation record table, the edge terminal obtains the target consumption confirmation level corresponding to the operation record table to which the target data belongs for each target data. This can include: the edge terminal determining whether a consumption confirmation level exists corresponding to the target data; if the edge terminal does not find a synchronization event record corresponding to the operation record table to which the target data belongs, or the consumption confirmation level in the found synchronization event record is empty, the edge terminal determines that the target consumption confirmation level has not been obtained; after determining that a valid target consumption confirmation level has not been obtained, the edge terminal determines not to perform the deletion action for the target data; if the edge terminal obtains the target consumption confirmation level, it performs the step of determining candidate cleanup records based on the end time, addition time, data change time, and target consumption confirmation level of the target data. This embodiment will not perform the deletion action for the target data when a valid target consumption confirmation level is not obtained. The reason is that when the synchronization event record does not exist or the consumption confirmation level is empty, the edge terminal cannot determine whether the central terminal has completed the synchronization consumption confirmation of the corresponding operation record. If the deletion action is continued at this time, data that has not yet been confirmed for consumption by the central terminal may be deleted. This embodiment reduces the risk of accidental deletion by retaining target data when no valid target consumption confirmation level is obtained, thus ensuring that the cleanup task follows the processing logic of not deleting without confirmation.
[0034] In one optional implementation, a distributed lock can be acquired before the cleanup task is executed. This embodiment does not limit the specific implementation of the distributed lock. For example, a Redis distributed lock can be used; alternatively, a database lock or a distributed coordination component can be used to implement mutual exclusion. In a specific example, the lock key of the distributed lock can be a preset cleanup task identifier, and the lock timeout can be 5 seconds. This embodiment prevents multiple cleanup tasks from processing the same running record table simultaneously by using a cleanup task lock, thereby reducing concurrent deletion conflicts. It should be noted that the distributed lock is used to improve the stability of cleanup task execution and does not change the main judgment logic of determining candidate cleanup records based on the target consumption confirmation level.
[0035] S104, the edge determines candidate cleanup records based on the end time, addition time, data change time, and target consumption confirmation level of the target data; wherein, the addition time of the candidate cleanup record is earlier than the retention time determined based on the current time and the preset retention period, and the data change time of the candidate cleanup record is earlier than the target consumption confirmation level.
[0036] This embodiment does not limit the specific value of the preset retention period. For example, the preset retention period can be 7 days; or, the preset retention period can be adjusted according to the storage capacity of the edge device, the business investigation cycle, or the system configuration. In a specific example, if the current time is 07:00:00 on May 20, 2026, and the preset retention period is 7 days, then the retention time is 07:00:00 on May 13, 2026. In this case, only adding target data with a time earlier than 07:00:00 on May 13, 2026 will satisfy the local retention period condition.
[0037] It should be noted that the aforementioned edge-end determination of candidate cleanup records based on the end time, addition time, data change time, and target consumption confirmation level of the target data may include: the edge-end filtering target data with an end time from the target data in the operation record table, and filtering target data with an addition time earlier than the retention time from the target data with an end time; the edge-end filtering target data with a data change time earlier than the target consumption confirmation level from the target data that meets the retention time, and determining the filtered target data as candidate cleanup records. In this embodiment, the end time is used to determine whether the running process of the bare metal instance corresponding to the target data has ended. If the target data does not have an end time, it means that the target data may still correspond to a running bare metal instance and cannot be determined as a candidate cleanup record. This embodiment filters target data by end time to prevent running instance records from being deleted. In this embodiment, the addition time is used to determine whether the target data meets the local retention requirements. If the addition time of the target data is not earlier than the retention time, it means that the target data is still within the preset retention period and should not be cleaned up. This embodiment compares the addition time and the retention time to ensure that recent operation records can be retained on the edge-end for local querying or problem troubleshooting. In this embodiment, the data change time is used to determine whether the most recent change to the target data has fallen within the scope of synchronous consumption confirmation at the central end. If the data change time of the target data is earlier than the target consumption confirmation level, it means that the data change time of the target data is earlier than the synchronous consumption confirmation progress at the central end, and it can enter the candidate cleanup scope. If this comparison is not performed, and target data is only filtered based on the addition time, there may be cases where the target data was added earlier but has recently undergone changes, resulting in the accidental deletion of changed data that has not yet been confirmed for consumption at the central end.
[0038] It should be further explained that before the edge end filters target data whose data change time is earlier than the target consumption confirmation level from the target data that meets the retention time, and determines the filtered target data as candidate cleanup records, it may also include: the edge end determines whether the addition time is less than the retention time from the target data with an end time; if it is less than the retention time, it determines to execute the process of filtering target data whose data change time is earlier than the target consumption confirmation level from the data that meets the retention time, and determines the filtered target data as candidate cleanup records. This embodiment pre-excludes recently changed data by comparing the data change time with the retention time, so that recently changed data will not enter the subsequent consumption confirmation level comparison process.
[0039] It should also be noted that when data push is interrupted at the edge due to master node anomaly or network partition, the central end stops advancing the consumption confirmation level. During the period when the consumption confirmation level is not advancing, the edge end uses the stopped consumption confirmation level as the target consumption confirmation level. In other words, during the data push interruption, the central end cannot continue to receive and persist new operation records, so the consumption confirmation level stops advancing. At this time, the edge end continues to use the stopped consumption confirmation level to judge candidate cleanup records, ensuring that target data whose data change time is no earlier than the stopped consumption confirmation level will not be identified as candidate cleanup records. For example, if the consumption confirmation level stops at 08:30:00 on May 13, 2026, and the data change time of a target data is 09:00:00 on May 13, 2026, then the data change time of this target data is no earlier than the consumption confirmation level and cannot be identified as a candidate cleanup record. Even if, over time, the addition time of this target data meets the retention time condition, this target data will still be excluded because its data change time is no earlier than the target consumption confirmation level. This embodiment automatically converges the cleanup range to the range of confirmed consumption at the central end by confirming that the water level has stopped during consumption, preventing the cleanup range from expanding only due to the passage of local time during the synchronization interruption.
[0040] To facilitate understanding, the process of determining candidate cleanup records is explained below with specific data. Under normal synchronization, assuming there are approximately 50,000 operation records in the operation record table, the corresponding consumption confirmation level at the central end is 08:30:00 on May 13, 2026, the current time is 07:00:00 on May 20, 2026, and the preset retention period is 7 days, then the retention time is 07:00:00 on May 13, 2026. The edge end can filter target data from the operation record table that has an end time, an addition time earlier than 07:00:00 on May 13, 2026, and a data change time earlier than 08:30:00 on May 13, 2026 as candidate cleanup records. If the data change time of a target data is 09:00:00 on May 13, 2026, then this target data is later than the consumption confirmation level and will not be included in the candidate cleanup records.
[0041] S105, the edge cleans up the running records based on the candidate cleanup records.
[0042] In this embodiment, cleanup refers to deleting or releasing storage space for candidate cleanup records in the runtime record table. This embodiment does not limit the specific cleanup method. For example, the edge device can directly delete candidate cleanup records; alternatively, the edge device can first archive the candidate cleanup records to a preset storage location before deleting them from the runtime record table. For example, the edge device can set a maximum limit for the number of records deleted in a single batch, retrieve data from the candidate cleanup records batch by batch, and execute deletions based on the primary key. By setting a maximum limit for a single batch and using a cyclic execution method, the number of records deleted at once is controlled within a small range, preventing excessive database pressure caused by deleting a large number of runtime records at once.
[0043] The operation record cleanup method provided in this embodiment of the invention generates operation records at the edge and pushes them to the central end. After the central end completes persistence, it updates the consumption confirmation level corresponding to the operation record table. When cleaning target data in the operation record table, the edge end obtains the target consumption confirmation level and determines candidate cleanup records based on the end time, addition time, data change time, and target consumption confirmation level of the target data. Compared with the current method of cleaning operation records based solely on local time attributes, this embodiment constrains the cleanup scope of the edge end by controlling the synchronous consumption confirmation progress of the central end, ensuring that data not falling within the synchronous consumption confirmation scope of the central end is not cleaned, thereby improving the accuracy of operation record cleanup.
[0044] For a clearer understanding of this invention, please refer to the following details. Figure 2 , Figure 2 A flowchart illustrating a method for cleaning up runtime records provided in this embodiment of the invention may specifically include: S201: The edge device generates a running record during the operation of the bare metal instance and writes the running record to the running record table; the running record includes the addition time, end time, and data change time.
[0045] This embodiment can use distributed locks for mutual exclusion control before each cleanup execution to prevent multiple cleanup tasks from processing the same running record table simultaneously. During the cleanup execution, a batch cyclic deletion method can be adopted, deleting only a preset number of candidate cleanup records each time. By executing in batches, the amount of data deleted in a single operation is controlled within a small range, thereby reducing the instantaneous pressure on the database from a single deletion operation.
[0046] S202: The edge terminal pushes the operation record to the central terminal through an asynchronous data synchronization channel; when there is a master node election at the edge terminal, the elected master node performs the data push.
[0047] S203: The central terminal receives and persists the operation records pushed by the edge terminal, and after the persistence is completed, updates the consumption confirmation level in the synchronization event table corresponding to the operation record table.
[0048] In this embodiment, the cloud-edge data synchronization framework maintains the `sync_event` table (synchronization consumption confirmation table), with `table_name` (the identifier of the `sync_event` table) as the primary key. The `last_push_time` field (consumption confirmation level) is written in real-time by the central-side synchronization consumption process after data persistence is completed. Its semantics are "the central side has fully received and persisted all data change times earlier than this time." The key point about this level is that it is maintained and updated by the data consumer (central side), not preset by the data producer (edge end)—it reflects the actual consumption progress of the remote system, not a local time estimate.
[0049] S204: The edge device responds to the cleanup task for the operation record table by querying the synchronization event record in the synchronization event table based on the business table name field corresponding to the operation record table, and obtains the consumption confirmation level corresponding to the operation record table from the synchronization event record.
[0050] In this embodiment, when the cleanup task is executed, it first queries the sync_event table for records whose business table name field is the running record table. If the record does not exist or lastPushTime is empty (the synchronization channel has never completed the first push confirmation), the cleanup task returns directly, which is the zero-trust principle, and no data is deleted before obtaining consumption confirmation from the central side.
[0051] S205: The edge device determines whether the corresponding synchronization event record is found and whether the consumption confirmation level is empty.
[0052] S206: If the edge terminal does not find the corresponding synchronization event record, or the consumption confirmation level is empty, the edge terminal will end the cleanup task and will not delete the running record in the running record table.
[0053] S207: If the edge terminal obtains the consumption confirmation level, the edge terminal filters candidate cleanup records based on the end time, addition time, data change time, and obtained consumption confirmation level of the running record; wherein, the candidate cleanup record has an end time, the addition time of the candidate cleanup record is earlier than the retention time determined based on the current time and the preset retention period, and the data change time of the candidate cleanup record is earlier than the obtained consumption confirmation level.
[0054] In this embodiment, the cleanup conditions are combined with three dimensions: (1) The record must be in an ended state—active records of running instances are not deleted. (2) The record was added earlier than the retention time determined based on the current time and the preset retention period. (3) The record's data change time is earlier than the consumption confirmation level confirmed by the central side—this condition ensures that local deletion is only allowed after the remote system has explicitly confirmed consumption, which is the core condition that distinguishes it from general resource cleanup schemes. This level changes dynamically with the consumption progress of the central side, which is fundamentally different from the fixed user preset time period.
[0055] S208: Delete candidate cleanup records at the edge to reduce redundant historical records in the run log table.
[0056] S209: When data push is interrupted at the edge due to master node abnormality or network partition, the central end stops advancing the consumption confirmation level; the edge end continues to filter candidate cleanup records based on the stopped consumption confirmation level.
[0057] This implementation ensures that operation records with data changes no earlier than the consumption confirmation level are not deleted. When data push is interrupted, the consumption confirmation level stops advancing, and the edge continues to use the stopped level as the target consumption confirmation level. Since the range of records with data changes earlier than the level no longer expands, the cleanup operation automatically converges to the range of confirmed consumption on the central side before the interruption—even if the local retention period continues to widen over time, newly added cleanable data will not exceed the level boundary. This protection does not require the cleanup task to explicitly perceive the master node status; after the master node recovers, the level advances again, and cleanup resumes normally.
[0058] For ease of understanding, assume the operation log table contains approximately 50,000 operation records, the consumption confirmation watermark is 8:30 AM on May 13, 2026, the current time is 7:00 AM on May 20, 2026, and the preset retention period is 7 days. After the scheduled task is triggered, the edge device acquires a distributed lock and reads the consumption confirmation watermark. Then, it performs a three-dimensional filtering: filtering records with an end time, records whose addition time is earlier than the retention time, and records whose data change time is earlier than the consumption confirmation watermark. If a record's data change time is later than the consumption confirmation watermark, that record will not be included in the candidate cleanup records. After filtering, the edge device performs batch-by-batch deletion of the candidate cleanup records until no more matching records are found.
[0059] For ease of understanding, a water level stagnation protection mechanism is implemented during synchronization interruptions. For example, if the master node experiences a network partition loss, causing push interruptions, the consumption confirmation water level will stagnate at 08:30:00 on May 13th. A scheduled task will trigger on May 14th: If the data update time is less than the consumption confirmation water level, the cleanable data will still be limited to data before 08:30:00 on May 13th. Run records generated after 08:30:00 on May 13th, even if completed and exceeding the retention period, will not be included in the cleanup set because the data update time is greater than or equal to the consumption confirmation water level. This protection is not available in user-preset time-period solutions, which will continuously expand the cleanable range over time, potentially leading to accidental deletion of unsynchronized data during the water level stagnation period. After a new master node is selected and push resumes, the consumption confirmation water level will be updated to the latest value, and cleanup will resume.
[0060] For ease of understanding, zero-trust protection is applied before water level initialization. Synchronization event records corresponding to the run record table in the synchronization event table are missing or the consumption confirmation water level is empty. The cleanup task returns immediately without performing any deletions. All run records, regardless of their age, are retained.
[0061] The beneficial effects of this invention are as follows: First, this invention introduces data change time as a consumption confirmation level explicitly maintained by the central side. This level is updated in real time by the synchronous consumption process on the central side after data persistence is completed, indicating that the central side has fully received all records of data change times at that point in time. This level is not a fixed time period preset by the user, but a confirmation signal that dynamically changes with the consumption progress on the central side. Second, compared with the current cleanup protection that lacks self-convergence, this invention stops advancing the consumption confirmation level on the central side when data push is interrupted due to network partitioning or master node changes, thereby achieving self-convergence cleanup protection based on the central side consumption confirmation level. Third, the cleanup task does not need to be aware of whether there is a master node or whether synchronization is interrupted; it automatically adjusts the cleanup range only through the natural evolution or stagnation of the level.
[0062] The following describes a running record cleaning system provided by an embodiment of the present invention. The running record cleaning system described below can be referred to in correspondence with the running record cleaning method described above.
[0063] Please refer to the details. Figure 3 , Figure 3 A schematic diagram of a runtime record cleaning system provided in an embodiment of the present invention may include: The central terminal 100 is used to receive and persist the operation records pushed by the edge terminal, and after the persistence is completed, update the consumption confirmation level corresponding to the operation record table; wherein, the consumption confirmation level is used to characterize the synchronous consumption confirmation progress of the central terminal on the operation record table. Edge terminal 200 is used to generate operation records during the operation of bare metal instances, write the operation records into an operation record table, and push the operation records to the central terminal through an asynchronous data synchronization channel; wherein, the operation records include addition time, end time, and data change time; and in response to a cleanup task for target data in the operation record table, for each target data, obtain the target consumption confirmation level corresponding to the operation record table to which the target data belongs; and determine candidate cleanup records based on the end time, addition time, data change time, and target consumption confirmation level of the target data; wherein, the addition time of the candidate cleanup records is earlier than the retention time determined based on the current time and a preset retention period, and the data change time of the candidate cleanup records is earlier than the target consumption confirmation level; and clean up the operation records based on the candidate cleanup records.
[0064] Compared with the current method of cleaning up running records based solely on local time attributes, the running record cleaning system provided in this embodiment of the invention can constrain the cleaning scope of the edge end by synchronizing the consumption confirmation progress at the central end, so that data that has not been confirmed for consumption by the central end will not be cleaned up, thereby improving the accuracy of running record cleaning.
[0065] The following describes an electronic device provided by an embodiment of the present invention. The electronic device described below can be referred to in correspondence with the running record cleaning method described above.
[0066] Please refer to Figure 4 , Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present invention may include: Memory 10 is used to store computer programs; The processor 20 is used to execute computer programs to implement the steps of the above-described method for cleaning up running records.
[0067] The memory 10, processor 20, and communication interface 30 all communicate with each other through the communication bus 40.
[0068] The aforementioned electronic devices can be central or peripheral.
[0069] In this embodiment of the invention, the memory 10 is used to store one or more programs. The programs may include program code, which includes computer operation instructions. In this embodiment of the invention, the memory 10 may store programs for implementing the following functions: The edge device generates a running record during the operation of the bare metal instance and writes the running record to the running record table. The running record is then pushed to the central device through an asynchronous data synchronization channel. The running record includes the addition time, end time, and data change time. The central terminal receives and persists the operation records pushed by the edge terminal, and after the persistence is completed, updates the consumption confirmation level corresponding to the operation record table; the consumption confirmation level is used to characterize the central terminal's synchronous consumption confirmation progress of the operation record table. The edge responds to the cleanup task for the target data in the operation record table, and for each target data, obtains the target consumption confirmation level corresponding to the operation record table to which the target data belongs; At the edge, candidate cleanup records are determined based on the end time, addition time, data change time, and target consumption confirmation level of the target data. Among these, the addition time of the candidate cleanup record is earlier than the retention time determined based on the current time and the preset retention period, and the data change time of the candidate cleanup record is earlier than the target consumption confirmation level. The edge device cleans up the running records based on the candidate cleanup records.
[0070] In one possible implementation, the memory 10 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; and the data storage area may store data created during use.
[0071] Furthermore, memory 10 may include read-only memory and random access memory, providing instructions and data to the processor. A portion of the memory may also include NVRAM. The memory stores operating systems and operating instructions, executable modules, or data structures, or subsets thereof, or extended sets thereof, wherein the operating instructions may include various operating instructions for implementing various operations. The operating system may include various system programs for implementing various basic tasks and handling hardware-based tasks.
[0072] Processor 20 can be a central processing unit (CPU), an application-specific integrated circuit, a digital signal processor, a field-programmable gate array, or other programmable logic device. Processor 20 can be a microprocessor or any conventional processor. Processor 20 can call programs stored in memory 10.
[0073] The communication interface 30 can be an interface for the communication module, used to connect with other devices or systems.
[0074] Of course, it should be noted that, Figure 4 The structure shown does not constitute a limitation on the electronic device in the embodiments of the present invention. In practical applications, the electronic device may include more than Figure 4 More or fewer components as shown, or combinations of certain components.
[0075] The following describes the computer-readable storage medium provided in the embodiments of the present invention. The computer-readable storage medium described below can be referred to in correspondence with the running record cleaning method described above.
[0076] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described method for cleaning up running records.
[0077] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0078] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.
[0079] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.
[0080] Finally, it should be noted that in this document, relationships such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.
[0081] The above provides a detailed description of the operation record cleaning method, system, device, and readable storage medium provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A method for cleaning up runtime records, characterized in that, include: The edge device generates a running record during the operation of the bare metal instance and writes the running record into a running record table. The running record is then pushed to the central device through an asynchronous data synchronization channel. The running record includes the addition time, end time, and data change time. The central terminal receives and persists the operation records pushed by the edge terminal, and after persistence is completed, updates the consumption confirmation level corresponding to the operation record table; wherein, the consumption confirmation level is used to characterize the synchronous consumption confirmation progress of the central terminal on the operation record table. The edge terminal responds to the cleanup task for the target data in the operation record table, and for each target data, obtains the target consumption confirmation level corresponding to the operation record table to which the target data belongs; The edge terminal determines candidate cleanup records based on the end time, addition time, data change time, and target consumption confirmation level of the target data; wherein, the addition time of the candidate cleanup record is earlier than the retention time determined based on the current time and a preset retention period, and the data change time of the candidate cleanup record is earlier than the target consumption confirmation level; The edge terminal cleans the running records based on the candidate cleanup records.
2. The method for cleaning up operation records according to claim 1, characterized in that, The edge terminal responds to a cleanup task for target data in the operation record table, and for each target data, obtains the target consumption confirmation level corresponding to the operation record table to which the target data belongs, including: The edge terminal determines the business table name field corresponding to the operation record table to which the target data belongs; The edge terminal queries the synchronization event record in the synchronization event table based on the business table name field; The edge device reads the consumption confirmation level from the queried synchronization event records and determines the read consumption confirmation level as the target consumption confirmation level corresponding to the target data.
3. The method for cleaning up operation records according to claim 1, characterized in that, The edge terminal responds to a cleanup task for target data in the operation record table, and for each target data, obtains the target consumption confirmation level corresponding to the operation record table to which the target data belongs, including: The edge terminal determines whether there is a consumption confirmation level corresponding to the target data; If the edge terminal does not find a synchronization event record corresponding to the running record table to which the target data belongs, or if the consumption confirmation level in the found synchronization event record is empty, then the edge terminal determines that it has not obtained the target consumption confirmation level. After determining that a valid target consumption confirmation level has not been obtained, the edge terminal decides not to perform the deletion action on the target data. If the edge terminal obtains the target consumption confirmation level, then the edge terminal executes the step of determining candidate cleanup records based on the end time, addition time, data change time, and target consumption confirmation level of the target data.
4. The method for cleaning up operation records according to any one of claims 1 to 3, characterized in that, The edge endpoint determines candidate cleanup records based on the end time, addition time, data change time, and target consumption confirmation level of the target data, including: The edge terminal filters target data with an end time from the target data in the operation record table, and filters target data with an addition time earlier than the retention time from the target data with an end time; The edge end filters target data whose data change time is earlier than the target consumption confirmation level from the target data that meets the retention time, and determines the filtered target data as the candidate cleanup record.
5. The method for cleaning up operation records according to claim 4, characterized in that, Before selecting target data with a data change time earlier than the target consumption confirmation level from the target data that meets the retention time at the edge end, and determining the selected target data as the candidate cleanup record, the process further includes: The edge terminal determines from the target data with an end time whether the addition time is less than the retention time; When the retention time is less than the specified retention time, it is determined that target data with a data change time earlier than the target consumption confirmation level will be selected from the data that meets the retention time, and the selected target data will be determined as the candidate cleanup record.
6. The method for cleaning up operation records according to claim 1, characterized in that, Before obtaining the target consumption confirmation level corresponding to the operation record table to which the target data belongs for each target data in response to a cleanup task for target data in the operation record table at the edge end, the method further includes: When the data push is interrupted at the edge due to a master node abnormality or network partition, the central end stops advancing the consumption confirmation level. The edge end uses the consumption confirmation water level at which the advance stops as the target consumption confirmation water level during the period when the consumption confirmation water level stops advancing.
7. The method for cleaning up operation records according to claim 1, characterized in that, The central terminal receives and persists the operation records pushed by the edge terminal, and after persistence is completed, updates the consumption confirmation level corresponding to the operation record table, including: After receiving and persisting the operation records pushed by the edge terminal, the central terminal updates the consumption confirmation level in the synchronization event table corresponding to the operation record table. The central terminal records the synchronous consumption confirmation progress of the operation record table at the central terminal through the consumption confirmation water level.
8. A system for cleaning up operation records, characterized in that, include: The central terminal is used to receive and persist the operation records pushed by the edge terminal, and after the persistence is completed, update the consumption confirmation level corresponding to the operation record table; wherein, the consumption confirmation level is used to characterize the synchronous consumption confirmation progress of the central terminal on the operation record table. At the edge, a process is used to generate operation records during the operation of a bare metal instance, write the operation records into an operation record table, and push the operation records to the central end via an asynchronous data synchronization channel. The operation records include an addition time, an end time, and a data change time. In response to a cleanup task for target data in the operation record table, for each target data, a target consumption confirmation level corresponding to the operation record table to which the target data belongs is obtained. Candidate cleanup records are determined based on the end time, addition time, data change time, and target consumption confirmation level of the target data. The addition time of the candidate cleanup records is earlier than the retention time determined based on the current time and a preset retention period, and the data change time of the candidate cleanup records is earlier than the target consumption confirmation level. The operation records are then cleaned based on the candidate cleanup records.
9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the runtime cleanup method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the runtime record cleaning method as described in any one of claims 1 to 7.