Data deletion method and device, equipment and storage medium

Abstract

The invention provides a data deletion method and device, equipment and a storage medium, and the method comprises the steps: obtaining to-be-deleted data in a storage system according to a data deletion instruction, and marking the state of the to-be-deleted data as a deletion state; according to the association relationship between the to-be-deleted data and each piece of target data, and the configuration information and the running log of the storage system, judging whether to trigger a deletion operation on the to-be-deleted data; the target data is data marked as a deleted state except the to-be-deleted data in the storage system; and deleting the to-be-deleted data from the physical storage space of the storage system under the condition that the deletion operation on the to-be-deleted data is determined to be triggered. According to the method and the device, the data is deleted by a multi-deletion strategy combining logic deletion and physical deletion, so that the risk of important data loss is reduced, and the data processing performance is guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a data deletion method, device, equipment and storage medium. Background Art

[0002] In the process of building and operating modern online information systems, the continuous growth and complexity of data is an inevitable trend. With the continuous enrichment and expansion of functions, the amount of data that the system needs to process has increased dramatically, and the correlation of data has become more complex. This trend not only puts higher requirements on data storage capabilities, but also brings challenges to the flexibility and security of data management. As an important part of data management, the design and implementation of data deletion mechanisms are directly related to the stability of the system and the security of data. Therefore, how to effectively delete data is an important issue that the industry needs to solve urgently.

[0003] In the related art, data is physically deleted. This deletion method is to actually remove the data records to be deleted from the information system. When the user deletes the data by mistake or accidentally, the data will be permanently deleted indiscriminately. As a result, some important data can be easily lost permanently due to user error, greatly increasing the risk of important data loss.

[0004] Therefore, there is an urgent need for a data deletion method, device, equipment and storage medium to solve the above technical problems. Summary of the invention

[0005] The present invention provides a data deletion method, device, equipment and storage medium to solve the defect in the prior art that physical deletion causes important data to be easily permanently lost due to user misoperation, which greatly increases the risk of important data loss, thereby reducing the risk of important data loss while ensuring data processing performance.

[0006] The present invention provides a data deletion method, comprising: According to the data deletion instruction, the data to be deleted is obtained in the storage system, and the state of the data to be deleted is marked as a deletion state; According to the association relationship between the data to be deleted and each target data, as well as the configuration information and operation log of the storage system, it is determined whether to trigger a deletion operation on the data to be deleted; the target data is data in the storage system that is marked as deleted except the data to be deleted; In the case where it is determined that a deletion operation on the data to be deleted is triggered, the data to be deleted is deleted from the physical storage space of the storage system.

[0007] According to a data deletion method provided by the present invention, judging whether to trigger a deletion operation on the data to be deleted based on the association relationship between the data to be deleted and each target data, and the configuration information and operation log of the storage system, includes: According to the association relationship between the data to be deleted and each of the target data, determining whether there is associated data associated with the data to be deleted in the plurality of target data, and obtaining a first determination result; According to the configuration information of the storage system, a target delayed deletion duration is obtained; Determine whether the duration for which the to-be-deleted data is recorded as deleted exceeds the target delayed deletion duration, and obtain a second determination result; According to the operation log of the storage system, the data call frequency of the storage system in each operation period is obtained; Among the plurality of operating time periods, obtaining an operating time period in which the frequency of data call is lower than a preset value as a target operating time period; Determine whether the current time belongs to the target operation time period, and obtain a third determination result; The first judgment result, the second judgment result and the third judgment result are combined to determine whether to trigger a deletion operation on the data to be deleted.

[0008] According to a data deletion method provided by the present invention, the combining of the first judgment result, the second judgment result and the third judgment result to determine whether to trigger a deletion operation on the data to be deleted includes: In the case where it is determined according to the first judgment result that there is no associated data associated with the data to be deleted among the multiple target data, if it is determined according to the second judgment result that the duration for which the data to be deleted is recorded as deleted exceeds the target delayed deletion duration, and it is determined according to the third judgment result that the current moment belongs to the target operating time period, then it is determined that the deletion operation of the data to be deleted is triggered.

[0009] According to a data deletion method provided by the present invention, the method further includes: In a case where it is determined according to the first judgment result that at least one associated data associated with the to-be-deleted data exists in the plurality of target data, acquiring a triggering status of each of the associated data; When it is determined that a deletion operation on any associated data is triggered according to the trigger state, the data to be deleted and the any associated data are deleted in batches from the physical storage space of the storage system.

[0010] According to a data deletion method provided by the present invention, obtaining a target delayed deletion duration according to configuration information of the storage system includes: According to the configuration information of the storage system, obtaining multiple items of data storage requirement information, storage capacity information, performance requirement information and data recovery requirement information of the storage system; Scoring each preset delayed deletion time according to multiple items of the data storage requirement information, the storage capacity information, the performance requirement information, and the data recovery requirement information to obtain a score value corresponding to each preset delayed deletion time; According to the score value, the target delayed deletion time length is obtained from the plurality of preset delayed deletion time lengths.

[0011] According to a data deletion method provided by the present invention, the method further includes: When performing a deletion action on the data to be deleted, recording a deletion timestamp of the data to be deleted, table information of a data table to which the data to be deleted belongs, index information of the data to be deleted in the data table, and a deletion result of the data to be deleted; When the deletion action corresponding to the data to be deleted is completed, marking the state of the data to be deleted as a deletion completed state; A deletion report of the data to be deleted is generated according to the deletion timestamp, the table information, the index information, and the deletion result, and the deletion report is output.

[0012] According to a data deletion method provided by the present invention, the method further includes: When it is determined that the data to be deleted is marked as being in a deletion completed state, searching the storage system for the data to be deleted according to the table information of the data table and the index information; If the search is empty, it is determined that the data to be deleted is deleted correctly; If the data to be deleted is found, the state of the data to be deleted is re-marked as a deleted state.

[0013] The present invention also provides a data deleting device, comprising: A marking module, used for obtaining the data to be deleted in the storage system according to the data deletion instruction, and marking the state of the data to be deleted as a deletion state; a judgment module, configured to judge whether to trigger a deletion operation on the data to be deleted according to the association relationship between the data to be deleted and each target data, as well as the configuration information and operation log of the storage system; the target data is data in the storage system that is marked as deleted except the data to be deleted; The deleting module is used to delete the data to be deleted from the physical storage space of the storage system when it is determined that a deletion operation on the data to be deleted is triggered.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-mentioned data deletion methods is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the data deletion method described in any one of the above methods is implemented.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned data deletion methods.

[0017] The data deletion method, device, equipment and storage medium provided by the present invention can first logically delete the data to be deleted after obtaining the data deletion instruction to ensure that the data can be restored at any time, which significantly reduces the risk of permanent loss of important data due to user error operation or accidental deletion. Before executing data deletion, the association relationship between the data to be deleted and other target data, system configuration information and operation logs are comprehensively evaluated to ensure that the physical deletion operation is triggered only under safe and appropriate circumstances, further reducing the risk of permanent loss of important data due to user error operation or accidental deletion, and realizing the timely release of storage space, effectively reducing storage costs and improving data processing performance, thereby improving the security, reliability and performance of data management. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 This is one of the flow charts of the data deletion method provided by the present invention.

[0020] Figure 2 This is the second flow chart of the data deletion method provided by the present invention.

[0021] Figure 3 It is a flowchart of deletion verification provided by the present invention.

[0022] Figure 4 It is a structural schematic diagram of the data deleting device provided by the present invention.

[0023] Figure 5It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] Usually, online information systems will add more functions and associated data as time goes by. In order to obtain the association and statistical data of these records, the association of data will become more and more complex. In addition to becoming larger and more complex over time, online data will also begin to need to be deleted. However, directly deleting data from the information system may cause data loss, performance problems or other unforeseen risks. Therefore, the design and implementation of the deletion mechanism is very important. This involves how to efficiently and securely handle data deletion operations while ensuring system stability and data consistency.

[0026] In the related art, data is physically deleted. This deletion method is to actually remove the data records to be deleted from the information system. When the user deletes the data by mistake or accidentally, the data will be permanently deleted indiscriminately. As a result, some important data can be easily lost permanently due to user error, greatly increasing the risk of important data loss.

[0027] Although some technologies have proposed logical deletion of data, this deletion method does not delete the data from the actual information system, but instead indicates that the data has been deleted by marking it. This method allows data recovery and is often used in scenarios where deletion records need to be retained. However, this deletion method, because it marks the data as deleted rather than physically deleting the data from the information system immediately, causes the deleted data to still exist in the table, which increases storage requirements and does not immediately release storage space in the information system, which may lead to a continuous increase in the amount of data in the information system and increase storage costs; and the deleted data still exists in the main data table, and additional conditions are required to filter out the deleted data when querying, especially when the amount of data is very large, which will lead to a decrease in query performance. Therefore, the data processing performance of this deletion method is low.

[0028] Some technologies also propose to soft delete data, that is, to move the data to an archive table or deleted table. Similar to logical deletion, the data is actually removed and can be restored later. However, this deletion method increases the complexity of information system maintenance and requires additional logic and maintenance to manage the archive table, which in turn leads to a decrease in query performance. Therefore, this deletion method has low data processing performance.

[0029] In this regard, the present application provides a data deletion method, which includes: when a data deletion instruction is obtained, first using a data marker to mark the data to be deleted without immediately performing physical deletion; in order to improve the performance of the deletion operation, a strategy of delayed deletion tasks and scheduled tasks is adopted, and the data marked as deleted is placed in a deletion queue, and Jenkins (continuous integration tool) is used for automatic scheduling to perform batch physical deletion operations during time periods when the storage system is less frequently used. In addition, in order to ensure the thoroughness and accuracy of data deletion, in addition to configuring deletion log records, a method for integrity verification query is established to ensure that the records marked for deletion have been completely deleted, thereby achieving data deletion through a strategy that combines logical deletion with physical deletion to ensure the security of important data while ensuring data processing performance.

[0030] Figure 1 This is one of the flow charts of the data deletion method provided by the present invention; the execution subject of the method is a data deletion device, which can be configured with a remote dictionary service (Remote Dictionary Server, Redis) database and a Jenkins automation tool. A scheduled task environment can be configured in the Jenkins automation tool to ensure that Jenkins can execute related Shell scripts or Java programs; in addition, a Redis client, such as Jedis, is integrated in the information system.

[0031] like Figure 1 As shown, the method includes: step 110, step 120 and step 130.

[0032] Step 110: According to the data deletion instruction, the data to be deleted is obtained from the storage system, and the state of the data to be deleted is marked as a deletion state.

[0033] The storage system here can be an information system for various data management, such as a Java information system, a file system, etc. This embodiment does not specifically limit this. The following describes the data deletion method provided in the embodiment of the present application using the Java information system as an example.

[0034] The data deletion instruction here is an instruction for deleting redundant data in the storage system. The data deletion instruction here can be user input entered by the user on the front-end interface, or it can be information generated under certain conditions by other controllers or other devices inside the storage system, such as a timer-triggered data deletion instruction. The so-called user input can be information input through a command line interface, a graphical interface, touch input, a drop-down selection input, voice input, gesture input, visual input, brain-computer input, etc. This embodiment does not specifically limit this.

[0035] Figure 2 FIG. 2 is a flow chart of the data deletion method provided by the present invention; Figure 2 As shown, after obtaining the data deletion instruction, the data deletion instruction may be parsed to obtain the identification information of the data to be deleted, so as to locate the location of the data to be deleted in the storage system according to the identification information, and find the data to be deleted accordingly. After finding the data to be deleted, the state of the data to be deleted is marked as deleted state.

[0036] The data marking here can add a status marking field "del_flag" for marking each data and a deletion timestamp marking field "del_time" for marking each data in the data table to which the data to be deleted belongs, so that after receiving the data deletion instruction, the value of the status marking field of the data to be deleted is marked as the value corresponding to the deletion state, and the state of the data to be deleted is marked as the deletion state, that is, the data to be deleted is logically deleted, instead of immediately deleting the data to be deleted from the storage system. When the background accesses the data in the storage system, it can filter the undeleted data according to the status marking field to improve the data processing performance. At the same time, in order to facilitate the subsequent execution of the deletion operation and verification operation, the status mark and the deletion timestamp mark of the data to be deleted can also be synchronized to the Redis database. The Redis database here may include fields including but not limited to fields for storing table information of the data table of the data to be deleted, fields for storing index information of the data to be deleted in the data table, fields for storing the status mark of the data to be deleted, fields for storing the deletion timestamp of the data to be deleted, and fields for storing the deletion results of the data to be deleted, etc., which are not specifically limited in this embodiment.

[0037] Step 120, judging whether to trigger a deletion operation on the data to be deleted based on the association relationship between the data to be deleted and each target data, as well as the configuration information and operation log of the storage system; the target data is the data in the storage system marked as deleted except the data to be deleted.

[0038] Optionally, after marking the status of the data to be deleted as deleted, data marked as deleted other than the data to be deleted can be acquired in real time in the storage system as target data, and the dependency between the data to be deleted and other data marked as deleted can be analyzed.

[0039] In addition, the configuration information and operation log of the storage system are obtained in real time; the configuration information here includes various pre-configured information associated with the deletion operation, including but not limited to multiple items of data storage requirement information, storage capacity information, performance requirement information, and data recovery requirement information. The operation log here is used to analyze the operation performance of the storage system, thereby formulating a more high-performance deletion operation.

[0040] Subsequently, a suitable deletion policy is configured for the data to be deleted based on the association between the data to be deleted and each target data, as well as the configuration information and operation log of the storage system, thereby determining whether to trigger the deletion operation on the data to be deleted based on the deletion policy.

[0041] The deletion policy here includes but is not limited to deletion conditions and scheduled deletion time, which is used to find a balance between logical deletion and physical deletion, or to add regular archiving and cleanup operations based on soft deletion.

[0042] The deletion strategy here can be determined by multiple logical judgments based on the association between the data to be deleted and each target data, as well as the configuration information and operation log of the storage system, or it can be determined by inputting the association between the data to be deleted and each target data, as well as the configuration information and operation log of the storage system into a classification model pre-trained based on a sample data set for classification, etc. This embodiment does not specifically limit this, and can be adaptively determined based on the computing performance of the execution subject of the data deletion method. Among them, the sample data set can be constructed with the association between the sample deletion data and each target data, the configuration information and operation log of the sample storage system as samples, and the deletion strategy actually used by the sample deletion data as labels.

[0043] In some embodiments, after the automatic deletion mechanism is started, the configuration information and operation log of the storage system can be regularly optimized and updated according to the real-time parameters of the storage system, so as to optimize the corresponding deletion strategy according to the updated configuration information and operation log of the storage system, thereby improving the performance and reliability of the data deletion operation. The real-time parameters here include but are not limited to real-time business adjustment parameters and real-time operation parameters. For example, the real-time business adjustment parameters and real-time operation parameters can be used as adjustment parameters to regularly update the configuration information and operation log of the storage system, so as to optimize the deletion strategy based on the updated configuration information and operation log, and then optimize the deletion conditions of the data deletion operation and the frequency of the scheduled deletion time trigger, thereby improving the performance and reliability of the data deletion operation.

[0044] Step 130: When it is determined that a deletion operation on the data to be deleted is triggered, the data to be deleted is deleted from the physical storage space of the storage system.

[0045] like Figure 2 As shown, in the case where it is determined that the deletion operation on the data to be deleted is triggered, it indicates that the data to be deleted has satisfied the deletion condition. At this time, the data to be deleted can be deleted from the physical storage space of the storage system, thereby realizing the physical deletion of the data to be deleted, and then freeing up the storage space of the storage system, reducing storage costs, and avoiding the problem of low data processing performance caused by the need to add additional filtering conditions when querying data, thereby improving data processing performance.

[0046] The method provided in this embodiment can first logically delete the data to be deleted after obtaining the data deletion instruction to ensure that the data can be restored at any time, which significantly reduces the risk of permanent loss of important data due to user error or accidental deletion. Before executing data deletion, the relationship between the data to be deleted and other target data, system configuration information and operation logs are comprehensively evaluated to ensure that the physical deletion operation is triggered only under safe and appropriate circumstances, further reducing the risk of permanent loss of important data due to user error or accidental deletion, and realizing the timely release of storage space, effectively reducing storage costs and improving data processing performance, thereby improving the security, reliability and performance of data management.

[0047] In some embodiments, the triggering judgment of the deletion operation in step 120 may be implemented based on the following steps: According to the association relationship between the data to be deleted and each of the target data, determining whether there is associated data associated with the data to be deleted in the plurality of target data, and obtaining a first determination result; According to the configuration information of the storage system, a target delayed deletion duration is obtained; Determine whether the duration for which the to-be-deleted data is recorded as deleted exceeds the target delayed deletion duration, and obtain a second determination result; According to the operation log of the storage system, the data call frequency of the storage system in each operation period is obtained; Among the plurality of operating time periods, obtaining an operating time period in which the frequency of data call is lower than a preset value as a target operating time period; Determine whether the current time belongs to the target operation time period, and obtain a third determination result; The first judgment result, the second judgment result and the third judgment result are combined to determine whether to trigger a deletion operation on the data to be deleted.

[0048] Optionally, after logically deleting the data to be deleted in step 110, the following steps may be performed to implement pre-judgment before physical deletion of the data to be deleted, so as to ensure data security and improve data processing performance: Traverse all target data in the storage system (that is, data marked as deleted except the data to be deleted), and determine whether there is data directly associated with the data to be deleted through the association relationship between the data, so as to realize the associated deletion judgment of the data and obtain the first judgment result, thereby ensuring the integrity of the data deletion.

[0049] In addition, in order to improve the performance of the deletion operation, the configuration information of the storage system can be queried to adaptively determine the delayed deletion strategy based on the configuration information of the storage system, so as to obtain the corresponding target delayed deletion duration based on the delayed deletion strategy. The duration is adaptively set according to the configuration information of the storage system, such as 20 days, 30 days, etc.; this duration can be used to ensure that after the data is logically deleted, there is still a period of time for it to be restored or otherwise processed, thereby ensuring the security of important data. After obtaining the target delayed deletion duration, the duration of time that the data to be deleted is recorded as deleted can be compared with the target delayed deletion duration to implement a delayed deletion judgment on the data and obtain a second judgment result, thereby ensuring the performance of the deletion operation and improving data security.

[0050] The target delayed deletion time here can be obtained by logically scoring multiple preset delayed deletion time periods based on the configuration information of the storage system, or it can be obtained by inputting the configuration information of the storage system into a pre-trained classification model for identification. This embodiment does not make specific limitations on this.

[0051] Exemplarily, in some embodiments, the step of obtaining the target delayed deletion duration specifically includes: According to the configuration information of the storage system, obtaining multiple items of data storage requirement information, storage capacity information, performance requirement information and data recovery requirement information of the storage system; Scoring each preset delayed deletion time according to multiple items of the data storage requirement information, the storage capacity information, the performance requirement information, and the data recovery requirement information to obtain a score value corresponding to each preset delayed deletion time; According to the score value, the target delayed deletion time length is obtained from the plurality of preset delayed deletion time lengths.

[0052] Optionally, in the process of obtaining the target delayed deletion duration, it is necessary to extract key data management requirement information from the detailed configuration information of the storage system, including multiple items of data storage requirement information, storage capacity information, performance requirement information, and data recovery requirement information. Subsequently, these data management requirement information are used as evaluation criteria to quantitatively score the multiple preset delayed deletion durations. By comparing the scoring results, the preset delayed deletion duration that best suits the current data management requirement information is intelligently selected from the multiple preset delayed deletion durations as the target delayed deletion duration, thereby dynamically optimizing the data deletion strategy according to the specific configuration information of the storage system, ensuring that the data is cleared at the right time, neither causing the risk of data loss too early nor occupying too many storage resources too late, thereby improving the efficiency and security of data management.

[0053] The scoring value corresponding to the preset delayed deletion time here can be the key information obtained from the configuration information and the preset delayed deletion time input into the scoring model, so that the scoring model can evaluate the impact of each preset delayed deletion time on each key information, thereby determining the coding value corresponding to each key information under each preset delayed deletion time, and scoring each preset delayed deletion time according to the coding value corresponding to each key information to obtain a corresponding scoring value.

[0054] In addition, in order to further improve the performance of the deletion operation, the operation log of the storage system can be queried to obtain the data call frequency of the storage system in each operating period, so as to select the operating period with a data call frequency lower than the preset value as the target operating period among multiple operating periods, that is, to obtain the off-peak period of data access, thereby determining the best time to perform the physical deletion operation, that is, the time with the least impact on system performance; then, after obtaining the target operating period, it can be determined whether the current moment belongs to the target operating period, so as to realize the timed deletion judgment of the data and obtain the third judgment result, thereby ensuring the performance of the deletion operation and improving data security.

[0055] It should be noted that the above-mentioned determination steps may be executed sequentially or synchronously, which may be specifically determined according to the operating performance of the execution subject, and this embodiment does not specifically limit this.

[0056] Subsequently, after obtaining the first judgment result, the second judgment result and the third judgment result, the above three judgment results can be comprehensively considered to perform multiple logical judgments to determine the deletion trigger status of the data to be deleted. Therefore, based on the deletion trigger status of the data to be deleted, it is effectively determined whether to trigger the deletion operation of the data to be deleted.

[0057] The method provided in the embodiment of the present application effectively improves the security and efficiency of data deletion operations through data marking and comprehensive consideration of data association relationships, delayed deletion duration and scheduled deletion tasks, ensuring that storage space is released in a timely manner and storage costs are reduced without affecting system performance and data integrity and ensuring data security.

[0058] In some embodiments, step 120 further includes: In the case where it is determined according to the first judgment result that there is no associated data associated with the data to be deleted among the multiple target data, if it is determined according to the second judgment result that the duration for which the data to be deleted is recorded as deleted exceeds the target delayed deletion duration, and it is determined according to the third judgment result that the current moment belongs to the target operating time period, then it is determined that the deletion operation of the data to be deleted is triggered.

[0059] Optionally, when it is determined according to the first judgment result that there is no associated data associated with the data to be deleted among the multiple target data, if it is determined according to the second judgment result that the duration for which the data to be deleted is recorded as deleted exceeds the target delayed deletion duration, that is, the duration of the deletion status of the data to be deleted is long, if it is further determined according to the third judgment result that the current time belongs to the off-peak operating period of the storage system (that is, the target operating period), such as 00:00-00:30 every day, that is, the scheduled deletion condition is met, at this time, the deletion operation of the data to be deleted is triggered to start the actual physical deletion mechanism to delete the data to be deleted from the physical storage space of the storage system, thereby ensuring that when the data to be deleted has no associated data and meets the conditions of a specific delay duration and operating period, the deletion operation of the data to be deleted is automatically triggered, thereby achieving safe and accurate data cleaning.

[0060] It should be noted that the above-mentioned scheduled deletion task can be configured in the Jenkins automation tool, so that the Jenkins automation tool executes the corresponding Shell script to trigger the corresponding data deletion operation during the off-peak operation period for the data to be deleted that has no associated data and has been in the deleted state for a long time, so as to select the time period with low system usage to trigger the data deletion operation, thereby ensuring the performance of the deletion operation.

[0061] Among them, the Jenkins automation tool builds a corresponding timer according to the off-peak operation period of the storage system. The exemplary construction code is as follows: "java -cp app.jar com.workspace.DataCleanJob".

[0062] And use Cron expressions to set the timer trigger period in the constructed timer so that the Jenkins automation tool can be triggered regularly during the corresponding period to perform data deletion operations.

[0063] In some embodiments, step 120 further includes: when it is determined according to the first judgment result that at least one associated data associated with the to-be-deleted data exists in the plurality of target data, obtaining a trigger status of each of the associated data; When it is determined that a deletion operation on any associated data is triggered according to the trigger state, the data to be deleted and the any associated data are deleted in batches from the physical storage space of the storage system.

[0064] Optionally, before physical deletion, if it is determined based on the first judgment result that there is at least one related data associated with the data to be deleted among the multiple target data, the trigger status of each related data is further queried, that is, whether each related data satisfies the deletion trigger judgment before physical deletion. The specific judgment method can be found in step 120, which will not be repeated here.

[0065] If it is detected that the trigger status of any associated data meets the deletion trigger judgment before physical deletion (that is, any associated data has no associated target data and meets the conditions of specific delay duration and off-peak operation period), these associated data and the data to be deleted will be removed from the physical storage space of the storage system through batch deletion operations, so as to reduce the overhead caused by individual deletion operations, while ensuring that the consistency and integrity of the data are not destroyed, thereby effectively improving the utilization rate of data storage space and the efficiency of data management, and reducing the impact of individual deletion operations on system performance.

[0066] In some implementations, step 120 further includes: In the case where it is determined according to the first judgment result that there is no associated data associated with the data to be deleted among the multiple target data, if it is determined according to the second judgment result that the duration for which the data to be deleted is recorded as deleted does not exceed the target delayed deletion duration, and / or it is determined according to the third judgment result that the current moment does not belong to the target operating time period, it is determined that the deletion operation of the data to be deleted is not triggered, and the process returns to step 120, and the above three judgment processes are iteratively executed until it is determined that there is associated data associated with the data to be deleted among the multiple target data, and the deletion operation of any associated data can be triggered, or it is determined that there is no associated data associated with the data to be deleted among the multiple target data, and it is determined that the duration for which the data to be deleted is recorded as deleted exceeds the target delayed deletion duration, and it is determined that the current moment belongs to the target operating time period, then it is determined that the deletion operation of the data to be deleted is triggered, and the data to be deleted is deleted from the physical storage space of the storage system.

[0067] In some embodiments, the method further comprises: When performing a deletion action on the data to be deleted, recording a deletion timestamp of the data to be deleted, table information of a data table to which the data to be deleted belongs, index information of the data to be deleted in the data table, and a deletion result of the data to be deleted; When the deletion action corresponding to the data to be deleted is completed, marking the state of the data to be deleted as a deletion completed state; A deletion report of the data to be deleted is generated according to the deletion timestamp, the table information, the index information, and the deletion result, and the deletion report is output.

[0068] Optionally, you can use the Redis database to save the deletion log. Every time you delete data, you can automatically record the deletion timestamp of the data to be deleted, that is, the timestamp of the deletion operation, the table information of the data table to which the data to be deleted belongs, the index information of the data to be deleted in the data table, and the deletion result of the data to be deleted. After the deletion action corresponding to the data to be deleted is completed, the status of the data to be deleted is marked as the deletion completed status, and an information document (or deletion report) of the data to be deleted is generated based on the deletion timestamp, table information, index information, and deletion result to the designated administrator, so that the administrator can have a basis for tracking data problems later, and if errors occur during the deletion process, they can also promptly discover and repair the problems as soon as possible, thereby improving maintenance efficiency.

[0069] In some embodiments, the method further comprises: When it is determined that the data to be deleted is marked as being in a deletion completed state, searching the storage system for the data to be deleted according to the table information of the data table and the index information; If the search is empty, it is determined that the data to be deleted is deleted correctly; If the data to be deleted is found, the state of the data to be deleted is re-marked as a deleted state.

[0070] like Figure 2 As shown, a verification mechanism can also be established within the execution body to perform an integrity verification query on the data to be deleted when it is determined that the data to be deleted is marked as deleted according to the status mark in the Redis database, so as to ensure that the data marked as deleted has been completely deleted.

[0071] Figure 3 : is a flowchart of the deletion verification provided by the present invention; Figure 3 As shown, the deletion verification steps specifically include: When it is determined that the data to be deleted is marked as deleted according to the status mark in the Redis database, the data to be deleted is searched in the storage system according to the table information and index information of the data table; if the search is empty, it is determined that the data to be deleted is correctly deleted; if the data to be deleted is found, it is determined that the data to be deleted is not correctly deleted. At this time, the state of the data to be deleted can be re-marked as deleted state to add it back to the list of data to be deleted, and return to step 120, iteratively execute the deletion judgment and deletion operation of the data to be deleted until it is determined that the data to be deleted is correctly deleted, thereby ensuring the integrity and irreversibility of data deletion through the verification mechanism to protect the error probability of the deletion operation and improve the efficiency of system operation.

[0072] The data deletion device provided by the present invention is described below. The data deletion device described below and the data deletion method described above can be referenced to each other.

[0073] Figure 4 Schematic diagram of the structure of the data deletion device provided by the present invention; Figure 4 As shown, the device includes: a marking module 410, a judging module 420 and a deleting module 430.

[0074] The marking module 410 is used to obtain the data to be deleted in the storage system according to the data deletion instruction, and mark the state of the data to be deleted as a deletion state; The judgment module 420 is used to judge whether to trigger a deletion operation on the data to be deleted according to the association relationship between the data to be deleted and each target data, as well as the configuration information and operation log of the storage system; the target data is the data in the storage system that is marked as deleted except the data to be deleted; The deleting module 430 is configured to delete the data to be deleted from the physical storage space of the storage system when it is determined that a deletion operation on the data to be deleted is triggered.

[0075] The device provided in this embodiment can first logically delete the data to be deleted after obtaining the data deletion instruction to ensure that the data can be restored at any time, which significantly reduces the risk of permanent loss of important data due to user error or accidental deletion. Before executing data deletion, the association relationship between the data to be deleted and other target data, system configuration information and operation logs are comprehensively evaluated to ensure that the physical deletion operation is triggered only under safe and appropriate circumstances, further reducing the risk of permanent loss of important data due to user error or accidental deletion, and realizing the timely release of storage space, effectively reducing storage costs and improving data processing performance, thereby improving the security, reliability and performance of data management.

[0076] The device provided by the present invention is used to execute the above-mentioned method embodiments. Please refer to the above-mentioned embodiments for the specific processes and detailed contents, which will not be repeated here.

[0077] Figure 5 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530 and a communication bus 540, wherein the processor 510, the communication interface 520 and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call the logic instructions in the memory 530 to execute the data deletion method, which includes: according to the data deletion instruction, obtaining the data to be deleted in the storage system, and marking the state of the data to be deleted as a deletion state; judging whether to trigger the deletion operation of the data to be deleted according to the association relationship between the data to be deleted and each target data, as well as the configuration information and operation log of the storage system; the target data is the data in the storage system that is marked as a deletion state except the data to be deleted; and in the case of determining that the deletion operation of the data to be deleted is triggered, deleting the data to be deleted from the physical storage space of the storage system.

[0078] In addition, the logic instructions in the above-mentioned memory 530 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0079] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the data deletion method provided by the above-mentioned methods, which method includes: according to the data deletion instruction, obtaining the data to be deleted in the storage system, and marking the state of the data to be deleted as a deletion state; according to the association relationship between the data to be deleted and each target data, as well as the configuration information and operation log of the storage system, determining whether to trigger a deletion operation on the data to be deleted; the target data is data in the storage system other than the data to be deleted that is marked as a deletion state; when it is determined that the deletion operation on the data to be deleted is triggered, deleting the data to be deleted from the physical storage space of the storage system.

[0080] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the data deletion method provided by the above-mentioned methods, the method comprising: according to a data deletion instruction, obtaining the data to be deleted in the storage system, and marking the state of the data to be deleted as a deletion state; judging whether to trigger a deletion operation on the data to be deleted according to the association relationship between the data to be deleted and each target data, as well as the configuration information and operation log of the storage system; the target data is the data in the storage system marked as a deletion state other than the data to be deleted; and when it is determined that the deletion operation on the data to be deleted is triggered, deleting the data to be deleted from the physical storage space of the storage system.

[0081] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0082] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0083] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data deletion method, characterized in that: include: According to the data deletion instruction, the data to be deleted is obtained in the storage system, and the state of the data to be deleted is marked as a deletion state; Determining whether to trigger a deletion operation on the data to be deleted according to the association relationship between the data to be deleted and each target data, as well as the configuration information and operation log of the storage system; The target data is data in the storage system that is marked as deleted except the data to be deleted; In the case where it is determined that a deletion operation on the data to be deleted is triggered, the data to be deleted is deleted from the physical storage space of the storage system.

2. The data deletion method according to claim 1, characterized in that: The determining whether to trigger a deletion operation on the data to be deleted according to the association relationship between the data to be deleted and each target data, and the configuration information and operation log of the storage system, includes: According to the association relationship between the data to be deleted and each of the target data, determining whether there is associated data associated with the data to be deleted in the plurality of target data, and obtaining a first determination result; According to the configuration information of the storage system, obtaining a target delayed deletion duration; Determine whether the duration for which the to-be-deleted data is recorded as deleted exceeds the target delayed deletion duration, and obtain a second determination result; According to the operation log of the storage system, the data call frequency of the storage system in each operation period is obtained; Among the plurality of operating time periods, obtaining an operating time period in which the frequency of data call is lower than a preset value as a target operating time period; Determine whether the current time belongs to the target operation time period, and obtain a third determination result; The first judgment result, the second judgment result and the third judgment result are combined to determine whether to trigger a deletion operation on the data to be deleted.

3. The data deletion method according to claim 2, characterized in that: The combining the first judgment result, the second judgment result and the third judgment result to judge whether to trigger a deletion operation on the to-be-deleted data includes: In the case where it is determined according to the first judgment result that there is no associated data associated with the data to be deleted among the multiple target data, if it is determined according to the second judgment result that the duration for which the data to be deleted is recorded as deleted exceeds the target delayed deletion duration, and it is determined according to the third judgment result that the current moment belongs to the target operating time period, then it is determined that the deletion operation of the data to be deleted is triggered.

4. The data deletion method according to claim 3, characterized in that: The method further comprises: In a case where it is determined according to the first judgment result that at least one associated data associated with the to-be-deleted data exists in the plurality of target data, acquiring a triggering status of each of the associated data; When it is determined that a deletion operation on any associated data is triggered according to the trigger state, the data to be deleted and the any associated data are deleted in batches from the physical storage space of the storage system.

5. The data deletion method according to claim 2, characterized in that: The obtaining, according to the configuration information of the storage system, a target delayed deletion duration, includes: According to the configuration information of the storage system, obtaining multiple items of data storage requirement information, storage capacity information, performance requirement information and data recovery requirement information of the storage system; Scoring each preset delayed deletion time according to multiple items of the data storage requirement information, the storage capacity information, the performance requirement information, and the data recovery requirement information to obtain a score value corresponding to each preset delayed deletion time; According to the score value, the target delayed deletion time length is obtained from the plurality of preset delayed deletion time lengths.

6. The data deletion method according to any one of claims 1 to 5, characterized in that: The method further comprises: When performing a deletion action on the data to be deleted, recording a deletion timestamp of the data to be deleted, table information of a data table to which the data to be deleted belongs, index information of the data to be deleted in the data table, and a deletion result of the data to be deleted; When the deletion action corresponding to the data to be deleted is completed, marking the state of the data to be deleted as a deletion completed state; A deletion report of the data to be deleted is generated according to the deletion timestamp, the table information, the index information, and the deletion result, and the deletion report is output.

7. The data deletion method according to claim 6, characterized in that: The method further comprises: When it is determined that the data to be deleted is marked as being in a deletion completed state, searching the storage system for the data to be deleted according to the table information of the data table and the index information; If the search is empty, it is determined that the data to be deleted is deleted correctly; If the data to be deleted is found, the state of the data to be deleted is re-marked as a deleted state.

8. A data deletion device, characterized in that: include: A marking module, used for obtaining the data to be deleted in the storage system according to the data deletion instruction, and marking the state of the data to be deleted as a deletion state; A judgment module, used to judge whether to trigger a deletion operation on the data to be deleted according to the association relationship between the data to be deleted and each target data, as well as the configuration information and operation log of the storage system; The target data is data in the storage system that is marked as deleted except the data to be deleted; The deleting module is used to delete the data to be deleted from the physical storage space of the storage system when it is determined that a deletion operation on the data to be deleted is triggered.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the data deletion method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the data deletion method according to any one of claims 1 to 7 is implemented.

Images