Data deletion method, device, electronic device and storage medium

Abstract

The embodiment of the present application proposes a data deletion method, device, electronic device and computer storage medium, the method comprising: obtaining a user's operation request, and when it is determined that the operation request is a DROP operation, obtaining start control information; when determining to start a garbage collection operation based on the start control information, determining an operation object of the DROP operation; the operation object includes a target database or a target data table; constructing asynchronous task information according to the operation object, and executing a garbage collection task based on the asynchronous task information to delete the data files in the operation object.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and in particular to a data deletion method, device, electronic device and computer storage medium. Background Art

[0002] As the number of databases or data tables in MySQL databases increases, large databases and tables are deleted. In the related technology, deletion is mainly based on the Data Definition Language (DDL) method, that is, the database or data table is deleted through the DROP command.

[0003] However, when the amount of data in the database and data table is relatively large, directly executing the DROP command will cause other operations related to the database or the data table to be blocked until the DROP operation is completed; the blocking time increases linearly with the amount of data in the database and the data table; at the same time, deleting the database or data table through the DROP command will also affect the execution efficiency of other databases, data tables and applications on the instance. Summary of the invention

[0004] The present application provides a data deletion method, device, electronic device and computer storage medium; it can solve the problem that when a DROP operation is directly performed on a database or data table with a large amount of data, other operations related to the database or data table are blocked and the disk IO surges.

[0005] The technical solution of this application is implemented as follows:

[0006] The present application provides a data deletion method, the method comprising:

[0007] Obtaining an operation request from a user, and when determining that the operation request is a DROP operation, obtaining opening control information;

[0008] When determining to start the garbage collection operation according to the start control information, determining the operation object of the DROP operation; the operation object includes a target database or a target data table;

[0009] Asynchronous task information is constructed according to the operation object, and a garbage collection task is executed based on the asynchronous task information to delete the data file in the operation object.

[0010] In some embodiments, the method further comprises:

[0011] When it is determined that the operation request is a DROP operation, obtaining a recycling interval;

[0012] Accordingly, the performing of the garbage collection task based on the asynchronous task information includes:

[0013] Obtaining the creation time of the garbage collection task in the asynchronous task information, and determining the time difference between the current time and the creation time;

[0014] When the time difference is greater than or equal to the recycling interval, it is determined to execute the garbage collection task.

[0015] In some embodiments, when it is determined that the operation object of the DROP operation is a target database, the step of constructing asynchronous task information according to the operation object includes:

[0016] Rename the target database to obtain a first database;

[0017] Asynchronous task information is constructed according to the first database.

[0018] In some embodiments, the method further comprises:

[0019] Creating a hard connection for the target database to obtain a first hard connection file, and pointing the first hard connection file to the first database;

[0020] According to the DROP operation, the first hard link file is deleted.

[0021] In some embodiments, when it is determined that the operation object of the DROP operation is a target data table, the asynchronous task information is constructed according to the operation object, and the method further includes:

[0022] Rename the target data table to obtain a first data table;

[0023] Asynchronous task information is constructed according to the first data table.

[0024] In some embodiments, the method further comprises:

[0025] Creating a hard link for the target data table to obtain a second hard link file, and pointing the second hard link file to the first data table;

[0026] According to the DROP operation, the second hard link file is deleted.

[0027] In some embodiments, deleting the data file in the operation object includes:

[0028] Splitting the data file in the first database or the first data table according to a set size to obtain at least two file blocks;

[0029] The at least two file blocks are deleted one by one.

[0030] In some embodiments, the method further comprises:

[0031] In the process of deleting the at least two file blocks one by one, updating the processing progress of the garbage collection task in real time;

[0032] According to the processing progress, the status information of the garbage collection task in the asynchronous task information is updated.

[0033] The embodiment of the present application also provides a data deletion device, which includes an acquisition module, a determination module and a deletion module, wherein:

[0034] An acquisition module, used for acquiring an operation request of a user, and when determining that the operation request is a DROP operation, acquiring opening control information;

[0035] A determination module, configured to determine an operation object of the DROP operation when determining to start the garbage collection operation according to the start control information; the operation object includes a target database or a target data table;

[0036] The deletion module is used to construct asynchronous task information according to the operation object, and execute garbage collection tasks based on the asynchronous task information to delete the data files in the operation object.

[0037] An embodiment of the present application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the data deletion method provided by one or more of the aforementioned technical solutions when executing the program.

[0038] An embodiment of the present application provides a computer storage medium, which stores a computer program; after the computer program is executed, it can implement the data deletion method provided by one or more of the above-mentioned technical solutions.

[0039] The embodiment of the present application proposes a data deletion method, device, electronic device and computer storage medium, the method comprising: obtaining a user's operation request, and when determining that the operation request is a DROP operation, obtaining start control information; when determining to start a garbage collection operation according to the start control information, determining an operation object of the DROP operation; the operation object includes a target database or a target data table; constructing asynchronous task information according to the operation object, and executing a garbage collection task based on the asynchronous task information to delete the data file in the operation object; in this way, when the data file in the operation object is large, instead of directly executing the DROP operation on the operation object, asynchronous task information is constructed according to the operation object, and the asynchronous task information is processed through an asynchronous garbage collection operation to achieve deletion of the data file of the operation object; furthermore, it can solve the problem that when executing the DROP operation, other operations related to the operation object are blocked and the disk IO soars, and at the same time, reduce the impact on system services and improve the execution efficiency of other databases, data tables and applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1a A schematic diagram of an application scenario of the data deletion method provided in an embodiment of the present application;

[0041] Figure 1b It is a flowchart of a data deletion method in an embodiment of the present application;

[0042] Figure 2a A flowchart of another data deletion method in an embodiment of the present application is shown in FIG.

[0043] Figure 2b A schematic diagram of a process for deleting a data file of an operation object in an embodiment of the present application;

[0044] Figure 3 A schematic diagram of the structure of a data deletion device according to an embodiment of the present application;

[0045] Figure 4 A schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0046] The present application will be further described in detail below in conjunction with the accompanying drawings and Examples. It should be understood that the embodiments provided herein are only used to explain the present application and are not intended to limit the present application. In addition, the embodiments provided below are partial embodiments for implementing the present application, rather than providing all embodiments for implementing the present application. In the absence of conflict, the technical solutions recorded in the embodiments of the present application can be implemented in any combination.

[0047] It should be noted that, in the embodiments of the present application, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a method or apparatus including a series of elements includes not only the elements explicitly recorded, but also includes other elements not explicitly listed, or also includes elements inherent to the implementation of the method or apparatus. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other related elements (such as steps in a method or units in a device, for example, a unit may be a portion of a circuit, a portion of a processor, a portion of a program or software, etc.) in the method or apparatus including the element.

[0048] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, I and / or J can represent the three situations that I exists alone, I and J exist at the same time, and J exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of I, J, and R can represent including any one or more elements selected from the set consisting of I, J, and R.

[0049] For example, the data deletion method provided in the embodiment of the present application includes a series of steps, but the data deletion method provided in the embodiment of the present application is not limited to the recorded steps. Similarly, the data deletion device provided in the embodiment of the present application includes a series of modules, but the data deletion device provided in the embodiment of the present application is not limited to including the modules explicitly recorded, and may also include modules required to obtain relevant time series data or perform processing based on time series data.

[0050] The embodiments of the present application can be applied to a computer system composed of a terminal device and a server, and can operate with many other general or special computing system environments or configurations. Here, the terminal device can be a thin client, a thick client, a handheld or laptop device, a microprocessor-based system, a set-top box, a programmable consumer electronic product, a network personal computer, a small computer system, etc., and the server can be a server computer system, a small computer system, a large computer system, and a distributed cloud computing technology environment including any of the above systems, etc.

[0051] Electronic devices such as terminal devices and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. In general, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices connected through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0052] In the related art, the conventional method for deleting data in a database and a data table is to delete the data through the DROP command. However, when the amount of data in the database and the data table is relatively large, there are the following technical problems when using this method to delete data:

[0053] 1) The execution time of the database and data table is long, which will cause other operations related to the database or data table to be blocked.

[0054] 2) Deleting databases and data tables will cause a surge in disk input and output (IO), affecting the normal service of the entire instance during the deletion period.

[0055] 3) Irregular DROP operations by users will seriously affect system services, and it is impossible to implement an asynchronous recycling mechanism that users are unaware of.

[0056] In view of the above technical problems, the following embodiments are proposed.

[0057] In some embodiments of the present application, the data deletion method can be implemented using a processor in a data deletion device, and the above-mentioned processor can be at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, and a microprocessor.

[0058] Figure 1a A schematic diagram of an application scenario of the data deletion method provided in an embodiment of the present application, such as Figure 1a As shown, user 10 sends an operation request to database server 11 to operate the data in MySQL database; after obtaining the operation request, the background processing thread in database server 11 determines whether the operation request is a DROP operation, and if it is determined that the operation request is a DROP operation, obtains the start control information; after determining to start the garbage collection operation according to the start control information, an asynchronous garbage collection operation is performed on the data files in the database or data table in the MySQL database.

[0059] Figure 1b is a flow chart of a data deletion method in an embodiment of the present application, such as Figure 1b As shown, the method comprises the following steps:

[0060] Step 100: Obtain the user's operation request, and when it is determined that the operation request is a DROP operation, obtain the start control information.

[0061] In an embodiment of the present application, a user's operation request may represent a user's request for adding, modifying, deleting or querying data in a database or data table in a MySQL database; illustratively, a user sends an operation request to a database server through a client to operate on data in a MySQL database, and the database server may display the operation result of the operation request through a client interface.

[0062] In one embodiment, the database server includes a database management system (DBMS); the database management system is a large-scale software for operating and managing databases, which is used to establish, use and maintain databases, and to uniformly manage and control databases to ensure the security and integrity of the databases.

[0063] Here, MySQL database is a common relational database management system, which includes multiple databases, and the database stores data in data tables as organizational units. For example, a database is a warehouse for storing data, and its essence is a file system that stores data in a specific format; a data table, referred to as a table, consists of three parts: table name, field, and record; each row in the table is called a record, which consists of several fields; each column in the table is called a field.

[0064] Exemplarily, the above user's operation request can be implemented through Structured Query Language (SQL); SQL is a database query and programming language used to access data and query, update and manage MySQL database.

[0065] In one embodiment, the user's operation request may be: CREATE DATABASE database name, CREATE TABLE data table name, DROP DATABASE database name, DROP TABLE data table name, etc. That is, creating a database, creating a data table, deleting a database, deleting a data table, etc. in a MySQL database all require the use of SQL statements.

[0066] In an embodiment of the present application, after obtaining the user's operation request, it is possible to determine whether the operation request is a DROP operation by identifying the SQL statement corresponding to the operation request. If so, the start control information is obtained; if not, the standard operation process is performed.

[0067] In the embodiment of the present application, the activation control information is a global parameter pre-added in the GLOBAL_VARIABLES table in the information_schema library; the global parameter is used to control subsequent garbage collection operations. Here, the information_schema library is an information database that comes with the MySQL database, which is used to store the metadata of the MySQL database, that is, data that describes the data, such as the database name, data table name, column data type, access rights, etc.

[0068] In the embodiment of the present application, there is no limitation on the method of obtaining the user operation request. It can be automatically obtained through the background processing thread of the MySQL database, or it can be obtained through other methods.

[0069] Step 101: when it is determined to start the garbage collection operation according to the start control information, the operation object of the DROP operation is determined; the operation object includes a target database or a target data table.

[0070] In an embodiment of the present application, the background processing thread monitors the start control information every set period to obtain the monitoring results; determines whether to start the garbage collection operation based on the monitoring results; if not, sleeps until the next set period to continue monitoring; if it is turned on, further determines the operation object of the DROP operation.

[0071] Exemplarily, the setting of the set period can be set according to the actual application scenario, and the embodiments of the present application are not limited to this; for example, it can be 5ms, 10ms, etc.

[0072] In one implementation, the background processing thread is a daemon thread running in the background, which can be used to delete garbage data and reclaim disk space; here, garbage data refers to data in the operation object of the DROP operation.

[0073] In one implementation, assuming that the start control information is 1, it indicates that the garbage collection operation is started; when the start control information is 0, it indicates that the garbage collection operation is not started; if the monitoring result is 0, it sleeps until the next set period to continue monitoring; if the monitoring result is 1, the operation object of the DROP operation is further determined.

[0074] Exemplarily, the start control information may be modified through relevant instructions; in this way, the start frequency of the garbage collection operation may be controlled more flexibly.

[0075] Step 102: construct asynchronous task information according to the operation object, and execute garbage collection tasks based on the asynchronous task information to delete data files in the operation object.

[0076] In one implementation, a recycling task table JOBS can be added in the information_schema library by modifying the MySQL source code; the asynchronous task information constructed according to the operation object is written into the JOBS table; wherein the design structure of the JOBS table is as follows:

[0077]

[0078]

[0079] Here, the asynchronous task information includes: id, status, created, payload and progress. Among them, id represents the identity document (ID) of the current task, status represents the status information of the current task, and the status information includes Pending / Running / Failure / Cancel / Success, which respectively represent pending / executing / failed / canceled / success; created represents the creation time of the current task; payload represents the specific script information of the current task execution, which mainly includes the library name or table name and the path of the data file to be cleaned, and progress represents the execution progress of the current task.

[0080] In some embodiments, when it is determined that the operation object of the DROP operation is the target database, constructing the asynchronous task information according to the operation object may include: renaming the target database to obtain a first database; and constructing the asynchronous task information according to the first database.

[0081] For example, the data in all data tables under the target database are in the same folder. When performing a DROP operation to delete the target database, the entire folder needs to be deleted. However, if the entire folder is deleted directly, it will cause a long period of lag and a large number of IO requests. For example, the target database can be deleted by modifying the code as follows.

[0082] In one implementation, assume that the operation request obtained according to the above steps is DROP DATABASE db; that is, the operation object of the DROP operation is the target database db, where db is the name of the target database; first, rename the target database db to obtain the first database db_ln; then, based on the basic information of the first database db_ln, construct the asynchronous task information and write it into the JOBS table. Here, the asynchronous task information includes the process of deleting the first database db_ln storing the actual data file through the background processing thread.

[0083] In some embodiments, the above method may further include: creating a hard connection for the target database to obtain a first hard connection file, and pointing the first hard connection file to the first database; and deleting the first hard connection file according to a DROP operation.

[0084] Furthermore, after the target database db is renamed, a hard link is created for the target database db to obtain a first hard link file; here, the first hard link file is equivalent to a pointer, pointing to the first database db_ln storing the actual data file; in this way, after executing the standard DROP DATABASE db deletion operation, only the first hard link file of the target database db will be deleted, the deletion speed is fast, and the IO request occupation is effectively reduced.

[0085] In some embodiments, when it is determined that the operation object of the DROP operation is the target data table, constructing the asynchronous task information according to the operation object may include: renaming the target data table to obtain a first data table; and constructing the asynchronous task information according to the first data table.

[0086] For example, when the target data table has a large amount of data, the entire data table needs to be deleted when performing a DROP operation to delete the target data table; similarly, if the entire data table is directly deleted, it will also cause a long period of lag and a large number of IO requests; for example, the target data table can be deleted by modifying the code as follows.

[0087] In one implementation, assume that the operation request obtained according to the above steps is DROP DATABASEtable; that is, the operation object of the DROP operation is the target data table table, where table is the name of the target data table; first, the target data table table is renamed to obtain the first data table table_ln; then, based on the basic information of the first data table table_ln, the asynchronous task information is constructed and written into the JOBS table. Here, the asynchronous task information includes the process of deleting the first data table table_ln storing the actual data file through the background processing thread.

[0088] In some embodiments, the above method may further include: creating a hard link for the target data table to obtain a second hard link file, and pointing the second hard link file to the first data table; and deleting the second hard link file according to a DROP operation.

[0089] Furthermore, after renaming the target data table table, a hard link is created for the target data table table to obtain a second hard link file; here, the second hard link file is equivalent to a pointer, pointing to the first data table table_ln storing the actual data file; in this way, after executing the standard DROP DATABASE table deletion operation, only the second hard link file of the target data table table is deleted, the deletion speed is fast, and the IO request occupancy is effectively reduced.

[0090] In an embodiment of the present application, after completing the above operation process, based on the asynchronous task information, the first database db_ln and the first data table table_ln storing the actual data files are asynchronously deleted through the background processing thread, thereby reducing the impact of the user's irregular operations on the overall system and other services of the same instance.

[0091] In some embodiments, the above method may also include: when it is determined that the operation request is a DROP operation, obtaining the recycling interval; accordingly, executing the garbage collection task based on the asynchronous task information may include: obtaining the creation time of the garbage collection task in the asynchronous task information, and determining the time difference between the current time and the creation time; when the time difference is greater than or equal to the recycling interval, determining to execute the garbage collection task.

[0092] In the embodiment of the present application, the recycling interval is also a global parameter pre-added in the GLOBAL_VARIABLES table in the information_schema library; the global parameter is used to control when the garbage collection task is executed after the DROP operation.

[0093] For example, the background processing thread periodically monitors the JOBS table, and determines whether to execute the current garbage collection task according to the time difference between the created time of the garbage collection task in the asynchronous task information and the current time; and determines whether to execute the current garbage collection task according to the comparison result of the time difference and the collection interval; when the time difference is greater than or equal to the collection interval, the garbage collection task is executed. When the time difference is less than the collection interval, the garbage collection task is not executed yet.

[0094] Here, by setting the recycling interval, even if the user mistakenly performs a DROP operation, the data in the target database or target data table will not be cleared immediately, and the data can be restored through commands; in this way, the security of the data can be ensured.

[0095] In some embodiments, deleting the data file in the operation object may include: splitting the data file in the first database or the first data table according to a set size to obtain at least two file blocks; and deleting the at least two file blocks one by one.

[0096] In an embodiment of the present application, when the garbage collection operation is started but the garbage collection task is not executed, the status information in the JOBS table is to be executed; after determining to execute the garbage collection task, the status information in the JOBS table is updated from to be executed to being executed; and according to the asynchronous task information in the JOBS table, the data file in the first database or the first data table is obtained, and the data file is split according to the set size to obtain at least two file blocks; and the at least two file blocks are deleted one by one.

[0097] Here, the set size can be set according to actual conditions, and the embodiment of the present application does not limit this; for example, it can be 1MB, 5MB, etc.

[0098] In an embodiment of the present application, by deleting data files in the operation object in blocks through a background processing thread, it is possible to reduce system lag and reduce the user's perception of the deletion operation.

[0099] In some embodiments, the above method may further include: updating the processing progress of the garbage collection task in real time during the process of deleting at least two file blocks one by one; and updating the status information of the garbage collection task in the asynchronous task information according to the processing progress.

[0100] In one implementation, during the process of deleting file blocks, it can be determined whether an exception occurs. If so, the status information in the JOBS table is updated from pending to failed; if not, the processing progress of the garbage collection task is updated in real time; and then the status information of the garbage collection task is further determined based on the processing progress.

[0101] Exemplarily, whether the garbage collection task has been completed can be determined based on the processing progress; if so, the status information in the JOBS table is updated from executing to successful; if not, the status information of the garbage collection task does not need to be updated, but the remaining file blocks continue to be deleted through the background processing thread until the current garbage collection task is completed.

[0102] Here, the setting of the processing progress can be represented by data between 0 and 1; wherein the data can be represented in the form of a decimal or a percentage.

[0103] In the implementation of this application, when the amount of data in the database and data table is relatively large, the control information and recycling interval configuration and the JOBS table are enabled through global parameters to control the background processing thread to delete the data files in the operation object, that is, the data files are deleted through asynchronous garbage collection operations to achieve disk recycling; thereby, reducing system lag and high IO occupancy.

[0104] The embodiment of the present application proposes a data deletion method, device, electronic device and computer storage medium, the method comprising: obtaining a user's operation request, and when determining that the operation request is a DROP operation, obtaining start control information; when determining to start a garbage collection operation according to the start control information, determining an operation object of the DROP operation; the operation object includes a target database or a target data table; constructing asynchronous task information according to the operation object, and executing a garbage collection task based on the asynchronous task information to delete the data file in the operation object; in this way, when the data file in the operation object is large, instead of directly executing the DROP operation on the operation object, asynchronous task information is constructed according to the operation object, and the garbage collection task is executed on the asynchronous task information through an asynchronous garbage collection operation to delete the data file of the operation object; further, it can solve the problem that when executing the DROP operation, other operations related to the operation object are blocked and the disk IO soars, and at the same time, reduce the impact on system services and improve the execution efficiency of other databases, data tables and applications.

[0105] In order to better reflect the purpose of the present application, further examples are given based on the above embodiments of the present application.

[0106] The application scenario of the embodiment of the present application is for the business in the MySQL database. In the production process, it is necessary to delete the database or data table with a very large amount of data. In order to reduce the impact on the service, an elegant background data cleaning method is provided; in order to provide convenience of use and ensure minor changes in the method of use, this method is provided by modifying the MySQL system source code.

[0107] Here, the main purpose is to design a garbage collection mechanism similar to that in programming languages. This mechanism is used to handle disk space recovery after a DROP operation. At the same time, this method can also handle scenarios of misoperation. If a DROP operation is performed incorrectly, the data will not be cleared immediately, and the data can be restored through commands. The implementation of this method requires modifying the MySQL source code, and the modification steps involved can be divided into the following three steps:

[0108] Step A1: Add global configuration.

[0109] In one implementation, since the frequency of garbage collection operations needs to be controlled, two global parameters GC_ENABLE and GC_INTERVAL need to be added to the GLOBAL_VARIABLES table in the information_schema library; here, GC_ENABLE represents the above-mentioned enabling control information; GC_INTERVAL represents the above-mentioned collection interval.

[0110] Step A2: Add a recycling task list.

[0111] In one implementation, the MySQL source code is modified to add a recycling task table JOBS in the information_schema library; wherein the JOBS table is mainly used to record asynchronous task information constructed according to the operation object. The above embodiment has already described the JOBS table, which will not be repeated here.

[0112] Step A3: Execute garbage collection tasks.

[0113] In one implementation, the background processing thread determines whether to enable the garbage collection operation based on the two global parameter information configured in the GLOBAL_VARIABLES table. If not, it will sleep until the next cycle to continue monitoring. If enabled, the background processing thread will periodically monitor the JOBS table and calculate whether to enable the current garbage collection task based on the value of the created time of the current task and the GC_INTERVAL parameter. After the recycling task starts, the large file will be split into very small file blocks and deleted piece by piece. In this way, the service can be completely unaware of the pressure of IO operations.

[0114] With the above source code modified database version, the following is a detailed description of the entire task execution process. Figure 2a FIG. 1 is a flow chart of another data deletion method in an embodiment of the present application. Figure 2a As shown, the process includes the following steps:

[0115] Step B1: Obtain the user's operation request.

[0116] In one implementation, after a user sends an operation request to a database server, a background processing thread of the MySQL database acquires the user's operation request.

[0117] Step B2: Determine whether the operation request is a DROP operation.

[0118] In one implementation, after obtaining the user's operation request, the operation request is parsed to obtain a parsing result, and it can be determined whether the user's operation request is a DROP operation based on the parsing result. If yes, execute step B3; if no, execute step B6.

[0119] Step B3: Determine whether to enable garbage collection operation.

[0120] In one implementation, whether to enable the garbage collection operation is determined according to the global variable GC_ENABLE in the GLOBAL_VARIABLES table. If yes, execute step B4; if no, execute step B6.

[0121] Step B4: Determine the operation object of the DROP operation.

[0122] In one embodiment, the operation object includes two types: a target database and a target data table; when it is determined that the operation object of the DROP operation is the target database, steps B40 to B42 are executed; the specific process is as follows: first, it is determined that the obtained operation request is DROP DATABASE db; then, the target database db is renamed to obtain the first database db_ln; then, a hard connection is created for the target database db, and the hard connection points to the first database db_ln storing the actual data file. When it is determined that the operation object of the DROP operation is the target data table, steps B43 to B45 are executed; the specific process is as follows: first, it is determined that the obtained operation request is DROP DATABASE table; then, the target data table table is renamed to obtain the first data table table_ln; then, a hard connection is created for the target data table table, and the hard connection points to the first data table table_ln storing the actual data file.

[0123] Step B5: Build asynchronous task information and write it into the JOBS table.

[0124] In one embodiment, after executing steps B40 to B42, asynchronous task information is constructed based on the basic information of the first database db_ln and written into the JOBS table. After executing steps B43 to B45, asynchronous task information is constructed based on the basic information of the first data table table_ln and written into the JOBS table.

[0125] Step B6: Standard MySQL operation.

[0126] In one embodiment, when the user's operation request is not a DROP operation, the standard MySQL operation indicates that the user's operation request is processed in a standard manner; when it is determined that the user's operation request is a DROP operation but the garbage collection operation is not enabled, the standard MySQL operation indicates that the DROP operation is processed in a standard manner.

[0127] Step B7: The process ends.

[0128] In one implementation, after the above standard MySQL operations are performed, the process ends.

[0129] After executing the above step B5, it is necessary to further perform asynchronous deletion on the first database db_ln and the first data table table_ln through a background processing thread loop; Figure 2b Schematic diagram of a process of deleting a data file of an operation object in an embodiment of the present application, such as Figure 2b As shown, the process includes the following steps:

[0130] Step B50: Obtain data file.

[0131] In one implementation, the background processing thread cyclically obtains the data file of the first database db_ln or the first data table table_ln from the JOBS table.

[0132] Step B51: Determine whether to start executing the garbage collection task.

[0133] In one embodiment, the background processing thread determines whether to start executing the garbage collection task according to the collection interval; if yes, execute step B52; if not, the background processing thread continues to determine whether to start executing the garbage collection task in the next time period.

[0134] Step B52: Split the data file.

[0135] In one implementation, the data file is split according to a set size, and at least two file blocks obtained by the split are deleted one by one.

[0136] Step B53: Determine whether there is any abnormality in the deletion process.

[0137] In one implementation, during the deletion of the file block, the background processing thread determines whether an exception occurs. If so, the status information in the JOBS table is updated from pending to failed; if not, step B54 is executed.

[0138] Step B54: Update the processing progress of the garbage collection task.

[0139] In one implementation, when there is no abnormality in the file block deletion process, the processing progress of the garbage collection task will be updated in real time.

[0140] Step B55: Determine whether the garbage collection task is completed.

[0141] In one implementation, it can be determined whether the garbage collection task has been completed based on the processing progress; if not, the remaining file blocks continue to be deleted through the background processing thread until the current garbage collection task is completed; if yes, execute step B56.

[0142] Step B56: Update status information.

[0143] In one implementation, if it is determined that the garbage collection task is completed, the status information in the JOBS table is updated from being executed to being successful.

[0144] It can be seen that the embodiment of the present application recycles the large libraries and tables of the MySQL database through an asynchronous garbage collection mechanism without the user's awareness, reducing the impact of the user's irregular operations on the overall system and other services of the same instance. Its basic principle is to introduce a garbage collection mechanism in the MySQL database, control the deletion tasks of large libraries and tables through the configuration of global parameters and the JOBS table, and perform block deletion of large libraries and tables through background processing threads, reducing system jams and high IO occupancy; here, large libraries and tables refer to the above-mentioned target databases and target data tables.

[0145] Figure 3 Schematic diagram of the structure of the data deletion device according to the embodiment of the present application. Figure 3 As shown, the device includes: an acquisition module 300, a determination module 301 and a deletion module 302, wherein:

[0146] The acquisition module 300 is used to acquire the operation request of the user, and when it is determined that the operation request is a DROP operation, acquire the opening control information;

[0147] The determination module 301 is used to determine the operation object of the DROP operation when the garbage collection operation is determined to be started according to the start control information; the operation object includes a target database or a target data table;

[0148] The deletion module 302 is used to construct asynchronous task information according to the operation object, and execute garbage collection tasks based on the asynchronous task information to delete the data files in the operation object.

[0149] In some embodiments, the acquisition module 300 is further configured to:

[0150] When it is determined that the operation request is a DROP operation, the recycling interval is obtained;

[0151] Accordingly, the deletion module 302 is used to perform the garbage collection task based on the asynchronous task information, including:

[0152] Get the creation time of the garbage collection task in the asynchronous task information, and determine the time difference between the current time and the creation time;

[0153] When the time difference is greater than or equal to the collection interval, it is determined to execute the garbage collection task.

[0154] In some embodiments, when it is determined that the operation object of the DROP operation is the target database, the deletion module 302 is used to construct asynchronous task information according to the operation object, including:

[0155] Rename the target database to obtain the first database;

[0156] Asynchronous task information is constructed according to the first database.

[0157] In some embodiments, the deletion module 302 is further configured to:

[0158] Creating a hard connection for the target database to obtain a first hard connection file, and pointing the first hard connection file to the first database;

[0159] According to the DROP operation, the first hard link file is deleted.

[0160] In some embodiments, when it is determined that the operation object of the DROP operation is the target data table, the deletion module 302 is used to construct asynchronous task information according to the operation object, including:

[0161] Rename the target data table to obtain the first data table;

[0162] Asynchronous task information is constructed according to the first data table.

[0163] In some embodiments, the deletion module 302 is further configured to:

[0164] Creating a hard link for the target data table to obtain a second hard link file, and pointing the second hard link file to the first data table;

[0165] According to the DROP operation, the second hard link file is deleted.

[0166] In some embodiments, the deletion module 302 is further used to delete the data file in the operation object, including:

[0167] Splitting the data file in the first database or the first data table according to the set size to obtain at least two file blocks;

[0168] Delete at least two file blocks one by one.

[0169] In some embodiments, the deletion module 302 is further used to: update the processing progress of the garbage collection task in real time during the process of deleting at least two file blocks one by one;

[0170] Update the status information of the garbage collection task in the asynchronous task information according to the processing progress.

[0171] In practical applications, the acquisition module 300, determination module 301 and deletion module 302 can all be implemented by a processor located in an electronic device, and the processor can be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor.

[0172] In addition, each functional module in this embodiment can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or software functional modules.

[0173] If the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or the part that contributes to the relevant technology or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the method of this embodiment. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc. Various media that can store program codes.

[0174] Specifically, a computer program instruction corresponding to a data deletion method in this embodiment can be stored on a storage medium such as a CD, a hard disk, or a USB flash drive. When the computer program instruction corresponding to a data deletion method in the storage medium is read or executed by an electronic device, any data deletion method in the aforementioned embodiments is implemented.

[0175] Based on the same technical concept as the above embodiments, see Figure 4 , which shows an electronic device 400 provided by the present application, which may include: a memory 401 and a processor 402; wherein,

[0176] Memory 401, used for storing computer programs and data;

[0177] The processor 402 is used to execute the computer program stored in the memory to implement any one of the data deletion methods in the foregoing embodiments.

[0178] In practical applications, the memory 401 may be a volatile memory, such as RAM; or a non-volatile memory, such as ROM, flash memory, hard disk drive (HDD) or solid-state drive (SSD); or a combination of the above types of memory, and provide instructions and data to the processor 402.

[0179] The processor 402 may be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor. It is understandable that for different data deletion devices, the electronic device used to implement the processor function may also be other, which is not specifically limited in the embodiments of the present application.

[0180] In some embodiments, the functions or modules included in the apparatus provided in the embodiments of the present application can be used to execute the method described in the above method embodiment. The specific implementation can refer to the description of the above method embodiment. For the sake of brevity, it will not be repeated here.

[0181] The above description of the various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other. For the sake of brevity, this article will not repeat them.

[0182] The methods disclosed in the various method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0183] The features disclosed in the various product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0184] The features disclosed in the various method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0185] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0186] The present application is described with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present application. It should be understood that each flow process and / or box in the flow chart and / or block diagram and the combination of the flow chart and / or box in the flow chart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processing machine or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one flow chart or multiple flows and / or one box or multiple boxes of the block diagram.

[0187] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0188] The above are only preferred embodiments of the present application and are not intended to limit the protection scope of the present application.

Claims

1. A data deletion method, characterized in that: The method comprises: Obtaining an operation request from a user, and when determining that the operation request is a DROP operation, obtaining a monitoring result and a recovery interval of the opening control information; When the garbage collection operation is started according to the monitoring result of the start control information, an operation object of the DROP operation is determined; the operation object includes a target database or a target data table; Constructing asynchronous task information according to the operation object, and executing a garbage collection task based on the asynchronous task information to delete the data file in the operation object; The performing of the garbage collection task based on the asynchronous task information includes: The creation time of the garbage collection task in the asynchronous task information is obtained, and the time difference between the current time and the creation time is determined; when the time difference is greater than or equal to the collection interval, it is determined to execute the garbage collection task.

2. The method according to claim 1, characterized in that When it is determined that the operation object of the DROP operation is the target database, the step of constructing the asynchronous task information according to the operation object includes: Rename the target database to obtain a first database; Asynchronous task information is constructed according to the first database.

3. The method according to claim 2, characterized in that The method further comprises: Creating a hard connection for the target database to obtain a first hard connection file, and pointing the first hard connection file to the first database; According to the DROP operation, the first hard link file is deleted.

4. The method according to claim 1, characterized in that: When it is determined that the operation object of the DROP operation is the target data table, the asynchronous task information is constructed according to the operation object, and the method further includes: Rename the target data table to obtain a first data table; Asynchronous task information is constructed according to the first data table.

5. The method according to claim 4, characterized in that The method further comprises: Creating a hard link for the target data table to obtain a second hard link file, and pointing the second hard link file to the first data table; According to the DROP operation, the second hard link file is deleted.

6. The method according to claim 2 or 3, characterized in that: Deleting the data file in the operation object includes: Splitting the data file in the first database according to a set size to obtain at least two file blocks; The at least two file blocks are deleted one by one.

7. The method according to claim 4 or 5, characterized in that: Deleting the data file in the operation object includes: Splitting the data file in the first data table according to a set size to obtain at least two file blocks; The at least two file blocks are deleted one by one.

8. The method according to claim 6 or 7, characterized in that: The method further comprises: In the process of deleting the at least two file blocks one by one, updating the processing progress of the garbage collection task in real time; According to the processing progress, the status information of the garbage collection task in the asynchronous task information is updated.

9. A data deletion device, characterized in that: The device comprises: An acquisition module, used to acquire an operation request from a user, and when it is determined that the operation request is a DROP operation, acquire a monitoring result and a recovery interval of the opening control information; A determination module, configured to determine an operation object of the DROP operation when determining to start the garbage collection operation according to the monitoring result of the start control information; the operation object includes a target database or a target data table; A deletion module, used to construct asynchronous task information according to the operation object, and execute a garbage collection task based on the asynchronous task information to delete the data file in the operation object; The deletion module is also used to: obtain the creation time of the garbage collection task in the asynchronous task information, determine the time difference between the current time and the creation time; when the time difference is greater than or equal to the recycling interval, determine to execute the garbage collection task.

10. An electronic device, characterized in that: The device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the method according to any one of claims 1 to 8 is implemented when the processor executes the program.

11. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images