A storage file deletion method, device, electronic device and storage medium
By generating a primary key based on file size and using historical deletion marks, the method efficiently manages and deletes files in cloud storage, addressing inefficiencies in traditional deletion strategies.
Patent Information
- Application Number
- CN202411275610.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-12
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-09-12
AI Technical Summary
Traditional data deletion strategies in cloud storage are inefficient and inaccurate when users subscribe to a fixed-capacity space and continuously upload files, as they struggle to quickly locate files for deletion and ensure sufficient space for new uploads, leading to potential storage space waste or data loss.
A method that generates a primary key based on file size to efficiently manage and delete files by size, using the historical deletion mark primary key to determine which files to delete, ensuring timely and accurate deletion based on the current storage space status.
This approach enhances the efficiency of file deletion by ensuring timely and accurate release of storage space, reducing the impact on system performance and improving user experience by avoiding unnecessary deletions or data loss.
Smart Images

Figure CN118778912B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of cloud storage technology, and in particular to a method, device, electronic device and storage medium for deleting stored files. Background Art
[0002] With the rapid development of Internet technology, the complexity of business scenarios has increased day by day, and data storage and management has become a crucial link. In various business scenarios, especially in scenarios where users subscribe to fixed-capacity space and continuously upload files, data management faces severe challenges. Traditional data deletion solutions are mostly based on time expiration strategies. This strategy is particularly inefficient when dealing with complex and changing business needs, especially when the user space reaches the threshold and needs to be rolled out in chronological order.
[0003] When users subscribe to a fixed amount of storage space and continue to upload files, the capacity of the storage space will gradually approach or even exceed the threshold over time. After reaching the threshold, the system needs to perform a rolling deletion of data to free up space for new files to upload. However, traditional deletion strategies usually need to check the upload time or expiration time of each file and determine whether it needs to be deleted. This method is not only inefficient, but also difficult to achieve efficient rolling deletion of files.
[0004] At the same time, in the scenario where the capacity is fixed and files need to be uploaded continuously, the system needs to give priority to deleting files uploaded earlier to ensure that there is enough space for newly uploaded files. However, traditional deletion strategies often cannot quickly locate the files that need to be deleted, and thus cannot ensure the efficiency and accuracy of the deletion operation.
[0005] Currently, no effective solution has been proposed in the related art on how to improve the efficiency of file deletion in a scenario where a user orders a fixed-capacity space and continuously uploads files. Summary of the invention
[0006] The embodiments of the present application provide a method, apparatus, system, electronic device and storage medium for deleting stored files, so as to at least solve the problem in the related art of how to improve the efficiency of file deletion in a scenario where a user subscribes to a fixed-capacity space and continuously uploads files.
[0007] In a first aspect, an embodiment of the present application provides a method for deleting a stored file, comprising:
[0008] Acquire a first file size of a first storage file stored in a preset storage space, and generate a first primary key based on the first file size;
[0009] Obtain a second file size of the second storage file, query a maximum primary key with the largest primary key value from the first primary keys, and obtain a historical deletion mark primary key;
[0010] Based on the maximum primary key, the historical deletion mark primary key and the second file size, a target capacity is acquired, and according to a comparison result between the target capacity and a preset fixed capacity, a deletion determination result of the first storage file is determined;
[0011] In the case where the deletion determination result indicates deletion, the to-be-deleted file in the first storage file is determined based on the historical deletion mark primary key, and the to-be-deleted file is deleted from the preset storage space.
[0012] In some embodiments, determining the to-be-deleted file in the first storage file based on the historical deletion mark primary key, and deleting the to-be-deleted file from the preset storage space includes:
[0013] Get the preset single deletion amount;
[0014] Based on the historical deletion mark primary key and the single deletion amount, determining the primary key of the file to be deleted in the first primary key;
[0015] Based on the primary key of the file to be deleted, the file to be deleted in the first storage file is determined, and the file to be deleted is deleted from the preset storage space.
[0016] In some embodiments, the determining the file to be deleted in the first storage file based on the primary key of the file to be deleted, and deleting the file to be deleted from the preset storage space includes:
[0017] Obtaining index information corresponding to the first storage file, the index information including the first storage file address;
[0018] Based on the index information and the primary key of the file to be deleted, determining the address of the file to be deleted in the first stored file address;
[0019] Based on the address of the file to be deleted, the file to be deleted is deleted from the preset storage space.
[0020] In some embodiments, determining the to-be-deleted file primary key in the first primary key based on the historical deletion mark primary key and the single deletion amount includes:
[0021] In the first primary key, a primary key whose primary key value is less than or equal to the primary key sum is determined as the primary key of the file to be deleted, and the primary key sum is the sum of the historical deletion mark primary key and the single deletion amount.
[0022] In some embodiments, the method further comprises:
[0023] When the deletion determination result indicates deletion, the to-be-deleted file is determined based on the historical deletion mark primary key, and the historical deletion mark primary key is updated based on the first primary key of the to-be-deleted file to obtain a new historical deletion mark primary key.
[0024] In some embodiments, the method further comprises:
[0025] If the deletion determination result does not indicate deletion, generating a second primary key of a second storage file based on the second file size;
[0026] The maximum primary key is updated based on the second primary key to obtain a new maximum primary key.
[0027] In some embodiments, obtaining the target capacity based on the maximum primary key, the historical deletion mark primary key, and the second file size includes:
[0028] Based on the maximum primary key and the historical deletion mark primary key, determine a space occupancy statistical value, and upload the space occupancy statistical value, the maximum primary key and the historical deletion mark primary key to a metadata table;
[0029] The target capacity is determined according to the second file size by querying the metadata table.
[0030] In a second aspect, an embodiment of the present application provides a storage file deletion device, comprising:
[0031] A first file storage module, configured to obtain a first file size of a first storage file stored in a preset storage space, and generate a first primary key based on the first file size;
[0032] An updating storage module is used to obtain a second file size of a second storage file, query a maximum primary key with a maximum primary key value from the first primary keys, and obtain a historical deletion mark primary key;
[0033] a capacity determination module, configured to obtain a target capacity based on the maximum primary key, the historical deletion mark primary key and the second file size, and determine a deletion determination result of the first storage file according to a comparison result between the target capacity and a preset fixed capacity;
[0034] The file deletion module is used to determine the file to be deleted in the first storage file based on the historical deletion mark primary key when the deletion judgment result indicates deletion, and delete the file to be deleted from the preset storage space.
[0035] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the storage file deletion method as described in the first aspect above is implemented.
[0036] In a fourth aspect, an embodiment of the present application provides a storage medium on which a computer program is stored, and when the program is executed by a processor, the storage file deletion method described in the first aspect above is implemented.
[0037] Compared with the related art, the storage file deletion method provided in the embodiment of the present application generates a primary key based on the file size, making the subsequent deletion of files by size more efficient, solving the problem of how to improve the efficiency of file deletion in the scenario where users order fixed-capacity space and continuously upload files, thereby achieving efficient use of storage space, stable operation of the system and improved user experience.
[0038] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0040] Figure 1 is a hardware structure block diagram of a terminal of a storage file deletion method according to an embodiment of the present invention;
[0041] Figure 2 is a flowchart of a method for deleting a stored file according to an embodiment of the present application;
[0042] Figure 3 is a flow chart of a storage file deletion method according to a preferred embodiment of the present application;
[0043] Figure 4 It is a structural block diagram of a storage file deletion device according to an embodiment of the present application. DETAILED DESCRIPTION
[0044] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application. In addition, it can also be understood that although the efforts made in this development process may be complex and lengthy, for ordinary technicians in the field related to the contents disclosed in the present application, some changes such as design, manufacturing or production based on the technical contents disclosed in the present application are only conventional technical means, and should not be understood as insufficient contents disclosed in the present application.
[0045] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.
[0046] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantity limitation, and may indicate the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application means greater than or equal to two. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. The terms "first", "second", "third" and the like involved in the present application are merely used to distinguish similar objects and do not represent a specific ordering of the objects.
[0047] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. Taking running on a terminal as an example, Figure 1 FIG. 1 is a hardware structure diagram of a terminal of a storage file deletion method according to an embodiment of the present invention. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.
[0048] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the storage file deletion method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0049] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.
[0050] This embodiment provides a method for deleting a stored file. Figure 2 is a flowchart of a method for deleting a stored file according to an embodiment of the present application. Figure 2 As shown, the process includes the following steps:
[0051] Step S201: Obtain a first file size of a first storage file stored in a preset storage space, and generate a first primary key based on the first file size.
[0052] The first storage file is a file that has been stored in the preset storage space, and the first file size is the number of bytes of the first storage file. In a computer system, byte (B for short) is the basic unit used to measure storage capacity in computer information technology. In addition to bytes, commonly used storage capacity units include kilobytes KB, megabytes MB, gigabytes GB, terabytes TB, etc. The conversion relationship of these units is based on 2 to the power of 10, that is, 1KB = 1024B, 1MB = 1024KB, etc. This step generates a first primary key of the first storage file based on the first file size of the first storage file. For example, if the file size is 1MB, the primary key ID of this file is 1MB = 1×1024×1024 = 1048576B, that is, 1048576B.
[0053] This step is the basis for file uploading and recording. By generating a primary key based on the file size, the uniqueness and order of the primary key are ensured, providing an efficient method for quickly querying and locating files, and also simplifying subsequent data management and maintenance.
[0054] Step S202, obtaining the second file size of the second storage file, searching for the largest primary key with the largest primary key value from the first primary keys, and obtaining the historical deletion mark primary key.
[0055] Among them, the historical deletion mark primary key refers to the maximum primary key in the first primary key corresponding to the last file to be deleted determined at the time of the last deletion. In addition, if the first storage file has not been deleted before, the historical deletion mark should be the initial value specified by the staff. The second storage file is the file to be stored in the storage space. Before storing in the preset storage space, it is necessary to obtain the second file size of the second storage file. Then, the maximum primary key with the current largest primary key value is queried from the first primary key, and the historical deletion mark primary key is obtained. After the deletion operation is performed, the largest primary key among the primary keys of all deleted files is the historical deletion mark primary key; when the deletion operation is not performed, the initial value of the historical deletion mark primary key is specified by the staff as 0, indicating that the deletion operation has not been performed. At this time, the capacity occupied by the file in the storage space is the primary key value of the maximum primary key, and then the storage operation of the new file is performed. Based on the initial value and the new file size, the corresponding target capacity is determined. If the corresponding target capacity is greater than the fixed capacity, a deletion operation is required to determine the current file to be deleted, and then delete it. Setting the initial value ensures that the target capacity required for storing files can be accurately and efficiently judged when no deletion operation is performed. For example, the first storage file contains two files A (size is a, primary key value is a) and B (size is b, primary key value is a+b). At this time, the deletion operation has not been performed. The initial value of the primary key of the historical deletion mark specified by the staff is 0. At this time, the space occupied is a+b. It is necessary to upload the second storage file C (size is c). The required target capacity is a+b+c. At this time, if the target capacity is greater than the fixed capacity of the storage space, the file C (size is c, primary key is a+b+c) is stored in it; then it is necessary to upload the second storage file D (size is d). The required target capacity is a+b+c+d. At this time, if the target capacity is less than the fixed capacity of the storage space, it needs to be deleted. At this time, the file to be deleted is A (size is a, primary key is Key value is a), delete the to-be-deleted file A, and store file D (size is c, primary key is a+b+c+d) in it; then you need to upload the second storage file E (size is e). In the last deletion, the largest primary key in the to-be-deleted file was a, so the historical deletion mark primary key is a, and the required target capacity is (a+b+c+d)+ea=b+c+d+e. At this time, if the target capacity is greater than the fixed capacity of the storage space, assuming that the to-be-deleted files are B and C, delete the to-be-deleted files B and C, and store file E (size is e, primary key is a+b+c+d+e); in the next deletion, the largest primary key in the to-be-deleted files B and C is a+b+c, so the historical deletion mark primary key is a+b+c, and the above operation is repeated.
[0056] When uploading a new file, this step queries the current largest primary key value and the historical deletion mark primary key to correctly reflect the current status of the entire storage space, obtain the largest primary key and the historical deletion mark primary key, and ensure that the management of files in the storage space can be carried out continuously and accurately. At the same time, it also avoids the waste of storage space or data loss caused by duplicate deletion or missed deletion of files.
[0057] Step S203, based on the maximum primary key, the historical deletion mark primary key and the second file size, the target capacity is obtained, and the deletion judgment result of the first storage file is determined according to the comparison result between the target capacity and the preset fixed capacity.
[0058] Among them, based on the maximum primary key and the historical deletion mark primary key, the current used space is calculated, and then the target capacity is obtained according to the second file size. The target capacity is the required capacity for storing the second storage file. The fixed capacity is a threshold preset by the system, which is used to limit the maximum usage of the storage space. By comparing the target capacity with the fixed capacity of the storage space, it can be determined whether the previously stored files need to be deleted.
[0059] This step calculates the target capacity based on the maximum primary key, the historical deletion mark primary key and the file size of the newly uploaded file. The target capacity is the key basis for determining whether a file deletion operation is required. By calculating the target capacity, it is possible to accurately determine whether the current storage space has reached the preset capacity threshold, thereby determining whether a file deletion operation is required, effectively avoiding storage space waste or data loss caused by deleting files too early or too late.
[0060] Step S204: When the deletion determination result indicates deletion, based on the historical deletion mark primary key, determine the file to be deleted in the first storage file, and delete the file to be deleted from the preset storage space.
[0061] If the deletion judgment result indicates deletion, the previously stored files need to be deleted. According to the primary key of the historical deletion mark, the primary keys of all files to be deleted are determined, and the primary keys of these files to be deleted can be queried according to the primary key, and finally the files to be deleted and their primary keys are deleted.
[0062] When this step detects that the target capacity is greater than the preset fixed capacity, the earliest uploaded file is determined and deleted according to the historical deletion mark primary key to free up storage space. The earliest uploaded file can be determined through the historical deletion mark primary key, that is, deletion starts from the earliest uploaded file. The actual effect is to determine all files to be deleted in the order of the primary key. Since the primary key is generated according to the file size, and the file size is positively correlated with the upload time, deleting files in ascending order of time according to the primary key order means deleting files in ascending order of time, thus realizing efficient rolling deletion of storage space. This not only ensures the effective use of storage space, but also ensures the timeliness and accuracy of data, greatly improves deletion efficiency, and reduces the impact on system performance.
[0063] Through the above steps, by generating a primary key based on the file size, a direct index of the file size is realized, making the subsequent deletion of files by size more efficient and ensuring that the files can be quickly retrieved and located; by obtaining the maximum primary key ID and the historical deletion mark primary key, the current state of the storage space is accurately grasped, providing an accurate basis for subsequent file deletion operations; by calculating the target capacity, it is ensured that the necessary deletion operation is performed when the storage space reaches the preset threshold, while avoiding unnecessary file deletion; by adopting a primary key-based fast deletion method, the deletion operation of the present invention has high efficiency and can significantly reduce the impact on system performance. Compared with the traditional deletion strategy that checks the upload time or expiration time of the file one by one, and determines whether it needs to be deleted and locates the file to be deleted based on this, the primary key-based fast deletion method of the present application replaces the upload time order of the file with the primary key size order, and links the upload time with the file size through the primary key. When it is detected that a deletion operation is required, the file to be deleted is determined by the relationship between the primary keys and deleted, realizing efficient rolling deletion of files in a fixed capacity scenario.
[0064] In some embodiments, based on the historical deletion mark primary key, determining the file to be deleted in the first storage file, and deleting the file to be deleted from the preset storage space includes:
[0065] Get the preset single deletion amount;
[0066] Based on the historical deletion mark primary key and the single deletion amount, determine the primary key of the file to be deleted in the first primary key;
[0067] Based on the primary key of the file to be deleted, the file to be deleted in the first storage file is determined, and the file to be deleted is deleted from the preset storage space.
[0068] Among them, a preset single deletion amount is set by the staff, and this value can be fixed or adjusted by the staff according to the current usage of the storage space. When the deletion operation needs to be performed, the selection starts from the primary key of the historical deletion mark until the selected file size reaches or exceeds the single deletion amount, thereby determining the primary key of the file to be deleted, and then determining the file to be deleted in the first storage file based on the primary key of the file to be deleted.
[0069] This embodiment can release sufficient storage space through the preset single deletion amount. The system allows the amount of files deleted at a time to be adjusted according to actual needs, which is also helpful to balance the speed of releasing storage space and system performance. When performing a deletion operation, the file to be deleted can be quickly located through a quick query of the primary key, which improves the efficiency of the deletion operation, avoids the traditional technology of directly locating files based on potentially unstable attributes such as file name, modification time, etc., and reduces the risk of accidental deletion or missed deletion.
[0070] In some embodiments, based on the primary key of the file to be deleted, determining the file to be deleted in the first storage file, and deleting the file to be deleted from the preset storage space, includes:
[0071] Obtaining index information corresponding to the first storage file, where the index information includes the address of the first storage file;
[0072] Based on the index information and the primary key of the file to be deleted, determine the address of the file to be deleted in the first storage file address;
[0073] Delete the file to be deleted from the preset storage space based on the address of the file to be deleted.
[0074] Among them, the index information includes but is not limited to the unique identifier of the file (such as the file name) and the address of the file in the storage space (such as the file path). When a file needs to be deleted, the system first queries the corresponding index information in the database based on the primary key of the file to be deleted, and extracts the address of the first storage file from the query result, that is, obtains the specific location of the file to be deleted in the preset storage space. According to the address of the file to be deleted, the file deletion operation is performed in the preset storage space to delete the file to be deleted and related information.
[0075] This embodiment uses index information to enable the system to quickly locate the specific address of any file in the storage space, thereby improving the efficiency of file access and management. The index information can be expanded according to actual needs, such as adding information such as the modification time of the file to support more complex query and management operations. By using unique identifiers such as primary keys to match file addresses, the addresses of files to be deleted can be directly located, reducing unnecessary directory traversal or file scanning operations, ensuring the accuracy of deletion operations, and improving deletion efficiency.
[0076] In some embodiments, based on the historical deletion mark primary key and the single deletion amount, determining the primary key of the file to be deleted in the first primary key includes:
[0077] In the first primary key, the primary key whose primary key value is less than or equal to the primary key sum is determined as the primary key of the file to be deleted, and the primary key sum is the sum of the historical deletion mark primary key and the single deletion amount.
[0078] For example, if the primary key of the historical deletion mark is c and the single deletion amount is y, the primary key whose primary key value is less than or equal to c+y in the first primary key is determined as the primary key of the file to be deleted. Specifically, if the MYSQL database is used, you only need to select From upload_table where id<=c+y, you can determine the file to be deleted.
[0079] This embodiment ensures that storage space is effectively released by accurately calculating the amount of files deleted each time, while avoiding performance degradation caused by excessive deletion. In addition, since the primary key is generated according to the file size (or upload time), the deletion operation will be performed in ascending order of time, which helps to maintain the timeliness and accuracy of the data and improves the deletion efficiency.
[0080] In some embodiments, it also includes:
[0081] When the deletion determination result indicates deletion, the file to be deleted is determined based on the historical deletion mark primary key, and the historical deletion mark primary key is updated based on the first primary key of the file to be deleted to obtain a new historical deletion mark primary key.
[0082] Among them, when the deletion judgment result indicates deletion, the primary key with the largest primary key value in the file to be deleted is selected, and the historical deletion mark primary key is updated to ensure that the deletion progress can continue to be tracked in subsequent deletions and the file deletion amount can be accurately calculated.
[0083] This embodiment tracks the progress and status of the deletion operation by updating the historical deletion primary key, which is helpful for troubleshooting and performance analysis and improves the accuracy of rolling deletion.
[0084] In some embodiments, it also includes:
[0085] If the deletion determination result does not indicate deletion, generating a second primary key of the second storage file based on the second file size;
[0086] Update the maximum primary key based on the second primary key to obtain a new maximum primary key.
[0087] Among them, when the deletion judgment result does not indicate deletion, the target capacity is less than or equal to the fixed capacity, indicating that the remaining capacity is sufficient to upload the second storage file, and then the second storage file is stored in the preset storage space together with the first storage file; the second primary key is generated according to the size of the second storage file in the same way as the first primary key generation process, for example, the current maximum primary key is 1048576B, and the size of the second storage file is 2MB, then the second primary key is 1048576B+2×1024×1024B=3145728B, and the size of the primary key is accumulated according to the order of file storage, and the primary keys of all storage files in the preset storage space are obtained one by one; after the second storage file is stored in the preset storage space, the maximum primary key in the preset storage space is the second primary key, so the second primary key is updated to the new maximum primary key.
[0088] This embodiment implements the method of continuously storing new files if there is sufficient remaining capacity when storing new files, and generating corresponding primary keys of new files in a cumulative form, so that files and their information can be quickly located through the primary keys, thereby improving data retrieval efficiency, ensuring that the primary keys in the system are continuous and incremental, and facilitating management and maintenance, and ensuring that the mappings of all files in the system are up to date by updating the mapping relationship, thereby avoiding data inconsistency problems.
[0089] In some embodiments, obtaining the target capacity based on the maximum primary key, the historical deletion mark primary key, and the second file size includes:
[0090] Based on the maximum primary key and the historical deletion mark primary key, determine the space occupancy statistics value, and upload the space occupancy statistics value, the maximum primary key and the historical deletion mark primary key to the metadata table;
[0091] The target capacity is determined according to the second file size by querying the metadata table.
[0092] Among them, the space occupancy statistics are determined according to the maximum primary key and the historical deletion mark primary key. The space occupancy statistics are the space occupied by the files in the current storage space. Specifically, the maximum primary key minus the historical deletion mark primary key is the space occupancy statistics. The space occupancy statistics, the maximum primary key and the historical deletion mark primary key are uploaded to the metadata table. By querying the metadata table, it is possible to accurately determine which file records currently need to be considered, read the space occupancy statistics, and obtain the target capacity required to store the file based on the second file size. Then, by comparing the target capacity and the fixed capacity, a comparison result is obtained, so that it can be determined whether a deletion operation is needed.
[0093] This embodiment can trace the deletion history of data by recording the primary key of historical deletion mark, which is helpful for data recovery or auditing; by centrally storing key information in the metadata table, it is convenient for subsequent query and analysis, improves the calculation efficiency of target capacity, and effectively supports the management and planning of storage capacity.
[0094] The embodiments of the present application are described and illustrated below through preferred embodiments.
[0095] Figure 3 is a flowchart of a storage file deletion method according to a preferred embodiment of the present application, such as Figure 3 As shown, the storage file deletion method includes the following steps:
[0096] The user has subscribed to 1TB of storage space and will continue to upload files. When the space capacity reaches 1TB each time, 10% of the files need to be deleted in chronological order, with older files being given priority.
[0097] This preferred embodiment sets up two data storage tables: a metadata table and an upload record table. The metadata table includes a deletion mark ID (marking the minimum ID in the list to be deleted, initially 0), a maximum primary key ID (the ID of the latest uploaded data in the upload record table), and space occupancy statistics (used to determine whether deletion is required); the upload record table includes a primary key ID (a primary key ID generated based on the size of the uploaded file in bytes), an uploaded file size (recording the space bytes of the uploaded file), and file index information (recording information such as the file name and file address).
[0098] When uploading a 1M file, a row of upload records is saved and the primary key ID of the file is 1M×1024×1024=1048576B. When uploading the second 3M file, you need to find the largest primary key ID in the current upload record table from the index table, which is 1048576B at this time. Therefore, the primary key ID of the second upload record is 1048576+3×1024×1024=4194304. Similarly, when the total size of the uploaded files exceeds 1T, you need to start deleting them. At this time, you need to query the minimum primary key ID in the index table. Because it is the first time and no deletion task has been executed, the minimum primary key ID is 0. At this time, the files to be deleted meet the ID less than or equal to 0+1×1024×1024×1024×10%=107374182.4B. Figure 3 The maximum number of deleted primary key auto-increment is 107374182.4B. If you use the MYSQL database, you only need to select from upload_table where id<=107374182.4, thus greatly improving the query efficiency.
[0099] This embodiment also provides a storage file deletion device, which is used to implement the above embodiments and preferred implementations, and will not be repeated here. As used below, the terms "module", "unit", "subunit", etc. can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.
[0100] Figure 4 is a structural block diagram of a storage file deletion device according to an embodiment of the present application, such as Figure 4 As shown, the device comprises:
[0101] A first file storage module 10 is used to obtain a first file size of a first storage file stored in a preset storage space, and generate a first primary key based on the first file size;
[0102] The storage module 20 is updated to obtain the second file size of the second storage file, query the maximum primary key with the largest primary key value from the first primary key, and obtain the historical deletion mark primary key;
[0103] The capacity determination module 30 is used to obtain the target capacity based on the maximum primary key, the historical deletion mark primary key and the second file size, and determine the deletion determination result of the first storage file according to the comparison result between the target capacity and the preset fixed capacity;
[0104] The file deletion module 40 is used to determine the file to be deleted in the first storage file based on the historical deletion mark primary key and delete the file to be deleted from the preset storage space when the deletion determination result indicates deletion.
[0105] The file deletion module 40 is also used to obtain a preset single deletion amount;
[0106] Based on the historical deletion mark primary key and the single deletion amount, determine the primary key of the file to be deleted in the first primary key;
[0107] Based on the primary key of the file to be deleted, the file to be deleted in the first storage file is determined, and the file to be deleted is deleted from the preset storage space.
[0108] The file deletion module 40 is further used to obtain index information corresponding to the first storage file, where the index information includes the address of the first storage file;
[0109] Based on the index information and the primary key of the file to be deleted, determine the address of the file to be deleted in the first storage file address;
[0110] Delete the file to be deleted from the preset storage space based on the address of the file to be deleted.
[0111] The file deletion module 40 is further used to determine, in the first primary key, a primary key whose primary key value is less than or equal to the primary key sum as the primary key of the file to be deleted, where the primary key sum is the sum of the historical deletion mark primary key and the single deletion amount.
[0112] The file deletion module 40 is also used to determine the file to be deleted based on the historical deletion mark primary key when the deletion determination result indicates deletion, and update the historical deletion mark primary key based on the first primary key of the file to be deleted to obtain a new historical deletion mark primary key.
[0113] The capacity determination module 30 is further configured to generate a second primary key of the second storage file based on the second file size when the deletion determination result does not indicate deletion;
[0114] Update the maximum primary key based on the second primary key to obtain a new maximum primary key.
[0115] The capacity determination module 30 is further used to determine the space occupancy statistics based on the maximum primary key and the historical deletion mark primary key, and upload the space occupancy statistics, the maximum primary key and the historical deletion mark primary key to the metadata table;
[0116] The target capacity is determined according to the second file size by querying the metadata table.
[0117] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.
[0118] This embodiment further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0119] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.
[0120] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:
[0121] S1, obtaining a first file size of a first storage file stored in a preset storage space, and generating a first primary key based on the first file size.
[0122] S2, obtaining the second file size of the second storage file, querying the maximum primary key with the largest primary key value from the first primary keys, and obtaining the historical deletion mark primary key.
[0123] S3, based on the maximum primary key, the historical deletion mark primary key and the second file size, the target capacity is obtained, and according to the comparison result between the target capacity and the preset fixed capacity, the deletion judgment result of the first storage file is determined.
[0124] S4, when the deletion determination result indicates deletion, based on the historical deletion mark primary key, determine the file to be deleted in the first storage file, and delete the file to be deleted from the preset storage space.
[0125] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and this embodiment will not be described in detail here.
[0126] In addition, in combination with the storage file deletion method in the above embodiment, the embodiment of the present application can provide a storage medium for implementation. The storage medium stores a computer program; when the computer program is executed by a processor, any storage file deletion method in the above embodiment is implemented.
[0127] Those skilled in the art should understand that the technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0128] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.
Claims
1. A method for deleting a stored file, characterized in that: include: Acquire a first file size of a first storage file stored in a preset storage space, and generate a first primary key based on the first file size; Obtain a second file size of the second storage file, query a maximum primary key with the largest primary key value from the first primary keys, and obtain a historical deletion mark primary key; Based on the maximum primary key, the historical deletion mark primary key and the second file size, a target capacity is acquired, and according to a comparison result between the target capacity and a preset fixed capacity, a deletion determination result of the first storage file is determined; In the case where the deletion determination result indicates deletion, the to-be-deleted file in the first storage file is determined based on the historical deletion mark primary key, and the to-be-deleted file is deleted from the preset storage space.
2. The method for deleting stored files according to claim 1, characterized in that: The determining the to-be-deleted file in the first storage file based on the historical deletion mark primary key, and deleting the to-be-deleted file from the preset storage space, includes: Get the preset single deletion amount; Based on the historical deletion mark primary key and the single deletion amount, determining the primary key of the file to be deleted in the first primary key; Based on the primary key of the file to be deleted, the file to be deleted in the first storage file is determined, and the file to be deleted is deleted from the preset storage space.
3. The method for deleting stored files according to claim 2, characterized in that: The determining the file to be deleted in the first storage file based on the primary key of the file to be deleted, and deleting the file to be deleted from the preset storage space includes: Obtaining index information corresponding to the first storage file, the index information including the first storage file address; Based on the index information and the primary key of the file to be deleted, determining the address of the file to be deleted in the first stored file address; Based on the address of the file to be deleted, the file to be deleted is deleted from the preset storage space.
4. The method for deleting stored files according to claim 2, characterized in that: The determining the to-be-deleted file primary key in the first primary key based on the historical deletion mark primary key and the single deletion amount includes: In the first primary key, a primary key whose primary key value is less than or equal to the primary key sum is determined as the primary key of the file to be deleted, and the primary key sum is the sum of the historical deletion mark primary key and the single deletion amount.
5. The method for deleting stored files according to claim 1, characterized in that: The method further comprises: When the deletion determination result indicates deletion, the to-be-deleted file is determined based on the historical deletion mark primary key, and the historical deletion mark primary key is updated based on the primary key with the largest primary key value in the to-be-deleted file to obtain a new historical deletion mark primary key.
6. The method for deleting stored files according to claim 1, characterized in that: The method further comprises: If the deletion determination result does not indicate deletion, generating a second primary key of a second storage file based on the second file size; The maximum primary key is updated based on the second primary key to obtain a new maximum primary key.
7. The method for deleting stored files according to claim 1, characterized in that: The acquiring the target capacity based on the maximum primary key, the historical deletion mark primary key and the second file size includes: Based on the maximum primary key and the historical deletion mark primary key, determine a space occupancy statistical value, and upload the space occupancy statistical value, the maximum primary key and the historical deletion mark primary key to a metadata table; The target capacity is determined according to the second file size by querying the metadata table.
8. A storage file deletion device, characterized in that: include: A first file storage module, configured to obtain a first file size of a first storage file stored in a preset storage space, and generate a first primary key based on the first file size; An updating storage module is used to obtain a second file size of a second storage file, query a maximum primary key with a maximum primary key value from the first primary keys, and obtain a historical deletion mark primary key; a capacity determination module, configured to obtain a target capacity based on the maximum primary key, the historical deletion mark primary key and the second file size, and determine a deletion determination result of the first storage file according to a comparison result between the target capacity and a preset fixed capacity; The file deletion module is used to determine the file to be deleted in the first storage file based on the historical deletion mark primary key when the deletion judgment result indicates deletion, and delete the file to be deleted from the preset storage space.
9. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the storage file deletion method according to any one of claims 1 to 7.
10. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the storage file deletion method according to any one of claims 1 to 7 when running.
Citation Information
Patent Citations
File management method, expired file deletion method and devices
CN108932236A
Data deleting method and device and electronic equipment
CN111552667A
