Object deletion method, data processing method and device
By presetting the specified metadata field in the target metadata database of the object storage server, determining whether the object expires according to the deletion conditions and setting its value, the problem of inaccurate statistics during object deletion is solved, and the accuracy of object statistics-related processing events is improved to ensure the accuracy of billing.
Patent Information
- Application Number
- CN202010831443.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-08-18
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2040-08-18
AI Technical Summary
The existing object deletion method has a delay in the deletion process, resulting in inaccurate object statistics and affecting the accuracy of billing.
The specified metadata field used to indicate whether the object has expired is preset in the target metadata database of the object storage server. It determines whether the object has expired based on the deletion conditions, and sets the value of the field when determining that the object is to be deleted, so that when deleting the object, it is deleted, the object is deleted. Reflect its expired status in a timely manner to ensure the accuracy of statistical results.
By setting the specified metadata field of the object, we ensure that only unexpired objects are counted during the object deletion process, which improves the accuracy of processing results of related processing events of the object statistics and avoids the problem of inaccurate billing.
Smart Images

Figure CN111966867B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data storage, and in particular to an object deletion method, a data processing method and a device. Background Art
[0002] Object storage is a highly reliable, highly available, low-cost, and infinitely scalable storage method suitable for storing massive amounts of unstructured data.
[0003] Even though cloud vendors offer very low prices for object storage, objects usually become worthless and expired data after a period of time. Therefore, in order to save costs, a large number of objects need to be deleted. In order to meet the deletion requirements, cloud vendors provide users with an interface for setting deletion conditions, so that users can set deletion conditions according to their own needs and delete objects in the cloud object storage server.
[0004] In the prior art, the object deletion method includes: using the deletion conditions set by the user to determine each object to be deleted belonging to the user in the object storage server, and then deleting each determined object to be deleted in turn.
[0005] However, after each object is determined, there is a delay in deleting each object, which may cause the following problems:
[0006] If a processing event related to object statistics occurs during the object deletion process, the processing result of the processing event may be inaccurate. For example, for the object capacity statistics based on storage billing, there may be a problem that objects that should have been deleted may be counted, affecting the accuracy of object capacity statistics and ultimately causing inaccurate billing. Summary of the invention
[0007] The purpose of the embodiments of the present invention is to provide an object deletion method, a data processing method and a device to improve the accuracy of the processing results of processing events related to object statistics. The specific technical solution is as follows:
[0008] In a first aspect, an embodiment of the present invention provides an object deletion method, comprising:
[0009] Determine at least one object to be analyzed in the object storage server; wherein metadata of the object in the object storage server is stored in a target metadata database, and a specified metadata field for indicating whether the object is expired is preset in the target metadata database;
[0010] For each object, when it is determined based on the set deletion condition that the object belongs to an object to be deleted, the object is determined as an object to be deleted, and the field value of the specified metadata field of the object is set using the specified content; wherein the specified content is content used to indicate that the object is expired;
[0011] Delete the object to be deleted in the object storage server.
[0012] Optionally, before determining the object as an object to be deleted, the method further includes:
[0013] Determine pre-set protection conditions;
[0014] Determining whether the object meets the protection condition;
[0015] If it is determined that the object does not satisfy the protection condition, the step of determining the object as an object to be deleted is executed; if it satisfies the condition, the deletion process of the object is terminated.
[0016] Optionally, the condition content of the deletion condition includes at least a first object range for indicating a range of objects to be deleted;
[0017] The first object range is represented by a combination of an operator and identification content, the operator includes match or mismatch, and the identification content includes: prefix and / or label information.
[0018] Optionally, the condition content of the protection condition includes at least a second object range for indicating a range of objects to be deleted;
[0019] The second object range is represented by a combination of an operator and identification content, the operator includes match or non-match, and the identification content includes: prefix and / or label information.
[0020] Optionally, determining the object as an object to be deleted includes:
[0021] Adding the object identifier of the object to a target list; wherein the target list is used to record a list of objects to be deleted;
[0022] The deleting the object to be deleted in the object storage server includes:
[0023] An object identifier is read from the target list, and an object having the read object identifier is deleted from the object storage server.
[0024] Optionally, deleting the object to be deleted in the object storage server includes:
[0025] The object to be deleted in the object storage server is deleted through an asynchronous deletion thread; wherein the asynchronous deletion thread is a thread running in a device with a load lower than a predetermined threshold.
[0026] In a second aspect, an embodiment of the present invention further provides a data processing method based on the object deletion method provided in the first aspect, comprising:
[0027] Obtaining a data processing instruction; wherein the data processing instruction is a processing instruction for a processing event related to object statistics of a target user;
[0028] Determine, from the target metadata database, metadata whose field value of the specified metadata field is not the specified content and whose owner is the target user;
[0029] Based on the determined metadata, a processing result of the data processing instruction is determined.
[0030] Optionally, obtaining the data processing instruction includes:
[0031] Get object capacity statistics command;
[0032] The determining, based on the determined metadata, a processing result of the data processing instruction includes:
[0033] The total space occupied by the object represented by the determined metadata is calculated as a processing result corresponding to the object capacity statistics instruction.
[0034] In a third aspect, an embodiment of the present invention further provides an object deletion device, comprising:
[0035] An object determination module, used to determine at least one object to be analyzed in the object storage server; wherein the metadata of the object in the object storage server is stored in a target metadata database, and the target metadata database is preset with a specified metadata field for indicating whether the object is expired;
[0036] The object processing module is used for determining, for each object, when it is determined based on the set deletion condition that the object belongs to an object to be deleted, the object as an object to be deleted, and setting the field value of the specified metadata field of the object by using the specified content; wherein the specified content is content used to indicate that the object is expired;
[0037] The object deletion module is used to delete the object to be deleted in the object storage server.
[0038] In a fourth aspect, an embodiment of the present invention further provides a data processing device based on the object deletion device provided in the third aspect, comprising:
[0039] An instruction acquisition module, used to obtain a data processing instruction; wherein the data processing instruction is a processing instruction for a processing event related to the target user and the object statistics;
[0040] A metadata determination module, used to determine, from a target metadata database, metadata whose field value of a specified metadata field is not specified content and whose owner is the target user;
[0041] The processing result determination module is used to determine the processing result of the data processing instruction based on the determined metadata.
[0042] In a fifth aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus;
[0043] Memory, used to store computer programs;
[0044] The processor is used to implement the method steps provided in the first aspect or the second aspect when executing the program stored in the memory.
[0045] In a sixth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps provided in the first aspect or the second aspect are implemented.
[0046] Beneficial effects of the embodiments of the present invention:
[0047] In the object deletion method provided by the embodiment of the present invention, a designated metadata field for indicating whether an object is expired is pre-set for the target metadata database; in this way, after determining at least one object to be analyzed, for each object, when it is determined that the object belongs to the object to be deleted based on the set deletion condition, the object is determined to be the object to be deleted, and the field value of the designated metadata field of the object is set using the designated content; wherein the designated content is the content used to indicate that the object is expired; and then, the object to be deleted in the object storage server is deleted. In this scheme, when the object is determined to belong to the object to be deleted, the field value of the designated metadata field of the object is set using the designated content, which can timely reflect whether the object is expired. In this way, even if a processing event related to object statistics occurs during the object deletion process, since the target database records whether the object is expired, the objects counted for the processing event can only include non-expired objects. It can be seen that this scheme can improve the accuracy of the processing results of processing events related to object statistics.
[0048] In addition, in the data processing method provided by the embodiment of the present invention, after obtaining the data processing instruction, the metadata whose field value of the specified metadata field is not the specified content and whose owner is the target user is determined from the target metadata database; then, based on the determined metadata, the processing result of the data processing instruction is determined. Since in the data processing process, the metadata whose field value of the specified metadata field is not the specified content and whose owner is the target user is determined from the target metadata database, and then the processing result is determined using the determined metadata, the objects counted for the processing event can only include non-expired objects. Therefore, the accuracy of the processing results of the processing events related to object statistics can be improved through the data processing method.
[0049] Of course, it is not necessary to achieve all of the advantages described above at the same time to implement any product or method of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can be obtained based on these drawings without paying creative work.
[0051] Figure 1 A flowchart of an object deletion method provided by an embodiment of the present invention;
[0052] Figure 2 Another flow chart of the object deletion method provided by an embodiment of the present invention;
[0053] Figure 3 A flowchart of a data processing method provided by an embodiment of the present invention;
[0054] Figure 4 A schematic diagram of the overall principle of the object deletion method and data processing method provided by an embodiment of the present invention;
[0055] Figure 5 A schematic diagram of the structure of an object deletion device provided by an embodiment of the present invention;
[0056] Figure 6 A schematic diagram of the structure of a data processing device provided by an embodiment of the present invention;
[0057] Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention;
[0058] Figure 8 A schematic diagram of the structure of another electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0060] In order to improve the accuracy of processing results of processing events related to object statistics, embodiments of the present invention provide an object deletion method, a data processing method and a device.
[0061] The object deletion method provided by the embodiment of the present invention is first introduced below.
[0062] The object deletion method provided in the embodiment of the present invention may be executed by an object deletion device. The object deletion device may be applied to an electronic device, wherein the electronic device may be an object storage server, or a management server for managing an object storage server, etc. The object storage server is a server for storing objects.
[0063] Furthermore, the objects mentioned in the embodiments of the present invention may be unstructured data such as pictures, videos or documents. For any object, it has metadata and object identifiers. Specifically, the metadata of an object refers to the relevant description of the object, such as the size of the object, the owner of the object, the last modification time of the object, etc., while the object identifier is a global unique identifier used to distinguish objects.
[0064] In addition, since the deletion is performed according to the needs of the target user, the deletion time is uncertain, so the object deletion provided by the embodiment of the present invention belongs to asynchronous deletion. In the object deletion process belonging to asynchronous deletion, the set deletion condition is used as the judgment basis for whether the object is deleted.
[0065] Specifically, the condition content of the set deletion condition may include at least two types of information: a deletion method and a first object range used to represent the range of the object to be deleted, that is, the deletion condition at least provides the deletion method and the first object range. Among them, the deletion method may include a first deletion method and a second deletion method, the first deletion method is a method of deleting at a specified expiration time, and the second deletion method is a method of deleting when a specified expiration time is reached. Among them, the first deletion method specifically refers to deleting at a specific time, and the second deletion method specifically refers to deleting after a specified time period starting from the last modification time of the object.
[0066] like Figure 1As shown, the embodiment of the present invention provides an object deletion method, which may include the following steps:
[0067] S101, determining at least one object to be analyzed in an object storage server;
[0068] The metadata of the object in the object storage server is stored in a target metadata database, and a designated metadata field for indicating whether the object is expired is preset in the target metadata database.
[0069] When the triggering time for deleting the target user's object is reached, at least one object to be analyzed can be first determined from the objects stored in the object storage server, and then the at least one object can be analyzed for deletion. The at least one object belongs to the target user.
[0070] The metadata of the objects in the object storage server is stored in the target metadata database. Since the metadata of the object includes the owner of the object, and the metadata of the object is stored in the target metadata database, at least one object belonging to the target user stored in the object storage server can be determined based on the target metadata database as the at least one object to be analyzed.
[0071] In addition, a designated metadata field is added to the target metadata database to indicate whether the object is expired. The designated metadata field can reflect whether the object is expired. Specifically, the designated metadata field can include initial content and designated content. The initial content is used to indicate that the object is not expired, and the designated content is used to indicate that the object is expired. The initial content and the designated content are different contents. For example, the initial content is a null value, and the designated content is a non-null value; or, the initial content is 0, and the designated content is 1, and so on.
[0072] Furthermore, there may be multiple triggering times for determining at least one object to be analyzed in the object storage server. Exemplarily, if the deletion method given by the deletion condition preset by the target user is the first deletion method, then the at least one object to be analyzed in the object storage server may be triggered to be executed when the specified expiration time is reached; if the deletion method given by the deletion condition preset by the target user is the second deletion method, then the at least one object to be analyzed in the object storage server may be periodically executed, that is, when each cycle is started, the at least one object to be analyzed in the object storage server may be triggered to be executed.
[0073] In addition, the number of pre-set deletion conditions may be one or more. In the case where the number of pre-set deletion conditions is multiple, if the deletion methods given by the multiple deletion conditions are all the first deletion method, then when the specified expiration time involved in the deletion method given by each deletion condition is reached, S101 can be triggered to execute; if the deletion methods given by the multiple deletion conditions include both the first deletion method and the second deletion method, then when the specified expiration time involved in the first deletion method is reached, S101 can be triggered to execute, and at the same time, S101 can be triggered to execute periodically; and if the deletion methods given by the multiple deletion conditions are all the second deletion method, S101 can be triggered to execute periodically.
[0074] S102, for each object, when it is determined based on the set deletion condition that the object belongs to an object to be deleted, the object is determined as an object to be deleted, and a field value of a specified metadata field of the object is set using specified content; wherein the specified content is content used to indicate that the object is expired;
[0075] After determining at least one object to be analyzed, each object can be traversed with the object as the traversal target. When traversing each object, it can be determined whether the object belongs to the object to be deleted based on the pre-set deletion condition, and when the judgment result is yes, not only the object is determined as the object to be deleted, but also the field value of the specified metadata field of the object is set using the specified content, that is, the content of the field value of the specified metadata field of the object is set to: the specified content; when the judgment result is no, the deletion process of the object is terminated. In this way, the object to be deleted can be known through step S102, and at the same time, which objects are expired objects are reflected in the target metadata database in a timely manner. It can be understood that if the field value of the specified metadata field in the metadata of a certain object is set to the specified content, the certain object cannot be accessed, which is specifically reflected in that it cannot be queried and downloaded. When there is a query or download request for the certain object, the querying party or the downloading party will obtain a 404 error page.
[0076] It can be understood that, for the deletion conditions given the first deletion method, in the specific implementation process of judging whether the object belongs to the object to be deleted based on the set deletion conditions, the first object range given by the deletion conditions can be used to judge whether the object belongs to the object to be deleted. Specifically, if the object is located within the first object range given by the deletion conditions, the object is judged to belong to the object to be deleted. For the deletion conditions given the second deletion method, in the specific implementation process of judging whether the object belongs to the object to be deleted based on the set deletion conditions, the deletion method and the first object range given by the deletion conditions can be used to judge whether the object belongs to the object to be deleted. Specifically, if the object meets the duration represented by the deletion method given by the deletion conditions, and the object is located within the first object range given by the deletion conditions, the object is judged to belong to the object to be deleted.
[0077] Exemplarily, in one implementation, determining the object record as an object to be deleted may include:
[0078] The object identifier of the object is added to a target list, wherein the target list is used to record a list of objects to be deleted.
[0079] In this implementation, the object identifier of the object to be deleted is recorded in the target list, so that when the object is subsequently deleted, the object identifier can be read from the target list, thereby deleting the object with the read object identifier.
[0080] The above specific implementation of determining the object as an object to be deleted is only an example and should not constitute a limitation on the embodiments of the present invention. For example, determining the object as an object to be deleted may also include: adding a tag to the object to indicate that it is to be deleted, or setting the object identifier of the object in the target metadata database to an identifier to indicate that it is to be deleted, etc.
[0081] In addition, there are multiple specific representation forms of the first object range given in the deletion condition. For the sake of clarity of layout and solution, the specific representation forms of the object range are introduced by examples below.
[0082] S103: Delete the object to be deleted in the object storage server.
[0083] After determining each object to be deleted, each object to be deleted in the object storage server can be deleted. Since the field value of the specified metadata field of the object to be deleted has been set to the specified content when the object is determined to be an object to be deleted, it can timely reflect whether the object is expired. Therefore, even if there is a delay in deleting the object, if a processing event related to object statistics occurs during the object deletion process, the objects counted in the processing event processing process will only be non-expired objects, thereby improving the accuracy of the processing results.
[0084] It can be understood that the specific implementation method of deleting the object to be deleted in the object storage server is affected by the specific implementation method of determining the object as the object to be deleted.
[0085] Exemplarily, with respect to the manner of adding the object identifier of the object to the target list, deleting the object to be deleted in the object storage server accordingly may include:
[0086] The object identifier is read from the target list, and the object having the read object identifier is deleted from the object storage server.
[0087] It is understandable that when an object is deleted, the electronic device can call its own deletion process to achieve object deletion. In addition, considering that the load of the electronic device where the object deletion device is located may be large, in order to further improve the efficiency of object deletion, other devices with lower loads can be triggered to delete the object to be deleted. Based on this processing idea, deleting the object to be deleted in the object storage server may include:
[0088] The object to be deleted in the object storage server is deleted through an asynchronous deletion thread; wherein the asynchronous deletion thread is a thread running in a device whose load is lower than a predetermined threshold.
[0089] The load of any electronic device may be calculated in any manner that can implement load calculation, and the embodiment of the present invention does not limit this.
[0090] Specifically, when each object identifier is read from the target list, and the object with the read object identifier is deleted from the object storage server, including:
[0091] Each object identifier is read from the target list through an asynchronous deletion thread, and the object with the read object identifier is deleted from the object storage server.
[0092] In this implementation, after being called, the asynchronous deletion thread can read each object identifier from the target list and delete the object with the read object identifier from the object storage server. In addition, after deleting an object, the asynchronous deletion thread can report to the object deletion device so that the object deletion device deletes the object identifier of the object in the target list, thereby avoiding repeated deletion of the same object.
[0093] In addition, if the object deletion fails, a deletion failure record may be generated and saved in a storage bucket, ie, a storage area, specified by the target user, so that the target user may analyze the failure cause based on the deletion failure record.
[0094] In this solution, when an object is determined to be an object to be deleted, the field value of the specified metadata field of the object is set using the specified content, which can timely reflect whether the object is expired. In this way, even if a processing event related to object statistics occurs during the object deletion process, since the target database records whether the object is expired, the objects counted for the processing event can only include non-expired objects. It can be seen that this solution can improve the accuracy of the processing results of processing events related to object statistics.
[0095] Considering that the target user has the need to delete objects, there is also a need to protect certain objects, that is, no matter whether the deletion conditions are met, these objects cannot be deleted. Therefore, object deletion not only meets the deletion needs of the target user, but also meets the protection needs of the target user. Based on this processing idea, Figure 2 As shown, the embodiment of the present invention provides an object deletion method, which may include the following steps:
[0096] S201, determining at least one object to be analyzed in an object storage server;
[0097] In this embodiment, step S201 is the same as step S101 in the above embodiment, and will not be described in detail here.
[0098] S202, for each object, when it is determined based on the set deletion condition that the object belongs to an object to be deleted, a pre-set protection condition is determined to determine whether the object satisfies the protection condition; if it is determined that the object does not satisfy the protection condition, the object is determined to be an object to be deleted, and the field value of the specified metadata field of the object is set using the specified content; if the condition is satisfied, the deletion process of the object is terminated;
[0099] In this embodiment, step S202 is similar to step S102 in the above embodiment, except that: when it is determined based on the set deletion condition that the object belongs to the object to be deleted, the object is not directly determined as the object to be deleted and the field value of the specified metadata field in the target metadata database is not set, but based on the set protection condition, it is determined whether the object is the object to be protected. If not, the object record is determined as the object to be deleted, and the field value of the specified metadata field of the object is set using the specified content. It can be understood that if it is determined that the object is the object to be protected, the step of deleting the object is terminated.
[0100] The condition content of the protection condition at least includes a second object range for indicating a range of objects to be deleted;
[0101] Exemplarily, in one implementation, the second object range is represented by a combination of an operator and an identification content, the operator includes a match or a mismatch, and the identification content includes: prefix and / or tag information. For a description of the operator and the identification content, see the corresponding content in the deletion condition.
[0102] For ease of understanding, the following example illustrates the protection conditions set based on this implementation method:
[0103] Protection condition 1: Matches objects with the prefix "1234".
[0104] S203: Delete the object to be deleted in the object storage server.
[0105] In this embodiment, step S203 is the same as step S103 in the above embodiment, and will not be described in detail here.
[0106] In the scheme provided by this embodiment, when an object is determined to be an object to be deleted, the field value of the specified metadata field of the object is set using the specified content, which can timely reflect whether the object is expired. In this way, even if a processing event related to object statistics occurs during the object deletion process, since the target database records whether the object is expired, the objects counted for the processing event can only include non-expired objects. It can be seen that this scheme can improve the accuracy of the processing results of processing events related to object statistics. In addition, in the scheme provided by this embodiment, the different needs of users can be met by setting deletion conditions and protection conditions.
[0107] The specific representation form of the first object range given by the deletion condition is introduced below by way of example.
[0108] Exemplarily, in order to ensure that the user can flexibly set the first object range according to needs, in one implementation, the first object range can be represented by a combination of an operator and identification content, the operator includes match or mismatch, and the identification content includes: prefix and / or tag information. The tag information is the important, backup, and other content pre-set for the object.
[0109] For ease of understanding, the following example illustrates the deletion conditions set based on this implementation method:
[0110] Deletion condition 1: Match the objects with the prefix "123" and specify to delete on 2020-12-12;
[0111] Deletion condition 2: objects with prefix "123" and tag information "backup" are deleted on 2020-12-12;
[0112] Deletion condition 3: Specifies to delete objects on 2020-12-12 that do not match the prefix "123" and have the tag information "Important".
[0113] For example, in another implementation, the object range can be represented by a storage space, an object prefix, or an object tag information. For ease of understanding, the deletion condition set based on this implementation is illustrated by way of example:
[0114] Deletion condition 1: For objects whose storage location is A, specify to be deleted on 2020-12-12; Deletion condition 2: For objects whose prefix is "123", specify to be deleted on 2020-12-12; Deletion condition 3: For objects whose tag information is "backup", specify to be deleted on 2020-12-12.
[0115] The specific representation forms of the above-mentioned object scope are only examples and should not constitute limitations on the embodiments of the present invention.
[0116] Based on the above object deletion method, an embodiment of the present invention further provides a data processing method.
[0117] The data processing method may be executed by a data processing device. The data processing device may be applied to an electronic device, wherein the electronic device may be an object storage server, or a management server for managing an object storage server, etc. The object storage server is a server that stores objects.
[0118] like Figure 3 As shown, the data processing method provided by the embodiment of the present invention may include:
[0119] S301, obtaining a data processing instruction; wherein the data processing instruction is a processing instruction for processing events related to object statistics of a target user;
[0120] The data processing instruction may be an object query instruction, an object quantity statistics instruction or an object capacity statistics instruction, but is not limited thereto. The object query instruction is used to query objects belonging to the target user; the object quantity statistics instruction is used to count the number of objects belonging to the target user; and the object capacity statistics instruction is used to count the total capacity of objects belonging to the target user, which can be used for capacity billing.
[0121] S302, from the target metadata database, determining metadata whose field value of the specified metadata field is not the specified content and whose owner is the target user;
[0122] After obtaining the data processing instruction, due to the delay in object deletion, some objects that should have been deleted have not been deleted, that is, the metadata of the objects that should have been deleted still exists in the target metadata database. Therefore, in order to ensure that only non-expired objects are counted, the metadata whose field value of the specified metadata field is not the specified content and whose owner is the target user can be determined from the target metadata database, and then the determined metadata can be used for subsequent processing.
[0123] S303: Determine a processing result of the data processing instruction based on the determined metadata.
[0124] Different data processing instructions determine different processing results. Specifically, if the data processing instruction is an object query instruction, the object represented by the determined metadata is determined as the processing result; if the data processing instruction is an object quantity statistics instruction, the number of objects represented by the determined metadata is counted as the processing result; if the data processing instruction is an object capacity statistics instruction, the total space occupied by the objects represented by the determined metadata is calculated as the processing result corresponding to the object capacity statistics instruction.
[0125] The total capacity of the object can be used to provide billing information to the user. Since the total capacity of the object is accurate, the billing information is accurate. In this way, the cloud vendor can give the user a billing commitment, that is, a commitment that the objects involved in the billing information are not expired objects.
[0126] In the data processing method provided by the embodiment of the present invention, after obtaining the data processing instruction, the metadata whose field value of the specified metadata field is not the specified content and whose owner is the target user is determined from the target metadata database; then, based on the determined metadata, the processing result of the data processing instruction is determined. Since in the data processing process, the metadata whose field content of the specified metadata field is not the specified value and whose owner is the target user is determined from the target metadata database, and then the processing result is determined using the determined metadata, the objects counted for the processing event can only include non-expired objects. Therefore, the accuracy of the processing results of the processing events related to object statistics can be improved through the data processing method.
[0127] In order to facilitate understanding of the above object deletion method and data processing method, Figure 4 The overall principle diagram of the object deletion method and data processing method is given. Figure 4 In the object deletion device, a designated metadata field is added to the target metadata database. When the object is determined to be an object to be deleted, the identification information of the object is recorded in the target list, and the field value of the designated metadata field of the object is set to the designated content; the object deletion device can call an asynchronous deletion thread, and access the object identification in the target list through the asynchronous deletion thread to delete the object in the disk of the object storage server; and the asynchronous deletion thread can synchronize the deletion operation information to the object deletion device. In addition, when the data processing device obtains the data processing instruction, it can access the target metadata database to determine the response result corresponding to the data processing instruction.
[0128] Based on the above object deletion method, the embodiment of the present invention also provides an object deletion device. Figure 5 As shown, the object deletion device provided in the embodiment of the present invention may include:
[0129] The object determination module 510 is used to determine at least one object to be analyzed in the object storage server; wherein the metadata of the object in the object storage server is stored in a target metadata database, and the target metadata database is preset with a specified metadata field for indicating whether the object is expired;
[0130] The object processing module 520 is used for, for each object, when it is determined based on the set deletion condition that the object belongs to the object to be deleted, determining the object as the object to be deleted, and setting the field value of the specified metadata field of the object by using the specified content; wherein the specified content is the content used to indicate that the object is expired;
[0131] The object deletion module 530 is used to delete the object to be deleted in the object storage server.
[0132] In the object deletion device provided by the embodiment of the present invention, a designated metadata field for indicating whether an object is expired is pre-set for the target metadata database; in this way, after determining at least one object to be analyzed, for each object, when it is determined that the object belongs to the object to be deleted based on the set deletion condition, the object is determined to be the object to be deleted, and the field value of the designated metadata field of the object is set using the designated content; wherein the designated content is the content used to indicate that the object is expired; and then, the object to be deleted in the object storage server is deleted. In this scheme, when the object is determined to belong to the object to be deleted, the field value of the designated metadata field of the object is set using the designated content, which can timely reflect whether the object is expired. In this way, even if a processing event related to object statistics occurs during the object deletion process, since the target database records whether the object is expired, the objects counted for the processing event can only include non-expired objects. It can be seen that this scheme can improve the accuracy of the processing results of processing events related to object statistics.
[0133] Optionally, the object processing module 520 is also used to determine a pre-set protection condition before determining the object as an object to be deleted; determine whether the object satisfies the protection condition; if it is determined that the object does not satisfy the protection condition, execute the step of determining the object as an object to be deleted; if it is satisfied, end the deletion process of the object.
[0134] Optionally, the condition content of the deletion condition includes at least a first object range for indicating a range of objects to be deleted;
[0135] The first object range is represented by a combination of an operator and identification content, the operator includes match or mismatch, and the identification content includes: prefix and / or label information.
[0136] Optionally, the condition content of the protection condition includes at least a second object range for indicating a range of objects to be deleted;
[0137] The second object range is represented by a combination of an operator and identification content, the operator includes match or non-match, and the identification content includes: prefix and / or label information.
[0138] Optionally, the object processing module 520 determines the object as an object to be deleted, including:
[0139] Adding the object identifier of the object to a target list; wherein the target list is used to record a list of objects to be deleted;
[0140] The object deletion module 530 is specifically used for:
[0141] An object identifier is read from the target list, and an object having the read object identifier is deleted from the object storage server.
[0142] Optionally, the object deletion module 530 is specifically used to: delete the to-be-deleted object in the object storage server through an asynchronous deletion thread; wherein the asynchronous deletion thread is a thread running in a device whose load is lower than a predetermined threshold.
[0143] Corresponding to the above-mentioned object deletion device, the embodiment of the present invention also provides a data processing device. Figure 6 As shown, the data processing device may include:
[0144] The instruction acquisition module 610 is used to obtain a data processing instruction; wherein the data processing instruction is a processing instruction for a processing event related to the target user and the object statistics;
[0145] The metadata determination module 620 is used to determine, from the target metadata database, metadata whose field value of the specified metadata field is not the specified content and whose owner is the target user;
[0146] The processing result determination module 630 is used to determine the processing result of the data processing instruction based on the object represented by the determined metadata.
[0147] In the data processing device provided by the embodiment of the present invention, after obtaining the data processing instruction, the metadata whose field value of the specified metadata field is not the specified content and whose owner is the target user is determined from the target metadata database; then, based on the determined metadata, the processing result of the data processing instruction is determined. Since in the data processing process, the metadata whose field content of the specified metadata field is not the specified value and whose owner is the target user is determined from the target metadata database, and then the processing result is determined using the determined metadata, the objects counted for the processing event can only include non-expired objects. Therefore, the accuracy of the processing results of the processing events related to object statistics can be improved through the data processing method.
[0148] Optionally, the instruction obtaining module 610 is specifically used to:
[0149] Get object capacity statistics command;
[0150] The processing result determination module 630 is specifically used to calculate the total space occupied by the object represented by the determined metadata as the processing result corresponding to the object capacity statistics instruction.
[0151] Based on the above object deletion method, an embodiment of the present invention further provides an electronic device, such as Figure 7As shown, it includes a processor 701, a communication interface 702, a memory 703 and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704.
[0152] Memory 703, used for storing computer programs;
[0153] The processor 701 is used to implement the steps of the object deletion method provided in the embodiment of the present invention when executing the program stored in the memory 703.
[0154] Based on the above data processing method, an embodiment of the present invention further provides an electronic device, such as Figure 8 As shown, it includes a processor 801, a communication interface 802, a memory 803 and a communication bus 804, wherein the processor 801, the communication interface 802, and the memory 803 communicate with each other through the communication bus 804.
[0155] Memory 803, used for storing computer programs;
[0156] The processor 801 is used to implement the steps of the data processing method provided in the embodiment of the present invention when executing the program stored in the memory 803.
[0157] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
[0158] The communication interface is used for communication between the above electronic device and other devices.
[0159] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0160] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0161] In another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned object deletion method are implemented.
[0162] In another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned data processing method are implemented.
[0163] In another embodiment of the present invention, a computer program product including instructions is provided, which, when executed on a computer, enables the computer to execute the steps of the object deletion method in the above embodiment.
[0164] In another embodiment of the present invention, a computer program product including instructions is provided, which, when executed on a computer, enables the computer to execute the steps of the data processing method in the above embodiment.
[0165] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.
[0166] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0167] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0168] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.
Claims
1. A method for deleting an object, It is characterized in that include: When a triggering timing for deleting an object of a target user is reached, determining at least one object to be analyzed in the object storage server; wherein metadata of the object in the object storage server is stored in a target metadata database, and a specified metadata field for indicating whether the object is expired is preset in the target metadata database; For each object, when it is determined based on the set deletion condition that the object belongs to an object to be deleted, the object is determined as an object to be deleted, and the field value of the specified metadata field of the object is set using the specified content; wherein the specified content is content used to indicate that the object is expired; Deleting the object to be deleted in the object storage server; Before determining the object as an object to be deleted, the method further includes: Determine pre-set protection conditions; Determining whether the object meets the protection condition; If it is determined that the object does not meet the protection condition, the step of determining the object as an object to be deleted is executed; if it meets the condition, the deletion process of the object is terminated.
2. The method according to claim 1, It is characterized in that The condition content of the deletion condition at least includes a first object range used to represent the range of objects to be deleted; The first object range is represented by a combination of an operator and identification content, the operator includes match or mismatch, and the identification content includes: prefix and / or label information.
3. The method according to claim 1, It is characterized in that The condition content of the protection condition at least includes a second object range for indicating a range of objects to be deleted; The second object range is represented by a combination of an operator and identification content, the operator includes match or non-match, and the identification content includes: prefix and / or label information.
4. The method according to claim 1, It is characterized in that Determining the object as an object to be deleted includes: Adding the object identifier of the object to a target list; wherein the target list is used to record a list of objects to be deleted; The deleting the object to be deleted in the object storage server includes: An object identifier is read from the target list, and an object having the read object identifier is deleted from the object storage server.
5. The method according to claim 1, It is characterized in that The deleting the object to be deleted in the object storage server includes: The object to be deleted in the object storage server is deleted through an asynchronous deletion thread; wherein the asynchronous deletion thread is a thread running in a device with a load lower than a predetermined threshold.
6. A data processing method based on the method according to any one of claims 1 to 5, It is characterized in that include: Obtaining a data processing instruction; wherein the data processing instruction is a processing instruction for a processing event related to object statistics of a target user; Determine, from the target metadata database, metadata whose field value of the specified metadata field is not the specified content and whose owner is the target user; Based on the determined metadata, a processing result of the data processing instruction is determined.
7. The method according to claim 6, It is characterized in that The obtaining of the data processing instruction comprises: Get object capacity statistics command; The determining, based on the determined metadata, a processing result of the data processing instruction includes: The total space occupied by the object represented by the determined metadata is calculated as a processing result corresponding to the object capacity statistics instruction.
8. An object deletion device, It is characterized in that include: An object determination module, configured to determine at least one object to be analyzed in the object storage server when a triggering timing for deleting the target user's object is reached; wherein the metadata of the object in the object storage server is stored in a target metadata database, and a specified metadata field for indicating whether the object is expired is preset in the target metadata database; The object processing module is used for determining, for each object, when it is determined based on the set deletion condition that the object belongs to an object to be deleted, the object as an object to be deleted, and setting the field value of the specified metadata field of the object by using the specified content; wherein the specified content is content used to indicate that the object is expired; An object deletion module, used to delete the object to be deleted in the object storage server; The object processing module is also used to determine a preset protection condition before determining the object as an object to be deleted; determine whether the object meets the protection condition; if it is determined that the object does not meet the protection condition, execute the step of determining the object as an object to be deleted; if it is satisfied, end the deletion process of the object.
9. A data processing device based on the device according to claim 8, It is characterized in that include: An instruction acquisition module, used to obtain a data processing instruction; wherein the data processing instruction is a processing instruction for a processing event related to the target user and the object statistics; A metadata determination module, used to determine, from a target metadata database, metadata whose field value of a specified metadata field is not specified content and whose owner is the target user; The processing result determination module is used to determine the processing result of the data processing instruction based on the determined metadata.
10. An electronic device, It is characterized in that It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method described in any one of claims 1 to 5, or the method described in any one of claims 6 to 7, when executing a program stored in a memory.
11. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of claims 1 to 5 or the method described in any one of claims 6 to 7 is implemented.
