Multi-version file management method and storage system

Abstract

The invention discloses a multi-version file management method and a storage system, relates to the technical field of distributed storage, is used for providing a multi-version file management scheme, and provides the multi-version file management method aiming at the problem caused by lack of an effective multi-version file management and organization mechanism at present. Version information, including historical version information and a current version number, of the file can be recorded; wherein the historical version information records the information when different versions of the same-name file are uploaded each time, and at least comprises the version number of the version file. The current version number is used for indicating the current version of the multi-version file. According to the method, the version information between different versions of the same-name file can be determined, and the version change track can be traced clearly, so that the problems of version conflicts, errors and the like are avoided, and the accuracy and availability of data in a storage system are improved.

Description

Technical Field

[0001] The present invention relates to the field of distributed storage technology, and in particular to a multi-version file management method and storage system. Background Art

[0002] Nowadays, with the rapid development of digital information technology, object multi-version management is of great significance in many application scenarios such as software development, document management, and database systems. However, the current storage system in multi-protocol intercommunication scenarios still has many deficiencies in handling object multi-versions. Among them, the most critical one is the version control confusion.

[0003] The current storage system lacks an effective organization and management mechanism in the version control link. This makes the version information unclear, which easily leads to version conflicts and errors. The relationship between versions is difficult to clarify, and the change track of versions cannot be clearly traced, which has a serious adverse impact on the accuracy and availability of data. In addition, the chaotic version control will further lead to problems such as slow access speed and low storage efficiency, which seriously affects the performance of the storage system.

[0004] Therefore, technicians in this field are in urgent need of a multi-version file management method to solve the version control confusion problem existing in the current storage system in a multi-protocol intercommunication scenario. Summary of the invention

[0005] The purpose of the present invention is to provide a multi-version file management method and storage system to solve the version control confusion problem existing in the current storage system in a multi-protocol intercommunication scenario.

[0006] In order to solve the above technical problems, the present invention provides a multi-version file management method in a multi-protocol intercommunication scenario, comprising:

[0007] When a file is uploaded, the metadata information of the uploaded file is recorded;

[0008] Determine, based on the file name in the metadata information, whether the uploaded file is uploaded for the first time;

[0009] If it is the first upload, create a version information item corresponding to the uploaded file, and save the version number of the uploaded file as the historical version information and the current version number in the version information item;

[0010] If it is not the first upload, the version number is used as the newly added historical version information and saved in the version information item created when the file with the same name is first uploaded, and the current version number in the version information item is updated according to the version number.

[0011] In a possible embodiment, the historical version information is a key-value pair, the key of the historical version information is a version number, and the value includes: an object name;

[0012] The object name is the object name of the uploaded file stored in the storage system.

[0013] In a possible embodiment, it further includes:

[0014] When receiving a file download request, parsing the file download request to obtain a file name of a target file;

[0015] According to the file name, query the current version number in the corresponding version information item;

[0016] According to the object name of the historical version information corresponding to the current version number, query the corresponding metadata information;

[0017] The data of the target file is read according to the data location information recorded in the metadata information.

[0018] In a possible embodiment, it further includes:

[0019] When receiving a request to download a file with a specified version number, parsing the request to download a file to obtain a file name and a target version number of the target file;

[0020] According to the file name, query the historical version information in the corresponding version information item;

[0021] According to the historical version information corresponding to the target version number, query the corresponding metadata information;

[0022] The data of the target file is read according to the data location information recorded in the metadata information.

[0023] In a possible embodiment, it further includes:

[0024] When receiving a file deletion request, parsing the file deletion request to obtain a file name of a target file;

[0025] According to the file name, query the current version number in the corresponding version information item;

[0026] According to the object name of the historical version information corresponding to the current version number, query the corresponding metadata information;

[0027] The data and metadata information of the target file are deleted according to the data location information recorded in the metadata information.

[0028] In a possible embodiment, the value of the historical version information further includes: a version attribute; the version attribute includes: a normal version and a deleted version, and the version attribute defaults to a normal version;

[0029] After finding the historical version information corresponding to the current version number, the method further includes:

[0030] The version attribute in the historical version information is set to the deleted version.

[0031] In a possible embodiment, the value of the historical version information further includes: a version attribute; the version attribute includes: a normal version and an empty version, and the version attribute defaults to a normal version;

[0032] The version information item also includes: version number; the version number defaults to 0;

[0033] When judging whether the uploaded file is uploaded for the first time according to the file name in the metadata information, the method further includes:

[0034] Determine whether the multi-version switch is in the on state; wherein the multi-version switch includes three states: on, off, and paused, and the on state and the paused state are prohibited from jumping to the off state;

[0035] If it is the open state, then if it is the first upload, the step of creating a version information item corresponding to the uploaded file and saving the version number of the uploaded file as the historical version information and the current version number in the version information item also includes:

[0036] The file name of the uploaded file is used as the object name, and the version attribute is set to the normal version, and the value of the version number is increased by 1;

[0037] If it is the open state, if it is not the first upload, the step of taking the version number as the newly added historical version information, saving it in the version information item created when the file with the same name is first uploaded, and updating the current version number in the version information item according to the version number also includes:

[0038] The version number is used as the object name, and the version attribute is set to a normal version, and the value of the version number is increased by 1;

[0039] If it is not in the open state, then if it is the first upload, the step of creating a version information item corresponding to the uploaded file and saving the version number of the uploaded file as the historical version information and the current version number in the version information item also includes:

[0040] The file name of the uploaded file is used as the object name, a preset mark is added to the object name, and the version attribute is set to an empty version;

[0041] If it is not the open state, then if it is not the first upload, the version number is used as the newly added historical version information, saved in the version information item created when the file with the same name is first uploaded, and the step of updating the current version number in the version information item according to the version number also includes:

[0042] Add the preset mark to the version number and use the version number as the object name;

[0043] Determine whether the uploaded file is the first version file, and if so, overwrite the historical version information in the version information item.

[0044] In a possible embodiment, it further includes:

[0045] When receiving a file deletion request with a specified version number, parsing the file deletion request to obtain the file name and target version number of the target file;

[0046] According to the file name, query the historical version information in the corresponding version information item;

[0047] According to the historical version information corresponding to the target version number, query the corresponding metadata information;

[0048] According to the data location information recorded in the metadata information, the data and metadata information of the target file are deleted, and the corresponding historical version information in the version information item is deleted.

[0049] To solve the above technical problems, the present invention also provides a storage system, including: an access interface layer, a version control layer and a data storage layer;

[0050] The access interface layer is used to provide a unified access interface to the outside world, and has a communication relationship of request and data with the version control layer and the data storage layer;

[0051] The version control layer is used to execute the steps of the multi-version file management method as described above;

[0052] The data storage layer is used to provide a data storage function.

[0053] In a possible embodiment, the storage medium of the data storage layer includes: a solid state hard disk and a mechanical hard disk;

[0054] The data storage layer is also used to: store data with access frequency higher than a preset access threshold or with response speed requirements higher than a preset standard in the solid state drive; and store the remaining data in the mechanical hard drive.

[0055] In a possible embodiment, it further includes: a cache layer;

[0056] The cache layer is arranged between the version control layer and the access interface layer, and is used to predict the version data that may be accessed according to the access mode and historical access data, and load the version data into the cache layer in advance.

[0057] In a possible embodiment, the data storage layer is further used to: deduplicate the same data stored in the data storage layer, and share the same data between different objects by means of pointers or references;

[0058] The data stored in the data storage layer is compressed.

[0059] The present invention provides a multi-version file management method, which, when uploading a file, in addition to recording the completion of file storage and recording the file's metadata information, also records the file's version information (i.e., the information in the version information item), including historical version information and the current version number. The historical version information is information recorded each time a different version of a file with the same name is uploaded, including at least the version number of the version file. The current version number is used to indicate the current version (i.e., the latest version) of the multi-version file. In addition, of all the information in the version information item, only the first version of the file is recorded, and the secondary version only updates the corresponding version number as the newly added historical version information to the version information item of the first version for easy management.

[0060] A version information management solution provided by this method can clarify the version information between different versions of files with the same name, and the version change trajectory can be clearly traced, thereby avoiding version conflicts and errors, which is conducive to improving the accuracy and availability of data in the storage system.

[0061] The storage system provided by the present invention corresponds to the above method and has the same effect as above. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0063] Figure 1 A flowchart of a multi-version file management method provided by an embodiment of the present invention;

[0064] Figure 2 A schematic diagram of a version information management solution provided by an embodiment of the present invention;

[0065] Figure 3 A flowchart of version information management when uploading a file provided by an embodiment of the present invention;

[0066] Figure 4 An architecture diagram of a storage system provided by an embodiment of the present invention;

[0067] Figure 5 A structural diagram of a multi-version file management device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0068] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0069] The core of the present invention is to provide a multi-version file management method and storage system.

[0070] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0071] In today's era of rapid development of digital information, data storage and management have become increasingly complex and critical. Object multi-version management is of great significance in many application scenarios, such as software development, document management, database systems, etc. However, current storage systems have many shortcomings when dealing with multi-version objects, such as confusing version control, low storage efficiency, and slow access speed.

[0072] The current storage system lacks an effective organization and management mechanism in the version control link. This makes the version information unclear, which easily leads to version conflicts and errors. The relationship between versions is difficult to clarify, and the change track of versions cannot be clearly traced, which poses a serious threat to the accuracy and availability of data.

[0073] In addition, due to the lack of the above version control link, when users need to find and access a specific version of data, the current storage system needs to traverse the object name of each object stored therein, determine the data to be read, and read it. This will bring a large number of disk input and output (IO) operations, which will inevitably lead to an increase in access latency, seriously affecting the performance of the system and the user experience. Users have to wait a long time to obtain the required data, which undoubtedly reduces work efficiency and hinders the smooth development of business.

[0074] Similarly, the failure to implement a reasonable optimized storage strategy for multi-version data has resulted in a large waste of storage space. This not only leads to unnecessary consumption of storage resources, but also significantly increases storage costs. Limited storage resources are not fully utilized, affecting the economy and sustainability of the entire storage system. Storage efficiency is low.

[0075] To solve the above problems, the present invention provides a multi-version file management method, such as Figure 1 As shown, including:

[0076] S11: When a file is uploaded, metadata information of the uploaded file is recorded.

[0077] S12: According to the file name in the metadata information, determine whether the uploaded file is uploaded for the first time; if it is the first time, go to step S13; if it is not the first time, go to step S14.

[0078] S13: Create a version information item corresponding to the uploaded file, and save the version number of the uploaded file as historical version information and current version number in the version information item.

[0079] S14: The version number is saved as newly added historical version information in the version information item created when the file with the same name is uploaded for the first time, and the current version number in the version information item is updated according to the version number.

[0080] From the above, it can be seen that the multi-version file management method provided by the present invention is as follows: Figure 2As shown, the uploaded file is distinguished according to whether it is the first version. Among them, when the first version file is uploaded, in addition to saving the basic metadata information (ObjectInfo) of the file, the version number (currect_version) of the current version file and the historical version information (version, version_stat) are also recorded, thereby realizing the multi-version management of the uploaded file. Whenever a file is uploaded, if this file is the first version, a version information item is created to facilitate version management. If this file is a secondary version, the version information item of the first version is updated synchronously. Among them, the version information item includes the version number of the current version file, that is, the version number indicating the current version of the file with the same name. When the user needs to obtain the data or metadata of the current version of a file, the current version number can be used to quickly determine which version of the multiple versions of the data is the target data that the user needs to read. Similarly, the historical version information records all the version information of each file and distinguishes them with the version number, thereby realizing the multi-version management mechanism of the file. When the subsequent user obtains the data or metadata of the specified version file, the version number can be used to quickly locate the version data to be read.

[0081] It should also be noted that although the above method provides several types of version information in the version information item, this embodiment does not limit the types of version information in the version information item, and other version information can be expanded according to actual needs. For example, in a possible implementation scheme, Figure 2 As shown, the version information item also includes a version number (version_num). The version number can be used to indicate that there are several different versions of the current file with the same name.

[0082] In addition, each type of version information can be further expanded according to actual needs. For example, the above-mentioned historical version information records the version number of each uploaded version file. Furthermore, the historical version information item can also include version attributes, and the version attributes can include null (empty), norm (normal version), and delete (deleted version). The version attribute can be used to indicate the status of the version information. For example, when the version attribute is norm, it means that the multi-version file management function implemented based on this method is being implemented normally. When the version attribute is delete, it means that the file data and metadata of this version have been deleted, and only the historical version information is retained to indicate that this version data once existed. When the version attribute is null, it means that the multi-version function implemented based on this method is closed at this time, that is, the multi-version file management function implemented by this method is a function that can be freely switched on and off based on user needs, and the implementation is more flexible.

[0083] Furthermore, it can be seen from the previous embodiment that the historical version information may include multiple data, and each piece of historical version information uniquely corresponds to a version of the file with the same name. Therefore, for the storage method of the historical version information, this embodiment also provides a possible implementation scheme:

[0084] The historical version information is a key-value pair (version, version_stat). The key (key) of the historical version information is the version number (version), and the value (value) version_stat includes but is not limited to: object name.

[0085] The object name is the object name of the uploaded file stored in the storage system. As in the previous embodiment, the historical version information also includes version attributes, and the version attributes can be used as a data in the value of the historical version information. That is, the value version_stat of the historical version information at this time includes: object name and version attributes.

[0086] Based on the historical version information provided by this embodiment, when a user needs to query the data or metadata of any version file, the user can quickly retrieve the corresponding historical version information key-value pair through the version number of the target version. And by reading the value to obtain the object name stored in the cache pool or the object name stored in the data pool, the storage location of the target version data can be determined for reading or other operations.

[0087] In addition, it can be seen from the above that the version number is an important basis for distinguishing different versions of files with the same name. Since there is no version management mechanism in the original storage system. Correspondingly, different versions of files with the same name do not necessarily have unique and different version numbers. Based on this, this embodiment provides a possible version number naming rule:

[0088] The version number is named as: v random character + t timestamp.

[0089] The random characters can be the file name of the uploaded file, which is used to distinguish the version data between different files. The timestamp can be the current system time when the version file is uploaded, which is used to distinguish different uploaded versions of the same-named file. Based on the two naming fields, the generated version number can be uniquely corresponding to a version of a file. This lays a reliable foundation for the multi-version file management method provided by the present invention.

[0090] In summary, the multi-version file management method provided by the present invention provides a version data management mechanism for the storage system, which can effectively solve the problem of the lack of an effective version data management mechanism in the current storage system. Using the file name as a distinction, a version information item is created in the first version uploaded with the same name to store and manage the version information of the file with the same name. This includes but is not limited to the current version number and historical version information. The historical version information includes the version number of each time a different version of data of the same name file is uploaded, that is, when this version of the file is uploaded, in addition to recording the file data and metadata, a trace of this version file upload is also left in the version information item of the first version of the same name file, which is convenient for subsequent processing operations on different versions of data.

[0091] On the other hand, the multi-version file management method provided in the above embodiment can facilitate traversal when subsequent users perform data processing operations (such as downloading, deleting, etc.) on multi-version data, thereby improving data query and access efficiency.

[0092] Specifically, this embodiment provides a data download implementation scheme, and the method further includes:

[0093] S21: When a file download request is received, the file download request is parsed to obtain a file name of a target file.

[0094] S22: According to the file name, query the current version number in the corresponding version information item.

[0095] S23: According to the object name of the historical version information corresponding to the current version number, query the corresponding metadata information.

[0096] S24: Reading the data of the target file according to the data location information recorded in the metadata information.

[0097] Furthermore, this embodiment also provides another data downloading implementation scheme, and the method further includes:

[0098] S31: When receiving a request to download a file with a specified version number, the request is parsed to obtain a file name and a target version number of a target file.

[0099] S32: According to the file name, query the historical version information in the corresponding version information item.

[0100] S33: According to the historical version information corresponding to the target version number, query the corresponding metadata information.

[0101] S34: Read the data of the target file according to the data location information recorded in the metadata information.

[0102] In order to distinguish the data downloading scheme provided by this embodiment from that provided by the previous embodiment, the two downloading schemes are respectively referred to as the general downloading scheme and the specified version number downloading scheme according to different user requests. Among them, steps S21~S24 correspond to the general downloading scheme, which is used to retrieve the data of the current version of the target file according to the current version number recorded in the first version file. Steps S31~S34 correspond to the downloading of the specified version number, which is used to find the storage address of the data of the corresponding version from the historical version information according to the version number specified by the user (i.e., determine the object name), and find the data of the corresponding version file for downloading. This embodiment and the above embodiment are aimed at two different file downloading scenarios, which basically cover the downloading scenarios in the daily application of multi-version files. And the file downloading implemented based on this embodiment and the above embodiment can effectively utilize the multi-version file management method provided by the above embodiment, and quickly locate the version data required to be accessed and queried through the stored historical version information and the current version number, without traversing the object names of all objects from the pool to determine the target data, which greatly improves the efficiency of multi-version data reading and querying.

[0103] On the other hand, for the scenario of deleting multiple versions of files, this embodiment also provides a possible implementation scheme, and the method further includes:

[0104] S41: When a file deletion request is received, the file deletion request is parsed to obtain a file name of a target file.

[0105] S42: According to the file name, query the current version number in the corresponding version information item.

[0106] S43: According to the object name of the historical version information corresponding to the current version number, query the corresponding metadata information.

[0107] S44: Delete the data and metadata information of the target file according to the data location information recorded in the metadata information.

[0108] In addition, this embodiment also provides another possible implementation scheme for deleting multiple versions of files. The method further includes:

[0109] S51: When a file deletion request with a specified version number is received, the file deletion request is parsed to obtain the file name and target version number of the target file.

[0110] S52: According to the file name, query the historical version information in the corresponding version information item.

[0111] S53: According to the historical version information corresponding to the target version number, query the corresponding metadata information.

[0112] S54: According to the data location information recorded in the metadata information, the data and metadata information of the target file are deleted, and the corresponding historical version information in the version information item is deleted.

[0113] Similar to downloading, the two multi-version file deletion scenarios provided in this embodiment and the previous embodiment are also implementation plans respectively provided for two common file deletion requirements of users in actual applications.

[0114] Among them, steps S41~S44 are aimed at the user's deletion of the current version data of a certain file. Therefore, the previous embodiment finds the storage address of the current version file data through the current version number and historical version information recorded in the version information item of the first version, sets the file version status to the delete status, and deletes the file data and metadata. Among them, the file version status is set to the delete status because the data of the current version of the file is deleted this time, that is, the file data and metadata are deleted, but the historical version information still needs to be retained. Therefore, only the file data and metadata are deleted, the corresponding historical version information is retained, and the version status is set to delete to indicate that the file version corresponding to this historical version information has been deleted.

[0115] Steps S51 to S54 are aimed at the user's deletion of a specific version of the file data, which can be the current version or not, without restriction. And this deletion scenario also corresponds to the user's desire to completely delete the traces of the target version in the storage system, that is, all data including file data, metadata, and historical version information are deleted, reflecting another deletion requirement of the user. In this deletion process, based on the target version number to be deleted, the storage address of the file data and metadata can be quickly determined through the historical version information. After deleting the file data and metadata, the corresponding historical version information is deleted. In this case, this version of data is equivalent to not being uploaded to the storage system, and no traces are left, achieving complete deletion.

[0116] As can be seen from the above, this embodiment provides a corresponding deletion solution for two common file deletion requests in practical applications. Both solutions can quickly determine the file data and metadata to be deleted and delete them, without traversing the object names of all objects in the pool to find the target data to be deleted, which can effectively improve the efficiency of file deletion and reflect the superiority of a multi-version file management method provided by the present invention.

[0117] Furthermore, it can be seen from the embodiment corresponding to the above steps S41 to S44 that when a user deletes a version of a file, he or she has the need to delete only the file data and metadata, but retain the historical version information record of this version. At this time, in order to distinguish the historical data information of the normal version from the historical data information of the deleted version to avoid version management confusion, the above embodiment provides an implementation plan for setting the corresponding version attribute to the deleted version. In view of this, this embodiment further provides a specific implementation plan:

[0118] The value of the historical version information also includes: version attributes. The version attributes include: normal version and deleted version, and the version attribute defaults to the normal version.

[0119] After finding the historical version information corresponding to the current version number, the method further includes:

[0120] S45: Setting the version attribute in the historical version information to a deleted version.

[0121] That is, in this embodiment, the value of the historical version information includes not only the object name but also the version status. The version attributes may include but are not limited to: norm (normal version) and delete (deleted version).

[0122] Among them, norm is used to indicate that the file data and metadata corresponding to this version of the file exist and have not been deleted. Delete is used to indicate that the file data and metadata corresponding to this version of the file have been deleted, and only the historical version information is retained for the user to analyze and obtain the multi-version file information later.

[0123] Furthermore, based on the implementation scheme provided in this embodiment, the deleted file versions and the undeleted file versions in the historical version information can be clearly distinguished. When it is necessary to query and access the file data of a specific version, if the file data of this version has been deleted, it can be directly determined based on the version attribute, without having to determine the data storage address and query to know whether the file data of this version still exists, which can further improve the efficiency of data access query.

[0124] Furthermore, with respect to the version attributes provided in the above embodiment, this embodiment also provides another management solution based on the version attributes:

[0125] The value of the historical version information also includes: version attribute. The version attribute includes: normal version and empty version (null), and the version attribute defaults to the normal version.

[0126] The version information item also includes: version number; the version number defaults to 0.

[0127] Then in step S11: judging whether the uploaded file is uploaded for the first time according to the file name in the metadata information, the following is also included:

[0128] S10: Determine whether the multi-version switch is turned on.

[0129] The multi-version switch includes three states: on, off, and pause. The on and pause states are prohibited from jumping to the off state.

[0130] If it is in the on state, step S12 is specifically as follows:

[0131] S12-A: The file name of the uploaded file is used as the object name, and the version attribute is set to the normal version and the value of the version number is increased by 1.

[0132] If it is in the on state, step S13 is specifically as follows:

[0133] S13-A: Use the version number as the object name, set the version attribute to the normal version, and increase the value of the version number by 1.

[0134] If it is not in the open state, step S12 is specifically as follows:

[0135] S12-B: The file name of the uploaded file is used as the object name, a preset tag is added to the object name, and the version attribute is set to the empty version.

[0136] If it is not in the open state, step S13 is specifically as follows:

[0137] S13-B1: Add a preset tag to the version number and use the version number as the object name.

[0138] S13-B2: Determine whether the uploaded file is the first version file, and if so, overwrite the historical version information in the version information item.

[0139] Based on the implementation scheme provided in this embodiment, the multi-version file management function can be freely controlled to be turned on and off, and a more flexible multi-version file management mechanism is realized. Based on the different states of turning on and off the multi-version function, when there is a file upload system, its specific possible scenarios and corresponding processing flow are as follows: Figure 3 As shown, there are five situations:

[0140] 1) The uploaded file is the first version and the multi-version switch is turned on.

[0141] At this time, the corresponding version information item is created, and a series of information in the version information item is updated.

[0142] Among them, since it is the first upload, the current version number is the version number of the file version uploaded this time, and the version number is 1. The key of the historical version information is: version number, and the value is: version attribute and object name. Specifically, the version number can be the version number calculated based on the version number naming rules provided in the above embodiment. The object name can directly use the file name itself (such as file1) as the object name. The version attribute is set to the normal state (norm).

[0143] 2) The uploaded file is the first version and the multi-version switch is turned off or paused.

[0144] This is similar to the above process 1), just create a version information item and update the version information in it. The difference is that the version number of this version does not need to be distinguished according to the above version number naming rules. The file name can be used as the version number and object name, but a preset tag (such as _null) needs to be added to distinguish the version information when the multi-version function is enabled, so as to update the current version number and the key in the historical version information. In addition, the version number in this scenario no longer increases by 1.

[0145] 3) The uploaded file is a minor version and the multi-version switch is turned on.

[0146] At this time, just update the information in the corresponding version information item.

[0147] Specifically, the version number can be calculated according to the naming rules, and the version number of this version is used as the current version number, the version number is increased by 1, and the historical version information is updated.

[0148] 4) The uploaded file is a minor version and the multi-version switch is suspended.

[0149] As can be seen from the above description, once the multi-version switch is turned on, it cannot be turned off and can only be paused. This setting is because the storage system can support additional writing when the multi-version switch is turned off (that is, in the original management scheme before the implementation of this method). However, when the multi-version switch is turned on, additional writing cannot be performed, otherwise there will be multiple data corresponding to the same version, which will cause confusion in version management. Therefore, in this embodiment, once the multi-version function is turned on, additional writing of version files is prohibited, and the multi-version function cannot be turned off after it is turned on, and can only be paused, thereby avoiding management confusion.

[0150] In addition, the update of the version information item in this scenario is the same as process 2). The difference is that since this version is this version, the version number needs to be calculated according to the naming rules, but an additional preset tag needs to be added.

[0151] 5) The uploaded file is a minor version, and the multi-version switch is suspended. In addition, the storage system already has this version of the file data, and this version of the file is the first version.

[0152] At this time, the version information in the version information item is the same as the above process 4), except that the object name in the historical version information does not need to use the calculated version number, and the file name can be directly used as the object name, and the historical version information is updated in an overwritten manner.

[0153] It should also be noted that this embodiment is compatible with the version attribute scheme including the deletion version (delete) in the previous embodiment, that is, the version attribute can include: null, norm, delete at the same time, and the implementation scheme provided by this embodiment and the previous embodiment can also be implemented together.

[0154] On the other hand, based on the multi-version file management method provided in the above embodiment, this embodiment also provides a storage system, such as Figure 4 As shown, it includes: an access interface layer 11, a version control layer 12 and a data storage layer 13.

[0155] Among them, the access interface layer is used to provide a unified access interface to the outside world, and there is a communication relationship of request and data between the version control layer and the data storage layer.

[0156] The version control layer is used to execute the steps of the multi-version file management method as described in the above embodiment, which will not be repeated here. For details, please refer to the embodiment of the above method part.

[0157] The data storage layer is used to provide data storage functions.

[0158] Among them, the access interface layer serves as a bridge for users to interact with the system, providing users with a unified and concise access interface. It can shield the complex technical details of the underlying layer, so that users do not need to care about the specific implementation methods of version control and data storage. Users initiate data acquisition requests through the access interface layer, and the interface layer passes the request to the version control layer and the data storage layer for processing. When acquiring data, the version control layer first determines the version information to be acquired based on the conditions provided by the user (such as version number, time range, etc.), and then the data storage layer extracts the actual data from the corresponding storage location according to the instructions of the version control layer, and returns it to the user through the access interface layer. In the entire process, users only need to issue clear instructions through the access interface layer, and the various layers within the system can work together to efficiently complete the management of multiple versions of objects and data acquisition tasks, providing users with a smooth and accurate data service experience.

[0159] The access control layer is the core of the system and is used to implement a multi-version file management method provided in the above embodiment. The specific embodiment can be found in the embodiment of the above method part, which will not be described in detail here.

[0160] When new data needs to be stored, the data storage layer can properly place the data according to the predetermined strategy and establish the corresponding index for fast retrieval. When the version control layer indicates that a certain version of data needs to be updated, queried or deleted, the data storage layer can accurately locate and operate the corresponding data block.

[0161] It is easy to know that the version control layer in this system is used to implement the multi-version file management method provided by the above embodiment, which can bring the advantages of the above multi-version file management method. In addition, this system implements the above multi-version file management method based on a layered design. This layered design and separation strategy enables the system to easily cope with different business needs and data scales. When it is necessary to add new functions or expand the system, only the corresponding layer needs to be modified and expanded without affecting other parts, thereby greatly improving the flexibility and scalability of the system.

[0162] In addition, for the data storage layer in the system, this embodiment also provides a further implementation scheme:

[0163] The storage media of the data storage layer include: solid state drives and mechanical hard drives.

[0164] The data storage layer is also used to: store data with access frequency higher than a preset access threshold or with response speed requirements higher than a preset standard in a solid-state drive; and store the rest of the data in a mechanical hard drive.

[0165] That is, the data storage layer of this embodiment can intelligently select the most appropriate storage medium and data structure according to factors such as the characteristics, scale, and access frequency of the data. For data that is frequently accessed and has extremely high speed requirements, a solid-state drive may be selected for storage. For large-scale data with relatively low access frequency, a mechanical hard disk may be used for storage. This achieves the purpose of using different storage media based on different data characteristics to further improve access speed.

[0166] On the other hand, this embodiment also provides another possible implementation scheme for the data storage layer:

[0167] The data storage layer is also used to: deduplicate the same data stored in the data storage layer, and share the same data between different objects through pointers or references; and compress the data stored in the data storage layer.

[0168] In this embodiment, in order to save storage space occupied by multiple historical version objects, data compression and data deduplication can be performed at the data storage layer. Data compression is to compress the stored data to reduce the storage space occupied. Data deduplication is to store only one copy of the same data content, share it through pointers or references, avoid repeated storage, and further improve the utilization of storage space.

[0169] On the other hand, this embodiment also provides a further embodiment, and the storage system also includes: a cache layer.

[0170] The cache layer is set between the version control layer and the access interface layer. It is used to predict the version data that may be accessed based on the access pattern and historical access data, and load the version data into the cache layer in advance.

[0171] This embodiment adds a cache layer between the version control layer and the access interface layer, and the cache layer implements pre-fetching, caching and other technologies to further improve the speed of version search and access and reduce the response time of the system. Pre-fetching can be: predicting the version data that the user may access based on the access pattern and historical data, and loading it into the cache in advance. Caching is: using a memory cache or a distributed cache system to store frequently accessed version data to improve access speed.

[0172] In a possible embodiment, the above prediction scheme specifically includes:

[0173] S61: For each registered user, a corresponding access behavior habit model is established.

[0174] Among them, registered users are legal users registered in the storage system. The access behavior habit model is trained based on the deep learning algorithm using the historical data of the corresponding user as the training data set. It should be noted that a set of data in the training data set includes: access mode, access time (system time when the access occurs) and access object (accessed version data). The historical access data of each registered user is used as an independent training data set to complete the training of the corresponding access behavior habit model. Among them, the access behavior habit model takes the access mode and access time as input and the access object as output.

[0175] When the access behavior habit model corresponding to each registered user is trained, it also includes:

[0176] S62: According to the current access mode and system time of the storage system, predict one or more versions of data that each registered user is most likely to access in the time period of the current system time as predicted access version data through each access behavior habit model.

[0177] The length of the time period can be freely set according to computing power, access requirements, etc., such as 1 minute, 1 hour, etc. And whenever the system time reaches the next time period, step S62 is repeated to update the predicted access version data pre-stored in the cache layer.

[0178] S63: Load the predicted access version data into the cache layer. When a version data access request takes any predicted access version data in the cache layer as an access object, the predicted access version data is read from the cache layer and returned as a request response result.

[0179] Based on the pre-storage solution provided in this embodiment, the registered users of the storage system can be taken as the object, and their access behavior habits can be analyzed based on the historical data of each registered user and a corresponding access behavior habit model can be established. Therefore, in the subsequent version data access management process, the storage system can predict one or more version data that each registered user is most likely to access in the current time period based on the current access mode and system time, and pre-store these version data in the cache layer. When a request with these predicted access version data as the access object is received, the version data can be directly read from the cache layer and returned, achieving higher access efficiency. In addition, in this solution, considering the limited computing power of the storage system, and in order to ensure the accuracy of the prediction and avoid resource waste (non-registered users may not have enough historical data as a training data set), only the registered users of the storage system are provided with pre-storage services to better meet the needs of actual storage system applications.

[0180] In addition to the embodiment of a multi-version file management method provided in the above embodiment, the present invention also provides an embodiment corresponding to a computer program product. A computer program product includes a computer program / instruction, and when the computer program / instruction is executed by a processor, the steps of the multi-version file management method described in any of the above embodiments can be implemented.

[0181] Since the embodiments of the computer program product part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the computer program product part, which will not be repeated here.

[0182] In the above embodiment, a multi-version file management method is described in detail, and the present invention also provides a corresponding embodiment of a multi-version file management device. It should be noted that the present invention describes the embodiment of the device part from two perspectives, one is based on the functional module perspective, and the other is based on the hardware perspective.

[0183] Based on the perspective of functional modules, this embodiment provides a multi-version file management device, including:

[0184] The metadata recording module is used to record the metadata information of the uploaded file when a file is uploaded;

[0185] The first version judgment module is used to judge whether the uploaded file is uploaded for the first time according to the file name in the metadata information. If it is the first upload, the first version management module is triggered; if it is not the first upload, the second version management module is triggered;

[0186] The first version management module is used to create a version information item corresponding to the uploaded file, and save the version number of the uploaded file as the historical version information and the current version number in the version information item;

[0187] The minor version management module is used to save the version number as the newly added historical version information in the version information item created when the file with the same name is first uploaded, and update the current version number in the version information item according to the version number.

[0188] Since the embodiments of the apparatus part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the apparatus part, which will not be repeated here.

[0189] Figure 5 A structural diagram of a multi-version file management device provided by another embodiment of the present invention is shown in FIG. Figure 5 As shown, a multi-version file management device includes: a memory 20, for storing computer programs;

[0190] The processor 21 is used to implement the steps of a multi-version file management method in the above embodiment when executing a computer program.

[0191] The multi-version file management device provided in this embodiment may include but is not limited to a mobile terminal, a personal computer, a workstation, etc.

[0192] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 can be implemented in at least one hardware form of a digital signal processor (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0193] The memory 20 may include one or more computer-readable storage media, which may be non-transitory. The memory 20 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In this embodiment, the memory 20 is at least used to store the following computer program 201, wherein, after the computer program is loaded and executed by the processor 21, it can implement the relevant steps of a multi-version file management method disclosed in any of the aforementioned embodiments. In addition, the resources stored in the memory 20 may also include an operating system 202 and data 203, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 202 may include Windows, Unix, Linux, etc. Data 203 may include, but is not limited to, a multi-version file management method, etc.

[0194] In some embodiments, a multi-version file management device may further include a display screen 22 , an input / output interface 23 , a communication interface 24 , a power supply 25 , and a communication bus 26 .

[0195] Those skilled in the art will understand that Figure 5 The structure shown in does not constitute a limitation on a multi-version file management device, and may include more or fewer components than those shown in the figure.

[0196] A multi-version file management device provided by an embodiment of the present invention includes a memory and a processor. When the processor executes a program stored in the memory, it can implement the following method: a multi-version file management method.

[0197] Finally, the present invention also provides an embodiment corresponding to a non-volatile storage medium. The non-volatile storage medium stores a computer program, and when the computer program is executed by a processor, the steps recorded in the above method embodiment are implemented.

[0198] It is understandable that if the method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc. Various media that can store program codes.

[0199] The above is a detailed introduction to a multi-version file management method, device, medium and storage system provided by the present invention. The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the embodiments can be referenced to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, the present invention can also be improved and modified in a number of ways, and these improvements and modifications also fall within the scope of protection of the present invention.

[0200] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A multi-version file management method, characterized in that: include: When a file is uploaded, the metadata information of the uploaded file is recorded; Determine, based on the file name in the metadata information, whether the uploaded file is uploaded for the first time; If it is the first upload, a version information item corresponding to the uploaded file is created, and the version number of the uploaded file is stored in the version information item as the historical version information and the current version number; If it is not the first upload, the version number is used as the newly added historical version information and saved in the version information item created when the file with the same name is first uploaded, and the current version number in the version information item is updated according to the version number.

2. The multi-version file management method according to claim 1, characterized in that: The historical version information is a key-value pair, the key of the historical version information is the version number, and the value includes: the object name; The object name is the object name of the uploaded file stored in the storage system.

3. The multi-version file management method according to claim 2, characterized in that: Also includes: When receiving a file download request, parsing the file download request to obtain a file name of a target file; According to the file name, query the current version number in the corresponding version information item; According to the object name of the historical version information corresponding to the current version number, query the corresponding metadata information; The data of the target file is read according to the data location information recorded in the metadata information.

4. The multi-version file management method according to claim 2, characterized in that: Also includes: When receiving a request to download a file with a specified version number, parsing the request to download a file to obtain a file name and a target version number of the target file; According to the file name, query the historical version information in the corresponding version information item; According to the historical version information corresponding to the target version number, query the corresponding metadata information; The data of the target file is read according to the data location information recorded in the metadata information.

5. The multi-version file management method according to claim 2, characterized in that: Also includes: When receiving a file deletion request, parsing the file deletion request to obtain a file name of a target file; According to the file name, query the current version number in the corresponding version information item; According to the object name of the historical version information corresponding to the current version number, query the corresponding metadata information; The data and metadata information of the target file are deleted according to the data location information recorded in the metadata information.

6. The multi-version file management method according to claim 5, characterized in that: The value of the historical version information also includes: version attributes; the version attributes include: normal version and deleted version, and the version attribute defaults to the normal version; After finding the historical version information corresponding to the current version number, the method further includes: The version attribute in the historical version information is set to the deleted version.

7. The multi-version file management method according to any one of claims 2 to 6, characterized in that: The value of the historical version information also includes: version attributes; the version attributes include: normal version and empty version, and the version attribute defaults to the normal version; The version information item also includes: version number; the version number defaults to 0; When judging whether the uploaded file is uploaded for the first time according to the file name in the metadata information, the method further includes: Determine whether the multi-version switch is in the on state; wherein the multi-version switch includes three states: on, off, and paused, and the on state and the paused state are prohibited from jumping to the off state; If it is the open state, the step of creating a version information item corresponding to the uploaded file if it is the first upload, and saving the version number of the uploaded file as the historical version information and the current version number in the version information item also includes: The file name of the uploaded file is used as the object name, and the version attribute is set to the normal version, and the value of the version number is increased by 1; If it is the open state, then if it is not the first upload, the version number is used as the newly added historical version information, saved in the version information item created when the file with the same name is first uploaded, and the step of updating the current version number in the version information item according to the version number also includes: The version number is used as the object name, and the version attribute is set to a normal version, and the value of the version number is increased by 1; If it is not in the open state, then if it is the first upload, the step of creating a version information item corresponding to the uploaded file and saving the version number of the uploaded file as the historical version information and the current version number in the version information item also includes: The file name of the uploaded file is used as the object name, a preset mark is added to the object name, and the version attribute is set to an empty version; If it is not the open state, then if it is not the first upload, the version number is used as the newly added historical version information, saved in the version information item created when the file with the same name is first uploaded, and the step of updating the current version number in the version information item according to the version number also includes: Add the preset mark to the version number and use the version number as the object name; Determine whether the uploaded file is the first version file, and if so, overwrite the historical version information in the version information item.

8. The multi-version file management method according to claim 2, characterized in that: Also includes: When receiving a file deletion request with a specified version number, parsing the file deletion request to obtain the file name and target version number of the target file; According to the file name, query the historical version information in the corresponding version information item; According to the historical version information corresponding to the target version number, query the corresponding metadata information; According to the data location information recorded in the metadata information, the data and metadata information of the target file are deleted, and the corresponding historical version information in the version information item is deleted.

9. A storage system, characterized in that: include: Access interface layer, version control layer and data storage layer; The access interface layer is used to provide a unified access interface to the outside world, and has a communication relationship of request and data with the version control layer and the data storage layer; The version control layer is used to execute the steps of the multi-version file management method according to any one of claims 1 to 8; The data storage layer is used to provide data storage function.

10. The storage system according to claim 9, characterized in that: The storage media of the data storage layer include: solid state hard disk and mechanical hard disk; The data storage layer is also used to: store data with access frequency higher than a preset access threshold or with response speed requirements higher than a preset standard in the solid state drive; and store the remaining data in the mechanical hard drive.

11. The storage system according to claim 9, characterized in that: Also includes: Cache layer; The cache layer is arranged between the version control layer and the access interface layer, and is used to predict the version data that may be accessed according to the access mode and historical access data, and load the version data into the cache layer in advance.

12. The storage system according to claim 9, characterized in that: The data storage layer is further used to: deduplicate the same data stored in the data storage layer, and share the same data between different objects by means of pointers or references; The data stored in the data storage layer is compressed.

Images